Python Sender

Last week I played my first Capture The Flag (CTF) where I really tried solving the challenges for a couple of hours. It was a regular jeopardy style CTF with binaries, web applications and other server ports. I don’t think CTFs are going to be my favourite hobby, as pentesting is similar but just a little bit more real life. However, CTFs are very nice for people who want to get into IT security, so I wanted to help a little bit in the team I joined. This particular CTF by Kaspersky really annoyed me though, as the servers were very often offline (HTTP 500 errors). Moreover, some challenges allowed easy Remote Command Execution (RCE) and I guess some teams took the chance to prevent other teams from scoring flags. As I just said I’m not very experienced with CTFs, maybe that’s how it’s supposed to be, but for me that’s silly. Anyway, this post is about something more positive: A Python script to play CTFs, but can also be used during pentests. For those who play CTFs very often, it’s probably better to use a full library such as pwntools, but if you just want a small script where you can delete whatever you don’t need and go with the POC||GTFO flow, you’ve come to the right place.

I think two of the mostly presented CTF challenges often look the same. You either get a URL to a challenge website and you have to do some HTTP magic or you get something like “nc www.example.org 1337” where you are supposed to talk to a server with netcat. Now both challenges usually use TCP/IP and maybe TLS. The website obviously uses HTTP(S) on top of that. So very often you find yourself sending a lot of HTTP requests or a lot of TCP packets to a certain port. Pentests also require the same sometimes.

To make sure we don’t have to fight if Python 2.7 is better than Python 3.6, the script I wrote works on both versions. But even then, people might argue that python’s urllib or urllib2 is sufficient or that they rather use the non-standard requests library. And others will simply say that only asynchronous network IO is really fast enough, so they prefer to use Python Twisted (or treq). However, I got all of these cases covered in the script.

The script allows arbitrary socket and HTTP(S) connections via:

  • socket and ssl-wrapped sockets – when you need bare bone or non-HTTP(S)
  • python urllib/urllib2 HTTP(S) library – when you need HTTP(S) and a little bit more automated HTTP feature handling
  • python requests HTTP(S) library – when you need HTTP(S) and full HTTP feature handling
  • python treq (uses Python Twisted and therefore asynchronous IO) – when you need full HTTP(S) feature handling and speed is important

The main features are:

  • Works under python 2.7 and python 3 (although treq here is untested under python 2.7)
  • You can just copy and paste an HTTP(S) request (e.g. from a proxy software) without worrying about the parsing and other details
  • You can also use the sockets functions to do non-HTTP related things
  • Ignores any certificate warnings for the server

It should be helpful when:

  • You want to script HTTP(S) requests (e.g. just copy-paste from a proxy like Burp), for example during a pentest or CTF
  • When you encounter a CTF challenge running on a server (like “nc example.org 1234”) or a proprietary TCP protocol during pentests

Howto:

  • Change the variables START, END and TLS
  • Optional: Change further configuration options, such as sending the HTTP(S) requests through a proxy
  • Change the ‘main’ function to send the request you would like to. By default it will send 3 HTTP requests to www.example.org with every library.

Enough words, head over to github to download the Python Sender.

Java Bugs with and without Fuzzing – AFL-based Java fuzzers and the Java Security Manager

This is a cross-posting of a blog post already published on the modzero company blog. If you’ve already read that, there is no need to read this, as both blog posts have the same content.

In the last half a year I have been doing some fuzzing with AFL-based Java fuzzers, namely Kelinci and JQF. I didn’t really work with java-afl. The contents of this post are:

Various AFL-based Java fuzzers are available that can be used to find more or less severe security issues. By combining these with sanitizers provided by the Java Security Manager, additional instrumentation can be achieved.

This blog post will mention several files, they are included on https://github.com/modzero/mod0javaFuzzingResults. Additionally, the zip file includes several other files that reproduce the same bugs.

AFL-based Java fuzzing tools

The AFL fuzzer is really popular nowadays as it performs instrumented fuzzing. If you are not familiar with AFL, it’s probably better if you at least quickly look at AFL before you read this post. It is especially important to understand how AFL handles hangs (test cases that take too much time to process) and crashes (e.g. target program segfault).

Kelinci is one of the first AFL for Java implementations and is very promising, although the approach with having two processes per fuzzing instance is a little clumsy and can get confusing. One process is the native C side, which takes mutated inputs produced by AFL and sends them to the second process via TCP socket. The second process is the Java process that feeds the input to the target program and sends back the code paths taken with this input. There are certain error messages in the Java part of this fuzzer that are not always exactly clear (at least to me), but they seem to indicate that the fuzzer is not running in a healthy state anymore. However, so far Kelinci worked very well for me and came up with a lot of results. There has not been any development for 7 months, so I hope the author will pick it up again.

JQF is actively maintained and the last changes were commited a couple of days ago. It does not take the classic fuzzer approach that most fuzzers for security researchers take, but instead is based on Java Unit Tests and focuses more on developers. It currently has only limited support of AFL’s -t switch for the timeout settings and there is also only rudimentary afl-cmin support. While this is perfect for developers using Unit Tests, it is not the most flexible fuzzer for security researchers fuzzing Java code.

java-afl has not been updated in four months. This is actually the fuzzer I didn’t successfully use at all. I tried to ask the developer about how to run it properly, but didn’t get an answer that would help me run it on the test case I had in mind. If you have better luck with java-afl, please let me know, it would be interesting to hear how this fuzzer performs.

First steps with Apache Commons

I started with the Apache Common’s Imaging JPEG parser as a target. The choice was simple because it was one of the examples explained for the Kelinci fuzzer. Apache Commons is a very popular library for all kind of things that are missing or incomplete in the Java standard library. When going through the author’s example, I realized that he gave the fuzzer only one input file containing the text “hello”, which is not a JPEG file and not a very good starting corpus. While it’s probably lcamtuf’s very interesting experiment that makes people believe using such corpus data is a valid choice, it is not a valid choice for proper fuzzing runs. lcamtuf’s experiment was good to proof the point that the fuzzer is smart, but for productive fuzzing proper input files have to be used to achieve good results. Fuzzing is all about corpus data in the end. So I took the JPEG files in lcamtuf’s corpus on the AFL website and some files from my private collection. The fuzzer quickly turned up with an additional ArrayIndexOutOfBoundsException which I reported to Apache (file ArrayIndexOutOfBoundsException_DhtSegment_79.jpeg). That was quite an easy start into Java fuzzing. If you would do the same for other parsers of Apache Commons (for example PNG parser), you would most probably find some more unchecked exceptions.

Goals: Taking a step back

After this quick experiment I gave the idea of fuzzing Java some more thoughts. Fuzzing is originally applied to programs that are not memory safe, hoping that we are able to find memory corruption issues. Out of bound read or writes in Java code simply do not result in memory corruption but in more or less harmless Exceptions such as IndexOutOfBoundsException. While it might be desirable to find (code robustness) issues and might result in Denial of Service issues, the severity of these issues is usually low. The question is what kind of behavior and fuzzing results are we looking for? There are different scenarios that might be of interest, but the attack vector (how does the attacker exploit the issue in the real world?) matters a lot when looking at them. Here is my rough (over)view on Java fuzzing:

  • Finding bugs in the JVM.
    • Arbitrary Java code as input. This could be helpful in more exotic scenarios, for example when you need to escape from a sandboxed JVM. In most other scenarios this attack vector is probably just unrealistic, as an attacker would be executing Java code already.
    • Feeding data into built-in classes/functions (fuzzing the standard library), such as strings. This is not very likely to come up with results, but you never know, maybe there are Java deserialization vulnerabilities lurking deep in the JVM code?
    • Finding low-severity or non-security issues such as code that throws an Exception it didn’t declare to throw (RuntimeExceptions).
  • Finding memory corruption bugs in Java code that uses native code (for example JNI or CNI). This is probably a very good place to use Java fuzzing, but I don’t encounter this situation very much except in Android apps. And fuzzing Android apps is an entirely different beast that is not covered here.
  • Fuzzing pure Java code.
    • We could go for custom goals. This might depend on your business logic. For example, if the code heavily uses file read/writes maybe there is some kind of race condition? Also the idea of differential fuzzing for crypto libraries makes a lot of sense.
    • Finding “ressource management” issues, such as Denial of Service (DoS) issues, OutOfMemoryExceptions, high CPU load, high disk space usage, or functions that never return.
    • Finding low-severity or non-security issues such as RuntimeExceptions.
    • Finding well-known security issues for Java code, such as Java deserialization vulnerabilities, Server Side Request Forgery (SSRF), and External Entity Injection (XXE).

I was especially interested in the last three points in this list: Finding ressource issues, RuntimeExceptions and well-known Java security issues. While I already found a RuntimeException in my little experiment described above, I was pretty sure that I would be able to detect certain ressource management issues by checking the “hangs” directory of AFL. However, the last point of finding well-known security issues such as SSRF seems tricky. The fuzzer would need additional instrumentation or sanitizers to detect such insecure behavior. Just as Address Sanitizer (ASAN) aborts on invalid memory access for native code (which then leads to a crash inside AFL), it would be nice to have sanitizers that take care about such areas in the Java world. A file sanitizer for example might take a whitelist of files that are allowed to be accessed by the process, but abort if any other file is accessed. This could be used to detect XXE and SSRF scenarios. A network sanitizer might do the same if sockets are used. Imagine a Java image file parsing library as a target. From a security perspective such a library should never open network sockets, as this would indicate Server Side Request forgery. This is a very realistic scenario, and I did find XXE issues in PNG XMP metadata parsing libraries before.

Java Security Manager

After doing some research it turned out that there is nothing like a file whitelist sanitizer for native code where AFL is usually used. So if we would fuzz any C/C++ code we would have to write our own parser and as stated by Jakub Wilk it might be tricky to implement due to async-signal-safe filesystem functions. So if you feel like writing one, please go ahead.

Back to Java I found out that there is already such a sanitizer. The best part is that it’s a built-in feature of the JVM and it’s called Java Security Manager. Look at this simple Java Security Manager policy file I created for running the Kelinci fuzzer with our simple Apache Commons JPEG parsing code:

grant {
    permission java.io.FilePermission "/tmp/*", "read,write,delete";
    permission java.io.FilePermission "in_dir/*", "read";
    permission java.io.FilePermission "/opt/kelinci/kelinci/examples/commons-imaging/out_dir/*", "read, write, delete";
    permission java.io.FilePermission "/opt/kelinci/kelinci/examples/commons-imaging/out_dir/master/*", "read, write, delete";
    permission java.io.FilePermission "/opt/kelinci/kelinci/examples/commons-imaging/out_dir/master0/*", "read, write, delete";
    permission java.io.FilePermission "/opt/kelinci/kelinci/examples/commons-imaging/out_dir/master1/*", "read, write, delete";
    permission java.io.FilePermission "/opt/kelinci/kelinci/examples/commons-imaging/out_dir/slave/*", "read, write, delete";
    permission java.io.FilePermission "/opt/kelinci/kelinci/examples/commons-imaging/out_dir/slave0/*", "read, write, delete";
    permission java.io.FilePermission "/opt/kelinci/kelinci/examples/commons-imaging/out_dir/slave1/*", "read, write, delete";
    permission java.net.SocketPermission "localhost:7007-", "accept, listen, resolve";
    permission java.lang.RuntimePermission "modifyThread";
};

All it does is allowing file access to the temporary directory, reading from the input directory (in_dir) and writing to the output directory (out_dir) of AFL. Moreover, it allows the Kelinci Java process to listen on TCP port 7007 as well as to modify other threads. As the Security Manager is built into every Java JVM, you can simply start it with your usual command line with two more arguments:

java -Djava.security.manager -Djava.security.policy=java-security-policy.txt

So in our case we can run the Kelinci fuzzer server process with:

java -Djava.security.manager -Djava.security.policy=java-security-policy.txt -Djava.io.tmpdir=/tmp/ -cp bin-instrumented:commons-imaging-1.0-instrumented.jar edu.cmu.sv.kelinci.Kelinci driver.Driver @@

I went back and ran the Kelinci fuzzer some more hours on the Apache Commons JPEG parser without getting any new results with the Java Security Manager. However, at this point I was convinced that the Java Security Manager would take Java fuzzing to the next level. I just needed a different target first.

Targeting Apache Tika

Fast forward several days later, I stumbled over the Apache Tika project. As Apache Tika was formerly part of Apache Lucene, I was convinced that a lot of servers on the Internet would allow users to upload arbitrary files to be parsed by Apache Tika. As I’m currently maintaining another related research about web based file upload functionalities (UploadScanner Burp extension) this got me even more interested.

Apache Tika is a content analysis toolkit and can extract text content from over a thousand different file formats. A quick’n’dirty grep-estimate turned out that it has about 247 Java JAR files as dependencies at compile time. Apache Tika also had some severe security issues in the past. So as a test target Apache Tika seemed to fit perfectly. On the other hand I also knew that using such a big code base is a bad idea when fuzzing with AFL. AFL will more or less quickly deplete the fuzzing bitmap when the instrumented code is too large. Afterwards, AFL will be unable to detect when an input results in an interesting code path being taken. I was also not sure if I could successfully use the Java fuzzers to instrument the huge Apache Tika project. However, I decided to go on with this experiment.

I first tried to get things running with Kelinci, but ran into multiple issues and ended up creating a “works-for-me” Kelinci fork. After Kelinci was running, I also tried to get the JQF fuzzer running, however, I ran into similar but distinct problems and therefore decided to stick with Kelinci at this point. For Tika I had to adopt the Java Security Manager Policy:

grant {
    //Permissions required by Kelinci
    permission java.lang.RuntimePermission "modifyThread";
    
    permission java.net.SocketPermission "localhost:7007", "listen, resolve";
    permission java.net.SocketPermission "localhost:7008", "listen, resolve";
    permission java.net.SocketPermission "localhost:7009", "listen, resolve";
    permission java.net.SocketPermission "localhost:7010", "listen, resolve";
    permission java.net.SocketPermission "[0:0:0:0:0:0:0:1]:*", "accept, resolve";
    
    permission java.io.FilePermission "in_dir/*", "read";
    permission java.io.FilePermission "corpus/*", "read, write";
    permission java.io.FilePermission "crashes/*", "read";
    permission java.io.FilePermission "out_dir/*", "read, write";
    
    //Permissions required by Tika
    permission java.io.FilePermission "tika-app-1.17.jar", "read";
    permission java.io.FilePermission "tika-app-1.17-instrumented.jar", "read";

    permission java.io.FilePermission "/tmp/*", "read, write, delete";
    
    permission java.lang.RuntimePermission "getenv.TIKA_CONFIG";
    
    permission java.util.PropertyPermission "org.apache.tika.service.error.warn", "read";
    permission java.util.PropertyPermission "tika.config", "read";
    permission java.util.PropertyPermission "tika.custom-mimetypes", "read";
    permission java.util.PropertyPermission "org.apache.pdfbox.pdfparser.nonSequentialPDFParser.eofLookupRange", "read";
    permission java.util.PropertyPermission "org.apache.pdfbox.forceParsing", "read";
    permission java.util.PropertyPermission "pdfbox.fontcache", "read";
    permission java.util.PropertyPermission "file.encoding", "read";

    //When parsing certain PDFs...
    permission java.util.PropertyPermission "user.home", "read";
    permission java.util.PropertyPermission "com.ctc.wstx.returnNullForDefaultNamespace", "read";
    
    //When parsing certain .mdb files...
    permission java.util.PropertyPermission "com.healthmarketscience.jackcess.resourcePath", "read";
    permission java.util.PropertyPermission "com.healthmarketscience.jackcess.brokenNio", "read";
    permission java.util.PropertyPermission "com.healthmarketscience.jackcess.charset.VERSION_3", "read";
    permission java.util.PropertyPermission "com.healthmarketscience.jackcess.columnOrder", "read";
    permission java.util.PropertyPermission "com.healthmarketscience.jackcess.enforceForeignKeys", "read";
    permission java.util.PropertyPermission "com.healthmarketscience.jackcess.allowAutoNumberInsert", "read";
    permission java.util.PropertyPermission "com.healthmarketscience.jackcess.timeZone", "read";
};

To produce this policy file manually was much more annoying than for Apache Commons. The reason is that the necessary permissions we need to whitelist depend on the input file. So if a PNG file is fed into Apache Tika, it will need other runtime property permissions than if a PDF file is fed into Apache Tika. This means that we have to do a dry run first that will go through the entire input corpus of files and run them once with the minimum policy file. If a security exception occurs, it might be necessary to whitelist another permission. This process takes a lot of time. However, as an article from 2004 states:

There’s currently no tool available to automatically generate a [Java security] policy file for specific code.

So that’s why I wrote another quick’n’dirty hack/tool to generate Java security policy files. As it’s not a beauty I gave it the ugly name TMSJSPGE on github. However, it does it’s job and generates a Java security policy file. It will feed each corpus file to the target process (Tika in this case) and add a new rule to the security policy.

If you look at the above property permissions, I’m still not sure what they are all doing. However, I just decided I’ll go with them and allow Tika to use them.

If you run your fuzzer with different input files, you might be required to adopt the Java Security policy, as other code paths might require new permissions. So the above security policy for Apache Tika is likely to be incomplete.

Apache Tika findings

As already explained, a good input corpus is vital for a successful fuzzing run. Additionally, I had to run Tika with as many files as possible to make sure the Java Security Policy covered most permissions necessary. Over the years I’ve collected many input sample files (around 100’000) by doing fuzzing runs with various libraries and by collecting third-party files (that’s actually a topic for another day). So I decided I will run the TMSJSPGE tool with each of these 100’000 files to create the best Security Policy I can. When I checked back on the TMSJSPGE I saw that the tool was stuck feeding a certain file to Apache Tika. This means that Apache Tika never returned a result and the process hung. And that meant I already found security issues in Apache Tika 1.17 before I even started fuzzing. After removing the file that resulted in a hang and restarting TMSJSPGE, Apache Tika hung with several other files as well. Some of the files triggered the same hang and after deduplicating, the following two security issues were reported to Apache Tika:

I was wondering where these input files I had in my collection were coming from. Several BPG files triggering the issue were from a fuzzing run I once did for libbpg, so they were produced by AFL when creating BPG files for a native library. But the chm file triggering the other issue was a file that I downloaded a long time ago from the fuzzing project. It was a file Hanno Böck provided that came out of a fuzzing run for CHMLib. Interesting.

So here I was and had already found an uncaught exception in Apache Commons and two low severity issues in Apache Tika without even starting to do proper fuzzing.

To get an idea of the Java classes causing the issue I ran Apache Tika with a debugger and the triggering file, stopped the execution during the infinite loop and printed a stack trace. But most of the hard work to figure out the actual root causes of these issues was done by the maintainers, most importantly by Tim Allison and the Apache Tika team. That is also true for all the upcoming issues.

Fuzzing Apache Tika with Kelinci

After sorting out the input files that resulted in a hang, I started a couple of afl-fuzz fuzzing instances and waited. The behavior of the Kelinci fuzzer is sometimes a little brittle, so I often got the “Queue full” error message. It means the fuzzer is not running properly anymore and that timeouts will occur. I had to restart the fuzzing instances several times and tried to tweak the command line settings to improve stability. However, over time the instances often managed to fill up the queue again. Anyway, a couple of instances ran fine and found several “AFL crashes”. Keep in mind that “AFL crashes” in this case just mean uncaught Java exceptions. After looking through and deduplicating issues, I reported the following non-security (or very low severity, a matter of definition) issues to the maintainers of the libraries used by Apache Tika:

The hang directory of AFL did not show any interesting results. After running each of the files in the hang directory with Apache Tika I found a PDF file that took nearly a minute to process, but none of the files lead to a full hang of the Tika thread. I suspect that the synchronization of the two processes was one of the reasons no infinite hangs were found by the fuzzer.

However, at this stage I was most disappointed that none of the crashes indicated that anything outside of the specified Java Security Manager policy was triggered. I guess this was a combination of my brittle configuration of Kelinci and the fact that it is probably not as easy to find arbitrary file read or write issues. But in the end you often simply don’t know what’s exactly the reason for not being successful with fuzzing.

JQF and a bug in Java

At one point I also wanted to try the JQF fuzzer on my ARM fuzzing machines with Apache Tika. It didn’t work for me at first and I found out that OpenJDK on ARM had horrible performance with JQF, so I switched to Oracle’s Java. Additionally, Apache Tika would simply not run with JQF. After the Tika 1.17 issues were fixed in Apache Tika I thought it was time to notify the maintainers of the fuzzers, so they could try to fuzz Apache Tika themselves. Rohan (maintainer of JQF) quickly fixed three independent issues and implemented a test case/benchmark for the fixed Tika 1.18 in JQF. After that I was able to fuzz Tika with my own corpus, but the performance was very bad for various reasons. One reason was the weak ARM boxes, but JQF couldn’t handle timeouts either (AFL’s -t switch). Rohan attempted a fix, but it’s only working sometimes. Rohan was also very quick to implement afl-cmin and said running with a Java Security Manager policy should be no problem. However, I couldn’t try those features properly due to the performance problems on the ARM machines. As I was not in the mood to switch fuzzing boxes, I just tried to get the fuzzer running somehow. After cutting down the input corpus and removing all PDF files that were taking potentially longer to be processed by Apache Tika, the fuzzer crept slowly forward. After not paying attention for 10 days, another hang was found by JQF in Apache Tika 1.18… I thought! However, after submitting this bug to Apache Tika, they pointed out that this was actually a bug in the Java standard libraries affecting Java before version 10 that I rediscovered:

The hang file was created by the JQF fuzzer by modifying a sample QCP file “fart_3.qcp” from the public ffmpeg samples. So without actively targeting Java itself, I had rediscovered a bug in Java’s standard libraries, as Tika used it. Quite an interesting twist.

Adding a x86 fuzzing box

At the same time I also realized that these ARM JQF fuzzer instances were stuck. The endless RIFF loop file was detected as a crash (which might just be bad behavior of JQF for hangs), so I didn’t really know the reason why they were stuck currently. I tried to run the current input file on another machine, but the testcase didn’t hang. So I didn’t figure out why the fuzzer got stuck, but as Rohan pointed out the timeout handling (AFL’s “hangs”) isn’t optimal yet. JQF will detect timeouts when the infinite loop hits instrumented part of the Java code, as it will be able to measure the time that passed. However, JQF will hang for now if a test file makes the code loop forever in non-instrumented code. I removed all RIFF/QCP input files so hopefully I wouldn’t rediscover the RIFF endless loop bug again (I never switched to Java 10) and restarted the fuzzing instances.

I decided to additionally use a 32bit x86 VMWare fuzzing box, maybe it would run more stable there. I setup JQF with Java 8 again and without RIFF files as inputs. The x86 virtual machine performed much better, executing around 10 testcases per second. So I let these instances run for several days… just to realize when I came back that both instances got stuck after 7 hours of running. I checked again if the current input file could be the reason and this time this was exactly the problem, so another bug. Rinse and repeat, the next morning another bug. So after a while (at least 5 iterations) I had a bag full of bugs:

  • An endless loop in Junrar (file 11_hang_junrar_zero_header2.rar), where the code simply never returned when the rar header size is zero. I contacted one of the maintainers, beothorn. It was fixed and this issue ended up as CVE-2018-12418.
  • Infinite loop in Apache Tika’s IptcAnpaParser for handling IPTC metadata (file 12_hang_tika_iptc.iptc), where the code simply never returned. This was fixed and assigned CVE-2018-8017.
  • Infinite loop in Apache PDFbox’ AdobeFontMetricsParser (file 16_570s_fontbox_OOM.afm), after nearly 10 minutes (on my machine) leading to an out of memory situation. This was fixed and assigned CVE-2018-8036.
  • An issue when a specially crafted zip content is read with Apache Commons Compress (file 14_69s_tagsoup_HTMLScanner_oom.zip) that leads to an out of memory exception. This was fixed in Apache Commons Compress and CVE-2018-11771 was assigned. Another zip file created (file 15_680s_commons_IOE_push_back_buffer_full.zip) runs for 11 minutes (on my machine) leading to IOException with a message that the push back buffer is full and is probably related to the issue. Also probably the same issue is a file where Tika takes an arbitrary amount of time (during the tests between 20 seconds and 11 minutes) to process a zip file (file 13_48s_commons_truncated_zip_entry3.zip). This last one is worth a note as JQF correctly detected this as a hang and put it in AFL’s hang directory. The underlying problem of CVE-2018-11771 was that a read operation started to return alternating values of -1 and 345 when called by an InputStreamReader with UTF-16. The minimal code to reproduce is:
@Test
public void testMarkResetLoop() throws Exception {
    InputStream is = Files.newInputStream(Paths.get("C:/14_69s_tagsoup_HTMLScanner_oom.zip"));
    ZipArchiveInputStream archive = new ZipArchiveInputStream(is);
    ZipArchiveEntry entry = archive.getNextZipEntry();
    while (entry != null) {
        if (entry.getName().contains("one*line-with-eol.txt")) {
            Reader r = new InputStreamReader(archive, StandardCharsets.UTF_16LE);
            int i = r.read();
            int cnt = 0;
            while (i != -1) {
                if (cnt++ > 100000) {
                    throw new RuntimeException("Infinite loop detected...");
                }
                i = r.read();
            }
        }
        entry = archive.getNextZipEntry();
    }
}

After all these fixes I ran the fuzzer again on a nightly build of Apache Tika 1.19 and it didn’t find any new issues in more than 10 days. So my approach of fuzzing Tika seems to be exhausted. As always, it doesn’t mean another approach wouldn’t find new issues.

Summary

This is where I stopped my journey of Java fuzzing for now. I was a little disappointed that the approach with the Java Security Manager still did not find any security issues such as SSRF and that I only found ressource management issues. However, I’m pretty sure this strategy is still the way to go, it probably just needs other targets. As you can see there are loose ends everywhere and I’m definitely planning to go back to Java fuzzing:

  • Use Kelinci/JQF with other Apache Commons parsers, e.g. for PNG
  • Write sanitizers such as file or socket opening for native code AFL
  • Contribute to the AFL-based Java fuzzers

However, for now there are other things to break on my stack.

I would like to thank Tim Allison of the Apache Tika project, it was a pleasure to do coordinated disclosure with him. And also a big thanks to Rohan Padhye who was really quick implementing new features in JQF.

Make sure you add the files included on https://github.com/modzero/mod0javaFuzzingResults to your input corpus collection, as we saw it’s worth having a collection of crashes for other libraries when targeting new libraries.

Activity wrap-up including polyglots, RIPS, UploadScanner and Java fuzzing

A tweet of takesako including a C/C++/Perl/Ruby/Python polyglot got me interested, so I created two follow-up polyglots based on his work and put them on github.

Recently I also evaluated the RIPS PHP scanner and I did that with some randomly chosen WordPress plugins. Afterwards I manually looked at the code of the plugins, to see if the scanner missed anything. Long story short, RIPS is probably going to have two new issue definition/checks in its future version, so hopefully it will find PHP type unsafe comparisons like the one I found in this WordPress plugin in the future. Additionally, they are planning to flag when a static string is used as an input for a hash function. Hashing a static string is pointless and bad from a performance perspective. But it might also indicate the creation of default or backdoor user accounts with static passwords. While discussing the idea of type unsafe comparisons, albinowax also added a new check for the backslash powered scanner Burp extension.

I will be giving a workshop on modzero’s yet unreleased Burp Proxy UploadScanner extension at the area41 conference in Zurich. I’ve been developing it for more than a year and I’m really looking forward to releasing it after the workshop (it will go public on github). It can be used to test HTTP based file uploads. The “presales” tickets are gone, but if you catch me at the conference in the morning you might be able to get one of the last seats.

I’ve also released a Java security manager policy generator, which is just a little hack but at least it works. I’m doing some research in the area of Java fuzzing at the moment, more about that later this year.

Schubser and his cookie dealing friend

I actually forgot to post this in February, so I’m a little late but the topic is as current as it was back then. One week in February my colleague at modzero AG, Jan Girlich and me took some time to review our tools and make three of them available on github.

Jan wrote a Proof of Concept (PoC) Android app that allows exploiting Java object deserialization vulnerabilities in Android and named this project modjoda (Modzero Java Object Deserialization on Android). To test the issue, he also wrote a vulnerable demo application to try the exploit.

I wrote mod0schubser, which provides a simple TCP- and TLS-level Man-In-The-Middle (MITM) proxy for people with python experience. It can be used when all the other proxy tools seem to be too complicated and you just want to do some modifications of the traffic in Python. Additionally, I wrote the mod0cookiedealer tool, a tool to demonstrate the impact of missing HTTP cookie flags (secure and HTTPonly). If you remember Firesheep, mod0cookiedealer is a modern implementation of Firesheep as a browser web-extension.

BSides Zurich – Nail in the JKS coffin

On Saturday I was happy to speak at the fabulous BSides Zurich about the Java Key Store topic. You can find my slides “Nail in the JKS coffin” as a PDF here. It was my second time at a BSides format and I really like the idea of having a short talk and then some more time to discuss the topic with interested people. I also included the “after the presentation” slides we used for roughly 50% of the discussion time. I hope you enjoyed the talk and I’m looking forward to hear some feedback. Although it was sold out, you should definitely come next year, it was one of my favorite public conferences.

cheers,
floyd

Android Nougat’s worst anti-security mechanism

If you are a pentester like me, you are doing mobile application reviews on Android. One of the most important things to check is the server API. On the other hand we might want to see what possibilities a server has to influence the Android app with its responses. For both the easiest and most straight forward method is to do a Man-In-The-Middle attack in the lab and look at the network traffic. How do we do this if the mobile app uses TLS? Easy, just install a user CA certificate.

Before Android 7 that was a good solution and straight forward. There was a nag screen showing up in the notifications every time you start up your phone (which was already a little funny), but it worked fine for everyone. However, starting with Android 7 it will not work, I tested that and the official announcement about this user-added certificate security is here. So let’s look at this new “security” feature of Google’s Android.

First of all who is affected? I think only the defender side has to jump through this hoop. Because every attack vector I can think of is ridiculous. First of all, a user would need to fully cooperate to let an attacker exploit this. As Android is not opening the security settings automatically when you download a certificate (like iOS), an attacker would have to convince the user to go to the settings dialogue, go to security, scroll down, tap on “install certificate” and choose the correct file from the file system. Let’s say an attacker will setup a Wi-Fi access point and forces the user to do this or otherwise the user won’t get internet access. This is the only scenario I can even think of where a user might at all consider installing such a certificate. You might say that can happen with non-technical users, but then why don’t we just add a big red warning that this is probably the worst idea ever? That would totally suffice in my opinion. If a user would be so stupid to install an unknown CA despite the warnings, everything is lost anyway. That user will also type all his passwords into any forms that look remotely like a known login form the attacker provides. Let’s also consider corporate Android phones. I can understand that administrators don’t want their users to decide on such a security critical topic. But why doesn’t Android just implement an Administrator API rule that would disable installation of user CA certificates and delete all already installed ones on managed phones?

Secondly, why the hell does Android think that a user installed certificate is less trusted than the hundreds of preinstalled, nation-state-attacker-owned CAs?

Android, you are raising the bar for defenders, not for attackers. You don’t defend against any attack vector. You are not doing security here, you pretend to.

And yes, I know how to disassemble an app and reassemble it to circumvent this “security”. I even consider building an Android app for rooted phones that will pull the CA certificate of Burp, remount the system partition and install the CA there automatically.

Maybe the Android team is just sour because they are losing the rooting-detection game with SafetyNet to Magisk root (good job Magisk guys!). I seriously don’t have a better explanation.

And by the way I’ve heard the joke “Android is open source, change it!” already.

I thought I’ve seen many stupid Android security decisions, but this is exceptionally stupid. Or it’s me, please enlighten me in the comments!

Java Key Store (JKS) format is weak and insecure (CVE-2017-10356)

While preparing my talk for the marvelous BSides Zurich I noticed again how nearly nobody on the Internet warns you that Java’s JKS file format is weak and insecure. While users only need to use very strong passwords and keep the Key Store file secret to be on the safe side (for now!), I think it is important to tell people when a technology is weak. People should stop using JKS now, as I predict a very long phase-out period. JKS was around and the default since Java had its first Key Store. Your security relies on a single SHA-1 calculation here.

Please note that I’m not talking about any other Key Store type (BKS, PKCS#12, etc.), but see the cryptosense website for articles about them.

I don’t want to go into the details “why” JKS is insecure, you can read all about it here:

I wrote an email to the Oracle security team, as I think assigning a CVE number would help people to refer to this issue and raise awareness for developers. My original email sent on September, 18 2017:

I would like to ask Oracle to assign a CVE Number for Java’s weak
encryption in JKS files for secure storage of private keys (Java Key
Store files). JKS uses a weak encryption scheme based on SHA1.

I think it is important to raise awareness that JKS is weak by assigning
a CVE number, even when it is going to be replaced in Java 1.9 with PKCS#12.

The details of the weakness are published on the following URLs:

– As an article in the POC||GTFO 0x15 magazine, I attached it to this
email, the full magazine can also be found on
https://www.alchemistowl.org/pocorgtfo/pocorgtfo15.pdf
– https://cryptosense.com/mighty-aphrodite-dark-secrets-of-the-java-keystore/
– https://github.com/floyd-fuh/JKS-private-key-cracker-hashcat

As the article states, no documentation anywhere in the Java world
mentions that JKS is a weak storage format. I would like to change this,
raise awareness and a CVE assignment would help people refer to this issue.

The timeline so far:

  • September, 18 2017: Notified Oracle security team via email
  • September, 18 2017: Generic response that my email was forwarded to the Oracle team that investigates these issues
  • September, 20 2017: Oracle assigned a tracking number (S0918336)
  • September, 25 2017: Automated email status report: Under investigation / Being fixed in main codeline
  • October, 10 2017: Requested an update and asked if they could assign a CVE number
  • October, 11 2017: Response, they are still investigating.
  • October, 13 2017: Oracle writes “We have confirmed the issue and will be addressing it in a future release”. In an automated email I get Oracle states “The following issue reported by you is fixed in the upcoming Critical Patch Update, due to be released at 1:00 PM, U.S. Pacific Time, on October 17, 2017.”.
  • October 17, 2017: Oracle assigned a CVE in their Oracle Critical Patch Update Advisory – October 2017: CVE-2017-10356. The guys from Cryptosense got credited too it seems. However, the documentation of Oracle so far didn’t change anywhere I could see it.
  • November 16, 2017: I asked again to clarify what the countermeasures are and what they are planning to do with JKS. They seem to be mixing my CVE and the JKS issues with other issue in other Key Store types.
  • November 17, 2017: Oracle replied (again, mixing-in issues of other Key Store types): “In JDK 9 the default keystore format is PKCS#11 which doesn’t have the limits of the JKS format — and we’ve put in some migration capability also. For all versions we have increased the iteration counts [sic!] used significantly so that even though the algorithms are weak, a brute-force search will take a lot longer. For older versions we will be backporting the missing bits of PKCS#11 so that it can be used as the keystore type.”. That was the good part of the answer, even though JKS has no iteration count. The second part where I asked if they could add some links to their Critical Path Update Advisory was: “In order to prevent undue risks to our customers, Oracle will not provide additional information about the specifics of vulnerabilities beyond what is provided in the Critical Patch Update (or Security Alert) advisory and pre-release note, the pre-installation notes, the readme files, and FAQs.”.

That’s it for me for now. I’m too tired to start arguing about keeping technical details secret. So basically I have to hope that everyone finds this blog posts when searching for CVE-2017-10356.

Cracking Java’s weak encryption – Nail in the JKS coffin

POC||GTFO journal edition 0x15 came out a while ago and I’m happy to have contributed the article “Nail in the JKS coffin”. You should really read the article, I’m not going to repeat myself here. I’ve also made the code available on my “JKS private key cracker hashcat” github repository.

For those who really need a TL;DR, the developed cracking technique relies on three main issues with the JKS format:

  1. Due the unusual design of JKS the key store password can be ignored and the private key password cracked directly.
  2. By exploiting a weakness of the Password Based Encryption scheme for the private key in JKS described by cryptosense, the effort to try a password is minimal (one SHA-1 calculation).
  3. As public keys are not encrypted in the JKS file format, we can determine the algorithm and key size of the public key to know the PKCS#8 encoded fingerprint we have to expect in step 2.

For a practical TL;DR, see the github repository on how JksPrivkPrepare.jar can be used together with the hashcat password cracking tool to crack passwords.

Not affected of the described issues are other key store file formats such as JCEKS, PKCS12 or BKS. It is recommended to use the PKCS12 format to store private keys and to store the files in a secure location. For example it is recommended to store Android app release JKS files somewhere else than a repository such as git.

Crash bash

Fuzzing Bash-4.4 patch 12 with AFL mainly fork bombed the fuzzing machine, but it also found this crash (they all have the same root cause):

<&-<${}
<&"-"<"$[~]"
<&"-"<"${}"
<&"-"<"${$0}"
<&"-"<$(())

It also works on a Bash 3.2.57, but some friends told me that they needed the following to reproduce:

echo -ne '<&-<${}'|bash

A Ubuntu user told me it was not reproducible at all, but I rather suspect his whoopsie didn’t want him to see it. Edit: As pointed out by Matthew in the comments it also works on Ubuntu.

It looks like a nullpointer dereference to me:

Program received signal SIGSEGV, Segmentation fault.
0x000912a8 in buffered_getchar () at input.c:565
565	  return (bufstream_getc (buffers[bash_input.location.buffered_fd]));
(gdb) bt
#0  0x000912a8 in buffered_getchar () at input.c:565
#1  0x0002f87c in yy_getc () at /usr/homes/chet/src/bash/src/parse.y:1390
#2  0x000302cc in shell_getc (remove_quoted_newline=1) at
/usr/homes/chet/src/bash/src/parse.y:2299
#3  0x0002e928 in read_token (command=0) at
/usr/homes/chet/src/bash/src/parse.y:3115
#4  0x00029d2c in yylex () at /usr/homes/chet/src/bash/src/parse.y:2675
#5  0x000262cc in yyparse () at y.tab.c:1834
#6  0x00025efc in parse_command () at eval.c:261
#7  0x00025de8 in read_command () at eval.c:305
#8  0x00025a70 in reader_loop () at eval.c:149
#9  0x0002298c in main (argc=1, argv=0xbefff824, env=0xbefff82c) at
shell.c:792
(gdb) p bash_input.location.buffered_fd
$1 = 0
(gdb) p buffers
$2 = (BUFFERED_STREAM **) 0x174808
(gdb) x/10x 0x174808
0x174808:	0x00000000	0x00000000	0x00000000	0x00000000
0x174818:	0x00000000	0x00000000	0x00000000	0x00000000
0x174828:	0x00000000	0x00000000

The maintainers of bash were notified.

iOS TLS session resumption race condition (CVE-2016-10511)

Roughly three months ago when iOS 9 was still the newest version available for the iPhone, we encountered a bug in the Twitter iOS app. When doing a transparent proxy setup for one of our iOS app security tests, a Twitter HTTPS request turned up in the Burp proxy log. This should never happen, as the proxy’s HTTPS certificate is not trusted on iOS and therefore connections should be rejected. Being shocked, we checked that certainly we did not install the CA certificate of the proxy on the iPhone and verified with a second non-jailbroken iPhone. The bug was repoducible on iOS 9.3.3 and 9.3.5.

After opening a Hackerone bug report with Twitter I took some time to further investigate the issue. Changing the seemingly unrelated location of the DHCP server in our test setup from the interception device to the WiFi access point made the bug non-reproducible. Moving the DHCP server back to the interception device the issue was reproducible again. This could only mean this was a bug that needed exact timing of certain network related packets. After a lot of back and forth, I was certain that this has to be a race condition/thread safety problem.

Dissecting the network packets with Wireshark, I was able to spot the bug. It seems that if the server certificate in the server hello packet is invalid, the TLS session is not removed fast enough/in a thread safe manner from the TLS connection pool. If the race condition is triggered, this TLS session will be reused for another TLS connection (TLS session resumption). During the TLS session resumption the server hello packet will not include a server certificate. The TLS session is already trusted and the client has no second opportunity to check the server certificate. If an attacker is able to conduct such an attack, the authentication mechanism of TLS is broken, allowing extraction of sensitive OAuth tokens, redirecting the Twitter app via HTTP redirect messages and other traffic manipulations.

I was not able to reproduce the issue on iOS 10. Twitter additionally fixed the issue on their side in Twitter iOS version 6.44, but noted that this was probably related to an Apple bug. We did not further investigate the issue, but the assumption seems plausible.

The issue was rated high severity by Twitter. The entire details are published on Hackerone.

Update: CVE-2016-10511 was assigned to this security issue.