Python Sender

Last week I played my first Capture The Flag (CTF) where I really tried solving the challenges for a couple of hours. It was a regular jeopardy style CTF with binaries, web applications and other server ports. I don’t think CTFs are going to be my favourite hobby, as pentesting is similar but just a little bit more real life. However, CTFs are very nice for people who want to get into IT security, so I wanted to help a little bit in the team I joined. This particular CTF by Kaspersky really annoyed me though, as the servers were very often offline (HTTP 500 errors). Moreover, some challenges allowed easy Remote Command Execution (RCE) and I guess some teams took the chance to prevent other teams from scoring flags. As I just said I’m not very experienced with CTFs, maybe that’s how it’s supposed to be, but for me that’s silly. Anyway, this post is about something more positive: A Python script to play CTFs, but can also be used during pentests. For those who play CTFs very often, it’s probably better to use a full library such as pwntools, but if you just want a small script where you can delete whatever you don’t need and go with the POC||GTFO flow, you’ve come to the right place.

I think two of the mostly presented CTF challenges often look the same. You either get a URL to a challenge website and you have to do some HTTP magic or you get something like “nc www.example.org 1337” where you are supposed to talk to a server with netcat. Now both challenges usually use TCP/IP and maybe TLS. The website obviously uses HTTP(S) on top of that. So very often you find yourself sending a lot of HTTP requests or a lot of TCP packets to a certain port. Pentests also require the same sometimes.

To make sure we don’t have to fight if Python 2.7 is better than Python 3.6, the script I wrote works on both versions. But even then, people might argue that python’s urllib or urllib2 is sufficient or that they rather use the non-standard requests library. And others will simply say that only asynchronous network IO is really fast enough, so they prefer to use Python Twisted (or treq). However, I got all of these cases covered in the script.

The script allows arbitrary socket and HTTP(S) connections via:

  • socket and ssl-wrapped sockets – when you need bare bone or non-HTTP(S)
  • python urllib/urllib2 HTTP(S) library – when you need HTTP(S) and a little bit more automated HTTP feature handling
  • python requests HTTP(S) library – when you need HTTP(S) and full HTTP feature handling
  • python treq (uses Python Twisted and therefore asynchronous IO) – when you need full HTTP(S) feature handling and speed is important

The main features are:

  • Works under python 2.7 and python 3 (although treq here is untested under python 2.7)
  • You can just copy and paste an HTTP(S) request (e.g. from a proxy software) without worrying about the parsing and other details
  • You can also use the sockets functions to do non-HTTP related things
  • Ignores any certificate warnings for the server

It should be helpful when:

  • You want to script HTTP(S) requests (e.g. just copy-paste from a proxy like Burp), for example during a pentest or CTF
  • When you encounter a CTF challenge running on a server (like “nc example.org 1234”) or a proprietary TCP protocol during pentests

Howto:

  • Change the variables START, END and TLS
  • Optional: Change further configuration options, such as sending the HTTP(S) requests through a proxy
  • Change the ‘main’ function to send the request you would like to. By default it will send 3 HTTP requests to www.example.org with every library.

Enough words, head over to github to download the Python Sender.

Sending generic HTTP(S) requests in python

During Web Application Penetration tests I always need to automate requests, e.g. for fuzzing. While most of the local proxy/testing softwares (Burp, WebScarab, w3af, etc.) include a repeater/fuzzer feature, I often want to do addtional computations in python (e.g. calculating a hash and sending it as a fuzzed value or comparing parts of the response). The following script will take an entire HTTP(S) request as a string, parse it and send it to the server. As I show with the POST parameter “fuzzableParam” in this example, values can easily be fuzzed.

def send_this_request(http_request_string, remove_headers=None):
    """
    Always HTTP/1.1
    """
    import urllib2
    if remove_headers is None:
        remove_headers=['content-length', 'accept-encoding', 'accept-charset', 
        'accept-language', 'accept', 'keep-alive', 'connection', 'pragma', 
        'cache-control']
    for i, remove_header in enumerate(remove_headers):
        remove_headers[i] = remove_header.lower()
    if '\n\n' in http_request_string:
        headers, body = http_request_string.split('\n\n',1)
    else:
        headers = http_request_string
        body = None
    headers = headers.split('\n')
    request_line = headers[0]
    headers = headers[1:]

    method, rest = request_line.split(" ", 1)
    url, protocol = rest.rsplit(" ", 1)

    merge_host_header_into_url = False
    if url.startswith("http"):
        merge_host_header_into_url = False
    elif url.startswith("/"):
        info("Warning: Defaulting to HTTP. Please write URL as https:// if you want SSL")
        merge_host_header_into_url = True
    else:
        fatalError("Protocol not supported. URL must start with http or /")

    header_tuples = []
    for header in headers:
        name, value = header.split(": ", 1)
        if merge_host_header_into_url and name.lower() == 'host':
            url = 'http://'+value+url
        if not name.lower() in remove_headers:
            header_tuples.append((name, value))
            
    opener = urllib2.build_opener()
    opener.addheaders = header_tuples
    urllib2.install_opener(opener)

    try:
        return urllib2.urlopen(url, body, 15).read()
    except urllib2.HTTPError, e:
        info('The server couldn\'t fulfill the request. Error code:', e.code)
    except urllib2.URLError, e:
        info("URLError:", e.reason)
    except Exception, e:
        error("DIDNT WORK:", e)
        
def info(*text):
    print "[PY-INFO] "+str(" ".join(str(i) for i in text))

def error(*text):
    print "[PY-ERROR] "+str(" ".join(str(i) for i in text))

request = '''POST http://example.com/ HTTP/1.1
Host: example.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Content-Type: application/x-www-form-urlencoded;charset=utf-8
Referer: http://example.com
Content-Length: 132
Cookie: test=somevalue; abc=123
DNT: 1
Connection: keep-alive
Pragma: no-cache
Cache-Control: no-cache

id=123&fuzzableParam='''

additionalValue = "&anotherParam=abc"

for i in ['78', '-1']:
    print send_this_request(request+i+additionalValue)