Edit: This evolved over years, see the CRASS project.
As a pentester you sometimes get access to the source code of the application you are reviewing. Sometimes you can look manually through the files, but sometimes you get million lines of code and you don’t have time to spend sitting there, reading line after line.
The first approach that came to my mind was to use static code analysis tools. There are a lot of them out there, you can find lists of tools on the flawfinder page or you can have a look at Wikipedia’s list of tools for static code analysis. It definitely makes sense to use these tools, but it needs time to download/compile them and most of them are written for one particular language.
You will be reading a lot of code. If you want to do yourself a favour, use an IDE like eclipse to look at the code. Especially when looking at Java code you will find yourself quite often changing from one class to another and changing between files. With eclipse you only need one click for that. But still, you need different IDEs for different programming languages. So this is still not a universal approach.
As a penetration tester you often want to find the interesting parts of the code. To name some interesting things: everything related to cryptography, encryption, SQL queries, file read and writes, URLs and sockets, obfuscation, passwords and so on. And there is one really universal tool that let us find these parts of the code: grep.
The script I wrote here is pretty focused on Java/Android and Objective-C/iOS. But I also got some JSP and spring java framework specific code. So here we go, no rocket science, but I hope it’s helpful for someone in the future.
#!/bin/bash # # A simple code greper... # # ---------------------------------------------------------------------------- # "THE BEER-WARE LICENSE" (Revision 42): # <floyd at floyd dot ch> wrote this file. As long as you retain this notice you # can do whatever you want with this stuff. If we meet some day, and you think # this stuff is worth it, you can buy me a beer in return # floyd http://floyd.ch @floyd_ch <floyd at floyd dot ch> # August 2012 # ---------------------------------------------------------------------------- # # Tested under MAC OSX ONLY! # # This script isn't very advanced - exactly what's needed if you don't know where to start. # It is not a real static analysis tool and it's not in any way a replacement for all the cool # tools out there (checkstyle, jlint, etc.) # if [ $# -ne 1 ] then echo "Usage: `basename $0` directory-to-grep-through" exit 0 fi ### #OPTIONS ### #Open the colored outputs with "less -R" or cat, otherwise remove --color=always ADDITIONAL_GREP_ARGUMENTS="-A 1 -B 3 --color=always" TARGET="./grep-output" #In my opinion I would always leave all the options below here on true, #because I did find strange android code in iphone apps and vice versa. I would only #change it if the greping needs very long, you are greping a couple of hundret apps #or if you have any other performance issues with this script. DO_JAVA=true DO_SPRING=true DO_JSP=true DO_ANDROID=true DO_IOS=true DO_PHP=true DO_GENERAL=true ### #END OPTIONS #Normally you don't have to change anything below here... ### GREP_ARGUMENTS="-nrP" STANDARD_GREP_ARGUMENTS=$ADDITIONAL_GREP_ARGUMENTS" "$GREP_ARGUMENTS SEARCH_FOLDER=$1 mkdir $TARGET echo "Your standard grep arguments: $STANDARD_GREP_ARGUMENTS" echo "Output will be put into this folder: $TARGET" echo "You are currently greping through folder: $SEARCH_FOLDER" sleep 2 #The Java stuff if [ $DO_JAVA ]; then SEARCH_STRING='javax.crypto|bouncy.*?castle|new\sSecretKeySpec\(|messagedigest' OUTFILE="java_general_crypto.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE SEARCH_STRING='toString\(\) *==|== *toString\(\)|" *==|== *"' OUTFILE="java_general_wrong_string_comparison.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #String comparison has to be done with .equals() in Java, not with == SEARCH_STRING='\.exec\(' OUTFILE="java_general_exec.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE SEARCH_STRING='java\.net\.|java\.io\.|javax\.servlet|org\.apache\.http' OUTFILE="java_general_io.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE SEARCH_STRING='@Entity|@ManyToOne|@OneToMany|@OneToOne|@Table|@Column' OUTFILE="java_persistent_beans.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #Special case, we only want to know matching files to know which beans get persisted, therefore -l to output matching files grep -l $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #Find out which Java Beans get persisted with javax.persistence SEARCH_STRING='@Table\(|@Column\(' OUTFILE="java_persistent_tables_and_columns_in_database.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #Case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #The source code shows the database table/column names... e.g. if you find a sql injection later on SEARCH_STRING='string .{0,10}password|string .{0,10}secret|string .{0,10}key|string .{0,10}cvv|string .{0,10}user|string .{0,10}hash(?!(table|map|set|code))|string .{0,10}passcode|string .{0,10}passphrase|string .{0,10}user|string .{0,10}pin|string .{0,10}credit' OUTFILE="java_confidential_data_in_strings.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #http://docs.oracle.com/javase/1.5.0/docs/guide/security/jce/JCERefGuide.html#PBEEx fi #The Java Spring specific stuff if [ $DO_SPRING ]; then SEARCH_STRING="DataBinder\.setAllowedFields" OUTFILE="java_spring_mass_assignment.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #see e.g. http://blog.fortify.com/blog/2012/03/23/Mass-Assignment-Its-Not-Just-For-Rails-Anymore fi #The JSP specific stuff if [ $DO_JSP ]; then SEARCH_STRING="escape\s*=\s*\"?\s*false|escape\s*=\s*\'?\s*false" OUTFILE="java_jsp_xss.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #can introduce XSS when using escape=false SEARCH_STRING="<s:file " OUTFILE="java_jsp_file_upload.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE fi #The Android specific stuff if [ $DO_ANDROID ]; then SEARCH_STRING='\.printStackTrace\(|Log\.(e|w|i|d|v)\(' OUTFILE="android_logging.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #printStackTrace Logs to Android log, information leakage, etc. SEARCH_STRING='MODE_|\.openFile\(|\.openOrCreate|\.getDatabase\(|\.openDatabase\(|\.getShared|\.getCache|\.getExternalCache|query\(|rawQuery\(|compileStatement\(' OUTFILE="android_access.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #Android file io and access things SEARCH_STRING='<intent-filter>|\.getIntent\(\)\.getData\(\)|RunningAppProcessInfo' OUTFILE="android_intents.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE fi #The iOS specific stuff if [ $DO_IOS ]; then SEARCH_STRING='NSFileProtection|NSFileManager|NSPersistantStoreCoordinator|NSData' #sqlite, see sql.txt OUTFILE="ios_file_access.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #File protection APIs SEARCH_STRING='kSecAttrAccessible|SecItemAdd|KeychainItemWrapper|Security\.h' OUTFILE="ios_keychain.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #Keychain stuff SEARCH_STRING='CFBundleURLSchemes|kCFStream|CFFTPStream|CFHTTP|CFNetServices|FTPURL|IOBluetooth' OUTFILE="ios_network.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #Network stuff SEARCH_STRING='NSLog\(' OUTFILE="ios_logging.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE SEARCH_STRING='initWithFormat:|informativeTextWithFormat:|format:|stringWithFormat:|appendFormat:|predicateWithFormat:|NSRunAlertPanel' OUTFILE="ios_string_format_functions.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #just check if the first argument to these functions are user controlled, that could be a format string vulnerability SEARCH_STRING='handleOpenURL:|openURL:' OUTFILE="ios_url_handler.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE fi #The PHP stuff if [ $DO_PHP ]; then SEARCH_STRING='\$_GET|\$_POST' OUTFILE="php_get_post.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE SEARCH_STRING='crypt\(' OUTFILE="php_crypt_call.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE fi #The general stuff if [ $DO_GENERAL ]; then SEARCH_STRING='\b[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,4}\b' OUTFILE="email.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #Email addresses SEARCH_STRING='todo|workaround' OUTFILE="todo_workaround.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE SEARCH_STRING='hack|crack|exploit|bypass|backdoor|backd00r' OUTFILE="exploit.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" | grep -vE 'Ack|setCdrBackdoor' | grep -viE 'imageshack' > $TARGET/$OUTFILE SEARCH_STRING='https?://' OUTFILE="https_and_http_urls.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #All URIs SEARCH_STRING='http://|ftp://|imap://|file://' OUTFILE="no_ssl_uris.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #Non-SSL URIs SEARCH_STRING='malloc\(|realloc\(' OUTFILE="initialisation.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #rather rare bug, but see issues CVE-2010-0041 and CVE-2010-0042... could also happen in java/android native code... SEARCH_STRING='memcpy\(|memset\(|strcat\(|strcpy\(|strncat\(|strncpy\(|sprintf\(|gets\(' OUTFILE="insecure_c_functions.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #memcpy #memset #strcat --> strlcat #strcpy --> strlcpy #strncat --> strlcat #strncpy --> strlcpy #sprintf --> snprintf #vsprintf --> vsnprintf #gets --> fgets SEARCH_STRING='default.?password|passwo?r?d|passcode|hash.?(?!(table|map|set|code))|pass.?phrase|salt|encryption.?key|encrypt.?key|BEGIN CERTIFICATE---|PRIVATE KEY---|Proxy-Authorization|pin' OUTFILE="keys.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE SEARCH_STRING='root.*?detection|rooted.*?Device|is.*?rooted|detect.*?root|jail.*?break' OUTFILE="root.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE SEARCH_STRING='sql.{0,10}injection|xss|click.{0,10}jacking|xsrf|directory.{0,10}listing|buffer.{0,10}overflow|obfuscate' OUTFILE="hacking_techniques.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #e.g. find prevention techniques... SEARCH_STRING='`.{2,100}`' OUTFILE="backticks.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #-I for binaries=without-match grep -I $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #anything between backticks is suspicious, thinking about command execution in perl scripts in cgi-bin directories... SEARCH_STRING='location\.hash|location\.href|location\.pathname|location\.search|eval\(|\.appendChild\(|document\.write\(|document\.writeln\(|\.innerHTML\s*?=|\.outerHTML\s*?=' OUTFILE="dom_xss.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" #case sensitive... grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE SEARCH_STRING='SELECT.*?FROM|INSERT.*?INTO|DELETE.*?WHERE|sqlite' OUTFILE="sql.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE SEARCH_STRING='^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{4})$' OUTFILE="base64.txt" #case sensitive, the regex is insensitive anyway echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #Sometimes developers try to hide stuff in base64... SEARCH_STRING='GNU\sGPL|GPLv2|GPLv3|GPL\sVersion|General\sPublic\sLicense' OUTFILE="gpl.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE #GPL violation, not security related, but your customer might be happy to know such stuff... SEARCH_STRING='stupid|fuck|shit|crap' OUTFILE="swear.txt" echo "Searching for $SEARCH_STRING --> writing to $OUTFILE" grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE fi echo "Done grep. Results in $TARGET. Have a grepy day."
Of course the script produces a lot of false positives, but it should be a tool that supports you in your manual analysis. I’m sure there are a million more interesting strings we can add to the script. If you think something is missing, leave it in the comments and I’ll add it.
Oh and if you already looked at one version of a source code and you get a new version, you better use the “diff” command line tool and first have a look at the parts that changed.
Hello Tobias,
Very nice blog post and nifty code! It’s very similar to my approach for a quick analysis. I’m also looking for all data input (get/post params), output (file access, sql statements), interesting functions (e.g. crypt() and log()) and comments (e.g. “todo”, “workaround”).
Regards,
Marc
Hi Marc
Thanks! I’m going to add those things, good input.
Best,
floyd