Monthly Archives: September 2012

Source Code Review Grep Script

Posted on by
2

Edit: This evolved over years, see the CRASS project.

As a pentester you sometimes get access to the source code of the application you are reviewing. Sometimes you can look manually through the files, but sometimes you get million lines of code and you don’t have time to spend sitting there, reading line after line.

The first approach that came to my mind was to use static code analysis tools. There are a lot of them out there, you can find lists of tools on the flawfinder page or you can have a look at Wikipedia’s list of tools for static code analysis. It definitely makes sense to use these tools, but it needs time to download/compile them and most of them are written for one particular language.

You will be reading a lot of code. If you want to do yourself a favour, use an IDE like eclipse to look at the code. Especially when looking at Java code you will find yourself quite often changing from one class to another and changing between files. With eclipse you only need one click for that. But still, you need different IDEs for different programming languages. So this is still not a universal approach.

As a penetration tester you often want to find the interesting parts of the code. To name some interesting things: everything related to cryptography, encryption, SQL queries, file read and writes, URLs and sockets, obfuscation, passwords and so on. And there is one really universal tool that let us find these parts of the code: grep.

The script I wrote here is pretty focused on Java/Android and Objective-C/iOS. But I also got some JSP and spring java framework specific code. So here we go, no rocket science, but I hope it’s helpful for someone in the future.

#!/bin/bash
#
# A simple code greper...
#
# ----------------------------------------------------------------------------
# "THE BEER-WARE LICENSE" (Revision 42):
# <floyd at floyd dot ch> wrote this file. As long as you retain this notice you
# can do whatever you want with this stuff. If we meet some day, and you think
# this stuff is worth it, you can buy me a beer in return
# floyd http://floyd.ch @floyd_ch <floyd at floyd dot ch>
# August 2012
# ----------------------------------------------------------------------------
#
# Tested under MAC OSX ONLY!
#
# This script isn't very advanced - exactly what's needed if you don't know where to start.
# It is not a real static analysis tool and it's not in any way a replacement for all the cool
# tools out there (checkstyle, jlint, etc.)
#
 
 
if [ $# -ne 1 ]
then
  echo "Usage: `basename $0` directory-to-grep-through"
  exit 0
fi
###
#OPTIONS
###
#Open the colored outputs with "less -R" or cat, otherwise remove --color=always
ADDITIONAL_GREP_ARGUMENTS="-A 1 -B 3 --color=always"
TARGET="./grep-output"
#In my opinion I would always leave all the options below here on true,
#because I did find strange android code in iphone apps and vice versa. I would only
#change it if the greping needs very long, you are greping a couple of hundret apps
#or if you have any other performance issues with this script.
DO_JAVA=true
DO_SPRING=true
DO_JSP=true
DO_ANDROID=true
DO_IOS=true
DO_PHP=true
DO_GENERAL=true
###
#END OPTIONS
#Normally you don't have to change anything below here...
###
 
GREP_ARGUMENTS="-nrP"
STANDARD_GREP_ARGUMENTS=$ADDITIONAL_GREP_ARGUMENTS" "$GREP_ARGUMENTS
SEARCH_FOLDER=$1
mkdir $TARGET
 
echo "Your standard grep arguments: $STANDARD_GREP_ARGUMENTS"
echo "Output will be put into this folder: $TARGET"
echo "You are currently greping through folder: $SEARCH_FOLDER"
sleep 2
 
#The Java stuff
if [ $DO_JAVA ]; then
    SEARCH_STRING='javax.crypto|bouncy.*?castle|new\sSecretKeySpec\(|messagedigest'
    OUTFILE="java_general_crypto.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
     
    SEARCH_STRING='toString\(\) *==|== *toString\(\)|" *==|== *"'
    OUTFILE="java_general_wrong_string_comparison.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #String comparison has to be done with .equals() in Java, not with ==
     
    SEARCH_STRING='\.exec\('
    OUTFILE="java_general_exec.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
     
    SEARCH_STRING='java\.net\.|java\.io\.|javax\.servlet|org\.apache\.http'
    OUTFILE="java_general_io.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
     
    SEARCH_STRING='@Entity|@ManyToOne|@OneToMany|@OneToOne|@Table|@Column'
    OUTFILE="java_persistent_beans.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #Special case, we only want to know matching files to know which beans get persisted, therefore -l to output matching files
    grep -l $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #Find out which Java Beans get persisted with javax.persistence
     
    SEARCH_STRING='@Table\(|@Column\('
    OUTFILE="java_persistent_tables_and_columns_in_database.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #Case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #The source code shows the database table/column names... e.g. if you find a sql injection later on
     
    SEARCH_STRING='string .{0,10}password|string .{0,10}secret|string .{0,10}key|string .{0,10}cvv|string .{0,10}user|string .{0,10}hash(?!(table|map|set|code))|string .{0,10}passcode|string .{0,10}passphrase|string .{0,10}user|string .{0,10}pin|string .{0,10}credit'
    OUTFILE="java_confidential_data_in_strings.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #http://docs.oracle.com/javase/1.5.0/docs/guide/security/jce/JCERefGuide.html#PBEEx
fi
 
#The Java Spring specific stuff
if [ $DO_SPRING ]; then
    SEARCH_STRING="DataBinder\.setAllowedFields"
    OUTFILE="java_spring_mass_assignment.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #see e.g. http://blog.fortify.com/blog/2012/03/23/Mass-Assignment-Its-Not-Just-For-Rails-Anymore
fi
 
#The JSP specific stuff
if [ $DO_JSP ]; then
    SEARCH_STRING="escape\s*=\s*\"?\s*false|escape\s*=\s*\'?\s*false"
    OUTFILE="java_jsp_xss.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #can introduce XSS when using escape=false
     
    SEARCH_STRING="<s:file "
    OUTFILE="java_jsp_file_upload.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
fi
 
#The Android specific stuff
if [ $DO_ANDROID ]; then
 
    SEARCH_STRING='\.printStackTrace\(|Log\.(e|w|i|d|v)\('
    OUTFILE="android_logging.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #printStackTrace Logs to Android log, information leakage, etc.
     
    SEARCH_STRING='MODE_|\.openFile\(|\.openOrCreate|\.getDatabase\(|\.openDatabase\(|\.getShared|\.getCache|\.getExternalCache|query\(|rawQuery\(|compileStatement\('
    OUTFILE="android_access.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #Android file io and access things
     
    SEARCH_STRING='<intent-filter>|\.getIntent\(\)\.getData\(\)|RunningAppProcessInfo'
    OUTFILE="android_intents.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
     
fi
 
#The iOS specific stuff
 
if [ $DO_IOS ]; then
    SEARCH_STRING='NSFileProtection|NSFileManager|NSPersistantStoreCoordinator|NSData' #sqlite, see sql.txt
    OUTFILE="ios_file_access.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #File protection APIs
 
    SEARCH_STRING='kSecAttrAccessible|SecItemAdd|KeychainItemWrapper|Security\.h'
    OUTFILE="ios_keychain.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #Keychain stuff
 
    SEARCH_STRING='CFBundleURLSchemes|kCFStream|CFFTPStream|CFHTTP|CFNetServices|FTPURL|IOBluetooth'
    OUTFILE="ios_network.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #Network stuff
 
    SEARCH_STRING='NSLog\('
    OUTFILE="ios_logging.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
 
    SEARCH_STRING='initWithFormat:|informativeTextWithFormat:|format:|stringWithFormat:|appendFormat:|predicateWithFormat:|NSRunAlertPanel'
    OUTFILE="ios_string_format_functions.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #just check if the first argument to these functions are user controlled, that could be a format string vulnerability
 
    SEARCH_STRING='handleOpenURL:|openURL:'
    OUTFILE="ios_url_handler.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
fi

#The PHP stuff

if [ $DO_PHP ]; then
    SEARCH_STRING='\$_GET|\$_POST'
    OUTFILE="php_get_post.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    
    SEARCH_STRING='crypt\('
    OUTFILE="php_crypt_call.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
fi

#The general stuff
 
if [ $DO_GENERAL ]; then
    SEARCH_STRING='\b[A-Za-z0-9._%+\-]+@[A-Za-z0-9.\-]+\.[A-Za-z]{2,4}\b'
    OUTFILE="email.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #Email addresses
    
    SEARCH_STRING='todo|workaround'
    OUTFILE="todo_workaround.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
 
    SEARCH_STRING='hack|crack|exploit|bypass|backdoor|backd00r'
    OUTFILE="exploit.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" | grep -vE 'Ack|setCdrBackdoor' | grep -viE 'imageshack' > $TARGET/$OUTFILE
 
    SEARCH_STRING='https?://'
    OUTFILE="https_and_http_urls.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #All URIs
 
    SEARCH_STRING='http://|ftp://|imap://|file://'
    OUTFILE="no_ssl_uris.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #Non-SSL URIs
 
    SEARCH_STRING='malloc\(|realloc\('
    OUTFILE="initialisation.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #rather rare bug, but see issues CVE-2010-0041 and CVE-2010-0042... could also happen in java/android native code...
 
    SEARCH_STRING='memcpy\(|memset\(|strcat\(|strcpy\(|strncat\(|strncpy\(|sprintf\(|gets\('
    OUTFILE="insecure_c_functions.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #memcpy
    #memset
    #strcat --> strlcat
    #strcpy --> strlcpy
    #strncat --> strlcat
    #strncpy --> strlcpy
    #sprintf --> snprintf
    #vsprintf --> vsnprintf
    #gets --> fgets
 
    SEARCH_STRING='default.?password|passwo?r?d|passcode|hash.?(?!(table|map|set|code))|pass.?phrase|salt|encryption.?key|encrypt.?key|BEGIN CERTIFICATE---|PRIVATE KEY---|Proxy-Authorization|pin'
    OUTFILE="keys.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
 
    SEARCH_STRING='root.*?detection|rooted.*?Device|is.*?rooted|detect.*?root|jail.*?break'
    OUTFILE="root.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
 
    SEARCH_STRING='sql.{0,10}injection|xss|click.{0,10}jacking|xsrf|directory.{0,10}listing|buffer.{0,10}overflow|obfuscate'
    OUTFILE="hacking_techniques.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #e.g. find prevention techniques...
 
    SEARCH_STRING='`.{2,100}`'
    OUTFILE="backticks.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #-I for binaries=without-match
    grep -I $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #anything between backticks is suspicious, thinking about command execution in perl scripts in cgi-bin directories...
    SEARCH_STRING='location\.hash|location\.href|location\.pathname|location\.search|eval\(|\.appendChild\(|document\.write\(|document\.writeln\(|\.innerHTML\s*?=|\.outerHTML\s*?='
    OUTFILE="dom_xss.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    #case sensitive...
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
 
    SEARCH_STRING='SELECT.*?FROM|INSERT.*?INTO|DELETE.*?WHERE|sqlite'
    OUTFILE="sql.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
 
    SEARCH_STRING='^(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=|[A-Za-z0-9+/]{4})$'
    OUTFILE="base64.txt"
    #case sensitive, the regex is insensitive anyway
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #Sometimes developers try to hide stuff in base64...
 
    SEARCH_STRING='GNU\sGPL|GPLv2|GPLv3|GPL\sVersion|General\sPublic\sLicense'
    OUTFILE="gpl.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
    #GPL violation, not security related, but your customer might be happy to know such stuff...
 
    SEARCH_STRING='stupid|fuck|shit|crap'
    OUTFILE="swear.txt"
    echo "Searching for $SEARCH_STRING --> writing to $OUTFILE"
    grep -i $STANDARD_GREP_ARGUMENTS "$SEARCH_STRING" "$SEARCH_FOLDER" > $TARGET/$OUTFILE
fi
 
echo "Done grep. Results in $TARGET. Have a grepy day."

Of course the script produces a lot of false positives, but it should be a tool that supports you in your manual analysis. I’m sure there are a million more interesting strings we can add to the script. If you think something is missing, leave it in the comments and I’ll add it.

Oh and if you already looked at one version of a source code and you get a new version, you better use the “diff” command line tool and first have a look at the parts that changed.