Let's consider a fictional application that stores user passwords as SHA digests:

import java.io.UnsupportedEncodingException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.*;

public class Authentication {

    public static byte[] encryptPassword(String password) 
        throws UnsupportedEncodingException, NoSuchAlgorithmException
    {
        String saltedPassword = password + "Add-Some-Salt";
        byte[] digestive = saltedPassword.getBytes("ISO-8859-1");
        MessageDigest md = MessageDigest.getInstance("SHA");
        md.update(digestive);
        return md.digest();
    }

    public static boolean checkPassword(String password, byte[] digest) 
        throws UnsupportedEncodingException, NoSuchAlgorithmException {
        
        if (Arrays.equals(encryptPassword(password),digest)) return true;
        System.out.println("Wrong password");
        return false;
    }
}

Compiling this class with the standard javac compiler and decompiling the resulting class file using one of the freely available decompilers yields:

import java.io.PrintStream;
import java.io.UnsupportedEncodingException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Arrays;

public class Authentication
{
    public static byte[] encryptPassword(String s)
        throws UnsupportedEncodingException, NoSuchAlgorithmException
    {
        String s1 = (new StringBuilder()).append(s).append("Add-Some-Salt").toString();
        byte abyte0[] = s1.getBytes("ISO-8859-1");
        MessageDigest messagedigest = MessageDigest.getInstance("SHA");
        messagedigest.update(abyte0);
        return messagedigest.digest();
    }

    public static boolean checkPassword(String s, byte abyte0[])
        throws UnsupportedEncodingException, NoSuchAlgorithmException
    {
        if(Arrays.equals(encryptPassword(s), abyte0))
        {
            return true;
        } else
        {
            System.out.println("Wrong password");
            return false;
        }
    }
}

As you may see, the only major differences in the decompiled source code are automatically generated names of parameters and local variables.

Let's run the above sample through a name obfuscator and decompile the resulting class:

import java.io.PrintStream;
import java.io.UnsupportedEncodingException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Arrays;

public class a
{
    public static byte[] a(String a)
        throws UnsupportedEncodingException, NoSuchAlgorithmException
    {
        String s = (new StringBuilder()).append(a).append("Add-Some-Salt").toString();
        byte abyte0[] = s.getBytes("ISO-8859-1");
        MessageDigest messagedigest = MessageDigest.getInstance("SHA");
        messagedigest.update(abyte0);
        return messagedigest.digest();
    }
    public static boolean a(String a, byte a[])
        throws UnsupportedEncodingException, NoSuchAlgorithmException
    {
        if(Arrays.equals(a(a), a))
        {
            return true;
        } else
        {
            System.out.println("Wrong password");
            return false;
        }
    }

Even though the obfuscator has replaced the public identifiers Authentication, encryptPassword() and checkPassword with meaningless, overloaded a, it is clear that these methods deal with the Security API and use the SHA algorithm. The salt string is also exposed.

Now, enabling string encryption makes the decompiled code a little bit more, well, cryptic:

import java.io.PrintStream;
import java.io.UnsupportedEncodingException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Arrays;

public class b
{
    public static byte[] a(String a)
        throws UnsupportedEncodingException, NoSuchAlgorithmException
    {
        String s = (new StringBuilder()).append(a).append(a.a("X|~4Nws|<Ksu!")).toString();
        byte abyte0[] = s.getBytes(a.a("]FX9(-&-1d"));
        MessageDigest messagedigest = MessageDigest.getInstance(a.a("E_\024"));
        messagedigest.update(abyte0);
        return messagedigest.digest();
    }
    public static boolean a(String a, byte a[])
        throws UnsupportedEncodingException, NoSuchAlgorithmException
    {
        if(Arrays.equals(a(a), a))
        {
            return true;
        } else
        {
            System.out.println(a.a("Cgxzw5cuofh{j1"));
            return false;
        }
    }
}

Okay, strings are now encrypted, but the import list is still there and it is perfectly clear that these methods use the Java Security API. So the hacker would still have little doubt over where to look for sensitive code, and s/he does not even need to reverse the encryption algorithm. All s/he needs is to extract the calls of the decryption method from the decompiled source:

// crack.java

public class crack {

    public static void main( String[] args ) {
        System.out.println(a.a("X|~4Nws|<Ksu!"));
        System.out.println(a.a("]FX9(-&-1d"));
        System.out.println(a.a("E_\024"));
        System.out.println(a.a("Cgxzw5cuofh{j1"));
    }
}

... and then compile and run the resulting code:

$ javac crack.java
$ java crack
Add-Some-Salt
ISO-8859-1
SHA
Wrong password
$

Voila!

Let's try code flow obfuscation now. On first sight, there is not much code to obfuscate here, and code and data flows are pretty simple: just a bunch of standard API calls without any loops or exception handling. Indeed, the obfuscator I was using could only make one change after I enabled code obfuscation:

        // Code ofbuscation disabled
        String s = (new StringBuilder()).append(a).append(a.a("X|~4Nws|<Ksu!")).toString();
        byte abyte0[] = s.getBytes(a.a("]FX9(-&-1d"));

        // Code ofbuscation enabled
        byte abyte0[] = (new StringBuilder()).append(a).append(a.a("X|~4Nws|<Ksu!")).toString().getBytes(a.a("]FX9(-&-1d"));

Perhaps that is just a weakness of the code obfuscation features implemented in a particular product? Indeed, it is possible to make the result of decompilation much less readable with JBCO. But before I move on, a word of caution:

DO NOT TRY THIS AT HOME!

or, more seriously, do not try to use JBCO in a production environment. It is a research project, and as such is aimed at enabling researchers to try their ideas. It is not meant to be scalable, robust, and well documented.

Anyway, I was able to push JBCO to the limits on the original version of Authentication.class using the following command line:

java -Xmx256m -cp sootclasses-2.2.4.jar;polyglotclasses-1.3.4.jar;jasminclasses-2.2.4.jar;. soot.jbco.Main -cp .;scimark2lib.jar;"C:\Program Files\Java\jre1.5.0_13\lib\rt.jar";"C:\Program Files\Java\jre1.5.0_13\lib\jce.jar" -t:9:wjtp.jbco_cr -t:9:wjtp.jbco_mr -t:9:wjtp.jbco_fr -t:9:wjtp.jbco_bapibm -t:9:wjtp.jbco_blbc -t:9:jtp.jbco_gia -t:9:jtp.jbco_adss -t:9:jtp.jbco_cae2bo -t:9:bb.jbco_cb2ji -t:9:bb.jbco_dcc -t:9:bb.jbco_rds -t:9:bb.jbco_riitcb -t:9:bb.jbco_iii -t:9:bb.jbco_plvb -t:9:bb.jbco_rlaii -t:9:bb.jbco_ctbcb -t:9:bb.jbco_ecvf -t:9:bb.jbco_ptss Authentication

Here is the output produced by the decompiler:

import java.io.PrintStream;
import java.io.UnsupportedEncodingException;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.Arrays;

public class Authentication
{

// JavaClassFileOutputException: Stack underflow

    public static byte[] II1(String s)
        throws UnsupportedEncodingException, NoSuchAlgorithmException
    {
_L2:
        StringBuilder stringbuilder;
        S$$(stringbuilder, s);
        stringbuilder = S$$(s);
        return stringbuilder;
        stringbuilder = JVM INSTR new #121 ;
        stringbuilder.StringBuilder();
        stringbuilder = S$$(s, stringbuilder);
        stringbuilder = S$$(S$$(ll1, stringbuilder));
        stringbuilder = S$$(S$$, stringbuilder);
        s = S$$(III);
        if(true) goto _L2; else goto _L1
_L1:
        throw ;
    }

    public static boolean S$$(String s, byte abyte0[])
        throws UnsupportedEncodingException, NoSuchAlgorithmException
    {
        if(!S$$(II1(s), abyte0)) goto _L2; else goto _L1
_L1:
        lll;
        lI1.booleanValue();
        JVM INSTR ifge 28;
           goto _L3 _L4
_L3:
        JVM INSTR pop ;
        break MISSING_BLOCK_LABEL_49;
_L5:
        throw ;
_L4:
        return;
_L2:
        JVM INSTR pop ;
        S$$(____, System.out);
        return II1;
        null;
          goto _L5
    }

    public static StringBuilder S$$(String s, StringBuilder stringbuilder)
    {
        return stringbuilder.append(s);
    }

    public static String S$$(StringBuilder stringbuilder)
    {
        return stringbuilder.toString();
    }

    public static byte[] S$$(String s, String s1)
    {
        return s1.getBytes(s);
    }

    public static MessageDigest S$$(String s)
    {
        return MessageDigest.getInstance(s);
    }

    public static void S$$(byte abyte0[], MessageDigest messagedigest)
    {
        // Placing a breakpoint here would reveal the salt string
        messagedigest.update(abyte0);  
    }

    public static byte[] S$$(MessageDigest messagedigest)
    {
        return messagedigest.digest();
    }

    public static boolean S$$(byte abyte0[], byte abyte1[])
    {
        return Arrays.equals(abyte0, abyte1);
    }

    public static void S$$(String s, PrintStream printstream)
    {
        printstream.println(s);
    }

    public static Boolean lI1;
    public static boolean ___;
    public static String ll1;
    public static String S$$;
    public static String III;
    public static String ____;
    public static int II1;
    public static int lll = 1;
        
    {
          goto _L1
_L3:
        Boolean boolean1;
        lI1 = boolean1;
        return;
_L1:
        ____ = "Wrong password";
        III = "SHA";
        S$$ = "ISO-8859-1";
        ll1 = "Add-Some-Salt";
        boolean1 = JVM INSTR new #15  ;
        boolean1.Boolean(true);
        if(true) goto _L3; else goto _L2
_L2:
        throw ;
    }
}

Reverting the above to a piece of Java source resembling the original Authentication.java is a very time-consuming task. But that is not necessarily what the attacker wants to achieve. Running the application under a debugger with a breakpoint set on MessageDigest.update() may reveal enough information about the password encryption scheme used in this fictional app.

Impact of Flow Obfuscation on Performance

You may now wonder what might be the degree of impact of such extensive transformations on application performance. So did I, therefore my next step was running a well-known benchmark suite through JBCO.

I have selected the SciMark 2.0a benchmark. It measures the performance of numerical computations typically found in scientific and engineering applications. These are the types of applications one may wish to protect against decompilation.

Another good thing about SciMark is that it validates the result of each test, which is useful for checking whether the transformations made by the obfuscator preserved the semantics of the original code. (Strictly speaking, I also had to disable obfuscation of the validation code so that it could serve as 100% proof.)

Here is the JBCO command line that I used:

java -Xmx384m -cp sootclasses-2.2.4.jar;polyglotclasses-1.3.4.jar;jasminclasses-2.2.4.jar;. soot.jbco.Main -cp .;scimark2lib.jar;"C:\Program Files\Java\jre1.5.0_13\lib\rt.jar";"C:\Program Files\Java\jre1.5.0_13\lib\jce.jar" -t:9:wjtp.jbco_cr -t:9:wjtp.jbco_mr -t:9:wjtp.jbco_fr -t:9:wjtp.jbco_bapibm -t:9:wjtp.jbco_blbc -t:9:jtp.jbco_gia -t:9:jtp.jbco_cae2bo -t:9:bb.jbco_cb2ji -t:9:bb.jbco_rds -t:9:bb.jbco_riitcb -t:9:bb.jbco_iii -app jnt.scimark2.commandline

(As you may see, I had to disable some of the transformations until finding a combination that does not cause JBCO to crash or hang on SciMark classes and results in emission of correct, verifiable bytecode. As I said above, JBCO is not meant to be production-ready.)

SciMark reports measurement results in terms of scores. A higher score is better. The original, non-obfuscated version produced the following output on Sun HotSpot 1.5.0_13:

SciMark 2.0a

Composite Score: 235.0564919826097
FFT (1024): 95.00221034941461
SOR (100x100):   457.27459020598775
Monte Carlo : 40.94500403786447
Sparse matmult (N=1000, nz=5000): 217.5829952655074
LU (100x100): 364.4776600542744
   .  .  .

Compared to that, the obfuscated version is slow like a worm:

SciMark 2.0a

Composite Score: 10.297235073178708
FFT (1024): 4.571244265099369
SOR (100x100):   14.39229057848735
Monte Carlo : 12.282002422350816
Sparse matmult (N=1000, nz=5000): 8.355773238345918
LU (100x100): 11.884864861610085
   .  .  .

The slowdown ranges from 3.3x for the Monte Carlo test to over 30x for SOR and LU. The composite score is 22.8x lower for the obfuscated version!

On the one hand, this means you have to be careful when obfuscating performance-sensitive code. On the other hand, if flow obfuscation has little or no impact on performance, it may be an indicator of obfuscation weakness. What an optimizing compiler such as Sun Hotspot may figure out, may also be figured out by a person of ordinary programming skills, especially if equipped with something like Understand for Java.

All that being said, I claim again that ahead-of-time compilation to native code is way better than flow obfuscation, and invite you to try Excelsior JET, a certified Java SE 6 AOT compiler that my company makes. What else you would have expected to find at the end of an article on a vendor site, anyway?

I update this article regularly, so if you have any comments or questions, or know of resource/tool URLs which I should have added, please send them to me.