Hand on laptop keyboard showing code obfuscation

Security Through Obscurity – How Code Obfuscation Works

What is code obfuscation?

A cornerstone of modern application security, obfuscation is an essential technique used to modify code or an executable file so that it is no longer transparent to foreign entities whilst still retaining functionality. Obfuscation therefore prevents the reverse-engineering of programs and is used to protect sensitive intellectual property (IP) such as algorithms that a company doesn’t want bad actors or competitors to see; the foremost example of this being security code. Despite the array of tools in a security professional’s arsenal, whether this be anti-debug, anti-tampering or hook detection, unless sufficiently hidden through obfuscation such measures are highly susceptible to interference and removal.

Code obfuscation has probably been around almost as long as programming itself but started to become noticed in 1984 with the launch of the 1st International Obfuscated C Code Contest. There is an industry rumour that military interest in code obfuscation arose after a US military helicopter crashed in China leaving systems software exposed to reverse engineering. Whether this is true or not, the widespread adoption of PCs and the need to protect high value applications led to the use of obfuscation within copy protection tools.

Today, with thousands of vulnerable banking and other apps being distributed in their billions to potentially insecure mobile devices, it is inevitable that higher levels of integrity are often needed. To achieve this aim, commercial obfuscation vendors like Promon and Arxan use obfuscation at one of three different levels: source based, native binary based level and at the intermediate level – the layer between compilers and the native code where optimizations are done.

Why is it still used?

Despite often being nothing more than UI’s on top of banking APIs, banking apps still need protection. Although mobile operating systems display a big improvement over Microsoft’s family of desktop operating systems in terms of security, they are still vulnerable. Even now both Apple and Google are finding security vulnerabilities and patching them. If any of those vulnerabilities allowed a root exploit (and plenty have so far) then a malicious app or remotely injected malware could interfere with the banking app. This might allow the theft of login credentials (often usable in another channel like web), rogue payments or misleading the user into signing a payment for the wrong recipient by changing the details displayed on the screen.

The first step in this kind of attack would be understanding the banking code. Code obfuscation hinders that. Although integrity checking and hook detection embedded in the app may prevent malicious code injection, the code that is doing that needs protection itself. This is why code obfuscation is now an integral part of wider software protection as well as protecting app code, for example, in the case of licence management or DRM.

What techniques are used?

Many sophisticated tools exist, both commercially and open-source, to aid attackers in the reverse engineering process. For instance, ‘apktool’ will unpack and disassemble an Android app allowing for an examination of how it works. Bad actors can then alter it and repack it to work with their modifications. App and code signing help protect against this but these can be disabled on a rooted device. For that reason, high integrity apps need their own OS-independent checks.

The job of code obfuscation is to make it hard to understand the code once it has been disassembled so it’s much harder to find what you’re looking for as a reverser. There are several techniques to help this:

  • Symbol renaming – this renames package, class, function, method and variable names to meaningless values so that their use can’t be inferred from their name. This technique is also often also used as an optimisation since shorter names reduce the size of the app. Deliberately shadowing names can also make variable lifetime unclear and make it hard to search for a particular reference.
  • String hiding – most apps contain lots of strings like URLs, user prompts, API keys, error text, etc. Seeing these in particular functions or methods is a big clue to what the function is doing. Strings are typically encrypted and then decrypted in memory at the point of use.
  • Control flow obfuscation – most programmers write well structured code that is logical and easy to maintain – this is a gift to the reverser. Obfuscators will often ‘flatten’ this structure so there is no procedural hierarchy or obvious logical flow between functions. Functions can be inlined, merged together and further confused by having redundant control flow structures like ‘opaque predicates’. Operations can be convoluted by making them more complicated e.g. pointlessly moving values between registers and the stack and vice versa or by adding convoluted arithmetic.
  • Literal hiding – numeric literals are often a big clue and so these are typically replaced with arithmetic operations that reconstruct them at runtime. An example are the constant vectors often used in crypto functions – finding these will pinpoint the crypto function allowing the key to be extracted by capturing the function arguments.
  • Whitebox cryptography – this is a data hiding technique where mathematical operations are carried out on data in a transformed domain so that original data can’t be seen in memory (only the transformed version). Entire cryptographic functions are then built from these primitives so that keys, and sometimes data, are entirely hidden.

A combination of these techniques, whilst interspersing with anti-debugging and integrity checking techniques, is the standard in software protection as it provides decent obscurity combined with self-defence. It’s good practice to use integrity checking checksums to derive control flow; then, if the code has been modified, for instance by inserting a breakpoint, it starts misbehaving, further frustrating reversing attempts. Direct checks with a ‘yes’ or ‘no’ answer are to be avoided as these can be easily found and defeated.

How effective are these techniques?

Well, certain approaches are difficult for bad actors to reverse, like symbol renaming, because information is discarded and can’t be recovered. Similarly, some control flow obfuscations would be pretty difficult to reverse since paths are merged. Whereas methods such as string encryption and numeric literal hiding can be reversed if you can stop the app at the right point and find the clear values in the memory – hence the importance of runtime controls like anti-debug.

A significant concern is that if some of the large transformer machine learning models could be trained on enough obfuscated apps, then they could start to automatically reverse some of these techniques by guessing at variable names and understanding how paths are merged. We haven’t really seen this yet but it is certainly coming. Mitigations against this sort of attack would probably require more use of code encryption but with virtualisation so that there’s no structure for the model to look at. These are areas that are being actively researched at the moment.

In any case, finding the most up-to-date methods of obfuscation is in a constant race against the clock. Only through a proactive and innovative approach, will security vendors be able to stay one step ahead and ensure sensitive IP remains secure.

 

Senior Technical Director at Promon