> ЗАМЕНТЕ ЈА ОВАА ЛИНИЈА СО ИНДЕНТИФИКАЦИОНИОТ БРОЈ НА ВАШИОТ ТРУД < 1

Software protection using Obfuscation

Aleksandar Ivanovski, Toni Stojanovski
aleksandar.ivanovski@live.com, toni.stojanovski@eurm.edu.mk

 valuable software. In the beginning we define code

Abstract— Much of the intellectual property of successful firms obfuscation. Also we are explaining measures for obfuscation,
is embedded in their software that is purchased or installed in potency- the extent to which man is confused when reading the
many of their clients. Such software is increasingly exposed to code, resilience, resilience- resistance from attacks from
attacks by those who use reversed engineering to try to
understand the structure and source code of the program and
automated deobfuscators and price- how much additional load
thus to gain access and use foreign intellectual property for their is added to obfuscated code. Further, we describe dynamic
own purposes. In this paper is reviewed software protection from obfuscation, methods and techniques for dynamic obfuscation.
reverse engineering attacks using method of obfuscation. Today it In the end, we give some examples and extend for dynamic
is accepted that automated obfuscation is most sustainable obfuscation, and also implementation of dynamic obfuscation
method for preventing reversed engineering. Obfuscator is based for malicious software, viruses etc.
on the application of code transformations, which in most cases
are similar to those used in the optimization of compilers. We are
describing most famous obfuscation transformations, and classify II. DEFINING OBFUSCATION
them according to their potency- the extent to which man is We have chosen two popular definitions for obfuscation.
confused when reading the code, resilience, resilience- resistance
from attacks from automated deobfuscators and price- how much A. Collberg et al
additional load is added to obfuscated code. Then we have First definition is from Collberg et al [1]:
explained some possible deobfuscation techniques. This paper 
discuses dynamic obfuscation in details, which is most popular Let P  P' be a transformation of a source program P

and most successful method for the protection of intellectual into a target program P ' . The transformation P   P' is
property stored in software code, but also used by hackers to hide
malicious code. Finally we recommend improvements to the an obfuscating transformation, if P and P ' have the same
methods of dynamic obfuscation. observable behavior. More precisely, in order for

P
 P' to be a legal obfuscating transformation the
Key words - obfuscation, deobfuscation, dynamic obfuscation, following conditions must hold:
software security, obfuscation techniques, software security.
 If P fails to terminate or terminates with an error
condition, then P ' may or may not terminate.
I. INTRODUCTION  Otherwise, P must terminate and produce the same
output as P ' .
S UPPOSE that a programmer develops a new piece of
software which contains a unique innovative algorithm. If
the programmer wants to sell this software, it is usually in the
B. Barak et al
Barak et al [2] takes a more formal approach to obfuscation —
programmer’s interest that the algorithm should remain secret their notion of obfuscation is as follows. An obfuscator O is a
from purchasers. What can the programmer do to stop the ―compiler‖ which takes as input a program P and produces a
algorithm from being seen or modified by a client? A similar
new program O(P) such that for every P :
issue arises when a piece of software contains a date stamp
that allows it to be used for a certain length of time. How can  Functionality — O(P) computes the same function as
we stop people from identifying (and changing) the date P.
stamp?  Polynomial Slowdown — the description length and
The method for protection of software that we are going to running time of O(P) are at most polynomial larger
present in this paper is obfuscation. This technique includes lot
than that of P .
of transformations, mostly from code optimization techniques.
 ―Virtual black box‖ property — ―Anything that can be
In this paper we put accent on dynamic obfuscation. Method
efficiently computed from O(P) can be efficiently
for software protection that is more resistant from attacks by
hacker. But also it is coming back like boomerang from computed given oracle access to P ‖ [2]
hacker’s side. It is also used to harm to users computers and With this definition, Barak et al construct a family of functions
which is unobfuscatable in the sense that there is no way of
obfuscating programs that compute these functions. Thus their
notion of obfuscation is impossible to achieve.
> ЗАМЕНТЕ ЈА ОВАА ЛИНИЈА СО ИНДЕНТИФИКАЦИОНИОТ БРОЈ НА ВАШИОТ ТРУД < 2

This definition of obfuscation, in particular the ―Virtual Black This transformation is valid under certain conditions such as
Box‖ property, is too strong — obfuscators will still be of ensuring that the statements S1 and S2 do not change the value
practical use even if they do not provide perfect black boxes. of q. We can easily extend this to obfuscate larger blocks with
This definition does not give an indication of the quality of a a bigger set of dynamically opaque predicates but the
proposed obfuscation technique (the measure says whether the conditions for ensuring that the transformation is valid become
transformation completely obfuscates or not). After the more complex.
publication of Barak et al [2] , the focus of obfuscation
B. Variable transformations
research has changed to designing obfuscations that are
difficult, but not necessarily impossible, for an attacker to In this section, we show how to transform an integer variable i
undo. within a method. To do this, we define two functions f and g:
f :: X  Y
III. OBFUSCATING TRANSFORMATIONS g :: Y  X
In this part we have explained some obfuscation techniques
where X  Z — this represents the set of values that i takes.
and also extend some of them.
We require that g is a left inverse of f (and so f needs to be
A. Opaque predicates injective). To replace the variable i with a new variable, j say,
One of the most valuable obfuscation techniques is the use of of type Y we need to perform two kinds of replacement
opaque predicates. An opaque predicate P is a predicate depending on whether we have an assignment to i or use of i .
An assignment to i is a statement of the form i = V and a use of
whose value is known at obfuscation time — P T denotes a
F ? i is an occurrence of i which is not an assignment. The two
predicate which is always True (similarly for P ) and P replacements are:
denotes a predicate which sometimes evaluates to True and  Any assignments of i of the form i = V are replaced by
sometimes to False. j = f (V ).
Here are some example predicates which are always True  Any uses of i are replaced by g(j ).
(supposing that x and y are integers): These replacements can be used to obfuscate a while loop.
x2  0 C. Loops
x ( x  1)
2 2
 0(mod 4) In this section, we show some possible obfuscations of this
simple while loop:
x2  7 y2  1 i = 1;
Opaque predicates can be used to transform a program block while (i < 100)
B as follows: {
T ...
 if ( P ) { B ; }This hides the fact that B will always
i + +;
be executed.
F }
 if ( P ) { B ' ; } else { B ; }. Since the predicate is We can obfuscate the loop counter i — one possible way is to
always false we can make the block B ' to be a copy use a variable transformation. We define functions f and g to
of B which may contain errors. be:
?
 if ( P ) { B ; } else { B ' ; }. In this case, we can have f(i) = (2i + 3)
two copies of B each with the same functionality. g(i) = (i − 3) div 2
When creating opaque predicates, we must ensure that they are and we can verify that g · f = id.
stealthy so that it is not obvious to an attacker that a predicate Using the rules for variable transformation, we obtain:
is in fact bogus. So we must choose predicates that match the j = 5;
―style‖ of the program and if possible use expressions that are while ((j − 3)/2 < 100)
already used within the program. {
1) Extensions of Opaque Predicates ...
As mentioned in the previous section, one of the limitations j = (2 * (((j − 3)/2) + 1)) + 3;
with using opaque predicates is that often the value of an }
opaque predicate is true for all possible inputs. With some simplifications, the loop becomes:
This lead to developing dynamically opaque predicates. These j = 5;
are a set of predicates which all evaluate to the same result in while (j < 203)
any given run, but in different runs they may evaluate to {
different values. For example, if we had a block of three ...
statements S1; S2; S3 and two linked predicate p and q (which j = j + 2;
evaluate to the same value in the same run) then we can }
obfuscate the block as follows: Another method we could use is to introduce a new variable, k
if (p) S1; else {S1; S2; } say, into the loop and put an opaque predicate (depending on
if (q) {S2; S3; } else S3; k) into the guard. The variable k performs no function in the
> ЗАМЕНТЕ ЈА ОВАА ЛИНИЈА СО ИНДЕНТИФИКАЦИОНИОТ БРОЈ НА ВАШИОТ ТРУД < 3

loop, so we can make any assignment to k. As an example, our { case 1 :

loop could be transformed to something of the form: init; var = 2; break;
i = 1; case 2 :
k = 20; if (cond) var = 3; else var = 4; break;
while (i < 100 && (k k (k + 1) (k + 1)%4 == 0)) case 3 :
{ loop body; var = 2; break;
... case 4 :
i + +; var = 1; end; }
k = k (i + 3); The variable var acts as a dispatcher and effectively controls
} the execution of the blocks. Each case contains an assignment
We could also put a false predicate into the middle of the loop to var (and in the last case the assignment is a dummy one).
that attempts to jump out of the loop. Here is a more complicated example which has conditional
Our last example changes the original single loop into two statements:
loops: init;
j = 0; while (cond)
k = 1; { loop body;
while (j < 10) if (test) stat1; else stat2; }
{ Again, we can use a switch block to convert the program to:
while (k < 10) var = 1;
{ switch (var)
<replace uses of i by (10 j ) + k > { case 1 :
k + +; init; var = 2; break;
} case 2 :
k = 0; if (cond) var = 3; else var = 7; break;
j + +; case 3 :
} loop body; var = 4; break;
case 4 :
D. Array transformations
if (test) var = 5; else var = 6; break;
There are many ways in which arrays can be obfuscated. One case 5 :
of the simplest ways is to change the array indices. Such a stat1; var = 2; break;
change could be achieved either by a variable transformation case 6 :
or by defining a permutation. stat2; var = 2; break;
Here is an example permutation for an array of size n: case 7 :
p =  i .(a × i + b (mod n)) where gcd (a, n) = 1 var = 1; end; }
Other array transformations involve changing the structure of It is quite easy to reconstruct the conditional statement and the
an array. One way of changing the structure is by choosing while loop from the switch block. But there are some extra
different array dimensions. We could fold a 1-dimensional levels of obfuscation. First, some dummy cases can be added
array of size m ×n into a 2-dimensional array of size [m, n]. to the switch to obscure the control-flow — for instance,
Similarly we could flatten an n-dimensional array into a 1- irreducible jumps could be added. Also, a global variable, such
dimensional array. as an array, can be used to dynamically determine the values of
Before performing array transformations, we must ensure that the dispatcher variable. Finally, pointers can be used so that
the arrays are safe to transform. For example, we may require static analysis of the control-flow becomes difficult to perform.
that a whole array is not passed to another method or that
elements of the array do not throw exceptions. F. Merging and Splitting
As well as obfuscating individual variables, we can also work
with obfuscating a number of variables together. Suppose
E. Control-Flow flattening that we want to merge two variables x and y. If we know
The idea of this transformation is to remove control-flow 0  x  N and y  0 then we can define z as follows:
constructs such as while loops so that all basic blocks have the
z=N*y+x
same predecessor and successor in the control-flow graph. So,
As well as merging two or more variables, we can split up one
for example, suppose we have this piece of code:
variable in two (or more) other variables.
init;
An integer variable x can be split into two variables a and b
while (cond)
such that
{ loop body; }
a = x div 10 and b = x mod 10
Then we could remove the while loop and replace it with a For example, under this transformation, the statement x++ is
switch block which loops until an end statement is reached: transformed to:
var = 1; a = (10 * a + b + 1) div 10;
switch (var) b = (b + 1) mod 10;
> ЗАМЕНТЕ ЈА ОВАА ЛИНИЈА СО ИНДЕНТИФИКАЦИОНИОТ БРОЈ НА ВАШИОТ ТРУД < 4

These assignments are equivalent to:

if (b == 9) {a = a + 1; b = 0; } else {b = b + 1; }
enter
IV. DYNAMIC OBFUSCATION
A dynamic obfuscator typically runs in two phases. The first A move origin, target
phase, at compile-time, transforms the program to an initial
configuration and then adds a runtime code transformer.
During the second phase, at runtime, the execution of the B
target
program code is interspersed with calls to this transformer. As
a consequence, a dynamic obfuscator turns a ―normal‖
program into a self-modifying one. C move bogus, target
A. Differences between static and dynamic obfuscation
 Static obfuscations transform the code prior to
execution. EXIT
 Dynamic algorithms transform the program at runtime.
 Static obfuscation counter attacks by static analysis. C. Dynamic obfuscation, aucsmith algorithm
 Dynamic obfuscation counter attacks by dynamic The idea of this algorithm is following:
analysis 1. Split the program into pieces
2. xor them with each other
 A dynamic obfuscator runs in two phases: 3. Add encryption to instructions
Illustration:
o At compile-time transform the program to an
initial configuration and add a runtime code- a
transformer. a
o At runtime, intersperse the execution of the b
program with calls to the transformer. d
o A dynamic obfuscator turns a normal program c e xor f
into a self-modifying one. split reorder
a xor b
B. Dynamic obfuscation: replacing instructions d
c
In following example we show how dynamic obfuscation is
e
used to dynamic change the way of how program is P={a,…,f} f
obfuscated.
f
Obfuscate (P):
1. Select three points A, B, and C in P, such that: In general sense:
a. A strictly dominates B, o (A ⊕ B)⊕ A =B
b. C strictly post-dominates B, and o (A ⊕ B)⊕ B =A
c. any path from B to A passes The idea is:
d. through C. o Reorder blocks such that upper and lower blocks are
2. Let orig be the instruction at B. alternated
3. Let orig be the instruction at B. o Execute a block and xor upper with lower if even
4. Let orig be the instruction at B. iterate
5. At point A insert the instruction move orig, v=B where o Execute a block and xor lower with upper if odd iterate
v is an opaque expression that evaluates to the Obfuscation steps:
address of point B. 1. Obfuscate(P):
6. Similarly, at point C insert the instruction move bogus, 2. Split P into n pieces: P0, . . . ,Pn−1.
v=B. 3. Let C0, . . . ,Cn−1 be the memory cells in which the
pieces will reside at runtime. The Ci ’s are of equal
size, large enough to fit the largest piece Pj .
4. Cells are, conceptually, divided into two spaces, upper
(C0, . . . ,Cn/2−1) and lower (Cn/2, . . . ,Cn−1). Each
cell in the upper space partners with a cell in the
lower space. Select a partner function PF(c) that maps
a cell number to the cell number of its partner, such
as PF(c) =c +K, for some constant K. Let IV0, . . . ,
IVn−1 be the initial values of cells C0, . . . ,Cn−1,
respectively.
> ЗАМЕНТЕ ЈА ОВАА ЛИНИЈА СО ИНДЕНТИФИКАЦИОНИОТ БРОЈ НА ВАШИОТ ТРУД < 5

5. Initialize a set of equations E1 ={C0 =IV0, . . . ,Cn1 programs need to detect that virus, and then build software or
=IVn−1} which expresses the current state of the tools for removing viruses from user pc.
memory cells as a function of their initial values. Dynamic obfuscation is a powerful tool in hands of hacker.
6. Initialize a set of equations E2 ={} which expresses And the second reason for this is that every time virus code is
how a piece Pi can be recovered in cleartext from the changed, dynamically, it can be detect from antivirus program.
initial values IV0, . . . , IVn−1. Because this paper is concentrated on methods for obfuscation,
7. Initialize a table next =hP0 =?, . . . ,Pn−1 =?i which we will provide some basic methods for implementing
maps each subprogram Pi to the cell it should jump to dynamic obfuscation in malware software.
in order to execute Pi+1. In the conclusion we will provide methods for deobfuscating
8. make obscure() and removal of malware software. However, only one solution
And the other function that is important for this algorithm id for detecting malware software is deobfuscaiton and then
make_obscure(). So for this method we have: providing unique removal software (antivirus) for that virus or
For p ∈ [0 . . . n −1] do worm etc.
1. Select a cell Cc to hold piece Pp in cleartext.
A. Obfuscation methods for malware software
2. Consult E1 to find the current contents V of Cc.
Update E2 :=E2[P −p =V]. Using Gaussian 1) Trojan.Clampi – Virtualization
elimination, try to invert E2 (i.e. find values for all Trojan.Clampi used a commercial tool to obfuscate its code.
the IVi ’s). If there is no solution select another cell Essentially this tool converts the existing instructions into an
for Pp. intermediate language. It also embeds a custom interpreter
3. next :=next[Pp−1 =Cc] For even (odd) p:s, simulate a known as a virtual machine to interpret this custom language.
mutation where every cell Ci in upper (lower) space Reverse engineering this code requires an understanding of
is xor:ed with its partner cell CPF(i) in lower (upper) how the virtual machine processes the custom code. While not
space. impossible this can be a very time-consuming task. To
4. E1 :=E1[CPF(i) =CPF(i) ⊕ Ci ]. understand Clampi one cannot simply rearrange blocks back
into a readable order, but must decipher essentially an entirely
D. Ideas for dynamic obfuscation new pseudo-language for each new sample.
Dynamic obfuscation is really great method in obfuscation 2) Entry point obfuscation
theory for protection of program code. Our opinion is that in Modifying an executable’s start address, or the code at the
this stage dynamic obfuscation hasn’t any good algorithm for original start address, constitutes extremely suspicious
code protection. Maybe the best combination of protection is behavior for anti-virus heuristics. A virus can try to get control
to combine two areas of protection, and that is obfuscation and elsewhere instead; this is called entry point obfuscation or
encryption. EPO.
We think that mixture of obfuscation and encryption can give 3) Packers
really good results for code protection. Reasons for this are: Packers are a throwback to days of yore when the Internet was
o If we use AES to encrypt already obfuscate code, we still a research toy and computer storage space was at a
can double protection level of a program premium. System RAM and disks were much smaller in the
o AES isn’t cracked till today. In other words, there is no 80’s and early 90’s. To keep the size of binary executables to
known methods that have cracked all rounds of an absolute minimum, so-called packing tools were
Rijndael algorithm. popularized that encrypted and compressed files. While
The main idea is not to obfuscate whole code that is executed, packers still have legitimate uses today (bundling executables
because that will significantly slow down all application. Code with component files and commercial software protection),
that needs to be transformed with transformer also should be this technique was adopted and extended by malware authors
encrypted and the transformer should also have encryption key to add polymorphism, armoring, metamorphism, EPO, and a
so it can in same time decrypt obfuscated code. Maybe it will host of other techniques aimed at evading AV scanners.
slow down code a little, but it doubles the protection of code. Packers offer powerful benefits to malware authors. When
It should be mentioned that in any case, dynamic obfuscation creating a new strain of an existing malware, if the malware
have higher level of protection that static obfuscation because author modifies most of the code but leaves parts of it intact
it dynamically change program every time that is executed. (or picks and chooses pieces from other existing malware), the
resultant executable will share patterns with its relatives. This
V. OBFUSCATING MALWARE means that if any signature exists for any piece of the
Today we face up with more and more sophisticated methods antecedent, an AV scanner can match this pattern. However,
for hiding malicious or malware software, so it can penetrate packing the file with a packer means that just a tiny change in
into user OS and programs. Antivirus program constantly are the source (for example, changing a register name) will result
trying to detect this kind of obfuscated viruses that are using in a radically different binary executable. This effect is akin to
dynamic obfuscation.Wethink that hackers are step forward. how a single letter change in a lengthy document will result in
Because fundamental theory says, we should have a reason so a completely different cryptographic hash. There are literally
we can act. In our case, companies that produce antivirus thousands of discrete packing tools out there that are used to
compress, encrypt and armor malware.
> ЗАМЕНТЕ ЈА ОВАА ЛИНИЈА СО ИНДЕНТИФИКАЦИОНИОТ БРОЈ НА ВАШИОТ ТРУД < 6

4) Polypack software protection lies in dynamic obfuscation. This method

In 2009, University of Michigan PhD student Jon Oberheide for protection can and will be more and more developed and
debuted Polypack, a web-based automated file packing updated in future. Reason for this is that in this moment it
service. Dubbed by pundits as a ‖Crimeware as a Service‖ hasn’t very good level of security for protecting software from
(CaaS) site, Polypack was in reality a service that evaluated hackers, and reverse engineers.
the effectiveness of AV scanners when detecting packed This can be compared with software, or hardware. Every
malware. While it did show the ease with which someone software and hardware in his early development is
could launch such a CaaS site, theirs is pro-bono only and rudimentary, and as years are going it is upgraded and
password protected from the masses. What makes Polypack updated. That is same with dynamic obfuscation. In this
notably notorious is that it offers (registered) users automated moment it is a ―baby‖ that needs to be upgraded.
access to a multitude of packers and AV scanners. The Further development should be concentrated on bigger security
submitted file is packed by each packer and then scanned by level of converting dynamically obfuscated code in runtime.
each of the AV engines and the results displayed. It offers Also further development should concentrate on:
users a quick way to determine an effective evasive packing  more methods and algorithms for dynamic obfuscation
solution. Malware authors can use this model for obvious  moving step forward from hackers and reverse
obfuscatory gain. engineers
B. Protecting from obfuscated malware software  add more measures for dynamic obfuscation such as:
o measuring incompleteness
Deobfustacating simple obfuscated malware software is very o measuring complexity of complete refinements
easy. Reason for this is because in obfuscated software: Finally, we conclude that at the moment obfuscation and its
 is added bogus code most promising variant - dynamic obfuscation are providing
 or maybe are added opaque predicates best protection of software.
 code is optimized, because lot of techniques for
compiler optimization are used in obfuscation VII. BIBLIOGRAPHY
 etc [1] A. V. Aho, R. Sethi, and J. D. Ullman. Compilers: Principles,
In other words, till now every method that is implemented in Techniques, and Tools. Addison-Wesley, 1986.
static code obfuscation have corresponding deobfuscation [2] Boaz Barak, Oded Goldreich, Russell Impagliazzo, Steven Rudich, Amit
method. That’s the biggest and only one reason for very low Sahai, Salil P. Vadhan, and Ke Yang. On the (im)possibility of
obfuscating programs. In Proceedings of the 21st Annual International
usage of static code obfuscation in malware software. Cryptology Conference on Advances in Cryptology, pages 1–18.
And the second obfuscation, dynamic obfuscation, is more Springer-Verlag, 2001.
used today in malware software, such are Trojans, worms etc. [3] Jesse C. Rabek Roger I. Khazan Scott M. Lewandowski Robert K.
The reason for this is definition given in IV. Cunningham. Detection of Injected, Dynamically Generated and
Obfuscated Malicious Code. Massachusetts Institute of Technology.
“The first phase, at compile-time, transforms the program to
[4] Roberto Giacobazzi. CODE OBFUSCATION- DEFENSE
an initial configuration and then adds a runtime code STRATEGIES. Dipartimento di Informatica,Universit`a degli Studi di,
transformer. During the second phase, at runtime, the Verona, Italy. 2009
execution of the program code is interspersed with calls to this [5] William Feng Zhu, Concepts and Techniques in Software Watermarking
transformer. As a consequence, a dynamic obfuscator turns a and Obfuscation. The Department of Computer Sciences, The
University of Auckland, New Zealand. August 2007
“normal” program into a self-modifying one.” [6] Stephen Drape, Obfuscation of Abstract Data-Types, St John’s College,
The prevention for malware lies in runtime. Precisely in University of Oxford. 2004
runtime code transformer. First step for program that detects [7] Monirul Sharif, Andrea Lanzi, Jonathon Giffin, Wenke Lee. Impeding
malware software should be looking when in runtime will be Malware Analysis Using Conditional Code Obfuscation. School of
Computer Science, College of Computing, Georgia Institute of
added transformer, and the second step is to monitor execution Technology, USA.
of program code that is calling that transformer. [8] Haruaki Tamada, Masahide Nakamura, Akito Monden, Ken-ichi
We think that this is the most secure way for preventing Matsumoto. INTRODUCING DYNAMIC NAME RESOLUTION
attacks from malware software. MECHANISM FOR OBFUSCATING SYSTEM-DEFINED NAMES IN
PROGRAMS. Proc. of IASTED International Conference on Software
Every algorithm till today, for preventing malware software, Engineering (IASTED SE 2008), Innsbruck, Austria, February 2008
has global idea as it was described above.

VI. CONCLUSION
Obfuscation as a field of research has still a long way to go till
find really stable methods for code protection. But in this
moment it is the most sustainable method compared to
watermarking, tampering and other techniques for software
and intellectual protection.
Static obfuscation, as predecessor of dynamic obfuscation,
provided excellent stable foundation for dynamic obfuscation.
As method we think that static obfuscation has still to be
developed, but it has no shiny future. The key method for