A method for detecting obfuscated calls in malicious binaries

IEEE Transactions on Software Engineering

Volumn 31, Issue 11, 2005, Pages 955-967

  Lakhotia, Arun ^a   Kumar, Eric Uday ^b   Venable, Michael ^a  

^a University of Louisiana at Lafayette   (United States)

^b Authentium Inc   (United States)

Author keywords

Abstract stack; Invasive software (viruses, worms); Obfuscation; Program analysis; Validation

Indexed keywords

ABSTRACT STACKS; INVASIVE SOFTWARES; OBFUSCATION; PROGRAM ANALYSIS;

ALGORITHMS; BINARY CODES; COMPUTER GRAPHICS; COMPUTER VIRUSES; DATA REDUCTION;

COMPUTER OPERATING SYSTEMS;

EID: 33751120104     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.2005.120     Document Type: Article

References (27)

1. "TESO, Burneye Elf Encryption Program," https://teso.scene.at, Nov. 2004.
2. "z0mbie," http://z0mbie.host.sk, Nov. 2004.
3. G. Balakrishnan and T. Reps, "Analyzing Memory Accesses in X86 Executables," Proc. Inl'l Conf. Compiler Construction (CO, 2004.
4. B. Barak, O. Goldreich, R. Impagliazzo, S. Rudich, A. Sahai, S. Vadhan, and K. Yang, "On the (Im)Possibility of Obfuscating Programs," Proc. Conf. Advances in Cryptology (CRYPTO '01), 2001.
5. D. Chess and S. White, "An Undetectable Computer Virus," Proc. Virus Bulletin Conf., 2000.
6. S. Cho, "Win32 Disassembler," http://www.geocities.com/sangcho/ disasm.html, Nov. 2004.
7. M. Christodorescu, S. Jha, S.A. Seshia, D. Song, and R.E. Bryant, "Semantics-Aware Malware Detection," Proc. IEEE Symp. Security and Privacy, 2005.
8. M. Christodrescu and S. Jha, "Static Analysis of Executables to Detect Malicious Patterns," Proc. 12th USENIX Security Symp. (Security '03), 2003.
9. C. Cifuentes and K.J. Gough, "Decompilation of Binary Programs," Software Practice and Experience, vol. 25, pp. 811-829, 1995.
10. F. Cohen, "Computational Aspects of Computer Viruses," Computers and Security, vol. 8, pp. 325-344, 1989.
11. C. Collberg and C. Thomborson, "Watermarking, Tamper-Proofing, and Obfuscation-Tools for Software Protection," IEEE Trans. Software Eng., vol. 28, pp. 735-746, 2002.
12. C. Collberg, C. Thomborson, and D. Low, "A Taxonomy of Obfuscating Transformations," technical report, Dept. of Computer Science, The University of Auckland 148, July 1997.
13. P. Cousot and R. Cousot, "Static Determination of Dynamic Properties of Programs," Proc. Second Int'l Symp. Programming, 1976.
14. N.D. Jones and F. Nielson, "Abstract Interpretation: A Semantics-Based Tool for Program Analysis," Handbook of Logic in Computer Science: Semantic Modelling, vol. 4, pp. 527-636, 1995.
15. C. Kruegel, W. Robertson, F. Valeur, and G. Vigna, "Static Disassembly of Obfuscated Binaries," Proc. USENIX Security Conf., 2004.
16. A. Lakhotia, M.E. Karim, A. Walenstein, and L. Parida, "Phylogeny Using Maximal pi-Patterns," Proc. 14th EICAR Conf., 2005.
17. A. Lakhotia and M. Mohammed, "Imposing Order on Program Statements and Its Implication to AV Scanners," Proc. 11th IEEE Working Conf. Reverse Eng., 2004.
18. A. Lakhotia and P.K. Singh, "Challenges in Getting Formal with Viruses," Virus Bull., pp. 14-18, 2003.
19. C. Linn and S. Debray, "Obfuscation of Executable Code to Improve Resistance to Static Disassembly," Proc. 10th ACM Conf. Computer and Comm. Security, 2003.
20. C. Nachenberg, "Computer Virus-Antivirus Coevolution," Comm. ACM, vol. 40, pp. 46-51, 1997.
21. D. Schmidt, "Abstract Interpretation and Static Analysis," http://www.cis.ksu.edu/santos/schmidt/Escuela03/, Feb. 2005.
22. B. Schwarz, S. Debray, and G. Andrews, "Disassembly of Executable Code Revisited," Proc. Ninth Working Conf. Reverse Eng. (WCRE '02), 2002.
23. Symantec, "Understanding Heuristics: Symantec's Bloodhound Technology," http://www.symantec.com/avcenter/reference/heuristc.pdf, July 2004.
24. P. Szor and P. Ferrie, "Hunting for Metamorphic," Proc. Virus Bull. Conf., 2001.
25. M. Venable, M. Chouchane, M.E. Karim, and A. Lakhotia, "Analyzing Memory Accesses in Obfuscated x86 Executables," Proc. Conf. Detection of Intrusions and Mahvare and VuInerability Assessment, 2005.
26. L. Vinciguerra, L. Wills, N. Kejriwal, P. Martino, and R. Vinciguerra, "An Experimentation Framework for Evaluating Disassembly and Decompilation Tools for C++ and Java," Proc. 10th Working Conf. Reverse Eng., 2003.
27. G. Wroblewski, "General Method of Program Code Obfuscation," technical report, Inst. of Eng. Cybernetics, Wroclaw Univ. of Technology, 2002.