Linear obfuscation to combat symbolic execution

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 6879 LNCS, Issue , 2011, Pages 210-226

  Wang, Zhi ^a   Ming, Jiang ^b   Jia, Chunfu ^a   Gao, Debin ^c  

^a NANKAI UNIVERSITY   (China)

^b PENNSYLVANIA STATE UNIVERSITY   (United States)

^c SINGAPORE MANAGEMENT UNIVERSITY   (Singapore)

Author keywords

malware analysis; Software obfuscation; symbolic execution

Indexed keywords

CODES (SYMBOLS); DATA OBFUSCATION; MODEL CHECKING;

CRYPTOGRAPHIC OPERATIONS; LINEAR OPERATIONS; MALICIOUS CODES; MALWARE ANALYSIS; MALWARES; SIMPLE LOOP; SOFTWARE OBFUSCATION; SYMBOLIC EXECUTION; TRIGGER CONDITIONS;

MALWARE;

EID: 80052983876     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-23822-2_12     Document Type: Conference Paper

References (40)

1. Boonstoppel, P., Cadar, C., Engler, D.: RWset: Attacking path explosion in constraint-based test generation. In: Ramakrishnan, C.R., Rehof, J. (eds.) TACAS 2008. LNCS, vol. 4963, pp. 351-366. Springer, Heidelberg (2008)
2. Brumley, D., Hartwig, C., Kang, M.G., Liang, Z., Newsome, J., Poosankam, P., Song, D., Yin., H.: Bitscope: Automatically dissecting malicious binaries. Technical report cs-07-133, School of Computer Science, Carnegie Mellon University (March 2007)
3. Brumley, D., Hartwig, C., Liang, Z., Newsome, J., Poosankam, J., Song, D., Yin, H.: Automatically identifying trigger-based behavior in malware. In: Botnet Analysis in Defense, vol. 36 (2007)
4. Brumley, D., Newsome, J., Song, D., Wang, H., Jha, S.: Towards automatic generation of vulnerability-based signatures. In: Proceedings of the 2006 IEEE Symposimu on Security and Privacy (2006)
5. Brumley, D., Wang, H., Jha, S., Song, D.: Creating vulnerability signature using weakest preconditions. In: Proceedings of the 2007 IEEE Symposium on Security and Privacy (2007)
6. Caballero, J., Liang, Z., Poosankam, P., Song, D.: Towards generating high coverage vulnerability-based signatures with protocol-level constraint-guided exploration. In: Proceedings of the 12th International Symposium on Recent Advances in Intrusion Detection (2009)
7. Caballero, J., McCamant, S., Barth, A., Song, D.: Extracting models of security-sensitive operations using string-enhanced white-box exploration on binaries. Tech. rep., Technical Report UCB/EECS-2009-36, EECS Department, University of California, Berkeley (March 2009)
8. Cadar, C., Dunbar, D., Engler, D.: Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs. In: Proceedings of the 2008 USENIX Symposium on Operating Systems Design and Implementation, OSDI 2008 (2008)
9. Cadar, C., Engler, D.: Execution generated test cases: How to make systems code crash itself. In: Proceedings of the 12th SPIN Workshop (2005)
10. Cadar, C., Ganesh, V., Pawlowski, P., Dill, D., Engler, D.: EXE:automatically generating inputs of death. In: Proceedings of the 2006 ACM Conference on Computer and Communications Security (CCS 2006) (2006)
11. Collberg, C., Thomborson, C., Low, D.: A taxonomy of obfuscating transformations. Technical report 148, Department of Computer Sciences, The University of Auckland (1997)
12. Comparetti, P.M., Salvaneschi, G., Kirda, E., Kolbitsch, C., Kruegel, C., Zanero, S.: Identifying dormant functionality in malware programs. In: Proceedings of the 2010 IEEE Symposium on Security and Privacy (2010)
13. Conway, J.H.: Unpredictable iterations. In: Proceedings of the 1972 Number Theorey Conference (1972)
14. Costa, M., Castro, M., Zhou, L., Zhang, L., Peinado, M.: Bouncer: Securing software by blocking bad input. In: Proceedings of the 2007 ACM Symposium on Operating Systems Principles (SOSP) (2007)
15. Crandall, R.E.: On the "3x + 1" problem. Mathematics of Computation 32, 1281-1292 (1978)
16. Ferrie, P.: W32.Mydoom (2004), http://www.symantec.com/security-response/ writeup.jsp?docid=2004-012612-5422-99&tabid=2
17. Gao, D., Reiter, M.K., Song, D.: BinHunt: Automatically finding semantic differences in binary programs. In: Chen, L., Ryan, M.D., Wang, G. (eds.) ICICS 2008. LNCS, vol. 5308, pp. 238-255. Springer, Heidelberg (2008)
18. Godefroid, P., Klarlund, N., Sen, K.: DART: Directed automated random testing. In: Proceedings of the ACM Conference on Programming Lanuguage Design and Implementation (2005)
19. Godefroid, P., Levin, M.Y., Molnar, D.: Automated whitebox fuzz testing. In: Proceedings of the 15th Annual Network and Distributed System Security Symposium (NDSS 2008) (2008)
20. Guy, R.K.: Unsolved problems in number theory. Problem Books in Mathematics (2004)
21. King, J.C.: Symbolic execution and program testing. Commun. ACM 19, 385-394 (1976), http://doi.acm.org/10.1145/360248.360252
22. Knowles, D., Perriott, F.: W32.Blaster (2003), http://www.symantec.com/ security-response/writeup.jsp?docid=2003-081113-0229-99&tabid=2
23. Lagarias, J.C.: The 3x+1 problem and its generations. Amer. Math. Monthly 92, 3-23 (1985)
24. Lee, B., Kim, Y., Kim, J.: binOb+: a framework for potent and stealthy binary obfuscation. In: Proceedings of the 5th ACM Symposium on Information, Computer and Communications Security (2010)
25. Lee, G., Morris, J., Parker, K., Bundell, G., Lam, P.: Using symbolic execution to guide test generation. Software Testing, Verification & Reliability 15(1), 41-61 (2005)
26. Linn, C., Debray, S.: Obfuscation of executable code to improve resistance to static disassembly. In: Proceedings of the 10th ACM Conference on Computer and Communications Security (2003)
27. Molnar, D., Li, X., Wagner, D.: Dynamic test generation to find integer bugs in x86 binary linux programs. In: Proceedings of the 2009 USENIX Security Symposium (2009)
28. Moser, A., Kruegel, C., Kirda, E.: Exploring multiple execution paths for malware analysis. In: Proceedings of the 2007 USENIX Security Symposium (2007)
29. Moser, A., Kruegel, C., Kirda, E.: Limits of static analysis for malware detection. In: Proceedings of the 23rd Annual Computer Security Applications Conference (2007)
30. Newsome, J., Brumley, D., Franklin, J., Song, D.: Replayer: Automatic protocol replay by binary analysis. In: Proceedings of the 13th ACM Conference on Computer and and Communications Security (CCS 2006) (2006)
31. Popov, I.V., Debray, S.K., Andrews, G.R.: Binary obfuscation using signals. In: Proceedings of the 2007 USENIX Security Symposium (2007)
32. Saxena, P., Poosankam, P., McCamant, S., Song, D.: Loop-extended symbolic execution on binary programs. In: Proceedings of the 18th International Symposium on Software Testing and Analysis (2009)
33. Schwartz, E.J., Avgerinos, T., Brumley, D.: All you ever wanted to know about dynamic taint analysis and forward symbolic execution (but might have been afraid to ask). In: Proceedings of the 2010 IEEE Symposium on Security and Privacy (2010)
34. Sen, K., Marinov, D., Agha, G.: CUTE: A concolic unit testing engine for c. In: Proceedings of 13th International Symposium on the Foundations of Software Engineering, FSE 2005 (2005)
35. Sharif, M., Lanzi, A., Giffin, J., Lee, W.: Impeding malware analysis using conditional code obfuscation. In: Proceedings of the 15th Annual Network and Distributed System Security Symposium (NDSS 2008) (2008)
36. Shinotsuka, H.: W32.NetSky (2004), http://www.symantec.com/security- response/writeup.jsp?docid=2004-030717-4718-99&tabid=2
37. Silva, T.O.: Computational verification of the 3x+1 conjecture. Tech. rep., Electronics, Telecommunications, and Informatics Department, University of Aveiro (November 2010), http://www.ieeta.pt/~tos/3x+1.html
38. Wang, T., Wei, T., Lin, Z., Zou, W.: Intscope: Automatically detecting integer overflow vulnerability in x86 binary using symbolic execution. In: Proceedings of the 16th Annual Network and Distributed System Security Symposium (NDSS 2009) (2009)
39. Xu, R.G., Godefroid, P., Majumdar, R.: Testing for buffer overflows with length abstraction. In: Proceedings of the 2008 International Symposium on Software Testing and Analysis (2008)
40. Yin, H., Song, D.: Panorama: capturing system-wide information flow for malware detection and analysis. In: ACM Conference on Computer and Communications Security (CCS 2007) (2007)