BitShred: Feature hashing malware for scalable triage and semantic analysis

Proceedings of the ACM Conference on Computer and Communications Security

Volumn , Issue , 2011, Pages 309-320

  Jang, Jiyong ^a   Brumley, David ^a   Venkataraman, Shobha ^b  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b AT AND T LABS RESEARCH   (United States)

Author keywords

Co clustering; Feature hashing; Hadoop; Malware triage

Indexed keywords

CO-CLUSTERING; FEATURE HASHING; HADOOP; HIGH-DIMENSIONAL FEATURE SPACE; MALWARE ANALYSIS; MALWARES; PARALLELIZED VERSION; SEMANTIC ANALYSIS; SIMILARITY ANALYSIS; SINGLE CPU;

SECURITY OF DATA; SEMANTICS;

COMPUTER CRIME;

EID: 80755168347     PISSN: 15437221     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2046707.2046742     Document Type: Conference Paper

References (39)

1. Apache hadoop. http://hadoop.apache.org/.
2. CMU Cloud Computer Cluster. http://www2.pdl.cmu.edu/~twiki/cgi-bin/view/ OpenCloud/ClusterOverview.
3. Malware Analysis System. http://mwanalysis.org/.
4. Offensive Computing. http://www.offensivecomputing.net/.
5. SimMetrics. http://sourceforge.net/projects/simmetrics/.
6. VirusTotal. http://www.virustotal.com/.
7. zynamics bindiff. http://www.zynamics.com/bindiff.html.
8. Symantec internet security threat report. http://www.symantec.com/ business/theme.jsp?themeid=threatreport, April 2010.
9. T. Abou-Assaleh, N. Cercone, V. Keselj, and R. Sweidan. N-gram-based detection of new malicious code. In COMPSAC '04: Proceedings of the 28th Annual International Computer Software and Applications Conference - Workshops and Fast Abstracts, 2004.
10. A. Andoni and P. Indyk. Near-optimal hashing algorithms for approximate nearest neighbor in high dimensions. Communications of the ACM, 51(1):177-122, 2008.
11. J. Attenberg, K. Weinberger, A. Dasgupta, A. Smola, and M. Zinkevich. Collaborative email-spam filtering with the hashing-trick. In Proceedings of the Sixth Conference on Email and Anti-Spam, 2009.
12. M. Bailey, J. Oberheide, J. Andersen, F. J. Z. Morley Mao, and J. Nazario. Automated classification and analysis of internet malware. In Proceedings of the Symposium on Recent Advances in Intrusion Detection, September 2007.
13. U. Bayer, P. M. Comparetti, C. Hlauschek, C. Kruegel, and E. Kirda. Scalable, behavior-based malware clustering. In Proceedings of the Network and Distributed System Security Symposium, 2009.
14. D. Bernstein. http://www.cse.yorku.ca/~oz/hash.html.
15. A. Broder and M. Mitzenmacher. Network applications of bloom filters: A survey. Internet Mathematics, 1(4):485-509, 2005.
16. D. Chakrabarti, S. Papadimitriou, D. Modha, and C. Faloutsous. Fully automatic cross associations. In Proceedings of ACM SIGKDD, August 2004.
17. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. In Proceedings of the USENIX Symposium on Operating System Design and Implementation, 2004.
18. A. Dinaburg, P. Royal, M. Sharif, and W. Lee. Ether: malware analysis via hardware virtualization extensions. In ACM CCS, 2008.
19. D. Eppstein. Fast hierarchical clustering and other applications of dynamic closest pairs. In Proceedings of ACM Symposium on Discrete Algorithms (SODA), 1998.
20. F. Guo, P. Ferrie, and T.-C. Chiueh. A study of the packer problem and its solutions. In Proceedings of the International Symposium on Recent Advances in Intrusion Detection, pages 98-115, 2008.
21. X. Hu, T. cker Chiueh, and K. G. Shin. Large-scale malware indexing using function call graphs. In Proceedings of the ACM Conference on Computer and Communications Security, 2009.
22. N. Jain, M. Dahlin, and R. Tewari. Using bloom filters to refine web search results. In Proceedings of Eighth International Workshop on the Web and Databases (WebDB 2005), June 2005.
23. M. Karim, A. Walenstein, A. Lakhotia, and L. Parida. Malware phylogeny generation using permutations of code. Journal in Computer Virology, 1(1):13-23, November 2005. (Pubitemid 44195063)
24. G. Karypis. CLUTO: a clustering toolkit, release 2.1.1. Technical report, University of Minnesota, 2003.
25. J. Z. Kolter and M. A. Maloof. Learning to detect and classify malicious executables in the wild. Journal of Machine Learning Research, 7:2721-2744, Dec. 2006. (Pubitemid 46011490)
26. P. Li, L. Lu, D. Gao, and M. Reiter. On challenges in evaluating malware clustering. In Proceedings of the International Symposium on Recent Advances in Intrusion Detection, 2010.
27. L. Martignoni, M. Christodorescu, and S. Jha. OmniUnpack: Fast, generic, and safe unpacking of malware. In In Proceedings of the Annual Computer Security Applications Conference, 2007.
28. A. Moser, C. Kruegel, and E. Kirda. Exploring multiple execution paths for malware analysis. In Proceedings of the IEEE Symposium on Security and Privacy, 2007.
29. A. Moser, C. Kruegel, and E. Kirda. Limits of static analysis for malware detection. In Proceedings of the Annual Computer Security Applications Conference, 2007.
30. S. Papadimitrou and J. Sun. Disco: Distributed co-clustering with map-reduce. In Proceedings of ICDM, 2008.
31. R. Perdisci, A. Lanzi, and W. Lee. Classification of packed executables for accurate computer virus detection. Pattern Recogn. Lett., 29(14):1941-1946, 2008.
32. R. Perdisci, W. Lee, and N. Feamster. Behavioral clustering of HTTP-based malware and signature generation using malicious network traces. In Proceedings of NSDI, 2010.
33. P. Royal, M. Halpin, D. Dagon, R. Edmonds, and W. Lee. PolyUnpack: Automating the hidden-code extraction of unpack-executing malware. In Proceedings of Computer Security Applications Conference, December 2006.
34. S. Schleimer, D. Wilkerson, and A. Aiken. Winnowing: Local algorithms for document fingerprinting. In Proceedings of the ACM SIGMOD/PODS Conference, 2003.
35. M. Sharif, A. Lanzi, J. Giffin, and W. Lee. Automatic reverse engineering of malware emulators. In Proceedings of the IEEE Symposium on Security and Privacy, 2009.
36. Q. Shi, J. Petterson, G. Dror, J. Langford, A. Smola, and S. Vishwanathan. Hash kernels for structured data. Journal of Machine Learning Research, 2009.
37. th International Conference on Artificial Intelligence and Statisics (AISTATS), 2009.
38. A. Walenstein and A. Lakhotia. The software similarity problem in malware analysis. In Duplication, Redundancy, and Similarity in Software, 2007.
39. K. Weinberger, A. Dasgupta, J. Langford, A. Smola, and J. Attenberg. Feature hashing for large-scale multitask learning. In Proceedings of ICML, 2009.