Opcode sequences as representation of executables for data-mining-based unknown malware detection

Information Sciences

Volumn 231, Issue , 2013, Pages 64-82

  Santos, Igor ^a   Brezo, Felix ^a   Ugarte Pedrero, Xabier ^a   Bringas, Pablo G ^a  

^a UNIVERSITY OF DEUSTO   (Spain)

Author keywords

Computer security; Data mining; Machine learning; Malware detection; Supervised learning

Indexed keywords

ANTI VIRUS; EMPIRICAL VALIDATION; EXECUTABLES; GLOBAL SECURITY; MALICIOUS CODES; MALWARE DETECTION; MALWARE FAMILIES; MALWARES; OP-CODE SEQUENCE; OR-NETWORKS;

COMPUTER VIRUSES; LEARNING SYSTEMS; SECURITY OF DATA; SECURITY SYSTEMS; SUPERVISED LEARNING;

DATA MINING;

EID: 84874105145     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2011.08.020     Document Type: Article

References (85)

1. S. Amari, and S. Wu Improving support vector machine classifiers by modifying kernel functions Neural Networks 12 1999 783 789
2. T. Bayes An essay towards solving a problem in the doctrine of chances Philosophical Transactions of the Royal Society 53 1763 370 418
3. J. Bergeron, M. Debbabi, M. Erhioui, B. Ktari, Static analysis of binary code to isolate malicious behaviors, in: Proceedings of the 1999 Workshop on Enabling Technologies on Infrastructure for Collaborative Enterprises, 1999, pp. 184-189.
4. D. Bilar Opcodes as predictor for malware International Journal of Electronic Security and Digital Forensics 1 2007 156 168
5. C. Bishop Pattern Recognition and Machine Learning 2006 Springer New York
6. A. Blum, and P. Langley Selection of relevant features and examples in machine learning Artificial Intelligence 97 1997 245 271
7. L. Breiman Random forests Machine Learning 45 2001 5 32
8. D. Bruschi, L. Martignoni, and M. Monga Detecting self-mutating malware using control-flow graph matching Lecture Notes in Computer Science 4064 2006 129
9. D. Cai, J. Theiler, M. Gokhale, Detecting a malicious executable without prior knowledge of its patterns, in: Proceedings of the 2005 Defense and Security Symposium. Information Assurance, and Data Network Security, vol. 5812, 2005, pp. 1-12.
10. J. Cano, F. Herrera, and M. Lozano On the combination of evolutionary algorithms and stratified strategies for training set selection in data mining Applied Soft Computing Journal 6 2006 323 332
11. E. Castillo, J.M. Gutiérrez, and A.S. Hadi Expert Systems and Probabilistic Network Models erste ed. 1996 Springer New York, NY, USA
12. O. Chapelle, B. Schölkopf, and A. Zien Semi-Supervised Learning 2006 MIT Press
13. M. Chouchane, and A. Lakhotia Using engine signature to detect metamorphic malware Proceedings of the 2006 ACM workshop on Recurring Malcode 2006 ACM New York, NY, USA 73 78
14. M. Christodorescu, Behavior-based malware detection, Ph.D. Thesis, 2007.
15. M. Christodorescu, and S. Jha Testing malware detectors ACM SIGSOFT Software Engineering Notes 29 2004 34 44
16. M. Christodorescu, S. Jha, Static analysis of executables to detect malicious patterns, in: Proceedings of the 12th USENIX Security Symposium, 2003, pp. 169-186.
17. M. Christodorescu, S. Jha, S. Seshia, D. Song, R. Bryant, Semantics-aware malware detection, in: Proceedings of the 2005 IEEE Symposium on Security and Privacy, 2005, pp. 32-46.
18. G.F. Cooper, E. Herskovits, A bayesian method for constructing bayesian belief networks from databases, in: Proceedings of the 1991 Conference on Uncertainty in Artificial Intelligence, 1991.
19. I. Czarnowski, P. Jedrzejowicz, Instance reduction approach to machine learning and multi-database mining, in: Proceedings of the 2006 Scientific Session organized during XXI Fall Meeting of the Polish Information Processing Society, Informatica, ANNALES Universitatis Mariae Curie-Skłodowska, Lublin, 2006, pp. 60-71.
20. M. Dash, and H. Liu Consistency-based search in feature selection Artificial Intelligence 151 2003 155 176
21. J. Derrac, S. Garcia, and F. Herrera A first study on the use of coevolutionary algorithms for instance and feature selection Proceedings of the 2009 International Conference on Hybrid Artificial Intelligence Systems 2009 Springer 557 564
22. C. Elkan, The foundations of cost-sensitive learning, in: Proceedings of the 2001 International Joint Conference on Artificial Intelligence, 2001, pp. 973-978.
23. D. Fisch, A. Hofmann, and B. Sick On the versatility of radial basis function neural networks: a case study in the field of intrusion detection Information Sciences 180 2010 2421 2439
24. E. Fix, J.L. Hodges, Discriminatory analysis: nonparametric discrimination: small sample performance, Technical Report Project 21-49-004, Report Number 11, 1952.
25. S. Garner, Weka: the Waikato environment for knowledge analysis, in: Proceedings of the 1995 New Zealand Computer Science Research Students Conference, 1995, pp. 57-64.
26. D. Geiger, M. Goldszmidt, G. Provan, P. Langley, P. Smyth, Bayesian network classifiers, in: Machine Learning 29 (1997) 131-163.
27. J. Huang, and I. Liao Shielding wireless sensor network using markovian intrusion detection system with attack pattern mining Information Sciences 2011
28. N. Idika, A. Mathur, A survey of malware detection techniques, Technical Report, Department of Computer Science, Purdue University, 2007.
29. G. Jacob, H. Debar, and E. Filiol Behavioral detection of malware: from a survey towards an established taxonomy Journal in Computer Virology 4 2008 251 266
30. X. Jin, A. Xu, R. Bie, and P. Guo Machine learning techniques and chi-square feature selection for cancer classification using SAGE gene expression profiles Lecture Notes in Computer Science 3916 2006 106 115
31. I. Jolliffe Principal Component Analysis 2002 Springer Verlag
32. M. Kang, P. Poosankam, H. Yin, Renovo: a hidden code extractor for packed executables, in: Proceedings of the 2007 ACM workshop on Recurring Malcode, 2007, pp. 46-53.
33. M. Karim, A. Walenstein, A. Lakhotia, and L. Parida Malware phylogeny generation using permutations of code Journal in Computer Virology 1 2005 13 23
34. J. Kent Information gain and a general measure of correlation Biometrika 70 1983 163
35. R. Kohavi, A study of cross-validation and bootstrap for accuracy estimation and model selection, in: Proceedings of the 1995 International Joint Conference on Artificial Intelligence, vol. 14, 1995, pp. 1137-1145.
36. J. Kolter, and M. Maloof Learning to detect malicious executables in the wild Proceedings of the 2004 ACM SIGKDD International Conference on Knowledge discovery and Data Mining 2004 ACM New York, NY, USA 470 478
37. S. Kotsiantis, Supervised machine learning: a review of classification techniques, in: Proceeding of the 2007 Conference on Emerging Artificial Intelligence Applications in Computer Engineering: Real Word AI Systems with Applications in eHealth, HCI, Information Retrieval and Pervasive Technologies, 2007, pp. 3-24.
38. S. Kotsiantis, and P. Pintelas Recent advances in clustering: a brief survey WSEAS Transactions on Information Science and Applications 1 2004 73 81
39. G. Kou, Y. Peng, Z. Chen, and Y. Shi Multiple criteria mathematical programming for multi-class classification and application in network intrusion detection Information Sciences 179 2009 371 381
40. N. Kuzurin, A. Shokurov, N. Varnovsky, and V. Zakharov On the concept of software obfuscation in computer security Lecture Notes in Computer Science 4779 2007 281
41. D. Lewis Naive (Bayes) at forty: the independence assumption in information retrieval Lecture Notes in Computer Science 1398 1998 4 18
42. W. Li, K. Wang, S. Stolfo, B. Herzog, Fileprints: identifying file types by n-gram analysis, in: Proceedings of the 2005 IEEE Workshop on Information Assurance and Security, 2005.
43. H. Liu, and H. Motoda Instance Selection and Construction for Data Mining 2001 Kluwer Academic Publication
44. H. Liu, and H. Motoda Computational Methods of Feature Selection 2008 Chapman and Hall/CRC
45. R. Lo, K. Levitt, and R. Olsson MCF: a malicious code filter Computers and Security 14 1995 541 566
46. R. Mántaras A distance-based attribute selection measure for decision tree induction Machine Learning 6 1991 81 92
47. L. Martignoni, M. Christodorescu, S. Jha, Omniunpack: fast, generic, and safe unpacking of malware, in: Proceedings of the 2007 Annual Computer Security Applications Conference (ACSAC), 2007, pp. 431-441.
48. M. McGill, and G. Salton Introduction to Modern Information Retrieval 1983 McGraw-Hill
49. P. Morley, Processing virus collections, in: Proceedings of the 2001 Virus Bulletin Conference (VB2001), Virus Bulletin, 2001, pp. 129-134.
50. R. Moskovitch, D. Stopel, C. Feher, N. Nissim, Y. Elovici, Unknown malcode detection via text categorization and the imbalance problem, in: Proceedings of the 2008 IEEE International Conference on Intelligence and Security Informatics (ISI), 2008, pp. 156-161.
51. G. Ollmann The evolution of commercial malware development kits and colour-by-numbers custom malware Computer Fraud and Security 2008 2008 4 7
52. J. Pearl, Reverend bayes on inference engines: a distributed hierarchical approach, in: Proceedings of the 1982 National Conference on Artificial Intelligence, 1982, pp. 133-136.
53. H. Peng, F. Long, and C. Ding Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy IEEE Transactions on Pattern Analysis and Machine Intelligence 2005 1226 1238
54. R. Perdisci, A. Lanzi, and W. Lee Classification of packed executables for accurate computer virus detection Pattern Recognition Letters 29 2008 1941 1946
55. R. Perdisci, A. Lanzi, W. Lee, McBoost: boosting scalability in malware collection and analysis using statistical classification of executables, in: Proceedings of the 23rd Annual Computer Security Applications Conference, 2008, pp. 301-310.
56. F. Perriot, P. Ferrie, Principles and practise of x-raying, in: Proceedings of the 2004 Virus Bulletin International Conference, 2004, pp. 51-66.
57. J. Platt Sequential minimal optimization: a fast algorithm for training support vector machines Advances in Kernel Methods-Support Vector Learning 208 1999
58. B. Potter, and G. Day The effectiveness of anti-malware tools Computer Fraud and Security 2009 2009 12 13
59. D. Pyle Data Preparation for Data Mining 1999 Morgan Kaufmann
60. J. Quinlan Induction of decision trees Machine Learning 1 1986 81 106
61. J. Quinlan C4. 5 Programs for Machine Learning 1993 Morgan Kaufmann Publishers
62. S. Robertson Understanding inverse document frequency: on theoretical arguments for IDF Journal of Documentation 60 2004 503 520
63. P. Royal, M. Halpin, D. Dagon, R. Edmonds, W. Lee, Polyunpack: automating the hidden-code extraction of unpack-executing malware, in: Proceedings of the 2006 Annual Computer Security Applications Conference (ACSAC), 2006, pp. 289-300.
64. S.J. Russell Norvig second ed. Artificial Intelligence: A Modern Approach 2003 Prentice Hall
65. G. Salton, A. Wong, and C. Yang A vector space model for automatic indexing Communications of the ACM 18 1975 613 620
66. I. Santos, F. Brezo, J. Nieves, Y. Penya, B. Sanz, C. Laorden, and P. Bringas Idea: opcode-sequence-based malware detection Engineering Secure Software and Systems LNCS 5965 2010 35 43 10.1007/978-3-642-11747-3-3
67. I. Santos, Y. Penya, J. Devesa, P. Bringas, N-Grams-based file signatures for malware detection, in: Proceedings of the 2009 International Conference on Enterprise Information Systems (ICEIS), vol. AIDSS, 2009, pp. 317-320.
68. R. Schapire The boosting approach to machine learning: an overview Lecture Notes in Statistics 2003 149 172
69. M. Schultz, E. Eskin, F. Zadok, S. Stolfo, Data mining methods for detection of new malicious executables, in: Proceedings of the 2001 IEEE Symposium on Security and Privacy, 2001, pp. 38-49.
70. A. Shabtai, R. Moskovitch, Y. Elovici, and C. Glezer Detection of malicious code by applying machine learning classifiers on static features: a state-of-the-art survey Information Security Technical Report 14 2009 16 29
71. M. Shafiq, S. Khayam, and M. Farooq Embedded malware detection using markov n-grams Lecture Notes in Computer Science 5137 2008 88 107
72. M. Sharif, V. Yegneswaran, H. Saidi, P. Porras, W. Lee, Eureka: a framework for enabling static malware analysis, in: Proceedings of the 2008 European Symposium on Research in Computer Security (ESORICS), 2008, pp. 481-500.
73. Y. Singh, A. Kaur, and R. Malhotra Comparative analysis of regression and machine learning methods for predicting fault proneness models International Journal of Computer Applications in Technology 35 2009 183 193
74. A. Sung, J. Xu, P. Chavez, S. Mukkamala, Static analyzer of vicious executables (save), in: Proceedings of the 2004 Annual Computer Security Applications Conference (ACSAC), 2004, pp. 326-334.
75. P. Ször The Art of Computer Virus Research and Defense 2005 Addison-Wesley Professional
76. K. Torkkola Feature extraction by non parametric mutual information maximization The Journal of Machine Learning Research 3 2003 1415 1438
77. C. Tsai, Y. Hsu, C. Lin, and W. Lin Intrusion detection by machine learning: a review Expert Systems with Applications 36 2009 11994 12000
78. E. Tsang, D. Yeung, and X. Wang OFFSS: optimal fuzzy-valued feature subset selection IEEE Transactions on Fuzzy Systems 11 2003 202 213
79. B. Üstün, W. Melssen, and L. Buydens Facilitating the application of support vector regression by using a universal Pearson VII function based kernel Chemometrics and Intelligent Laboratory Systems 81 2006 29 40
80. V. Vapnik The Nature of Statistical Learning Theory 2000 Springer
81. P. Vinod, R. Jaipur, V. Laxmi, M. Gaur, Survey on malware detection methods, in: Hack, 2009.
82. J. Xu, A. Sung, P. Chavez, S. Mukkamala, Polymorphic malicious executable scanner by API sequence analysis, in: Proceedings of the 2004 International Conference on Hybrid Intelligent Systems, 2004, pp. 378-383.
83. C. Zhai, and J. Lafferty A study of smoothing methods for language models applied to information retrieval ACM Transactions on Information Systems 22 2004 179 214
84. Q. Zhang, D. Reeves, Metaaware: identifying metamorphic malware, in: Proceedings of the 2007 Annual Computer Security Applications Conference (ACSAC), 2007, pp. 411-420.
85. Y. Zhou, and W. Inge Malware detection using adaptive data compression Proceedings of the 2008 ACM Workshop on AISec 2008 ACM New York, NY, USA 53 60