Employing Program Semantics for Malware Detection

IEEE Transactions on Information Forensics and Security

Volumn 10, Issue 12, 2015, Pages 2591-2604

  Naval, Smita ^a   Laxmi, Vijay ^a   Rajarajan, Muttukrishnan ^b   Gaur, Manoj Singh ^a   Conti, Mauro ^c  

^a MALAVIYA NATIONAL INSTITUTE OF TECHNOLOGY   (India)

^b CITY UNIVERSITY   (United Kingdom)

^c UNIVERSITY OF PADOVA   (Italy)

Author keywords

Malware; Malware Detection; Semantically relevant paths; System call injection attacks; System calls

Indexed keywords

BINS; COMPUTER CRIME; FEATURE EXTRACTION; INFORMATION THEORY; INTRUSION DETECTION; NETWORK SECURITY; SEMANTICS;

ASYMPTOTIC EQUIPARTITION PROPERTY (AEP); EXECUTION SEQUENCES; EXTRACT INFORMATIONS; MALWARE DETECTION; PROPAGATION DYNAMICS; SEMANTICALLY-RELEVANT PATHS; SIGNATURE-BASED APPROACH; SYSTEM CALLS;

MALWARE;

EID: 84960933928     PISSN: 15566013     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIFS.2015.2469253     Document Type: Article

References (56)

1. A. Kapravelos, Y. Shoshitaishvili, M. Cova, C. Kruegel, G. Vigna, "Revolver: An automated approach to the detection of evasive Web-based malware," in Proc. 22nd USENIX Conf. Secur. (SEC), 2013, pp. 637-652.
2. AV-Test. (Nov. 2014). AV-Test Malware Statistics. [Online]. Available: http://www.av-test.org/en/statistics/malware/
3. W. Wang, I. Murynets, J. Bickford, C. Van Wart, G. Xu, "What you see predicts what you get-Lightweight agent-based malware detection," Secur. Commun. Netw., vol. 6, no. 1, pp. 33-48, 2013.
4. D. Spinellis, "Reliable identification of bounded-length viruses is NP-complete," IEEE Trans. Inf. Theory, vol. 49, no. 1, pp. 280-284, Jan. 2003.
5. H. Qiu and F. C. C. Osorio, "Static malware detection with segmented sandboxing," in Proc. IEEE 8th Int. Conf. Malicious Unwanted Softw. (MALWARE), Oct. 2013, pp. 132-141.
6. Z. Shan and X. Wang, "Growing grapes in your computer to defend against malware," IEEE Trans. Inf. Forensics Security, vol. 9, no. 2, pp. 196-207, Feb. 2014.
7. M. Christodorescu, S. Jha, S. A. Seshia, D. Song, R. E. Bryant, "Semantics-aware malware detection," in Proc. IEEE Symp. Secur. Privacy (SP), May 2005, pp. 32-46.
8. M. Fredrikson, S. Jha, M. Christodorescu, R. Sailer, X. Yan, "Synthesizing near-optimal malware specifications from suspicious behaviors," in Proc. 8th IEEE Symp. Secur. Privacy (SP), May 2010, pp. 45-60.
9. G. Jacob, R. Hund, C. Kruegel, T. Holz, "JACKSTRAWS: Picking command and control connections from bot traffic," in Proc. 20th USENIX Conf. Secur. (SEC), 2011, pp. 29-48.
10. H. Lu, X. Wang, B. Zhao, F. Wang, J. Su, "ENDMal: An antiobfuscation and collaborative malware detection system using syscall sequences," Math. Comput. Model., vol. 58, nos. 5-6, pp. 1140-1154, 2013.
11. Y. Park, D. S. Reeves, M. Stamp, "Deriving common malware behavior through graph clustering," Comput. Secur., vol. 39, pp. 419-430, Nov. 2013.
12. D. Mutz, F. Valeur, G. Vigna, C. Kruegel, "Anomalous system call detection," ACM Trans. Inf. Syst. Secur., vol. 9, no. 1, pp. 61-93, Feb. 2006.
13. W. Ma, P. Duan, S. Liu, G. Gu, J.-C. Liu, "Shadow attacks: Automatically evading system-call-behavior based malware detection," J. Comput. Virol., vol. 8, nos. 1-2, pp. 1-13, 2012.
14. T. Barabosch, S. Eschweiler, E. Gerhards-Padilla, "Bee master: Detecting host-based code injection attacks," in Proc. 11th Int. Conf. Detection Intrusions Malware, Vulnerability Assessment (DIMVA), vol. 8550. 2014, pp. 235-254.
15. G. S. Kc, A. D. Keromytis, V. Prevelakis, "Countering code-injection attacks with instruction-set randomization," in Proc. 10th ACM Conf. Comput. Commun. Secur. (CCS), 2003, pp. 272-280.
16. Y. Ji, Y. He, D. Zhu, Q. Li, D. Guo, "A mulitiprocess mechanism of evading behavior-based bot detection approaches," in Information Security Practice and Experience (Lecture Notes in Computer Science), vol. 8434. Springer-Verlag, 2014, pp. 75-89.
17. M. Ramilli, M. Bishop, S. Sun, "Multiprocess malware," in Proc. 6th Int. Conf. Malicious Unwanted Softw. (MALWARE), Oct. 2011, pp. 8-13.
18. T. M. Cover and J. A. Thomas, Elements of Information Theory (Wiley Series in Telecommunications and Signal Processing), 2nd ed. New York, NY, USA: Wiley, Jul. 2006.
19. C. Cui, Z. Dang, T. R. Fischer, "Typical paths of a graph," Fundam. Inf., vol. 110, nos. 1-4, pp. 95-109, Jan. 2011.
20. R. Islam, R. Tian, L. M. Batten, S. Versteeg, "Classification of malware based on integrated static and dynamic features," J. Netw. Comput. Appl., vol. 36, no. 2, pp. 646-656, 2013.
21. N. Kheir, "Behavioral classification and detection of malware through HTTP user agent anomalies," J. Inf. Secur. Appl., vol. 18, no. 1, pp. 2-13, 2013.
22. R. Moskovitch, Y. Elovici, L. Rokach, "Detection of unknown computer worms based on behavioral classification of the host," Comput. Statist. Data Anal., vol. 52, no. 9, pp. 4544-4566, 2008.
23. D. Balzarotti, M. Cova, C. Karlberger, E. Kirda, C. Kruegel, G. Vigna, "Efficient detection of split personalities in malware," in Proc. Int. Conf. Netw. Distrib. Syst. Secur. Symp. (NDSS), 2010, pp. 1-16.
24. M. Lindorfer, C. Kolbitsch, P. M. Comparetti, "Detecting environment-sensitive malware," in Proc. 14th Int. Conf. Recent Adv. Intrusion Detection (RAID), vol. 6961. 2011, pp. 338-357.
25. S. Naval, V. Laxmi, M. S. Gaur, S. Raja, M. Rajarajan, M. Conti, "Environment-reactive malware behavior: Detection and categorization," in Data Privacy Management, Autonomous Spontaneous Security, Security Assurance. New York, NY, USA: Springer-Verlag, 2015, pp. 167-182.
26. G. Suarez-Tangil, M. Conti, J. E. Tapiador, P. Peris-Lopez, "Detecting targeted smartphone malware with behavior-triggering stochastic models," in Proc. 19th Eur. Symp. Res. Comput. Secur. (ESORICS), vol. 8712. 2014, pp. 183-201.
27. C. E. Shannon and W. Weaver, The Mathematical Theory of Communication. Urbana, IL, USA: Univ. Illinois Press, 1963.
28. A. Dinaburg, P. Royal, M. Sharif, W. Lee, "Ether: Malware analysis via hardware virtualization extensions," in Proc. 15th ACM Conf. Comput. Commun. Secur. (CCS), 2008, pp. 51-62.
29. G. Pék, B. Bencsáth, L. Buttyán, "nEther: In-guest detection of out-of-the-guest malware analyzers," in Proc. 4th Eur. Workshop Syst. Secur. (EUROSEC), 2011, Art. ID 3.
30. B. Anderson, D. Quist, J. Neil, C. Storlie, T. Lane, "Graph-based malware detection using dynamic analysis," J. Comput. Virol., vol. 7, no. 4, pp. 247-258, 2011.
31. D. Babíc, D. Reynaud, D. Song, "Recognizing malicious software behaviors with tree automata inference," Formal Methods Syst. Design, vol. 41, no. 1, pp. 107-128, Aug. 2012.
32. M. J. Jurczyk. (Nov. 2014). Windows WIN32K.SYS System Call Table. [Online]. Available: http://j00ru.vexillium.org/win32k-syscalls/
33. J. R. Norris, Markov Chains. Cambridge, U.K.: Cambridge Univ. Press, 1998.
34. M. J. Quinn and N. Deo, "Parallel graph algorithms," ACM Comput. Surv., vol. 16, no. 3, pp. 319-348, Sep. 1984.
35. S. Edelkamp and R. E. Korf, "The branching factor of regular search spaces," in Proc. 15th Nat./10th Conf. Artif. Intell./Innov. Appl. Artif. Intell. (AAAI/IAAI), 1998, pp. 299-304.
36. S. Maji, A. C. Berg, J. Malik, "Classification using intersection kernel support vector machines is efficient," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2008, pp. 1-8.
37. L. Breiman, "Random forests," Mach. Learn., vol. 45, no. 1, pp. 5-32, 2001.
38. T. K. Ho, "The random subspace method for constructing decision forests," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 8, pp. 832-844, Aug. 1998.
39. P. Barham et al., "Xen and the art of virtualization," ACM SIGOPS Oper. Syst. Rev., vol. 37, no. 5, pp. 164-177, Dec. 2003.
40. M. Ahmadi, A. Sami, H. Rahimi, B. Yadegari, "Malware detection by behavioural sequential patterns," Comput. Fraud Secur., vol. 2013, no. 8, pp. 11-19, 2013.
41. VirusTotal. (Feb. 2015). File Types Statistics. [Online]. Available: https://www.virustotal.com/en/statistics/
42. K. Rieck, T. Holz, C. Willems, P. Düssel, P. Laskov, "Learning and classification of malware behavior," in Proc. 5th Int. Conf. Detection Intrusions Malware Vulnerability Assessment (DIMVA), 2008, pp. 108-125.
43. P. Vinod, V. Laxmi, M. S. Gaur, G. Chauhan, "Momentum: Metamorphic malware exploration techniques using MSA signatures," in Proc. Int. Conf. Innov. Inf. Technol. (IIT), Mar. 2012, pp. 232-237.
44. D. Quist, L. Liebrock, J. Neil, "Improving antivirus accuracy with hypervisor assisted analysis," J. Comput. Virol., vol. 7, no. 2, pp. 121-131, May 2011.
45. T. Fawcett, "An introduction to ROC analysis," Pattern Recognit. Lett., vol. 27, no. 8, pp. 861-874, 2006.
46. J. R. Ullmann, "An algorithm for subgraph isomorphism," J. ACM, vol. 23, no. 1, pp. 31-42, Jan. 1976.
47. H. Bunke, P. Foggia, C. Guidobaldi, M. Vento, "Graph clustering using the weighted minimum common supergraph," in Graph Based Representations in Pattern Recognition (Lecture Notes in Computer Science), vol. 2726. Berlin, Germany: Springer-Verlag, 2003, pp. 235-246.
48. Microsoft. (Feb. 2015). Kernel Object. [Online]. Available: https://msdn. microsoft.com/en-us/library/ms724485%28VS.85%29.aspx
49. Boost-Software. (Feb. 2015). Graph Library. [Online]. Available: http://sourceforge.net/projects/boost/files/boost/1.57.0/boost-1-57-0.tar. gz/download
50. K. Kaczmarski, P. Przymus, P. Rzazewski, "Improving high-performance GPU graph traversal with compression," in New Trends in Database and Information Systems II (Advances in Intelligent Systems and Computing), vol. 312. Berlin, Germany: Springer-Verlag, 2015, pp. 201-214.
51. A. Lanzi, D. Balzarotti, C. Kruegel, M. Christodorescu, E. Kirda, "AccessMiner: Using system-centric models for malware protection," in Proc. 17th ACM Conf. Comput. Commun. Secur. (CCS), 2010, pp. 399-412.
52. Y. Wu and R. H. C. Yap, "Experiments with malware visualization," in Proc. 9th Int. Conf. Detection Intrusions Malware, Vulnerability Assessment (DIMVA), vol. 7591. 2013, pp. 123-133.
53. P. Trinius, T. Holz, J. Gobel, F. C. Freiling, "Visual analysis of malware behavior using treemaps and thread graphs," in Proc. 6th Int. Workshop Visualizat. Cyber Secur. (VizSec), Oct. 2009, pp. 33-38.
54. D. A. Quist and L. M. Liebrock, "Visualizing compiled executables for malware analysis," in Proc. 6th Int. Workshop Visualizat. Cyber Secur. (VizSec), Oct. 2009, pp. 27-32.
55. J. Saxe, D. Mentis, C. Greamo, "Visualization of shared system call sequence relationships in large malware corpora," in Proc. 9th Int. Symp. Visualizat. Cyber Secur. (VizSec), 2012, pp. 33-40.
56. S.-T. Liu, H.-C. Huang, Y.-M. Chen, "A system call analysis method with mapreduce for malware detection," in Proc. IEEE 17th Int. Conf. Parallel Distrib. Syst. (ICPADS), Dec. 2011, pp. 631-637.