A survey of network anomaly detection techniques

Journal of Network and Computer Applications

Volumn 60, Issue , 2016, Pages 19-31

  Ahmed, Mohiuddin ^a   Naser Mahmood, Abdun ^a   Hu, Jiankun ^a  

^a UNIVERSITY OF NEW SOUTH WALES   (Australia)

Author keywords

Anomaly detection; Classification; Clustering; Computer security; Information theory; Intrusion detection

Indexed keywords

CLASSIFICATION (OF INFORMATION); ECONOMIC AND SOCIAL EFFECTS; ECONOMICS; INFORMATION THEORY; MERCURY (METAL); NATIONAL SECURITY; SECURITY OF DATA; SIGNAL DETECTION;

ANOMALY DETECTION; CLUSTERING; INFORMATION AND COMMUNICATION TECHNOLOGIES; MOBILE COMMUNICATION DEVICES; NETWORK ANOMALY DETECTION; NETWORK INTRUSION DETECTION; NETWORK INTRUSIONS; RESEARCH CHALLENGES;

INTRUSION DETECTION;

EID: 84950256593     PISSN: 10848045     EISSN: 10958592     Source Type: Journal    
DOI: 10.1016/j.jnca.2015.11.016     Document Type: Review

References (100)

1. Adfa intrusion detection datasets, accessed: 2014-12-29. URL (http://seit.unsw.adfa.edu.au/staff/sites/hu/).
2. Ahmed M, Mahmood A. Clustering based semantic data summarization technique: a new approach. In: 2014 IEEE 9th conference on industrial electronics and applications (ICIEA), 2014, p. 1780-5.
3. Ahmed M, Mahmood A. Network traffic analysis based on collective anomaly detection. In: 2014 IEEE 9th conference on industrial electronics and applications (ICIEA), 2014. p. 1141-46.
4. Ahmed M, Mahmood AN. Network traffic pattern analysis using improved information-theoretic co-clustering based collective anomaly detection. In: Security and privacy in communication networks, Lecture notes of the institute for computer sciences, social informatics and telecommunications engineering, vol. 153. Springer International Publishing, 2014. p. 1-16.
5. M. Ahmed, and A. Mahmood Novel approach for network traffic pattern analysis using clustering-based collective anomaly detection Ann. Data Sci. 2 1 2015 111 130
6. Ahmed M, Naser A. A novel approach for outlier detection and clustering improvement. In: 2013 8th IEEE conference on industrial electronics and applications (ICIEA), 2013, p. 577-82.
7. Ahmed M, Mahmood AN, Hu J. Outlier Detection, CRC Press, New York, USA, 2014. p. 3-21, Chapter 1 (in book: The State of the Art in Intrusion Prevention and Detection).
8. M. Ahmed, A.N. Mahmood, and M.R. Islam A survey of anomaly detection techniques in financial domain Future Gener Comp Syst 55 2016 278 288
9. M. Ahmed, A. Anwar, A.N. Mahmood, Z. Shah, and M.J. Maher An investigation of performance analysis of anomaly detection techniques for big data in scada systems EAI Endorsed Trans Ind Netw Intell Syst 15 3 2015 1 16
10. M. Ahmed, A.N. Mahmood, and M.J. Maher An efficient technique for network traffic summarization using multiview clustering and statistical sampling EAI Endorsed Trans Scalable Inf Syst 15 5 2015 1 9
11. M. Ahmed, A. Mahmood, and M. Maher Heart disease diagnosis using co-clustering J.J. Jung, C. Badica, A. Kiss, Scalable information systems, Lecture notes of the institute for computer sciences, Social informatics and telecommunications engineering vol. 139 2015 Springer International Publishing 61 70
12. M. Ahmed, A. Mahmood, and M. Maher A novel approach for network traffic summarization J.J. Jung, C. Badica, A. Kiss, Scalable information systems, Lecture notes of the institute for computer sciences, Social informatics and telecommunications engineering vol. 139 2015 Springer International Publishing 51 60
13. M. Ahmed, A. Mahmood, and M. Maher An efficient approach for complex data summarization using multiview clustering J.J. Jung, C. Badica, A. Kiss, Scalable information systems, lecture notes of the institute for computer sciences, Social informatics and telecommunications engineering vol. 139 2015 Springer International Publishing 38 47
14. M.A. Ambusaidi, Z. Tan, X. He, P. Nanda, L.F. Lu, and A. Jamdagni Intrusion detection method based on nonlinear correlation measure Int J Internet Protoc Technol 8 2/3 2014 77 86
15. Axelsson S. Technical report: Research in intrusion-detection systems: A survey, no. 98-17, SE-412 96, Göteborg, Sweden, 1998.
16. Balabine I, Velednitsky A. Method and system for confident anomaly detection in computer network traffic. Google Patents, 2015.
17. A. Banerjee, I. Dhillon, J. Ghosh, S. Merugu, and D.S. Modha A generalized maximum entropy approach to bregman co-clustering and matrix approximation J Mach Learn Res 8 2007 1919 1986
18. R.J. Beckman, and R.D. Cook Outlier.s Technometrics 25 2 1983 119 149
19. Caida, accessed: 2014-12-29. URL (www.caida.org).
20. V. Chandola, A. Banerjee, and V. Kumar Anomaly detection: a survey ACM Comput Surv 41 3 2009 15:1 15:58
21. Creech G, Hu J. Generation of a new ids test dataset: time to retire the kdd collection, In: Wireless communications and networking conference (WCNC), 2013 IEEE, 2013. p. 4487-92.
22. N. Cristianini, and J. Shawe-Taylor An introduction to support vector machines: and other kernel-based learning methods 2000 Cambridge University Press New York, NY, USA
23. H. Debar, M. Dacier, and A. Wespi A revised taxonomy for intrusion-detection systems Ann Des Télécommun 55 7-8 2000 361 378
24. Defcon, accessed: 2014-12-29. URL (www.defcon.org).
25. Željko Deljac, M. Randic, and G. Krcelic Early detection of network element outages based on customer trouble calls Decis Support Syst 73 2015 57 73
26. Eskin E. Anomaly detection over noisy data using learned probability distributions. In: Proceedings of the seventeenth international conference on machine learning, ICML '00, San Francisco, CA, USA, Morgan Kaufmann Publishers Inc., 2000. p. 255-62.
27. Eskin E, Arnold A, Prerau M, Portnoy L, Stolfo S. A geometric framework for unsupervised anomaly detection. in: Barbará D, Jajodia S (editors). Applications of data mining in computer security, Adv Inf Secur, vol. 6. Springer US, 2002. p. 77-101.
28. Ester M, Kriegel H-P, Sander J, Xu X. A density-based algorithm for discovering clusters in large spatial databases with noise. In: KDD'96, 1996. p. 226-31.
29. J.M. Estevez-Tapiador, P. Garcia-Teodoro, and J.E. Diaz-Verdejo Anomaly detection methods in wired networks: a survey and taxonomy Comput Commun 27 16 2004 1569 1584
30. S. Furnell, C. Tucker, S. Furnell, B. Ghita, and P. Brooke A new taxonomy for comparing intrusion detection systems Internet Res 17 1 2007 88 98
31. P. Gogoi, D. Bhattacharyya, B. Borah, and J.K. Kalita A survey of outlier detection methods in network anomaly identification Comput J 54 4 2011 570 588
32. G. Govaert, and M. Nadif Block clustering with Bernoulli mixture models: comparison of different approaches Comput Stat Data Anal 52 6 2008 3233 3245
33. Guan Y, Ghorbani A, Belacel N. Y-means: a clustering method for intrusion detection. In: IEEE CCECE 2003 Canadian conference on electrical and computer engineering, 2003, vol. 2; 2003. p. 1083-6.
34. Hacking and cracking tools, accessed: 2014-12-29. URL (http://hackingncrackingtools.blogspot.com.au/).
35. D. Hawkins Identification of outliers (monographs on statistics and applied probability) 1st ed. 1980 Springer Netherlands
36. S. Hawkins, H. He, G. Williams, and R. Baxter Outlier detection using replicator neural networks Y. Kambayashi, W. Winiwarter, M. Arikawa, Data warehousing and knowledge discovery, lecture notes in computer science vol. 2454 2002 Springer Berlin, Heidelberg 170 180
37. Heller KA, Svore KM, Keromytis AD, Stolfo SJ. One class support vector machines for detecting anomalous windows registry accesses. In: Proceedings of the workshop on data mining for computer security, 2003.
38. V. Hodge, and J. Austin A survey of outlier detection methodologies Artif Intell Rev 22 2 2004 85 126
39. D. Hoplaros, Z. Tari, and I. Khalil Data summarization for network traffic monitoring J Netw Comput Appl 37 0 2014 194 205
40. Hu W, Liao Y, Vemuri VR. Robust anomaly detection using support vector machines. In: Proceedings of the international conference on machine learning; 2003.
41. ICS Attack Dataset, 2014, accessed: 2015-02-27. URL (http://www.ece.msstate.edu/wiki/).
42. Identifying outliers via clustering for anomaly detection, accessed: 2014-12-29. URL (https://repository.lib.fit.edu/handle/11141/126).
43. Internet traffic archive, accessed: 2014-12-29. URL (http://ita.ee.lbl.gov/).
44. A.K. Jain, M.N. Murty, and P.J. Flynn Data clustering: a review ACM Comput Surv 31 3 1999 264 323
45. Kendall K. A database of computer attacks for the evaluation of intrusion detection systems. In: Proceedings of DARPA information survivability conference and exposition (DISCEX); 1999. p. 12-26.
46. Kruegel C, Mutz D, Robertson W, Valeur F. Bayesian event classification for intrusion detection. In: Proceedings of 19th annual computer security applications conference; 2003. p. 14-23.
47. Krügel C, Toth T, Kirda E. Service specific anomaly detection for network intrusion detection. In: Proceedings of the 2002 ACM symposium on applied computing, SAC '02, ACM, New York, NY, USA; 2002. p. 201-8.
48. Kyoto Dataset, accessed: 2014-12-29. URL (www.takakura.com).
49. Lee W, Xiang D. Information-theoretic measures for anomaly detection. In: Proceedings of 2001 IEEE symposium on security and privacy, 2001 S P 2001; 2001. p. 130-43.
50. Lee W, Stolfo S, Mok K. A data mining framework for building intrusion detection models. In: Proceedings of the 1999 IEEE Symposium on Security and Privacy; 1999. p. 120-32.
51. Leung K, Leckie C. Unsupervised anomaly detection in network intrusion detection using clusters. In: Proceedings of the twenty-eighth Australasian conference on computer science - vol. 38, ACSC '05, Australian Computer Society, Inc., Darlinghurst, Australia, Australia; 2005. p. 333-42.
52. H.-J. Liao, C.-H.R. Lin, Y.-C. Lin, and K.-Y. Tung Intrusion detection system: a comprehensive review J Netw Comput Appl 36 1 2013 16 24
53. Lin J, Keogh E, Fu A, Van Herle H. Approximations to magic: finding unusual medical time series. In: Proceedings of the 18th IEEE symposium on computer-based medical systems, CBMS '05, IEEE Computer Society, Washington, DC, USA; 2005. p. 329-334.
54. A. Mahmood, C. Leckie, and P. Udaya An efficient clustering scheme to exploit hierarchical data in network traffic analysis IEEE Trans Knowl Data Eng 20 6 2008 752 767
55. A.N. Mahmood, J. Hu, Z. Tari, and C. Leckie Critical infrastructure protection: resource efficient sampling to improve detection of less frequent patterns in network traffic J Netw Comput Appl 33 4 2010 491 502
56. M. Mahoney, and P. Chan An analysis of the 1999 darpa/lincoln laboratory evaluation data for network anomaly detection G. Vigna, C. Kruegel, E. Jonsson, Recent advances in intrusion detection, Lecture notes in computer science vol. 2820 2003 Springer Berlin, Heidelberg 220 237
57. M. Markou, and S. Singh Novelty detection: a review; part 2: neural network based approaches Signal Process 83 12 2003 2499 2521
58. Münz G, Li S, Carle G. Traffic anomaly detection using kmeans clustering. In: In GI/ITG Workshop MMBnet, 2007.
59. Noble CC, Cook DJ. Graph-based anomaly detection. In: Proceedings of the ninth ACM SIGKDD international conference on knowledge discovery and data mining, KDD '03, ACM, New York, NY, USA; 2003. p. 631-6.
60. NSL-KDD, accessed: 2014-12-29. URL (http://nsl.cs.unb.ca/NSL-KDD/).
61. J. Oldmeadow, S. Ravinutala, and C. Leckie Adaptive clustering for network intrusion detection H. Dai, R. Srikant, C. Zhang, Advances in knowledge discovery and data mining, Lecture notes in computer science vol. 3056 2004 Springer Berlin, Heidelberg 255 259
62. C.M. Papadimitriou Computational complexity 1994 Addison-Wesley Reading, MA
63. Papalexakis EE, Beutel A, Steenkiste P. Network anomaly detection using co-clustering. In: Proceedings of the 2012 international conference on advances in social networks analysis and mining (ASONAM 2012), ASONAM '12, IEEE Computer Society, Washington, DC, USA; 2012. p. 403-10.
64. A. Patcha, and J.-M. Park An overview of anomaly detection techniques: existing solutions and latest technological trends Comput Netw 51 12 2007 3448 3470
65. Pelleg D, Moore AW. X-means: extending k-means with efficient estimation of the number of clusters. In: Proceedings of the seventeenth international conference on machine learning, ICML '00, San Francisco, CA, USA; Morgan Kaufmann Publishers Inc., 2000. p. 727-34.
66. Petrovic S, Alvarez G, Orfila A, Carbo J. Labelling clusters in an intrusion detection system using a combination of clustering evaluation techniques. In: Proceedings of the 39th annual Hawaii international conference on System Sciences, 2006. HICSS '06, vol. 6; 2006. p. 129b-129b.
67. PHP: Hypertext processor, accessed: 2014-12-29. URL (http://www.php.net).
68. Phua C, Lee VCS, Smith-Miles K, Gayler RW. A comprehensive survey of data mining-based fraud detection research, CoRR abs/1009.6119. URL arxiv.org/abs/1009.6119.
69. Platt JC. Advances in kernel methods. In: Fast training of support vector machines using sequential minimal optimization. MIT Press, Cambridge, MA, USA; 1999. p. 185-208.
70. Poojitha G, Kumar K, Reddy P. Intrusion detection using artificial neural network, In: 2010 International conference on computing communication and networking technologies (ICCCNT); 2010. p. 1-7.
71. Portnoy L, Eskin E, Stolfo S. Intrusion detection with unlabeled data using clustering. In: Proceedings of ACM CSS workshop on data mining applied to security (DMSA); 2001.
72. Predict, accessed: 2014-12-29. URL (www.predict.org).
73. T. Qin, X. Guan, W. Li, P. Wang, and Q. Huang Monitoring abnormal network traffic based on blind source separation approach J. Netw. Comput. Appl. 34 5 2011 1732 1742 Dependable multimedia communications: systems, services, and applications
74. M. Ramadas, S. Ostermann, and B. Tjaden Detecting anomalous network traffic with self-organizing maps G. Vigna, C. Kruegel, E. Jonsson, Recent advances in intrusion detection, Lecture notes in computer science vol. 2820 2003 Springer Berlin, Heidelberg 36 54
75. Rubner Y, Tomasi C, Guibas L. A metric for distributions with applications to image databases. In: Sixth international conference on computer vision, 1998; 1998. p. 59-66.
76. K. Shafi, and H. Abbass Evaluation of an adaptive genetic-based signature extraction system for network intrusion detection Pattern Anal Appl 16 4 2013 549 566
77. A. Shiravi, H. Shiravi, M. Tavallaee, and A.A. Ghorbani Toward developing a systematic approach to generate benchmark datasets for intrusion detection Comput Secur 31 3 2012 357 374
78. Shyu M-L, Chen S-C, Sarinnapakorn K, Chang L. A novel anomaly detection scheme based on principal component classifier. In: IEEE foundations and new directions of data mining workshop, in conjunction with ICDM'03, 2003. p. 171-9.
79. M.-Y. Su Using clustering to improve the knn-based classifiers for online anomaly network traffic identification J Netw Comput Appl 34 2 2011 722 730 (efficient and robust security and services of wireless mesh networks)
80. I. Syarif, A. Prugel-Bennett, and G. Wills Unsupervised clustering approach for network anomaly detection R. Benlamri, Networked digital technologies communications in computer and information science vol. 293 2012 Springer Berlin Heidelberg 135 145
81. Symantec internet security threat report, accessed: 2014-12-29. URL (http://www.symantec.com/).
82. Z. Tan, A. Jamdagni, X. He, P. Nanda, and R.P. Liu A system for denial-of-service attack detection based on multivariate correlation analysis IEEE Trans Parallel Distrib Syst 25 2 2014 447 456
83. Tan Z, Nagar UT, He X, Nanda P, Liu RP, Wang S, Hu J. Enhancing big data security with collaborative intrusion detection. IEEE Cloud Comput 2014:27-33.
84. Testing intrusion detection systems: a critique of the 1998 and 1999 darpa intrusion detection system evaluations as performed by lincoln laboratory. ACM Trans Inf Syst Secur 2000:3(4):262-94.
85. Thc-hydra, accessed: 2014-12-29. URL (http://www.thc.org/thc-hydra/).
86. The apache software foundation, accessed: 2014-12-29. URL (http://apache.org).
87. The Global State of Information Security Survey 2015, accessed: 2015-01-19. URL (http://www.pwc.com).
88. M. Thottan, and C. Ji Anomaly detection in ip networks IEEE Trans Signal Process 51 8 2003 2191 2204
89. Tikiwiki cms groupware remote php code injection, accessed: 2014-12-29. URL (http://www.exploit-db.com/exploits/18265/).
90. Tikiwiki: Cms groupware, accessed: 2014-12-29. URL (http://info.tiki.org/Tiki+Wiki+CMS+Groupware).
91. Towards a taxonomy of intrusion-detection systems, Comput. Netw. 31 (9) (1999) 805-822.
92. Ubuntu Linux, accessed: 2014-12-29. URL (http://www.ubuntu.com).
93. E. Vasilomanolakis, S. Karuppayah, M. Mühlhäuser, and M. Fischer Taxonomy and survey of collaborative intrusion detection ACM Comput Surv 47 4 2015 551 5533
94. Verizon's data breach investigation report 2014, accessed: 2014-12-29. URL (http://www.verizonenterprise.com/DBIR/2014/).
95. Q. Wang, Y. Shen, and J.Q. Zhang A nonlinear correlation measure for multivariable data set Phys D: Nonlinear Phenom 200 3-4 2005 287 295
96. M. Xie, S. Han, B. Tian, and S. Parvin Anomaly detection in wireless sensor networks: a survey J Netw Comput Appl 34 4 2011 1302 1325 (advanced topics in cloud computing)
97. Yang Y, McLaughlin K, Littler T, Sezer S, Wang H. Rule-based intrusion detection system for scada networks. In: Renewable power generation conference (RPG 2013), 2nd IET; 2013. p. 1-4.
98. N. Ye, and Q. Chen An anomaly detection technique based on a chi-square statistic for detecting intrusions into information systems Qual Relaib Eng Int 17 2001 105 112
99. Zhang Z, Li J, Manikopoulos CN, Jorgenson J, Ucles J. Hide: a hierarchical network intrusion detection system using statistical preprocessing and neural network classification. In: Proceedings of IEEE workshop on information assurance and security; 2001. p. 85-90.
100. R. Zhu Intelligent rate control for supporting real-time traffic in wlan mesh networks J Netw Comput Appl 34 5 2011 1449 1458 (dependable multimedia communications: systems, services, and applications)