A multi-step outlier-based anomaly detection approach to network-wide traffic

Information Sciences

Volumn 348, Issue , 2016, Pages 243-271

  Bhuyan, Monowar H ^a   Bhattacharyya, D K ^b   Kalita, J K ^c  

^a KAZIRANGA UNIVERSITY   (India)

^b TEZPUR UNIVERSITY   (India)

^c University of Colorado at Colorado Springs   (United States)

Author keywords

Anomaly detection; Clustering; Network wide traffic; Outlier score; Reference point

Indexed keywords

DATA MINING; DIAGNOSIS; INFORMATION MANAGEMENT; NETWORK MANAGEMENT; SIGNAL DETECTION; STATISTICS;

ANOMALY DETECTION; CLUSTERING; DISTRIBUTED FEATURES; FALSE POSITIVE RATES; GENERALIZED ENTROPIES; NETWORK ANOMALY DETECTION; OUTLIER SCORE; REFERENCE POINTS;

FEATURE EXTRACTION;

EID: 84959432825     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2016.02.023     Document Type: Article

References (60)

1. A. Agrawal Local subspace based outlier detection Proceedings of the Communication in Computer and Information Science vol. 40 2009 Springer 149 157
2. F. Amiri, M.M.R. Yousefi, C. Lucas, A. Shakery, and N. Yazdani Mutual information-based feature selection for intrusion detection systems J. Netw. Comput. Appl. 34 4 2011 1184 1199
3. R. Andersen, D.F. Gleich, and V. Mirrokni Overlapping clusters for distributed computation Proceedings of the 5th ACM International Conference on Web Search and Data Mining 2012 ACM New York, USA 273 282
4. F. Angiulli, S. Basta, and C. Pizzuti Distance-based detection and prediction of outliers IEEE Trans. Knowl. Data Eng. 18 2 2006 145 160
5. K. Bache, M. Lichman, UCI machine learning repository, 2013. URL http://archive.ics.uci.edu/ml (accessed 10.05.14).
6. D. Barbara, J. Couto, S. Jajodia, L. Popyack, and N. Wu ADAM: detecting intrusions by data mining Proceedings of the IEEE Workshop on Information Assurance and Security 2001 West Point, NY 11 16
7. D. Barbara, N. Wu, and S. Jajodia Detecting novel network intrusions using Bayes estimators Proceedings of the 1st SIAM Conference on Data Mining, Chicago, IL 2001
8. S. Bay, and M. Schwabacher Mining distance-based outliers in near linear time with randomization and a simple pruning rule Proceedings of the 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining 2003 29 38
9. R. Beghdad Critical study of supervised learning techniques in predicting attacks Inf. Secur. J. A Glob. Perspect. 19 1 2010 22 35
10. J.C. Bezdek, R. Ehrlich, and W. Full FCM: the fuzzy c-means clustering algorithm Comput. Geosci. 10 2-3 1984 191 203
11. D.K. Bhattacharyya, and J.K. Kalita Network Anomaly Detection: A Machine Learning Perspective 2013 CRC Press
12. M.H. Bhuyan, D. Bhattacharyya, and J. Kalita Surveying port scans and their detection methodologies Comput. J. 54 10 2011 1565 1581
13. M.H. Bhuyan, D.K. Bhattacharyya, and J.K. Kalita NADO: network anomaly detection using outlier approach Proceedings of the International Conference on Communication, Computing & Security 2011 ACM Odisha, India 531 536
14. M.H. Bhuyan, D.K. Bhattacharyya, and J.K. Kalita Towards generating real-life datasets for network intrusion detection Int. J. Netw. Secur. 17 6 2015 675 693
15. M.M. Breunig, H.-P. Kriegel, R.T. Ng, and J. Sander LOF: identifying density-based local outliers Proceedings of the ACM SIGMOD International Conference on Management of Data 2000 386 395
16. W. Bul'ajoul, A. James, and M. Pannu Improving network intrusion detection system performance through quality of service configuration and parallel technology J. Comput. Syst. Sci. 81 6 2015 981 999
17. P. Casas, J. Mazel, and P. Owezarski Unsupervised network intrusion detection systems: detecting the unknown without knowledge Comput. Commun. 35 7 2012 772 783
18. K.A.P. Costa, L.A.M. Pereira, R.Y.M. Nakamura, C.R. Pereira, J.P. Papa, and A.X. Falcão A nature-inspired approach to speed up optimum-path forest clustering and its application to intrusion detection in computer networks Inf. Sci. 294 2015 95 108
19. G. D'angelo, F. Palmieri, M. Ficco, and S. Rampone An uncertainty-managing batch relevance-based approach to network anomaly detection Appl. Soft. Comput. 36 2015 408 418
20. L. Ertoz, E. Eilertson, A. Lazarevic, P. Tan, J. Srivastava, V. Kumar, and P. Dokas Next Generation Data Mining, Chap. The MINDS-Minnesota Intrusion Detection System 2004 MIT Press
21. M. Ester, H. Kriegel, S. Jörg, and X. Xu A density-based algorithm for discovering clusters in large spatial databases with noise Proceedings of 2nd International Conference on Knowledge Discovery and Data Mining 1996 AAAI Press Portland, Oregon 226 231
22. G. Fernandes, J.J. Rodrigues, and M.L. Proença Autonomous profile-based anomaly detection system using principal component analysis and flow analysis Appl. Soft. Comput. 34 2015 513 525
23. A. Frank, A. Asuncion, UCI machine learning repository, 2010, URL http://archive.ics.uci.edu/ml (accessed 03.04.14).
24. C. Freeman, D. Kulic, and O. Basir An evaluation of classifier-specific filter measure performance for feature selection Pattern Recognit. 48 5 2015 1812 1826
25. J. Ha, S. Seok, and J.S. Lee A precise ranking method for outlier detection Inf. Sci. 324 2015 88 107
26. J.A. Hartigan, and M.A. Wong Algorithm AS 136: a K-means clustering algorithm J. R. Stat. Soc. Ser. C (Appl. Stat.) 28 1 1979 100 108
27. S.J. Horng, M.Y. Su, Y.H. Chen, T.W. Kao, R.J. Chen, J.L. Lai, and C.D. Perkasa A novel intrusion detection system based on hierarchical clustering and support vector machines Expert Syst. Appl. 38 1 2011 306 313
28. A.K. Jain, and R.C. Dubes Algorithms for Clustering Data 1988 Prentice-Hall Upper Saddle River, USA
29. S.Y. Ji, B.K. Jeong, S. Choi, and D.H. Jeong A multi-level intrusion detection method for abnormal network behaviors J. Netw. Comput. Appl. 62 2016 9 17
30. S. Jiang, X. Song, H. Wang, J.-J. Han, and Q.-H. Li A clustering-based method for unsupervised intrusion detections Pattern Recognit. Lett. 27 7 2006 802 810
31. L. Kaufman, and P.J. Rousseeuw Finding Groups in Data: An Introduction to Cluster Analysis vol. 344 2009 John Wiley & Sons
32. E.M. Knorr, and R.T. Ng Algorithms for mining distance-based outliers in large datasets Proceedings of the 24th International Conference on VLDB 1998 Morgan Kaufmann USA 392 403
33. E.M. Knorr, R.T. Ng, and V. Tucakov Distance-based outliers: algorithms and applications VLDB J. 8 3-4 2000 237 253
34. A. Koufakou, and M. Georgiopoulos A fast outlier detection strategy for distributed high-dimensional data sets with mixed attributes Data Mining Knowl. Discov. 20 2 2010 259 289
35. W. Lee, and S.J. Stolfo Data mining approaches for intrusion detection Proceedings of the USENIX Security Symposium 1998 USENIX Association
36. T. Li, and N.F. Xiao Novel heuristic dual-ant clustering algorithm for network intrusion outliers detection Optik - Int. J. Light Electron Optics 126 4 2015 494 497
37. W.C. Lin, S.W. Ke, and C.F. Tsai CANN: an intrusion detection system based on combining cluster centers and nearest neighbors Knowl. Based Syst. 78 2015 13 21
38. H. Liu, S. Shah, and W. Jiang On-line outlier detection and data cleaning Comput. Chem. Eng. 28 9 2004 1635 1647
39. Z. Liu, R. Wang, M. Tao, and X. Cai A class-oriented feature selection approach for multi-class imbalanced network traffic datasets based on local and global metrics fusion Neurocomputing 168 2015 365 381
40. M.M. Masud, Q. Chen, L. Khan, C.C. Aggarwal, J. Gao, J. Han, A. Srivastava, and N.C. Oza Classification and adaptive novel class detection of feature-evolving data streams IEEE Trans. Knowl. Data Eng. 25 7 2013 1484 1497
41. H.D.K. Moonesinghe, and P.N. Tan OutRank: a graph-based outlier detection framework using random walk Int. J. Artif. Intell. Tools 17 1 2008 19 36
42. S. Mukkamala, G. Janoski, and A. Sung Intrusion detection using neural networks and support vector machines Proceedings of the International Joint Conference on Neural Networks vol. 2 2002 IEEE 1702 1707
43. R.T. Ng, and J. Han CLARANS: a method for clustering objects for spatial data mining IEEE Trans. Knowl. Data Eng. 14 5 2002 1003 1016
44. G.H. Orair, C.H.C. Teixeira, J.W. Meira, Y. Wang, and S. Parthasarathy Distance-based outlier detection: consolidation and renewed bearing Proceedings of the VLDB Endowment 3 2010 1469 1480
45. L. Ozçelik, and R.R. Brooks Deceiving entropy based DoS detection Comput. Secur. 48 2015 234 245
46. N. Paulauskas, and A.F. Bagdonas Local outlier factor use for the network flow anomaly detection Secur. Commun. Netw. 8 18 2015 4203 4212
47. Y. Pei, O.R. Zaiane, and Y. Gao An efficient reference-based approach to outlier detection in large datasets Proceedings of the 6th International Conference on Data Mining 2006 IEEE USA 478 487
48. L. Portnoy, E. Eskin, and S. Stolfo Intrusion detection with unlabeled data using clustering Proceedings of the ACM CSS Workshop on Data Mining Applied to Security 2001 Philadelphia, PA 5 8
49. K. Prakobphol, and J. Zhan A novel outlier detection scheme for network intrusion detection systems Proceedings of the International Conference on Information Security and Assurance 2008 IEEE Washington, USA 555 560
50. F. Shaari, A.A. Bakar, and A.R. Hamdan Outlier detection based on rough sets theory Intell. Data Anal. 13 2 2009 191 206
51. J. Song, H. Takakura, Y. Okabe, and K. Nakao Toward a more practical unsupervised anomaly detection system Inf. Sci. 231 0 2013 4 14
52. B. Swingle Rényi entropy, mutual information, and fluctuation properties of fermi liquids Phys. Rev. B 86 2012 045109
53. C.C. Szeto, and E. Hung Mining outliers with faster cutoff update and space utilization Pattern Recognit. Lett. 31 11 2010 1292 1301
54. M. Thottan, and C. Ji Anomaly detection in IP networks IEEE Trans. Signal Process. 51 8 2003 2191 2204
55. C.F. Tsai, and C.Y. Lin A triangle area based nearest neighbors approach to intrusion detection Pattern Recognit. 43 1 2010 222 229
56. Z. Wang, M. Li, and J. Li A multi-objective evolutionary algorithm for feature selection based on mutual information with a new redundancy measure Inf. Sci. 307 2015 73 88
57. J. Zhang, H. Li, Q. Gao, H. Wang, and Y. Luo Detecting anomalies from big network traffic data using an adaptive detection approach Inf. Sci. 318 2015 91 110
58. J. Zhang, and M. Zulkernine Anomaly based network intrusion detection with unsupervised outlier detection Proceedings of the IEEE International Conference on Communications vol. 5 2006 2388 2393
59. Y. Zhang, S. Yang, and Y. Wang LDBOD: a novel local distribution based outlier detector Pattern Recognit. Lett. 29 7 2008 967 976
60. N. Zhou, Y. Xu, H. Cheng, J. Fang, and W. Pedrycz Global and local structure preserving sparse subspace learning: an iterative approach to unsupervised feature selection Pattern Recognit. 53 2016 87 101