A methodological overview on anomaly detection

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 7754, Issue , 2013, Pages 148-183

  Callegari, Christian ^a   Coluccia, Angelo ^b   D'alconzo, Alessandro ^c   Ellens, Wendy ^d   Giordano, Stefano ^a   Mandjes, Michel ^e,f,g   Pagano, Michele ^a   Pepe, Teresa ^a   Ricciato, Fabio ^b,c   Zìšuraniewski, Piotr ^e,h  

^a UNIVERSITY OF PISA   (Italy)

^b UNIVERSITY OF SALENTO   (Italy)

^c TELECOMMUNICATIONS RESEARCH CENTER VIENNA FTW   (Austria)

^d TNO   (Netherlands)

^e UNIVERSITY OF AMSTERDAM   (Netherlands)

^f EINDHOVEN UNIVERSITY OF TECHNOLOGY   (Netherlands)

^g CWI   (Netherlands)

^h AGH UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Poland)

Author keywords

[No Author keywords available]

Indexed keywords

QUALITY OF SERVICE;

GENERAL KNOWLEDGE;

ANOMALY DETECTION;

EID: 84875951216     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-36784-7_7     Document Type: Review

References (104)

1. Darpa intrusion detection evaluation data set, http://www.ll.mit.edu/ mission/ communications/ist/corpora/ideval
2. Kdd cup (1999), data, http://kdd.ics.uci.edu/databases/kddcup99/kddcup99. html
3. Arthur, D., Vassilvitskii, S.: K-means++: The advantages of careful seeding. In: SODA 2007: Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 1027-1035. Society for Industrial and Applied Mathematics, Philadelphia (2007
4. Barford, P., Kline, J., Plonka, D., Ron, A.: A signal analysis of network traffic anomalies. In: Proceedings of the 2nd ACM SIGCOMM Workshop on Internet Measurment, IMW 2002, pp. 71-82. ACM, New York (2002
5. Borgnat, P., Dewaele, G., Fukuda, K., Abry, P., Cho, K.: Seven years and one day: Sketching the evolution of internet traffic. In: INFOCOM (April 2009
6. Bouzida, Y., Cuppens, F., Cuppens-Boulahia, N.A., Gombault, S.N.: Efficient intrusion detection using principal component analysis. In: 3̀eme Conf'erence sur la S'ecurit'e et Architectures R'eseaux, La Londe, France, Juin, RSM-D'ept. R'eseaux, S'ecurit'e et Multim'edia (Institut T'el'ecom-T'el'ecom Bretagne) (2004
7. Breunig, M.M., Kriegel, H.-P., Ng, R.T., Sander, J.: Lof: Identifying density-based local outliers. ACM SIGMOD Record 29(2), 93-104 (2000
8. Brodsky, B., Darkhovsky, B.: Nonparametric Methods in Change-point Problems. Kluwer (1993
9. Brown, C., Cowperthwaite, A., Hijazi, A., Somayaji, A.: Analysis of the 1999 darpa/lincoln laboratory ids evaluation data with netadhict. In: CISDA 2009: Proceedings of the Second IEEE International Conference on Computational Intelligence for Security and Defense Applications, pp. 67-73. IEEE Press, Piscataway (2009
10. Bucklew, J.: Large Deviation Techniques in Decision, Simulation, andEstimation. Wiley (1985
11. Burgess, M., Haugerud, H., Straumsnes, S., Reitan, T.: Measuring system normality. ACM Trans. Comput. Syst. 20(2), 125-160 (2002
12. Callegari, C., Gazzarrini, L., Giordano, S., Pagano, M., Pepe, T.: A novel multi time-scales pca-based anomaly detection system. In: 2010 International Symposium on Performance Evaluation of Computer and Telecommunication Systems, SPECTS (2010
13. Callegari, C., Gazzarrini, L., Giordano, S., Pagano, M., Pepe, T.: When randomness improves the anomaly detection performance. In: Proceedings of 3rd International Symposium on Applied Sciences in Biomedical and Communication Technologies, ISABEL (2010
14. Callegari, C., Giordano, S., Pagano, M.: Application of Wavelet Packet Transform to Network Anomaly Detection. In: Balandin, S., Moltchanov, D., Koucheryavy, Y. (eds.) NEW2AN 2008. LNCS, vol. 5174, pp. 246-257. Springer, Heidelberg (2008
15. Callegari, C., Giordano, S., Pagano, M., Pepe, T.: On the use of sketches and wavelet analysis for network anomaly detection. In: IWCMC 2010: Proceedings of the 6th International Wireless Communications and Mobile Computing Conference, pp. 331-335. ACM, New York (2010
16. Callegari, C., Giordano, S., Pagano, M., Pepe, T.: Combining sketches and wavelet analysis for multi time-scale network anomaly detection. Computers & Security 30(8), 692-704 (2011
17. Callegari, C., Giordano, S., Pagano, M., Pepe, T.: Detecting heavy change in the heavy hitter distribution of network traffic. In: IWCMC, pp. 1298-1303. IEEE Press (2011
18. Callegari, C., Giordano, S., Pagano, M., Pepe, T.: Detecting anomalies in backbone network traffic: A performance comparison among several change detection methods. IJSNet 11(4), 205-214 (2012
19. Carl, G., Brooks, R.R., Rai, S.: Wavelet based denial-of-service detection. Computers & Security 25(8), 600-615 (2006
20. Chandola, V., Banerjee, A., Kumar, V.: Anomaly detection: A survey. ACM Comput. Surv. 41(3), 15:1-15:58 (2009
21. Charikar,M., Chen, K., Farach-Colton, M.: Finding frequent items in data streams. In: Proc. VLDB Endow, pp. 693-703 (2002
22. Chatzigiannakis, V., Papavassiliou, S., Androulidakis, G.: Improving network anomaly detection effectiveness via an integrated multi-metric-multi- link (m3l) pca-based approach. Security and Communication Networks 2(3), 289-304 (2009
23. Chen, J., Gupta, A.: Testing and locating variance change points with application to stock prices. J. Am. Statist. Assoc. 92, 739-747 (1997
24. Cheung-Mon-Chan, P., Clerot, F.: Finding hierarchical heavy hitters with the count min sketch. In: Proceedings of 4th InternationalWorkshop on Internet Performance, Simulation, Monitoring and Measurement, IPS-MOME (2006
25. Coifman, R.R., Wickerhauser, M.V.: Entropy-based algorithms for best basis selection. IEEE Transactions on Information Theory 38(2), 713-718 (1992
26. Cormode, G., Muthukrishnan, S.: What's hot and what's not: Tracking most frequent items dynamically. In: Proceedings of ACM Principles of Database Systems, pp. 296-306 (2003
27. Cormode, G., Muthukrishnan, S.: What's new: Finding significant differences in network data streams. In: Proc. of IEEE Infocom, pp. 1534-1545 (2004
28. Cormode, G., Muthukrishnan, S.: An improved data stream summary: The count-min sketch and its applications. Journal of Algorithms 55(1), 58-75 (2005
29. Cormode, G., Muthukrishnan, S., Srivastava, D.: Finding hierarchical heavy hitters in data streams. In: Proc. of VLDB, pp. 464-475 (2003
30. Dainotti, A., Pescape, A., Ventre, G.: Wavelet-based detection of dos attacks. In: Proceedings of Global Telecommunications Conference, GLOBECOM2006, pp. 1-6. IEEE (2006
31. D'Alconzo, A., Coluccia, A., Romirer-Maierhofer, P.: Distribution-based anomaly detection in 3g mobile networks: From theory to practice. Int. J. Netw. Manag. 20(5), 245-269 (2010
32. Daubechies, I.: Orthonormal bases of compactly supported wavelets. Communications on Pure and Applied Mathematics 41, 909-996 (1988
33. Daubechies, I.: Ten lectures on Wavelets. CBMS-NSF Series in Applied Mathematics, vol. 61. SIAM, Philadelphia (1992
34. Dembo, A., Zeitouni, O.: Large Deviations Techniques and Applications. Springer (1998
35. Denning, D.E.: An intrusion-detection model. IEEE Transactions on Software Engineering 13(2), 222-232 (1987
36. Dewaele, G., Fukuda, K., Borgnat, P., Abry, P., Cho, K.: Extracting hidden anomalies using sketch and non gaussian multiresolution statistical detection procedures. In: LSAD 2007: Proceedings of the 2007 Workshop on Large Scale Attack Defense, pp. 145-152. ACM, New York (2007
37. Ensafi, R., Dehghanzadeh, S., Akbarzadeh, T.M.R.: Optimizing fuzzy k-means for network anomaly detection using pso. In: AICCSA 2008: Proceedings of the 2008 IEEE/ACS International Conference on Computer Systems and Applications, pp. 686-693. IEEE Computer Society, Washington, DC (2008
38. Ertöz, L., Eilertson, E., Lazarevic, A., Tan, P.N., Kumar, V., Srivastava, J.P., Dokas, P.: MINDS-Minnesota Intrusion Detection System. MIT Press (2004
39. Eskin, E., Arnold, A., Prerau, M., Portnoy, L., Stolfo, S.: A geometric framework for unsupervised anomaly detection: Detecting intrusions in unlabeled data. In: Applications of Data Mining in Computer Security. Kluwer (2002
40. Estan, C., Varghese, G.: New directions in traffic measurement and accounting: Focusing on the elephants, ignoring the mice. ACM Transactions on Computer Systems 21, 270-313 (2003
41. Ester, M., Kriegel, H.-P., Sander, J., Xu, X.: A density-based algorithm for discovering clusters in large spatial databases with noise, pp. 226-231. AAAI Press (1996
42. Fox, K.L., Henning, R.R., Reed, J.H., Simonian, R.P.: A neural network approach towards intrusion detection. In: Proc. 13th National Computer Security Conference. Information Systems Security. Standards-The Key to the Future, vol. I, pp. 124-134 (1990
43. Maier, G., Feldmann, A., Paxson, V., Allman,M.: On dominant characteristics of residential broadband internet traffic. In: IEEE IMC (2009
44. Gao, J., Hu, G., Yao, X.: Anomaly detection of network traffic based on wavelet packet (2006
45. Gilbert, A.C.:Multiscale analysis and data networks. Applied and Computational Harmonic Analysis 10, 185-202 (2001
46. Hodge, V., Austin, J.: A survey of outlier detection methodologies. Artif. Intell. Rev. 22(2), 85-126 (2004
47. Hsu, D.: Tests for variance shift at an unknown time point. Appl. Statist. 26, 279-284 (1977
48. Huang, P., Feldmann, A., Willinger, W.: A non-instrusive, wavelet-based approach to detecting network performance problems. In: IMW 2001: Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement, pp. 213-227 (2001
49. Incl'an, C., Tiao, G.: Use of cumulative sums of squares for retrospective detection of changes of variance. J. Am. Statist. Assoc. 89, 913-923 (1994
50. Zaki, M.J., Sequeira, K.: Admit: Anomaly-base data mining for intrusions. In: 8th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Jul. 2002
51. Karp, R.M., Papadimitriou, C.H., Shenker, S.: A simple algorithm for finding frequent elements in streams and bags. ACM Transactions on Database Systems 28 (2003
52. Kim, S.S.,Narasimha Reddy, A.L., Vannucci,M.: Detecting traffic anomalies using discrete wavelet transform. In: Proceedings of International Conference on Information Networking (ICOIN), Busan, Korea, pp. 1375-1384 (2003
53. Lakhina, A.: Diagnosing network-wide traffic anomalies. In: ACM SIGCOMM, pp. 219-230 (2004
54. Lakhina, A., Crovella, M., Diot, C.: Characterization of network-wide anomalies in traffic flows. In: ACM Internet Measurement Conference, pp. 201-206 (2004
55. Lakhina, A., Crovella, M., Diot, C.: Mining anomalies using traffic feature distributions. In: ACM SIGCOMM (2005
56. Lakhina, A., Papagiannaki, K., Crovella, M., Christophe, D., Kolaczyk, E.D., Taft, N.: Structural analysis of network traffic flows. In: Proceedings of the Joint International Conference on Measurement and Modeling of Computer Systems, SIGMETRICS 2004/Performance 2004, pp. 61-72. ACM, New York (2004
57. Lazarevic, A., Ozgur, A., Ertoz, L., Srivastava, J., Kumar, V.: A comparative study of anomaly detection schemes in network intrusion detection. In: Proceedings of the Third SIAM International Conference on Data Mining (2003
58. Leland, W.E., Taqqu, M.S., Willinger, W., Wilson, D.V.: On the self-similar nature of ethernet traffic (extended version). IEEE/ACMTrans. Netw. 2(1), 1-15 (1994
59. Lin, S.-Y., Liu, J.-C., Zhao,W.: Adaptive cusum for anomaly detection and its application to detect shared congestion. Texas A&M University. Technical Report TAMU-CS-TR-2007- 1-2 (2007
60. Liu, Y., Zhang, L., Guan, Y.: Sketch-based streaming pca algorithm for network-wide traffic anomaly detection. In: Proceedings of International Conference on Distributed Computing Systems (2010
61. Lorden, G.: Procedures for reacting to a change in distribution. Ann. Math. Statist. 42, 1897-1908 (1971
62. Lu, W., Ghorbani, A.: Network anomaly detection based on wavelet analysis. EURASIP Journal on Advances in Signal Processing (1), 837601 (2009
63. Mallat, S.G.: A theory for multiresolution signal decomposition: The wavelet representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 11(7), 674-693 (1989
64. Mandjes, M.: Large Deviations for Gaussian Queues. Wiley (2007
65. Mandjes, M., Zuraniewski, P.:M/g/∞transience, and its applications to overload detection. Performance Evaluation 68, 507-527 (2011
66. Manku, G.S., Motwani, R.: Approximate frequency counts over data streams. In: VLDB, pp. 346-357 (2002
67. Mata, F., Zuraniewski, P., Mandjes, M., Mellia, M.: Anomaly detection in voip traffic with trends. In: Proceedings of the 24th International Teletraffic Congress (2012
68. Matteoli, S., Diani, M., Corsini, G.: A tutorial overview of anomaly detection in hyperspectral images. IEEE Aerospace and Electronic Systems Magazine 25(7), 5-28 (2010
69. Münz, G., Carle, G.: Application of forecasting techniques and control charts for traffic anomaly detection. In: Proceedings of the 19th ITC Specialist Seminar on Network Usage and Traffic (2008
70. Munz, G., Li, S., Carle, G.: Traffic anomaly detection using k-means clustering. In: GI/ITGWorkshop MMBnet (2007
71. Muthukrishnan, S.: Data streams: Algorithms and applications. In: Proceedings of the Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 413-413. Society for Industrial and Applied Mathematics, Philadelphia (2003
72. Oldmeadow, J., Ravinutala, S., Leckie, C.: Adaptive Clustering for Network Intrusion Detection. In: Dai, H., Srikant, R., Zhang, C. (eds.) PAKDD 2004. LNCS (LNAI), vol. 3056, pp. 255-259. Springer, Heidelberg (2004
73. Page, E.: Continuous inspection scheme. Biometrika 41, 100-115 (1954
74. Pollak, M.: Optimal detection of a change in distribution. Ann. Statist. 13, 206-227 (1985
75. Portnoy, L., Eskin, E., Stolfo, S.J.: Intrusion detection with unlabeled data using clustering. In: Proceedings of ACM CSS Workshop on Data Mining Applied to Security (November 2001
76. Pukkawanna, S., Fukuda, K.: Combining sketch and wavelet models for anomaly detection. In: 2010 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP), pp. 313-319 (August 2010
77. Ramaswamy, S., Rastogi, R., Shim, K.: Efficient algorithms for mining outliers from large data sets. SIGMOD Rec. 29(2), 427-438 (2000
78. Resnick, S.: Adventures in Stochastic Processes. Birkhäuser (2002
79. Ricciato, F., Coluccia, A., D'Alconzo, A., Veitch, D., Borgnat, P., Abry, P.: On the role of flows and sessions in internet traffic modeling: An explorative toy-model. In: IEEE Globecom (2009
80. Ricciato, F., Coluccia, A., D'Alconzo, A.: A review of dos attack models for 3g cellular networks from a system-design perspective. Computer Communications 33(5), 551-558 (2010
81. Ringberg, H., Soule, A., Rexford, J., Diot, C.: Sensitivity of pca for traffic anomaly detection. SIGMETRICS Perform. Eval. Rev. 35(1), 109-120 (2007
82. Roughan, M., Veitch, D., Abry, P.: Real-Time estimation of the parameters of long-range dependence. IEEE/ACM Trans. Netw. 8(4), 467-478 (2000
83. Schweller, R., Gupta, A., Parsons, E., Chen, Y.: Reversible sketches for efficient and accurate change detection over network data streams. In: Proceedings of the 4th ACM SIGCOMM Conference on Internet Measurement, IMC 2004, pp. 207-212. ACM, New York (2004
84. Shiryaev, A.: On optimum methods in quickest detection problems. Theory Probab. Appl. 8, 22-46 (1963
85. Shiryaev, A.: On Markov sufficient statistics in non-Additive Bayes problems of sequential analysis. Theory Probab. Appl. 9, 604-618 (1964
86. Shlens, J.: A tutorial on principal component analysis (December 2005), http://www.snl.salk.edu/?shlens/pub/notes/pca.pdf
87. Shyu, M., Chen, S., Sarinnapakorn, K., Chang, L.: A novel anomaly detection scheme based on principal component classifier. In: In IEEE Foundations and New Directions of Data Mining Workshop, in Conjunction with ICDM 2003, pp. 172-179 (2003
88. Siegmund, D.: Sequential Analysis. Springer (1985
89. Sperotto, A., Mandjes, M., Sadre, R., de Boer, P.T., Pras, A.: Autonomic parameter tuning of anomaly-based IDSs: An SSH case study. IEEE Transactions on Network and Service Management 9, 128-141 (2012
90. Subhabrata, B.K., Krishnamurthy, E., Sen, S., Zhang, Y., Chen, Y.: Sketch-based change detection: Methods, evaluation, and applications. In: Internet Measurement Conference, pp. 234-247 (2003
91. Svoboda, P., Ricciato, F., Hasenleithner, E., Pilz, R.: Composition of gprs/umts traffic: Snapshots from a live network. In: 4th IntlWorkshop on Internet Performance, Simulation, Monitoring and Measurement, IPS-MOME 2006, Salzburg (2006
92. Tartakovsky, A., Veeravalli, V.: Changepoint detection in multi-channel and distributed systems with applications. In: Applications of Sequential Methodologies, pp. 331-363 (2004
93. Thorup, M., Zhang, Y.: Tabulation based 4-universal hashing with applications to second moment estimation. In: SODA 2004: Proceedings of the Fifteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pp. 615-624. Society for Industrial and AppliedMathematics, Philadelphia (2004
94. Thottan, M., Ji, C.: Anomaly detection in IP networks. IEEE Trans. on Signal Processing 51(8) (August 2003
95. Thottan, M., Liu,G., Ji, C.: Anomaly detection approaches for communication networks. In: Cormode, G., Thottan, M., Sammes, A.J. (eds.) Algorithms for Next Generation Networks. Computer Communications and Networks, pp. 239-261. Springer, London (2010
96. Tolle, J., Niggemann, O.: Supporting intrusion detection by graph clustering and graph drawing. Springer (2000
97. Traynor, P., McDaniel, P., La Porta, T.: On attack causality in internet-connected cellular networks. In: USENIX Security (August 2007
98. Traynor, P., McDaniel, P., La Porta, T.: Security for Telecommunications Networks. Springer (2008
99. Vetterli,M., Kovacevic, J.:Wavelets and subband coding. Prentice-Hall, Inc., Upper Saddle River (1995
100. Wang, L., Potzelberger, K.: Boundary crossing probability for Brownian motion and general boundaries. J. Appl. Probab. 34, 54-65 (1997
101. Wang, W., Battiti, R.: Identifying intrusions in computer networks with principal component analysis. In: ARES 2006: Proceedings of the First International Conference on Availability, Reliability and Security, pp. 270-279. IEEE Computer Society, Washington, DC (2006
102. Wang, W., Guan, X., Zhang, X.: A Novel Intrusion Detection Method Based on Principle Component Analysis in Computer Security. In: Yin, F.-L., Wang, J., Guo, C. (eds.) ISNN 2004, Part II. LNCS, vol. 3174, pp. 657-662. Springer, Heidelberg (2004
103. Yang, H., Ricciato, F., Lu, S., Zhang, L.: Securing a wireless world. Proceedings of the IEEE 94(2), 442-454 (2006
104. Ye, N.: A markov chain model of temporal behavior for anomaly detection. In: Proceedings of the Workshop on Information Assurance and Security (2000