Fast outlier detection for very large log data

Expert Systems with Applications

Volumn 38, Issue 8, 2011, Pages 9587-9596

  Kim, Seung ^a   Cho, Nam Wook ^b   Kang, Bokyoung ^a   Kang, Suk Ho ^a  

^a Seoul National University   (South Korea)

^b Seoul National University of Science and Technology   (South Korea)

Author keywords

anomaly) detection; Approximated k nearest neighbors; Density based outlier detection; Intrusion (novelty; Kd tree

Indexed keywords

ANOMALY) DETECTION; DENSITY-BASED OUTLIER DETECTION; INTRUSION (NOVELTY; K-NEAREST NEIGHBORS; KD-TREE;

ALGORITHMS; COMPUTATIONAL COMPLEXITY; TREES (MATHEMATICS);

INTRUSION DETECTION;

EID: 79953720253     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2011.01.162     Document Type: Article

References (30)

1. C.C. Aggarwal, and P.S. Yu Outlier detection for high dimensional data Proceedings of the 2001 ACM SIGMOD International Conference on Management of Data 2001 ACM press New York 37 46
2. Agrawal, R.; Gehrke, J.; Gunopulos, D.; & Raghavan P. (1998). Automatic Subspace Clustering of High Dimensional Data for Data Mining Applications Automatic Subspace Clustering of High Dimensional Data for Data Mining Applications, In Proceeding of the ACM SIGMOD International Conference on Management of Data, Seattle, Washington, pp. 94-105. (Pubitemid 128655960)
3. M. Agyemang, and C.I. Ezeife LSC-Mine: Algorithm for Mining Local Outliers Proceedings of the 15th Information Resource Management Association (IRMA) International Conference, New Orleans 2004 IRM press Hershey, PA U.S.A. 5 8
4. S. Arya, D.M. Mount, N.S. Netanyahu, R. Silverman, and A. Wu An optimal algorithm for approximate nearest neighbor searching Journal of the ACM 45 6 1998 891 923
5. Asuncion, A.; & Newman, D.J. (2007). UCI Machine Learning Repository, available at http://www.ics.uci.edu/∼mlearn/MLRepository.html, Irvine, CA: University of California, School of Information and Computer Science.
6. V. Barnett, and T. Lewis Outliers in Statistical Data 1994 John Wiley
7. I. Ben-Gal Outlier detection O. Maimon, L. Rokach, Data Mining and Knowledge Discovery Handbook: A Complete Guide for Practitioners and Researchers 2005 Kluwer Academic Publishers
8. J.L. Bentley K-d trees for semidynamic point sets Proceedings of 6th Annual ACM Symposium Computational Geometry 1990 ACM Press New York 187 197
9. M.M. Breunig, H.P. Kriegel, R.T. Ng, and J. Sander LOF: Identifying Density Based Local Outliers Proceedings of the ACM SIGMOD Conference, Dallas, Texas 2000 ACM Press New York 93 104
10. M.M. Ester, H.P. Kriegel, J. Sander, and X. Xu A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise Proceeding for 2nd International Conference on Knowledge Discovery and Data Mining(KDD '96), Portland, Oregon 1996 AAAI Press 226 231
11. K.J. Ezawa, and S.W. Norton Constructing Bayesian Networks to predict Uncollectible Telecommunications Accounts IEEEExpert 11 5 1996 45 51 (Pubitemid 126796868)
12. C. Faloutsos, and K.I. Lin FastMap: A fast algorithm for indexing, data-mining and visualization of traditional and multimedia datasets Proceedings of the 1995 ACM SIGMOD International Conference on Management of Data, San Jose, California 1995 ACM Press New York 163 174
13. J.H. Friedman, J.L. Bentley, and R.A. Finkel An algorithm for finding best matches in logarithmic expected time ACM Transaction on Mathematical Software 3 3 1977 209 226
14. K. Fukunaga Introduction to Statistical Pattern Recognition 2nd ed. 1990 Academic Press
15. S. Guha, R. Rastogi, and K. Shim Cure: An Efficient Clustering Algorithm for Large Databases Proceeding of the ACM SIGMOD International Conference on Management of Data, Seattle, Washington 1998 ACM Press New York 73 84 (Pubitemid 128655958)
16. D. Hawkins Identification of Outliers 1980 Chapman and Hall
17. D.V. Hinkley On the Ratio of Two Correlated Normal Random Variables Biometrika 56 3 1969 635 639
18. S.S. Hwang, S. Cho, and S. Park Keystroke dynamics-based authentication for mobile devices Computers & Security 28 2009 85 93
19. Lazarevic, A.; Ertoz, L.; Kumar, V.; Ozgur, A.; & Srivastava, J. (2003). A comparative study of anomaly detection schemes in network intrusion detection, In Proceedings of the Third SIAM International Conference on Data Mining, San Francisco, CA.
20. J. MacQueen Some Methods for Classification and Analysis of Multivariate Observations Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, University of California, Berkeley 1967 University of California Press Berkeley, California 291 297
21. M.A. Maloof Machine Learning and Data Mining for Computer Security: Methods and Applications 2005 Springer
22. G. Medioni, I. Cohen, S. Hongeng, F. Bremond, and R. Nevatia Event Detection and Analysis from Video Streams IEEE Transactions on Pattern Analysis and Machine Intelligence 8 23 2001 873 889 (Pubitemid 32780014)
23. Mount, D.M.; & Arya, S. (2006). ANN: A Library for Approximate Nearest Neighbor Searching available at http://www.cs.umd.edu/∼mount/ANN/.
24. D. Pokrajac, A. Lazarevic, and L.J. Latecki Incremental Local Outlier Detection for Data Streams IEEE Symposium on Computational Intelligence and Data Mining (CIDM), Honolulu, Hawaii 2007 IEEE Press New York 504 515 (Pubitemid 47431481)
25. W.H. Press, B.P. Flannery, S.A. Teukolsky, and W.T. Vetterling Numerical Recipes in C 1988 Cambridge University Press
26. S. Ramaswamy, R. Rastogi, and K. Shim Efficient algorithms for mining outliers from large data sets Proceedings of the International Conference on Management of Data, Dallas, Texas 2000 ACM Press New York 427 438
27. G. Strang Linear Algebra and its Applications 2nd ed. 1980 Academic Press
28. Yue, D.; Wu, X.; Wang, Y.; Li, Y.; & Chu, C.H. (2007). A Review of Data Mining-Based Financial Fraud Detection Research, In Proceedings of 2007 International Conference on Wireless Communications, Networking and Mobile Computing, Shanghai, P.R. China. Sponsored by IEEE, pp. 5514-5517.
29. T. Zhang, R. Ramakrishnan, and M. Livny Birch: An Efficient data clustering method for very large databases Proceedings for the ACM SIGMOD Conference on Management of Data, Montreal, Canada 1996 ACM Press New York 103 114 (Pubitemid 126440760)
30. J. Zhang, and M. Zulkernine Anomaly Based Network Intrusion Detection with Unsupervised Outlier Detection Proceedings of 2006 IEEE International Conference on Communications, Istanbul, Turkey 2006 IEEE Press New York 2388 2393 (Pubitemid 351575588)