A fast outlier detection strategy for distributed high-dimensional data sets with mixed attributes

Data Mining and Knowledge Discovery

Volumn 20, Issue 2, 2010, Pages 259-289

  Koufakou, Anna ^a,b   Georgiopoulos, Michael ^b  

^a FLORIDA GULF COAST UNIVERSITY   (United States)

^b University of Central Florida   (United States)

Author keywords

Anomaly detection; Data mining; Distributed data sets; High dimensional data sets; Mixed attribute data sets; Outlier detection

Indexed keywords

ANOMALY DETECTION; ATTRIBUTE DATA; DISTRIBUTED DATA; HIGH DIMENSIONAL DATA; OUTLIER DETECTION;

MEASUREMENT THEORY;

INTRUSION DETECTION;

EID: 77649275031     PISSN: 13845810     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10618-009-0148-z     Document Type: Article

References (44)

1. Acuna E, Rodriguez C (2004) A meta analysis study of outlier detection methods in classification. Technical paper, Department of Mathematics, University of Puerto Rico at Mayaguez. Available at http://academic.uprm. edu~eacuna/paperout.pdf
2. C Aggarwal P Yu 2001 Outlier detection for high dimensional data ACM SIGMOD Record 30 2 37 46 10.1145/376284.375668
3. Agrawal R, Srikant R (1994) Fast algorithms for mining association rules in large databases. In: Proceedings of the international conference on very large data bases, pp 487-499
4. Aha D, Bankert R (1994) Feature selection for case-based classification of cloud types: an empirical comparison. In: Proceedings of the 1994 AAAI workshop on case-based reasoning, pp 106-112
5. F Angiulli C Pizzuti 2005 Outlier mining in large high-dimensional data sets IEEE Transac Knowl Data Engin 17 2 203 215 10.1109/TKDE.2005.31 2231536 (Pubitemid 40320179)
6. Barnett V, Lewis T (1978) Outliers in statistical data. Wiley, NY
7. Bay S, Schwabacher M (2003) Mining distance-based outliers in near linear time with randomization and a simple pruning rule. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, pp 29-38
8. Beyer K, Goldstein J, Ramakrishnan R, Shaft U (1999) When is "nearest neighbor" meaningful? In: Proceedings of the 7th international conference on database theory, pp 217-235
9. Biba M, Esposito F, Ferilli S, Di Mauro N, Basile T (2007) Unsupervised discretization using kernel density estimation. In: Proceedings of the 20-th international conferece on artificial intelligence, pp 696-701
10. Blake C, Merz C (1998) UCI repository of machine learning databases. http://archive.ics.uci.edu. Accessed Sept 2008
11. R Bolton D Hand 2002 Statistical fraud detection: a review Stat Sci 17 3 235 255 1013.62115 10.1214/ss/1042727940 1963313
12. Branch J, Szymanski B, Giannella C, Wolff R, Kargupta H (2006) In-network outlier detection in wireless sensor networks. In: Proceedings 26th international conference on distributed computing systems
13. M Breunig H Kriegel R Ng J Sander 2000 LOF: identifying density-based local outliers ACM SIGMOD Record 29 2 93 104 10.1145/335191.335388
14. T Calders C Rigotti J Boulicaut 2004 A survey on condensed representations for frequent sets LNCS Constraint-Based Mining and Inductive Databases 3848 64 80 10.1007/11615576-4
15. Catlett J (1991) Megainduction: machine learning on very large databases, PhD thesis, Basser Department of Computer Science, University of Sydney, Australia
16. Dean J, Ghemawat S (2004) MapReduce: simplified data processing on large clusters. In: USENIX symposium on operating systems design and implementation OSDI
17. Dokas P, Ertoz L, Kumar V, Lazarevic A, Srivastava J, Tan P (2002) Data mining for network intrusion detection. In: Proceedings NSF workshop on next generation data mining, pp 21-30
18. Ertoz L, Steinbach M, Kumar V (2003) Finding clusters of different sizes, shapes, and densities in noisy, high dimensional data. In: SIAM international conference on data mining, pp 47-58
19. F Geerts B Goethals J Van den Bussche 2005 Tight upper bounds on the number of candidate patterns ACM Transac Database System (TODS) 30 2 333 363 10.1145/1071610.1071611 (Pubitemid 41204993)
20. Hawkins D (1980) Identification of outliers. Chapman and Hall, London
21. Hawkins S, He H, Williams G, Baxter R (2002) Outlier detection using replicator neural networks. In: Proceedings of the 4th international conference on data warehousing and knowledge discovery, pp 170-180
22. Hays C (2004) What Wal-Mart knows about customers habits. The New York Times, November 14
23. He Z, Xu X, Deng S, Calvanese D, De Giacomo G, Lenzerini M (2006) A fast greedy algorithm for outlier mining. In: Proceedings of 10th Pacific-Asia conference on knowledge and data discovery, pp 567-576
24. Hettich S, Bay S (1999) The UCI KDD archive. http://kdd.ics.uci.edu
25. V Hodge J Austin 2004 A survey of outlier detection methodologies Artif Intell Rev 22 2 85 126 1101.68023 10.1023/B:AIRE.0000045502.10941.a9
26. Knorr E, Ng R (1998) Algorithms for mining distance-based outliers in large datasets. In: Proceedings of the 24th international conference on very large data bases, pp 392-403
27. E Knorr R Ng V Tucakov 2000 Distance-based outliers: algorithms and applications Int J Very Large Data Bases VLDB 8 3 237 253 10.1007/s007780050006
28. Knuth D (1968) The art of computer programming, vol 1. Addison-Wesley, Reading, MA
29. Koufakou A, Georgiopoulos M, Anagnostopoulos G (2008b) Detecting outliers in high-dimensional datasets with mixed attributes. In: International conference on data mining DMIN, pp 427-433
30. Koufakou A, Ortiz E, Georgiopoulos M, Anagnostopoulos G, Reynolds K (2007) A scalable and efficient outlier detection strategy for categorical data. In: IEEE international conference on tools with artificial intelligence ICTAI, pp 210-217
31. Koufakou A, Secretan J, Reeder J, Cardona K, Georgiopoulos M (2008a) Fast parallel outlier detection for categorical datasets using MapReduce. In: IEEE world congress on computational intelligence international joint conference on neural networks IJCNN, pp 3298-3304
32. L Latecki A Lazarevic D Pokrajac 2007 Outlier detection with kernel density functions Lecture Notes in Computer Science 4571 61 10.1007/978-3-540- 73499-4-6
33. Lazarevic A, Ertoz L, Kumar V, Ozgur A, Srivastava J (2003) A comparative study of anomaly detection schemes in network intrusion detection. In: Proceedings of the 3rd SIAM international conference on data mining, p 25
34. S Mehta S Parthasarathy H Yang 2005 Toward unsupervised correlation preserving discretization IEEE Transac Knowl Data Engin 17 9 1174 1185 10.1109/TKDE.2005.153 (Pubitemid 46488935)
35. M Otey A Ghoting S Parthasarathy 2006 Fast distributed outlier detection in mixed-attribute data sets Data Mining Knowl Discov 12 2 203 228 10.1007/s10618-005-0014-6 2227402
36. Papadimitriou S, Kitagawa H, Gibbons P, Faloutsos C, (2003) LOCI: fast outlier detection using the local correlation integral. In: Proceedings 19th international conference on data engineering, pp 315-326
37. K Penny I Jolliffe 2001 A comparison of multivariate outlier detection methods for clinical laboratory safety data The Statistician 50 3 295 308 1856382
38. Preparata F, Shamos M (1985) Computational geometry: an introduction. Springer, Berlin
39. S Roberts L Tarassenko 1994 A probabilistic resource allocating network for novelty detection Neural Comput 6 2 270 284 10.1162/neco.1994.6.2.270
40. P Rousseeuw 1985 Multivariate estimation with high breakdown point Math Stat Appl 8 283 297 851060
41. Rousseeuw P, Leroy A (1987) Robust regression and outlier detection. Wiley, NY
42. Tan P, Steinbach M, Kumar V (2005) Introduction to data mining. Pearson Addison Wesley, London
43. D Tax R Duin 2004 Support vector data description Mach Learn 54 1 45 66 1078.68728 10.1023/B:MACH.0000008084.60811.49
44. J Yu W Qian H Lu A Zhou 2006 Finding centric local outliers in categorical/numerical spaces Knowl Inform Syst 9 3 309 338 10.1007/s10115-005- 0197-6