Fast parallel outlier detection for categorical datasets using MapReduce

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2008, Pages 3298-3304

  Koufakou, Anna ^a   Secretan, Jimmy ^a   Reeder, John ^a   Cardona, Kelvin ^b   Georgiopoulos, Michael ^a  

^a University of Central Florida   (United States)

^b UNIVERSITY OF PUERTO RICO   (Puerto Rico)

Author keywords

[No Author keywords available]

Indexed keywords

ATTRIBUTE VALUES; CATEGORICAL DATASETS; CATEGORICAL DATUMS; CREDIT CARD FRAUDS; DATASET SCANS; LOAD BALANCING; MAPREDUCE; MASSIVE DATASETS; NETWORK ATTACKS; NUMBER OF CLUSTERS; OUTLIER DETECTIONS; PARALLEL SYSTEMS; QUADRATIC TIMES;

COMPUTER CRIME; FAULT TOLERANCE; NUMERICAL METHODS; PARALLEL PROGRAMMING; QUALITY ASSURANCE; RELIABILITY; SIGNAL DETECTION;

NEURAL NETWORKS;

EID: 56349095269     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2008.4634266     Document Type: Conference Paper

References (23)

1. Bolton, R.J., Hand, D.J., "Statistical fraud detection: A review", Statistical Science, 17, pp. 235-255, 2002.
2. Penny, K.I., Jolliffe, I.T., "A comparison of multivariate outlier detection methods for clinical laboratory safety data", The Statistician, Journal of the Royal Statistical Society, 50, pp. 295-308, 2001.
3. Koufakou, A., Ortiz, E., Georgiopoulos, M., Anagnostopoulos, G., Reynolds, K., "A Scalable and Efficient Outlier Detection Strategy for Categorical Data", Int'l Conference on Tools with Artificial Intelligence ICTAI, October, 2007.
4. Agrawal, R., Srikant, R., "Fast algorithms for mining association rules", Proc. of the Int'l Conference on Very Large Data Bases VLDB, pp. 487-499, 1994.
5. He, Z., Xu, X., Huang, J., Deng, J., "FP-Outlier: Frequent Pattern Based Outlier Detection", Computer Science and Information System, pp. 103-118, 2005.
6. Otey, M.E., Ghoting, A., Parthasarathy, A., "Fast Distributed Outlier Detection in Mixed-Attribute Data Sets", Data Mining and Knowledge Discovery, 2006.
7. Dean, J., and Ghemawat, S., "Mapreduce: Simplified data processing on large clusters," Proceedings of OSDI'04: Symposium on Operating System Design and Implementation, 2004.
8. Chu, C.-T., Kim, S., Lin, Y.-A., Yu, Y., Bradski, G., Ng, A. Y., and Olukotun, K., "Map-reduce for machine learning on multicore," Proceedings of NIPS, 19, 2006.
9. Barnett, V., Lewis, T. Outliers in Statistical Data. John Wiley, 1994.
10. Knorr, E., Ng, R., and Tucakov, V., "Distance-based outliers: Algorithms and applications", Very Large Databases VLDB Journal, 2000.
11. Bay, S.D. Schwabacher, M., "Mining distance-based outliers in near linear time with randomization and a simple pruning rule", Proc. of ACM SIGKDD Int'l Conf. on Knowledge Discovery and Data Mining, 2003.
12. Breunig, M. M., Kriegel, H.-P., Ng, R. T., and Sander, J., "LOF: Identifying density-based local outliers", Proc. of the ACM SIGMOD Int'l Conference on Management of Data, 2000.
13. Papadimitriou, S., Kitawaga, H., Gibbons, P., Faloutsos, C., "LOCI: Fast outlier detection using the local correlation integral", Proceedings of the International Conference on Data Engineering, 2003.
14. Wei, L., Qian, W., Zhou, A., Jin, W., "HOT: Hypergraph-based Outlier Test for Categorical Data", Proc. of 7th Pacific-Asia Conference on Knowledge Discovery and Data Mining PAKDD, pp. 399-410, 2003.
15. Tax, D., Duin, R., "Support Vector Data Description", Machine Learning, pp. 45-66, 2004.
16. Harkins, S., He, H., Williams, G., Baster, R., "Outlier Detection Using Replicator Neural Networks", Data Warehousing and Knowledge Discovery, 4th International Conference, DaWaK, pp. 170-180, 2002.
17. Pei, Y., Zaiane, O., Gao, Y., "An Efficient Reference-based Approach to Outlier Detection in Large Dataset", IEEE Int'l Conference on Data Mining, 2006.
18. Hodge, V., Austin, J., "A Survey of Outlier Detection Methodologies", Artificial Intelligence Review, pp. 85, 2004.
19. He, Z., Deng, S., Xu, X., "A Fast Greedy algorithm for outlier mining", Proceedings of PAKDD, 2006.
20. Ghemawat, S., Gobioff, H., and Leung, S.-T., "The google file system," In Proceedings of 19th ACM Symposium on Operating Systems Principles, October, 2003.
21. Blake, C., Merz, C. UCI machine learning repository: www.ics.uci.edu/~mlearn/MLRepository.html.
22. Cristofor, D., and Simovici, D., "Finding Median Partitions Using Information-Theoretical Algorithms", Journal of Universal Computer Science, pp. 153-172, 2002 (software at http://www.cs.umb.cdu/~dana/GAClust/ index.html)
23. Hadoop, "Welcome to hadoop!", http://lucene.apache.org/hadoop/, 2007.