A 2-tier clustering algorithm with Map-Reduce

Proceedings - 5th Annual ChinaGrid Conference, ChinaGrid 2010

Volumn , Issue , 2010, Pages 160-166

  Zhang, Jing ^a   Wu, Gongqing ^a   Li, Haiguang ^a   Hu, Xuegang ^a   Wu, Xindong ^a,b  

^a HEFEI UNIVERSITY OF TECHNOLOGY   (China)

^b University of Vermont   (United States)

Author keywords

2 tier clustering; Distributed computing; K means; Map reduce

Indexed keywords

2-TIER CLUSTERING; CLUSTERING MECHANISM; COMPUTING MODEL; DISTANCE-BASED CLUSTERING ALGORITHM; DISTRIBUTED CLUSTERING; DISTRIBUTED COMPUTING; DISTRIBUTED ENVIRONMENTS; DISTRIBUTED PROCESSING; K-MEANS; MAP-REDUCE; NUMBER OF CLUSTERS; OPEN-SOURCE; PARALLEL AND DISTRIBUTED COMPUTING; PARALLEL AND DISTRIBUTED PROGRAMMING; SCALABLE CLUSTERING; TIME PERFORMANCE;

CLUSTER ANALYSIS; PARALLEL PROGRAMMING;

CLUSTERING ALGORITHMS;

EID: 77958181344     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ChinaGrid.2010.14     Document Type: Conference Paper

References (18)

1. U. Fayyad, R. Uthurusamy, "Data Mining and knowledge discovery in databases, " Communications of the ACM, vol. 39, no. 11, pp. 24-26, November 1996.
2. X. Wu, V. Kumar, Ross, J. Ghosh, Q. Yang, H. Motoda, G. Mclachlan, A. Ng, B. Liu, P. Yu, Z.-H. Zhou, M. Steinbach, D. Hand, and D. Steinberg, "Top 10 algorithms in data mining, " Knowledge and Information Systems, vol. 14, no. 1, pp. 1-37, January 2008.
3. S. Papadimitriou, J. Sun, "DisCo: Distributed Co-clustering with Map-Reduce: A Case Study Towards Petabyte-Scale End-to-End Mining, " in: Eighth IEEE International Conference on Data Ming, pp. 512-521, December 2008.
4. K. Cardona, J. Secretan, M. Georgiopoulos, G. Anagnostopoulos, "A Grid Based System for Data Mining Using MapReduce, " Technical Report, University of Puerto Rico, July 2007.
5. E. Januzaj, H. Kriege, M. Pfeifle, "Towards effective and efficient distributed clustering, " in: Proceedings of the 3th IEEE International Conference on Data Mining, pp. 49-58, 2003.
6. W. Ni, G. Chen, Y. Wu, Z. Sun, "Local Density Based Distributed Clustering Algorithm, " Journal of Software, vol. 19, no. 9, pp. 2339-2348, September 2008.
7. G. Ji, X. Ling, M. Yang, "Distributed clustering algorithm based on ensemble learning, " Journal of Southeast University (Natural Science Edition), vol. 37, no. 4, pp. 585-588, July 2007. (Pubitemid 47270617)
8. J. Dean, S. Ghemawat, "MapReduce: Simplified data processing on large clusters, " in: Proceedings of the 6th Symposium on Operating System Design and Implementation. San Francisco, California, USA. USENIX Association, pp. 137-150, December 2004.
9. J. B. MacQueen, "Some methods for classification and analysis of multivariate observations, " in: LeCam LM, Neyman J, eds. Proc. of the 5th Berkeley Symp. On Mathematical Statistics and Probability. Berkeley: University of California Press, pp. 281-297, 1967.
10. P. S. Bradley, U. M. Fayyad, "Refining initial points for K-Means clustering, " in: Proceedings of the IJCAI-93, San Mateo, CA (1983), pp. 1058-1063.
11. C. Moretti, K. Steinhaeuser, D. Thain, N. V. Chawla, "Scaling up classifiers to cloud computers, " in: ICDM'08: Proceedings of the 8th IEEE International Conference on Data Mining, pp. 472-481, 2008.
12. H. Yang, A. Dasdan, R. Hsiao, D. S. Parker, "Mapreduce-merge: simplified relational data processing on large clusters, " in: Proceedings of the 2007 ACM SIGMOD international conference on Management of data, pp. 1029-1040, June 2007.
13. J. Dean, S. Ghemawat, " MapReduce: A flexible data processing tool, " Communications of the ACM, vol. 53, no. 1, pp. 72-77, January 2010.
14. Apache, "Hadoop, " http://lucene.apache.org/hadoop/, 2006.
15. L. Barroso, J. Dean, U. Holzle, "Web search for a planet: The Google cluster architecture, " in: IEEE Micro, vol. 23, no. 2, pp. 22-28, April 2003.
16. D. Borthaku, "The hadoop distributed file system: Architecture and design, " Retrieved from http://lucene.apache.org/hadoop, 2007.
17. G. Wu, H. Li, X. Hu, Y. Bi, J. Zhang, X. Wu, "MReC4.5: C4.5 Ensemble Classification with MapReduce, " in: Proceedings of the 4th ChinaGrid Annual Conference (ChinaGrid'09), pp. 249-255, August 2009.
18. C. Blake, E. Keogh, C. Merz, "UCI Repository of machine learning databases, " Irvine: Department of Information and Computer Science, University of California, 1998.