High performance big data clustering

Advances in Parallel Computing

Volumn 23, Issue , 2013, Pages 192-211

  Agrawal, Ankit ^a   Patwary, Md Mostofa Ali ^a   Hendrix, William ^a   Liao, Wei Keng ^a   Choudhary, Alok ^a  

^a NORTHWESTERN UNIVERSITY   (United States)

Author keywords

big data; clustering; density based clustering; hierarchical clustering

Indexed keywords

EID: 84895107082     PISSN: 09275452     EISSN: None     Source Type: Book Series    
DOI: 10.3233/978-1-61499-322-3-192     Document Type: Article

References (63)

1. Parallel k-means data clustering, 2005. http://users.eecs.northwestern. edu/wkliao/Kmeans/.
2. Cluto-clustering high-dimensional datasets, 2006. http://glaros.dtc.umn. edu/gkhome/cluto/cluto/.
3. Ankit Agrawal, Sanchit Misra, Daniel Honbo, et al. Parallel pairwise statistical significance estimation of local sequence alignment using message passing interface library. Concurrency and Computation: Practice and Experience, 23(17):2269-2279, 2011.
4. R. Agrawal and R. Srikant. Quest synthetic data generator. IBM Almaden Research Center, 1994.
5. D. Arlia and M. Coppola. Experiments in parallel clustering with DBSCAN. In Euro-Par 2001 Parallel Processing, pages 326-331. Springer, LNCS, 2001.
6. N. Beckmann, H.P. Kriegel, R. Schneider, and B. Seeger. The r*-tree: an efficient and robust access method for points and rectangles. Proceedings of the 1990 ACM SIGMOD international conference on Management of data, 19(2):322-331, 1990.
7. J.L. Bentley. Multidimensional binary search trees used for associative searching. Communications of the ACM, 18(9):509-517, 1975.
8. S. Bertone, G. De Lucia, and P.A. Thomas. The recycling of gas and metals in galaxy formation: predictions of a dynamical feedback model. Monthly Notices of the Royal Astronomical Society, 379(3):1143-1154, 2007.
9. D. Birant and A. Kut. ST-DBSCAN: An algorithm for clustering spatial-temporal data. Data & Knowledge Engineering, 60(1):208-221, 2007.
10. RG Bower, AJ Benson, R. Malbon, JC Helly, CS Frenk, CM Baugh, S. Cole, and CG Lacey. Breaking the hierarchy of galaxy formation. Monthly Notices of the Royal Astronomical Society, 370(2):645-655, 2006.
11. S. Brecheisen, H.P. Kriegel, and M. Pfeifle. Parallel density-based clustering of complex objects. Advances in Knowledge Discovery and Data Mining, pages 179-188, 2006.
12. M. Chen, X.D. Gao, and H.F. Li. Parallel DBSCAN with priority r-tree. In Information Management and Engineering (ICIME), 2010 The 2nd IEEE International Conference on, pages 508-511. IEEE, 2010.
13. Jatin Chhugani, Changkyu Kim, Hemant Shukla, Jongsoo Park, Pradeep Dubey, John Shalf, and Horst D. Simon. Billion-particle simd-friendly two-point correlation on large-scale hpc cluster systems. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, SC '12, pages 1:1-1:11, Los Alamitos, CA, USA, 2012. IEEE Computer Society Press.
14. M. Coppola and M. Vanneschi. High-performance data mining with skeleton-based structured parallel programming. Parallel Computing, 28(5):793-813, 2002.
15. T.H. Cormen. Introduction to algorithms. The MIT press, 2001.
16. Manoranjan Dash, Simona Petrutiu, and Peter Scheuermann. Efficient parallel hierarchical clustering. In Marco Danelutto, Marco Vanneschi, and Domenico Laforenza, editors, Euro-Par 2004 Parallel Processing, volume 3149 of Lecture Notes in Computer Science, pages 363-371. Springer Berlin/Heidelberg, 2004.
17. G. De Lucia and J. Blaizot. The hierarchical formation of the brightest cluster galaxies. Monthly Notices of the Royal Astronomical Society, 375(1):2-14, 2007.
18. Zhihua Du and Feng Lin. A novel parallelization approach for hierarchical clustering. Parallel Computing, 31(5):523-527, 2005.
19. M. Ester, H.P. Kriegel, J. Sander, and X. Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the 2nd International Conference on Knowledge Discovery and Data mining, volume 1996, pages 226-231. AAAI Press, 1996.
20. U. Fayyad, G. Piatetsky-Shapiro, and P. Smyth. From data mining to knowledge discovery in databases. AI magazine, 17(3):37, 1996.
21. Y.X. Fu,W.Z. Zhao, and H.F. Ma. Research on parallel DBSCAN algorithm design based on mapreduce. Advanced Materials Research, 301:1133-1138, 2011.
22. B.A. Galler and M.J. Fisher. An improved equivalence algorithm. Communications of the ACM, 7:301-303, 1964.
23. J. Han, M. Kamber, and J. Pei. Data mining: concepts and techniques. Morgan Kaufmann, 2011.
24. William Hendrix, Ali Patwary, Md Mostofa, Ankit Agrawal, Wei-keng Liao, and Alok Choudhary. Parallel hierarchical clustering on shared memory platforms. pages 1-9, 2012.
25. Tony Hey, Stewart Tansley, and Kristin Tolle, editors. The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research, Redmond, Washington, 2009.
26. H. Kargupta and J. Han. Next generation of data mining, volume 7. Chapman & Hall/CRC, 2009.
27. Matthew B. Kennel. KDTREE 2: Fortran 95 and C++ software to efficiently search for near neighbors in a multi-dimensional Euclidean space, 2004. Institute for Nonlinear Science, University of California.
28. S. K. Khaitan, Chuan Fu, and J. D. McCalley. Fast parallelized algorithms for on-line extended-term dynamic cascading analysis. In IEEE PES Power Systems Conference and Exposition, pages 15-18, 2009.
29. S. K. Khaitan and J. D. McCalley. Dynamic load balancing and scheduling for parallel power system dynamic contingency analysis. In High Performance Computing in Power and Energy Systems, pages 189-209. Springer-Verlag Inc., 2012.
30. S. K. Khaitan and J. D. McCalley. High performance computing for power system dynamic simulation. In High Performance Computing in Power and Energy Systems, pages 43-69. Springer-Verlag Inc., 2012.
31. Siddhartha Kumar Khaitan, James D. McCalley, and Arun Somani. Proactive task scheduling and stealing in master-slave based load balancing for parallel contingency analysis. Electric Power Systems Research, 103(0):9-15, 2013.
32. G. Lemson and the Virgo Consortium. Halo and galaxy formation histories from the millennium simulation: Public release of a VO-oriented and SQL-queryable database for studying the evolution of galaxies in the LambdaCDM cosmogony. Arxiv preprint astro-ph/0608019, 2006.
33. Ying Liu, Wei-keng Liao, and Alok Choudhary. Design and evaluation of a parallel HOP clustering algorithm for cosmological simulation. In Proceedings of the 17th International Symposium on Parallel and Distributed Processing (IPDPS 2003), page 82.1, Washington, DC, USA, 2003. IEEE.
34. J. MacQueen et al. Some methods for classification and analysis of multivariate observations. In Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, volume 1, pages 281-297. USA, 1967.
35. S.C. Madeira and A.L. Oliveira. Biclustering algorithms for biological data analysis: a survey. Computational Biology and Bioinformatics, IEEE/ACM Transactions on, 1(1):24-45, 2004.
36. F. Manne and M. Patwary. A scalable parallel union-find algorithm for distributed memory computers. In Parallel Processing and Applied Mathematics, pages 186-195. Springer, LNCS, 2010.
37. Sparsh Mittal, Saket Gupta, and Sudeb Dasgupta. FPGA: An Efficient And Promising Platform For Real-Time Image Processing Applications. In National Conference On Research and Development In Hardware Systems (CSI-RDHS), Kolkata, India, 2008.
38. Sparsh Mittal, Saket Gupta, and Sudeb Dasgupta. System Generator: The State-Of-Art FPGA Design Tool For DSP Applications. In International Innovative Conference On Embedded Systems, Mobile Communication And Computing (ICEMC2 2008). Global Education Center, India, 2008.
39. A. Mukhopadhyay and U. Maulik. Unsupervised satellite image segmentation by combining SA based fuzzy clustering with support vector machine. In Proceedings of 7th ICAPR'09, pages 381-384. IEEE, 2009.
40. Amit Pande and Joseph Zambreno. Embedded Multimedia Security Systems-Algorithms and Architectures. Springer, 2012.
41. Amit Pande and Joseph Zambreno. The secure wavelet transform. J. of Real-Time Image Process, 7:131-142, 2012.
42. Amit Pande, Joseph Zambreno, and Prasant Mohapatra. Architectures for simultaneous coding and encryption using chaotic maps. In ISVLSI, pages 351-352, 2011.
43. H.S. Park and C.H. Jun. A simple and fast algorithm for K-medoids clustering. Expert Systems with Applications, 36(2):3336-3341, 2009.
44. M. Patwary, J. Blair, and F. Manne. Experiments on union-find algorithms for the disjoint-set data structure. In Proceedings of the 9th International Symposium on Experimental Algorithms (SEA 2010), pages 411-423. Springer, LNCS 6049, 2010.
45. M. Patwary, A. Gebremedhin, and A. Pothen. New multithreaded ordering and coloring algorithms for multicore architectures. In Proceedings of 17th International European Conference on Parallel and Distributed Computing (Euro-Par 2011), pages 250-262. Springer, LNCS 6853, 2011.
46. Md Patwary, Mostofa Ali, Diana Palsetia, Ankit Agrawal, Wei-keng Liao, Fredrik Manne, and Alok Choudhary. A new scalable parallel dbscan algorithm using the disjoint-set data structure. pages 1-11, 2012.
47. Md. Mostofa Ali Patwary, Peder Refsnes, and Fredrik Manne. Multi-core spanning forest algorithms using the disjoint-set data structure. volume 0, pages 827-835, Los Alamitos, CA, USA, 2012. IEEE Computer Society.
48. J. Pisharath, Y. Liu, W. Liao, A. Choudhary, G. Memik, and J. Parhi. Nu-minebench 3.0. Technical report, Technical Report CUCIS-2005-08-01, Northwestern University, 2010.
49. M. Raju and S. K. Khaitan. High performance computing using out-of-core sparse direct solvers. Intl. J. of Mathematical, Physical and Engineering Sciences, 3(2).
50. M. Raju and S. K. Khaitan. Domain decomposition based high performance parallel computing. Intl. J. Computer Science Issues, 5:691-704, 2009.
51. M. Raju and S. K. Khaitan. Implementation of shared memory sparse direct solvers for three dimensional finite element codes. J. of Computing, 1(8):699-706, 2010.
52. G. Sheikholeslami, S. Chatterjee, and A. Zhang. WaveCluster: a wavelet-based clustering approach for spatial data in very large databases. The VLDB Journal, 8(3):289-304, 2000.
53. Robert Sibson. Slink: An optimally efficient algorithm for the single-link cluster method. The Computer Journal, 16(1):30-34, 1973.
54. Tyler A. Simon, William A. Ward, Jr., and Alan P. Boss. Performance analysis of intel multiprocessors using astrophysics simulations. Concurr. Comput. : Pract. Exper., 24(2):155-166, February 2012.
55. V. Springel, S.D.M. White, A. Jenkins, C.S. Frenk, N. Yoshida, L. Gao, J. Navarro, R. Thacker, D. Croton, J. Helly, et al. Simulations of the formation, evolution and clustering of galaxies and quasars. Nature, 435(7042):629-636, 2005.
56. M. Surdeanu, J. Turmo, and A. Ageno. A hybrid unsupervised approach for document clustering. In Proceedings of the 11th ACM SIGKDD, pages 685-690. ACM, 2005.
57. R.E. Tarjan. A class of algorithms which require nonlinear time to maintain disjoint sets. Journal of computer and system sciences, 18(2):110-127, 1979.
58. W.Wang, J. Yang, and R. Muntz. STING: A statistical information grid approach to spatial data mining. In Proceedings of the International Conference on Very Large Data Bases, pages 186-195. IEEE, 1997.
59. X. Xu, J. J̈ager, and H.P. Kriegel. A fast parallel clustering algorithm for large spatial databases. High Performance Data Mining, pages 263-290, 2002.
60. Ling Sing Yung, Can Yang, Xiang Wan, and Weichuan Yu. Gboost: a gpu-based tool for detecting genegene interactions in genomewide case control studies. Bioinformatics, 27(9):1309-1310, 2011.
61. T. Zhang, R. Ramakrishnan, and M. Livny. BIRCH: an efficient data clustering method for very large databases. In ACM SIGMOD Record, volume 25(2), pages 103-114. ACM, 1996.
62. Y. Zhang, S. Misra, A. Agrawal, et al. Accelerating pairwise statistical significance estimation for local alignment by harvesting GPU's power. BMC Bioinformatics, 13(Suppl 3):S1, 2012.
63. A. Zhou, S. Zhou, J. Cao, Y. Fan, and Y. Hu. Approaches for scaling DBSCAN algorithm to large spatial databases. Journal of computer science and technology, 15(6):509-526, 2000.