A highly efficient multi-core algorithm for clustering extremely large datasets

BMC Bioinformatics

Volumn 11, Issue , 2010, Pages

  Kraus, Johann M ^a,b   Kestler, Hans A ^a,b  

^a UNIVERSITY OF ULM   (Germany)

^b UNIVERSITY HOSPITAL ULM   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

CLUSTER NUMBER ESTIMATION; COMPUTATIONAL ACCURACY; COMPUTATIONAL BIOLOGY; DUAL CORE-PROCESSORS; GENE EXPRESSION MICROARRAY; HIGH-THROUGHPUT TECHNOLOGIES; MULTI-CORE ALGORITHM; SHARED-MEMORY PARALLELS;

BIOINFORMATICS; GENE EXPRESSION; PERSONAL COMPUTERS;

CLUSTERING ALGORITHMS;

ALGORITHM; ARTICLE; CLUSTER ANALYSIS; DNA MICROARRAY; FACTUAL DATABASE; GENE EXPRESSION; GENE EXPRESSION PROFILING; METHODOLOGY; SINGLE NUCLEOTIDE POLYMORPHISM;

ALGORITHMS; CLUSTER ANALYSIS; DATABASES, FACTUAL; GENE EXPRESSION; GENE EXPRESSION PROFILING; OLIGONUCLEOTIDE ARRAY SEQUENCE ANALYSIS; POLYMORPHISM, SINGLE NUCLEOTIDE;

EID: 77950541370     PISSN: None     EISSN: 14712105     Source Type: Journal    
DOI: 10.1186/1471-2105-11-169     Document Type: Article

References (57)

1. Andreopoulos B, An A, Wang X, Schroeder M. A roadmap of clustering algorithms: finding a match for a biomedical application. Brief Bioinformatics 2009, 10(3):297-314. 10.1093/bib/bbn058, 19240124., http://bib.oxfordjournals.org/cgi/content/full/10/3/297
2. Ng M, Li M, Ao S, Sham P, Cheung Y, Huang J. Clustering of SNP Data with Application to Genomics. Proceedings of the Sixth IEEE International Conference on Data Mining 2006, 158-162. Los Alamitos: IEEE Computer Society.
3. Sham P, Ao S, Kwan J, Kao P, Cheung F, Fong P, Ng M. Combining functional and linkage disequilibrium information in the selection of tag SNPs. Bioinformatics 2007, 23:129-131. 10.1093/bioinformatics/btl532, 17060359.
4. Feng W, Balaji P. Tools and Environments for Multicore and Many-Core Architectures. IEEE Computer 2009, 42(12):26-27.
5. Gropp W, Lusk E, Skjellum A. Using MPI: Portable Parallel Programming with the Message Passing Interface 1999, Cambridge: MIT Press.
6. Kraj P, Sharma A, Garge N, Podolsky R, McIndoe RA. ParaKmeans: Implementation of a parallelized K-means algorithm suitable for general laboratory use. BMC Bioinformatics 2008, 9(200). 2375128, 18416829.
7. R Development Core Team R: A Language and Environment for Statistical Computing 2009, R Foundation for Statistical Computing, Vienna, R Development Core Team., http://www.R-project.org
8. Gentleman RC, Carey VJ, Bates DM, Bolstad B, Dettling M, Dudoit S, Ellis B, Gautier L, Ge Y, Gentry J, Hornik K, Hothorn T, Huber W, Iacus S, Irizarry R, Leisch F, Li C, Maechler M, Rossini AJ, Sawitzki G, Smith C, Smyth G, Tierney L, Yang JYH, Zhang J. Bioconductor: Open software development for computational biology and bioinformatics. Genome Biology 2004, 5:R80. 10.1186/gb-2004-5-10-r80, 545600, 15461798.
9. Hill J, Hambley M, Forster T, Mewissen M, Sloan TM, Scharinger F, Trew A, Ghazal P. SPRINT: A new parallel framework for R. BMC Bioinformatics 2008, 9(558). 2628907, 19114001.
10. Chapman B, Jost G, Pas R. Using OpenMP: Portable Shared Memory Parallel 'Programming 2007, Cambridge: MIT Press.
11. Duan R, Strey A. Data Mining Algorithms on the Cell Broadband Engine. Euro-Par 2008 2008, 665-675. Berlin: Springer-Verlag, Luque E, Margalef T, Benitez D.
12. Lea D. Concurrent Programming in Java: Design Principles and Patterns 2000, Boston: Addison Wesley, 2.
13. Peyton-Jones S. Beautiful concurrency. Beautiful code 2007, Sebastopol: O'Reilly, Oram A, Wilson G.
14. Adl-Tabatabai AR, Kozyrakis C, Saha B. Unlocking Concurrency. ACM Queue 2006, 4(10):24-33.
15. Jain A, Dubes R. Algorithms for Clustering Data 1988, New Jersey: Prentice Hall.
16. MacQueen J. Some methods for classification and analysis of multivariate observations. Proceedings of the 5th Berkeley Symposium on Math, Statistics and Probability 1967, 1:281-297. Berkely: University of California Press, Neyman J, Cam LL.
17. Gao X, Starmer J. Human population structure detection via multilocus genotype clustering. BMC Genetetics 2007, 8:34.
18. Edwards AWF. Human genetic diversity: Lewontin's fallacy. Bioessays 2003, 25(8):798-801. 10.1002/bies.10315, 12879450.
19. Gao X, Martin ER. Using allele sharing distance for detecting human population stratification. Human Heredity 2009, 68(3):182-91. 10.1159/000224638, 19521100.
20. Xu R, Wunsch DC. Clustering 2009, IEEE Press Series on Computational Intelligence, New York: Wiley.
21. Anderson M. Sun's Rock CPU Could Be a Gem for Oracle. IEEE Spectrum 2009, June:10-11.
22. Drepper U. Parallel Programming with Transactional Memory. ACM Queue 2008, 6(5):38-45.
23. Shavit N, Touitou D. Software Transactional Memory. Proceedings of the 14th ACM Symposium on Principles of Distributed Computing 1995, 204-213. New York: ACM Press.
24. Rajwar R, Goodman J. Transactional Execution: Toward Reliable, High-Performance Multithreading. IEEE Micro 2003, 23(6):117-125.
25. Larus J, Kozyrakis C. Transactional Memory. Communications of the ACM 2008, 51(7):80-88.
26. Bernstein PA, Newcomer E. Principles of transaction processing 2009, Burlington, MA: Morgan Kaufmann.
27. Herzeel C, Costanza P, D'Hondt T. Reusable Building Blocks for Software Transactional Memory. Proceedings of the 2nd European Lisp Symposium, Milan 2009,
28. Handl J, Knowles J, Kell D. Computational cluster validation in post-genomic data analysis. Bioinformatics 2005, 21(15):3201-3212. 10.1093/bioinformatics/bti517, 15914541.
29. Dalgin G, Alexe G, Scanfeld D, Tamayo P, Mesirov J, Ganesan S, DeLisi C, Bhanot G. Portraits of breast cancer progression. BMC Bioinformatics 2007, 8:291. 10.1186/1471-2105-8-291, 1978212, 17683614.
30. Kuncheva L. Combining Pattern Classifiers: Methods and Algorithms 2004, New York: Wiley.
31. Jain AK, Moreau JV. Bootstrap Technique In Cluster Analysis. Pattern Recognition 1987, 20(5):547-568.
32. Kestler HA, Müller A, Schwenker F, Gress T, Mattfeldt T, Palm G. Cluster analysis of comparative genomic hybridization data. Lecture Notes NATO ASI: Aritificial Intelligence and Heuristic Methods for Bioinformatics 2001, S-40. [Abstract].
33. Lange T, Roth V, Braun ML, Buhmann JM. Stability-Based Validation of Clustering Solutions. Neural Computation 2004, 16(6):1299-1323. 10.1162/089976604773717621, 15130251.
34. Bertoni A, Valentini G. Random projections for assessing gene expression cluster stability. Proceedings of the IEEE-International Joint Conference on Neural Networks (IJCNN) 2005, 1:149-154. IEEE Computer Society.
35. Smolkin M, Ghosh D. Cluster stability scores for microarray data in cancer studies. BMC Bioinformatics 2003, 4(36). 200969, 12959646.
36. Rand W. Objective criteria for the evaluation of clustering methods. Journal of the American Statistical Association 1971, 66:846-850.
37. Jaccard P. Nouvelles recherches sur la distribution florale. Bulletin de la Société Vaudoise des sciences naturelles 1908, 44:223-270.
38. Kuhn HW. The Hungarian Method for the assignment problem. Naval Research Logistics Quarterly 1955, 2(1-2):83-97.
39. Jonker R, Volgenant A. A shortest augmenting path algorithm for dense and sparse linear assignment problems. Computing 1987, 38(4):325-340.
40. Conover WJ. Practical Nonparametric Statistics 1999, New York: Wiley, 2.
41. Hubert L, Arabie P. Comparing Partitions. Journal of Mathematical Classification 1985, 2:193-218.
42. Graham R, Knuth D, Patashnik O. Concrete Mathematics 1994, Boston: Addison Wesley, 2.
43. Levine E, Domany E. Resampling Method for Unsupervised Estimation of Cluster Validity. Neural Computation 2001, 13(11):2573-2593. 10.1162/089976601753196030, 11674852.
44. Fridlyand J, Dudoit S. Applications of resampling methods to estimate the number of clusters and to improve the accuracy of a clustering method. Technical report 600 2001, University of California, Berkeley.
45. Kerr MK, Churchill GA. Bootstrapping cluster analysis: assessing the reliability of conclusions from microarray experiments. Proceedings of the National Academy of Sciences 2001, 98(16):8961-8965.
46. Ben-Hur A, Elisseeff A, Guyon I. A stability based method for discovering structure in clustered data. Pac Symp Biocomput 2002, 6-17.
47. Kestler HA, Müller A, Buchholz M, Palm G, Gress TM. Robustness evaluation of clusterings. Currents in Computational Molecular Biology 2003 2003, 253-254. Spang R, Béziat P, Vingron M, [Abstract].
48. Selim S, Ismail M. K-means-type algorithms: A generalized convergence theorem and characterization of local optimality. IEEE Transactions on Pattern Analysis and Machine Intelligence 1984, 6:81-87.
49. Barr DR, Slezak NL. A Comparison of Multivariate Normal Generators. Communications of the ACM 1972, 15(12):1048-1049.
50. Fukunaga K. Introduction to Statistical Pattern Recognition 1990, Academic Press, 2.
51. Smirnov D, Morley M, Shin E, Spielman R, Cheung V. Genetic analysis of radiation-induced changes in human gene expression. Nature 2009, 459:587-591. 10.1038/nature07940, 19349959.
52. Consortium IH The International HapMap Project. Nature 2003, 426(6968):789-96. 10.1038/nature02168, 14685227, Consortium IH.
53. Tibshirani R, Walther G, Hastie T. Estimating the number of clusters in a data set via the gap statistic. Journal of the Royal Statistical Society (Series B) 2001, 63(2):411-423.
54. Monti S, Tamayo P, Mesirov J, Golub T. Consensus clustering: A resampling-based method for class discovery and visualization of gene expression microarray data. Machine Learning 2003, 52(1-2):91-118.
55. Müller F, Laurent L, Kostka D, Ulitsky I, Williams R, Lu C, Park I, Rao M, Shamir R, Schwartz P, Schmidt N, Loring J. Regulatory networks define phenotypic classes of human stem cell lines. Nature 2008, 455(7211):401-405. 10.1038/nature07213, 2637443, 18724358.
56. Sammon JW. A nonlinear mapping for data structure analysis. IEEE Transactions on Computers 1969, 18(5):401-409.
57. Kohonen T. Self-Organizing Maps, of Springer Series in Information Sciences 2001, 30. Berlin: Springer, 3.