Clustering based on pairwise distances when the data is of mixed dimensions

IEEE Transactions on Information Theory

Volumn 57, Issue 3, 2011, Pages 1692-1706

  Arias Castro, Ery ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Clustering; detection in point clouds; extracting connected components; hierarchical clustering with single linkage; manifold learning; minimax rates; nearest neighbor search; neighborhood graphs; random geometric graphs; spectral methods

Indexed keywords

CLUSTERING; DETECTION IN POINT CLOUDS; EXTRACTING CONNECTED COMPONENTS; HIER-ARCHICAL CLUSTERING; MANIFOLD LEARNING; MINIMAX; NEAREST NEIGHBOR SEARCH; NEIGHBORHOOD GRAPHS; RANDOM GEOMETRIC GRAPHS; SPECTRAL METHODS;

CLOUDS; OPTIMIZATION; SPECTROSCOPY;

GRAPHIC METHODS;

EID: 79951912473     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2011.2104630     Document Type: Article

References (67)

1. A. Y. Ng, M. I. Jordan, and Y. Weiss, "On spectral clustering: Analysis and an algorithm," in Advances in Neural Information Processing Systems 16. Cambridge, MA: MIT Press, 2002.
2. M. Maier, M. Hein, and U. von Luxburg, "Optimal construction of k-nearest-neighbor graphs for identifying noisy clusters," Theor. Comput. Sci., vol. 410, no. 19, pp. 1749-1764, 2009.
3. M. R. Brito, E. L. Chávez, A. J. Quiroz, and J. E. Yukich, "Connectivity of the mutual k-nearest-neighbor graph in clustering and outlier detection," Stat. (Pubitemid 127161265)
4. D. Pollard, "Strong consistency of -means clustering," Ann. Statist., vol. 9, no. 1, pp. 135-140, 1981.
5. S. Dasgupta, "Learning mixtures of gaussians," in Proc. Foundations Comput. Science 40th Annu. Symp., 1999, pp. 634-644. (Pubitemid 30539944)
6. S. Vempala and G. Wang, "A spectral algorithm for learning mixture models," J. Comput. Syst. Sci., vol. 68, no. 4, pp. 841-860, 2004.
7. S. Dasgupta and P. M. Long, "Performance guarantees for hierarchical clustering," J. Comput. Syst. Sci., vol. 70, no. 4, pp. 555-569, 2005. (Pubitemid 40537244)
8. D. Achlioptas and F. McSherry, "On spectral learning of mixtures of distributions," in Learning Theory, ser. Lecture Notes in Computer Science. Berlin/Heidelberg, Germany: Springer, 2005, vol. 3559, pp. 458-469.
9. R. Valdarnini, "Detection of non-random patterns in cosmological gravitational clustering," Astron. Astrophys., vol. 366, pp. 376-386, 2001. (Pubitemid 32960342)
10. J. D. Barrow, S. P. Bhavsar, and D. H. Sonoda, "Minimal spanning trees, filaments and galaxy clustering," MNRAS, vol. 216, pp. 17-35, 1985.
11. S. P. Bhavsar and J. D. Barrow, "Percolation analyses of observed and simulated galaxy clustering," Roy. Astron. Soc., Monthly Notices, vol. 205, pp. 61-161, 1983.
12. A. Klypin and S. Shandarin, "Percolation technique for galaxy clustering," Astrophys. J., vol. 413, no. 1, pp. 48-58, 1993.
13. Y. Ma, A. Y. Yang, H. Derksen, and R. Fossum, "Estimation of subspace arrangements with applications in modeling and segmenting mixed data," SIAM Rev., vol. 50, no. 3, pp. 413-458, 2008.
14. Z. Fu, W. Hu, and T. Tan, "Similarity based vehicle trajectory clustering and anomaly detection," in Proc. IEEE Int. Conf. Image Processing, 11-14, 2005, vol. 2, pp. II-602-5-II-602-5.
15. R. Vidal and Y. Ma, "A unified algebraic approach to 2-D and 3-D motion segmentation and estimation," J Math. Imag. Vis., vol. 25, no. 3, pp. 403-421, 2006. (Pubitemid 44728465)
16. G. Chen, S. Atev, and G. Lerman, "Kernel spectral curvature clustering (KSCC)," presented at the 4th Int. Workshop Dynamical Vision (ICCV'09), 2009.
17. J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science, vol. 290, no. 5500, pp. 2319-2323, 2000. (Pubitemid 32041577)
18. S. Roweis and L. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol. 290, no. 5500, pp. 2323-2326, 2000. (Pubitemid 32041578)
19. M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Comput., vol. 15, pp. 1373-1396, 2003. (Pubitemid 37049796)
20. M. Filippone, F. Camastra, F. Masulli, and S. Rovetta, "A survey of kernel and spectral methods for clustering," Pattern Recognit., vol. 41, no. 1, pp. 176-190, Jan. 2008. (Pubitemid 47284212)
21. E. Mammen and A. B. Tsybakov, "Asymptotical minimax recovery of sets with smooth boundaries," Ann. Statist., vol. 23, no. 2, pp. 502-524, Apr. 1995.
22. U. Luxburg, "A tutorial on spectral clustering," Stat. Comput., vol. 17, no. 4, pp. 395-416, Sep. 10, 2007.
23. E. Arias-Castro, D. L. Donoho, X. Huo, and C. A. Tovey, "Connect the dots: How many random points can a regular curve pass through?," Adv. Appl. Probab., vol. 37, no. 3, pp. 571-603, 2005. (Pubitemid 41488720)
24. E. Arias-Castro, "Graphical structures for geometric detection," Ph.D. dissertation, Stanford Univ., Stanford, CA, 2004.
25. L. Zelnik-Manor and P. Perona, "Self-tuning spectral clustering," Adv. Neural Inf. Process. Syst. 17, pp. 1601-1608, 2004.
26. M. Penrose, Random Geometric Graphs, ser. Oxford Studies in Probability. Oxford, U.K.: Oxford Univ. Press, 2003, vol. 5.
27. M. Maier, M. Hein, and U. Von Luxburg, "Cluster identification in nearest-neighbor graphs," in Algorithmic Learning Theory. New York: Springer, 2007, pp. 196-210.
28. G. Biau, B. Cadre, and B. Pelletier, "A graph-based estimator of the number of clusters," ESAIM Probab. Stat., vol. 11, pp. 272-280, 2007. (Pubitemid 47177024)
29. R. Kannan, S. Vempala, and A. Vetta, "On clusterings: Good, bad and spectral," J. ACM, vol. 51, no. 3, pp. 497-515, 2004.
30. U. von Luxburg, M. Belkin, and O. Bousquet, "Consistency of spectral clustering," Ann. Statist., vol. 36, no. 2, pp. 555-586, 2008.
31. B. Pelletier and P. Pudlo, Strong consistency of spectral clustering on level sets, 2008, Tech. Rep..
32. H. Narayanan, M. Belkin, and P. Niyogi, , B. Schölkopf, J. Platt, and T. Hoffman, Eds., "On the relation between low density separation, spectral clustering and graph cuts," in Advances in Neural Information Processing Systems 19. Cambridge, MA: MIT Press, 2007.
33. B. Fischer, V. Roth, and J. M. Buhmann, , S. Thrun, L. Saul, and B. Schölkopf, Eds., "Clustering with the connectivity kernel," in Advances in Neural Information Processing Systems 16. Cambridge, MA:MIT Press, 2004.
34. G. Chen and G. Lerman, "Foundations of a multi-way spectral clustering framework for hybrid linear modeling," Found. Comput. Math., vol. 9, no. 5, pp. 517-558, 2009.
35. Y. Yang and A. Barron, "Information-theoretic determination of minimax rates of convergence," Ann. Statist., vol. 27, no. 5, pp. 1564-1599, 1999.
36. D. L. Donoho and I. M. Johnstone, "Minimax estimation via wavelet shrinkage," Ann. Statist., vol. 26, no. 3, pp. 879-921, 1998.
37. A. P. Korostelev and A. B. Tsybakov, Minimax Theory of Image Reconstruction, ser. Lecture Notes in Statistics. New York: Springer- Verlag, 1993, vol. 82.
38. L. Birgé, "Estimating a density under order restrictions: Nonasymptotic minimax risk," Ann. Statist., vol. 15, no. 3, pp. 995-1012, 1987.
39. Y. Yang, "Minimax nonparametric classification. I. rates of convergence," IEEE Trans. Inf. Theory, vol. 45, pp. 2271-2284, 1999. (Pubitemid 30514188)
40. C. Scott and R. D. Nowak, "Minimax-optimal classification with dyadic decision trees," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1335-1353, Apr. 2006.
41. M. D. Penrose, "Single linkage clustering and continuum percolation," J. Multivariate Anal., vol. 53, no. 1, pp. 94-109, 1995.
42. F. R. K. Chung, Spectral Graph Theory, ser. Regional Conference Series in Mathematics. Providence, RI: Amer. Math. Soc., 1997.
43. G. Haro, G. Randall, and G. Sapiro, "Stratification learning: Detecting mixed density and dimensionality in high dimensional point clouds," Adv. Neural Inf. Process. Syst., vol. 19, pp. 553-1553, 2007.
44. E. Levina and P. Bickel, "Maximum likelihood estimation of intrinsic dimension," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2005, vol. 17, pp. 777-784.
45. G. Chen and G. Lerman, "Spectral curvature clustering (SCC)," Int. J. Comput. Vis., vol. 81, no. 3, pp. 317-330, 2009.
46. C. Ma and C.-H. Lee, "Unsupervised anchor shot detection using multimodal spectral clustering," in Proc. IEEE Int. Conf. Acoustics, Speech Signal Processing, Apr. 31, 2008, pp. 813-816.
47. Y. Xu, V. Olman, and D. Xu, "Clustering gene expression data using a graph-theoretic approach: An application of minimum spanning trees," Bioinformatics (Oxford), vol. 18, no. 4, pp. 536-545, Apr. 1, 2002. (Pubitemid 34521043)
48. A. Ben-Dor and Z. Yakhini, "Clustering gene expression patterns," in Proc. 3rd Annu. Int. Conf. Computational Molecular Biol., New York, 1999, pp. 33-42.
49. B. Larsen and C. Aone, "Fast and effective text mining using lineartime document clustering," in Proc. 5th Int. Conf. ACM SIGKDD, New York, 1999, pp. 16-22.
50. F. Beil, M. Ester, and X. Xu, "Frequent term-based text clustering," in Proc. 8th ACM Int. Conf. SIGKDD, New York, 2002, pp. 436-442.
51. J. Ho, M. Yang, J. Lim, K. Lee, and D. Kriegman, "Clustering appearances of objects under varying illumination conditions," in Proc. Int. Conf. Comput. Vision Pattern Recognition, 2003, vol. 1, pp. 11-18.
52. G. Biau, L. Devroye, and G. Lugosi, "On the performance of clustering in hilbert spaces," IEEE Trans. Inf. Theory, vol. 54, pp. 781-790, 2008. (Pubitemid 351314665)
53. S. Ghosal, J. K. Ghosh, and R. V. Ramamoorthi, "Non-informative priors via sieves and packing numbers," in Advances in Statistical Decision Theory and Applications, ser. Stat. Ind. Technol.. Boston, MA: Birkhäuser, 1997, pp. 119-132.
54. A. Beygelzimer, S. Kakade, and J. Langford, "Cover trees for nearest neighbor," in Proc. 23rd Int. Conf. Mach. Learning, New York, 2006, pp. 97-104.
55. B. Chazelle, D. Liu, and A. Magen, "Approximate range searching in higher dimension," Comput. Geom. Theor. Appl., vol. 39, no. 1, pp. 24-29, 2008. (Pubitemid 47393372)
56. J. K. Cullum and R. A. Willoughby, Lanczos Algorithms For Large Symmetric Eigenvalue Computations. Vol. 1, ser. Classics in Applied Mathematics. Philadelphia, PA: SIAM, 2002, vol. 41.
57. Y. Zhao, Y. Zhao, and G. Karypis, "Hierarchical clustering algorithms for document datasets," in Data Min Knowl Disc. Reading, MA: ACM Press, 2002, pp. 515-524.
58. D. Kushnir, M. Galun, and A. Brandt, "Fast multiscale clustering and manifold identification," Pattern Recognit., vol. 39, no. 10, pp. 1876-1891, 2006. (Pubitemid 43947093)
59. R. Souvenir and R. Pless, "Manifold clustering," in Proc. 10th IEEE Int. Conf. Comput. Vision, 2005, vol. 1, pp. 648-653. (Pubitemid 44055017)
60. M. Talagrand, The Generic Chaining, ser. Springer Monographs in Mathematics. Berlin, Germany: Springer-Verlag, 2005.
61. E. L. Lehmann and J. P. Romano, Testing Statistical Hypotheses, ser. Springer Texts in Statistics, 3rd ed. New York: Springer, 2005.
62. d," Adv. Appl. Probab., vol. 29, no. 3, pp. 582-594, 1997. (Pubitemid 127539020)
63. G. W. Stewart and J. G. Sun, Matrix Perturbation Theory, ser. Computer Science and Scientific Computing. Boston, MA: Academic, 1990.
64. C. Davis and W. M. Kahan, "The rotation of eigenvectors by a perturbation. III," SIAM J. Numer. Anal., vol. 7, pp. 1-46, 1970.
65. E. Arias-Castro, E. J. Candès, H. Helgason, and O. Zeitouni, "Searching for a trail of evidence in a maze," Ann. Statist., vol. 36, no. 4, pp. 1726-1757, 2008.
66. G. H. Golub and C. F. Van Loan, Matrix Computations, ser. Johns Hopkins Studies in the Mathematical Sciences, 3rd ed. Baltimore, MD: Johns Hopkins Univ. Press, 1996.
67. A. Edelman, T. A. Arias, and S. T. Smith, "The geometry of algorithms with orthogonality constraints," SIAM J. Matrix Anal. Appl., vol. 20, no. 2, pp. 303-353, 1999. (Pubitemid 129333771)