Consistency of spectral clustering

Annals of Statistics

Volumn 36, Issue 2, 2008, Pages 555-586

  Von Luxburg, Ulrike ^a   Belkin, Mikhail ^b,d   Bousquet, Olivier ^c,e  

^a MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

^b OHIO STATE UNIVERSITY   (United States)

^c Pertinence ^*

^d The Ohio State University   (United States)

^e Pertinence   (France)

Author keywords

Consistency; Convergence of eigen vectors; Graph Laplacian; Spectral clustering

Indexed keywords

EID: 51049104834     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/009053607000000640     Document Type: Article

References (51)

1. ABDULLAEV, Z. and LAKAEV, S. (1991). On the spectral properties of the matrix-valued Friedrichs model. In Many-Particle Hamiltonians: Spectra and Scattering. Adv. Soviet Math. 5 1-37. Amer. Math. Soc., Providence, RI, MR 1130183
2. ALPERT, C. J. and YAO, S.-Z. (1995). Spectral partitioning: The more eigenvectors, the better. In Proceedings of the 32nd ACM/IEEE Conference on Design Automation 195-200. ACM Press, New York.
3. ANSELONE, P. (1971). Collectively Compact Operator Approximation Theory. Prentice-Hall, Englewood Cliffs, NJ. MR0443383
4. ANTHONY, M. (2002). Uniform Glivenko - Cantelli theorems and concentration of measure in the mathematical modelling of learning. Research Report LSE-CDAM-2002-07.
5. ATKINSON, K. (1967). The numerical solution of the eigenvalue problem for compact integral operators. Trans. Amer. Math. Soc. 129 458-465. MR0220105
6. BAI, Z. D. (1999). Methodologies in spectral analysis of large dimensional random matrices. Statist. Sinica 9 611-677. MR1711663
7. BAKER, C. ( 1977). The Numerical Treatment of Integral Equations. Clarendon Press, Oxford. MR0467215
8. BARNARD, S., POTHEN, A. and SIMON, H. (1995). A spectral algorithm for envelope reduction of sparse matrices. Numer. Linear Algebra Appl. 2 317-334, MR 1349276
9. BARTLETT, P., LINDER, T. and LUGOSI, G. (1998). The minimax distortion redundancy in empirical quantizer design. IEEE Trans. Inform. Theory 44 1802-1813. MR1664098
10. BELKIN. M. and NIYOGI, P. (2003). Laplacian eigenmaps for dimensionality reduction and data representation. Neural Comput.151373-1396. MR0203473
11. BEN-DAVID, S. (2004). A framework for statistical clustering with constant time approximation algorithms for K-median clustering. In Proceedings of the 17th Annual Conference on Learning Theory (COLT) (J. Shawe-Taylor and Y. Singer, eds.) 415-426. Springer, Berlin, MR2177925
12. BENGIO, Y., VINCENT, P., PAIEMENT, J.-F., DELALLEAU, O. M. OUIMET, and LE ROUX, N. (2004). Learning eigenfunctions links spectral embedding and kernel PCA. Neural Comput. 16 2197-2219.
13. CHATELIN, F. (1983). Spectral Approximation of Linear Operators. Academic Press, New York. MR0716134
14. CHUNG, F. (1997). Spectral Graph Theory. Conference Board of the Mathematical Sciences, Washington. MR 1421568
15. DHILLON, I. (2001). Co-clustering documents and words using bipartite spectral graph partitioning. In Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) 269-274. ACM Press, New York.
16. DING, C., HE, X., ZHA, H., GU, M. and SIMON, H. (2001). A min-max cut algorithm for graph partitioning and data clustering. In Proceedings of the First IEEE International Conference on Data Mining (ICDM) 107-114. IEEE Computer Society, Washington, DC.
17. DONATH, W. E. and HOFFMAN, A. J. (1973). Lower bounds for the partitioning of graphs. IBM J. Res. Develop. 17 420-425. MR0329965
18. DUDLEY, R. M. (1999). Uniform Central Limit Theorems. Cambridge Univ. Press. MR 1720712
19. FIEDLER, M. (1973). Algebraic connectivity of graphs. Czechoslovak Math. J. 23 298-305. MR0318007
20. GUATTERY, S. and MILLER, G. (1998). On the quality of spectral separators. SIAM J. Matrix Anal. Appl. 19 701-719. MR1611179
21. HAGEN, L. and KAHNG, A. (1992). New spectral methods for ratio cut partitioning and clustering. IEEE Trans. Computer-Aided Design 11 1074-1085.
22. HARTIGAN, J. ( 1981). Consistency of single linkage for high-density clusters. J. Amer. Statist. Assoc. 76 388-394. MR0624340
23. HASTIE, T., TIBSHIRANI, R. and FRIEDMAN, J. (2001). The Elements of Statistical Learning. Springer, New York. MR 1851606
24. HENDRICKSON, B. and LELAND, R. (1995). An improved spectral graph partitioning algorithm for mapping parallel computations. SIAM J. Sci. Comput. 16 452-469. MR 1317066
25. HIGHAM, D. and KIBBLE, M. (2004). A unified view of spectral clustering. Mathematics Research Report 2, Univ. Strathclyde.
26. IKROMOV, I. and SHARIPOV, F. (1998). On the discrete spectrum of the nonanalytic matrix-valued Friedrichs model. Funct. Anal. Appl 32 49-50. Available at http://www.arxiv.org/ pdf/funct-an/9502004, MR 1627263
27. JAIN, A. K., MURTY. M. N. and FLYNN, P. J. (1999). Data clustering: A review. ACM Comput. Surv. 31 264-323.
28. KANNAN, R., VEMPALA, S. and VETTA. A. (2000). On clusterings - good, bad and spectral. In 41st Annual Symposium on Foundations Of Computer Science (Redondo Beach, CA, 2000) 367-377. IEEE Comput. Soc. Press, Los Alamitos, CA. MR 1931834
29. KATO, T. (1966). Perturbation Theory for Linear Operators. Springer, Berlin.
30. KOLTCHINSKII, V. (1998). Asymptotics of spectral projections of some random matrices approximating integral operators. In Progr. Probab. 43 191-227. Birkhäuser. Basel. MR 1652327
31. KOLTCHINSKII, V. and GINÉ, E. (2000). Random matrix approximation of spectra of integral operators. Bernoulli 6 113-167. MR1781185
32. LAKAEV, S. N. (1979). The discrete spectrum of a generalized Friedrichs model. Dokl. Akad. Nauk UzSSR 4 9-10. MR0539360
33. MEILA, M. and SHI, J. (2001). A random walks view of spectral segmentation. In 8th International Workshop on Artificial Intelligence and Statistics (AISTATS).
34. MENDELSON, S. (2003). A few notes on statistical learning theory. Advanced Lectures in Machine Learning. Lecture Notes in Comput. Sci. 2600 1-40. Springer, Berlin.
35. MOHAR, B. (1991). The Laplacian spectrum of graphs. In Graph Theory, Combinatorics, and Applications 2 (Kalamazoo, MI, 1988) 871-898. Wiley, New York. MR1170831
36. NG, A., JORDAN, M. and WEISS, Y. (2002). On spectral clustering: Analysis and an algorithm. In Advances in Neural Information Processing Systems (T. Dietterich, S. Becker and Z. Ghahramani, eds.) 14 849-856. MIT Press.
37. NIYOGI, P. and KARMARKAR, N. K. (2000). An approach to data reduction and clustering with theoretical guarantees. In Proceedings of the Seventeenth International Conference on Machine Learning (P. Langley, ed.) 679-686. Morgan Kaufmann, San Francisco, CA.
38. POLLARD, D. (1981). Strong consistency of k-means clustering. Ann. Statist. 9 135-140. MR0600539
39. POLLARD, D. ( 1984). Convergence of Stochastic Processes. Springer. New York. MR0762984
40. POTHEN, A., SIMON, H. D. and LIOU. K. P. (1990). Partitioning sparse matrices with eigenvectors of graphs. SIAM J. Matrix Anal. Appl. 11 430-452. MR 1054210
41. REED, M. and SIMON, B. (1980). Functional Analysis. I, 2nd ed. Academic Press, New York. MR0751959
42. SHAWE-TAYLOR, J., WILLIAMS. C., CRISTIANINI, N. and KANDOLA. J. (2002). On the eigenspectrum of the Gram matrix and its relationship to the operator eigenspectrum. In Proceedings of the 13th International Conference on Algorithmic Learning Theory (N. Cesa-Bianchi, M. Numao and R. Reischuk. eds.) 23-40. Springer, Berlin. MR2071605
43. SHI, J. and MALIK. J. (2000). Normalized cuts and image segmentation. IEEE Trans. Pattern Analysis and Machine Intelligence 22 888-905.
44. SPIELMAN, D. and TENG, S. (1996). Spectral partitioning works: planar graphs and finite element meshes. In 37th Annual Symposium on Foundations of Computer Science (Burling-ton, VT, 1996) 96-105. IEEE Comput. Soc. Press. Los Alamitos, CA. MR1450607
45. VAN DER VAART, A. W. and WELLNER, J. A. (1996). Weak Convergence and Empirical Processes. Springer, New York. MR 1385671
46. VAN DRIESSCHE, R. and ROOSE, D. (1995). An improved spectral bisection algorithm and its application to dynamic load balancing. Parallel Comput. 21 29-48. MR 1314376
47. VON LIJXBURG, U. (2004). Statistical learning with similarity and dissimilarity functions. Ph.D. thesis, Technical Univ. Berlin.
48. VON LUXBURG, U. (2007). A tutorial on spectral clustering. Stat. Comput. 17. To appear.
49. WEISS, Y. (1999). Segmentation using eigenvectors: A unifying view. In Proceedings of the International Conference on Computer Vision 975-982.
50. WILLIAMS, C. K. I. and SEEGER, M. (2000). The effect of the input density distribution on kernel-based classifiers. In Proceedings of the 17th International Conference on Machine Learning (P. Langley, ed.) 1159-1166. Morgan Kaufmann, San Francisco.
51. ZHOU, D.-X. (2002). The covering number in learning theory. J. Complexity 18 739-767. MR 1928805