Data spectroscopy: Eigenspaces of convolution operators and clustering

Annals of Statistics

Volumn 37, Issue 6 B, 2009, Pages 3960-3984

  Shi, Tao ^a   Belkin, Mikhail ^b   Yu, Bin ^c  

^a OHIO STATE UNIVERSITY   (United States)

^b Ohio State University   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Gaussian kernel; Kernel principal component analysis; Spectral clustering; Support vector machines; Unsupervised learning

Indexed keywords

EID: 73949117460     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/09-AOS700     Document Type: Article

References (22)

1. BELKIN, M. and NIYOGI, P. (2003). Using manifold structure for partially labeled classification. In Advances in Neural Information Processing Systems (S. Becker, S. Thrun and K. Obermayer, eds.) 15 953-960. MIT Press, Cambridge, MA.
2. DHILLON, I., GUAN, Y. and KULIS, B. (2005). A unified view of kernel k-means, spectral clustering, and graph partitioning. Technical Report UTCS TF-04-25, Univ. Texas, Austin.
3. DIACONIS, P., GOEL, S. and HOLMES, S. (2008). Horseshoes in multidimensional scaling and kernel methods. Ann. Appl. Stat. 2 777-807.
4. KOLTCHINSKII, V. and GINÉ, E. (2000). Random matrix approximation of spectra of integral operators. Bernoulli 6 113-167. MR1781185
5. LE CUN, Y., BOSER, B., DENKER, J., HENDERSON, D., HOWARD, R., HUBBARD, W. and JACKEL, L. (1990). Handwritten digit recognition with a backpropogation network. In Advances in Neural Information Processing Systems (D. Touretzky, ed.) 2. Morgan Kaufman, Denver, CO.
6. MALIK, J., BELONGIE, S., LEUNG, T. and SHI, J. (2001). Contour and texture analysis for image segmentation. International Journal of Computer Vision 43 7-27.
7. NADLER, B. and GALUN, M. (2007). Fundamental limitations of spectral clustering. In Advances in Neural Information Processing Systems (B. Schölkopf, J. Platt and T. Hoffman, eds.) 19 1017-1024. MIT Press, Cambridge, MA.
8. NG, A., JORDAN, M. and WEISS, Y. (2002). On spectral clustering: Analysis and an algorithm. In Advances in Neural Information Processing Systems (T. Dietterich, S. Becker and Z. Ghahramani, eds.) 14 955-962. MIT Press, Cambridge, MA.
9. PARLETT, B. N. (1980). The Summetric Eigenvalue Problem. Prentice Hall, Englewood Cliffs, NJ. MR0570116
10. PERONA, P. and FREEMAN, W. T. (1998). A factorization approach to grouping. In Proceedings of the 5th European Conference on Computer Vision 655-670. Springer, London.
11. SCHÖLKOPF, B. and SMOLA, A. (2002). Learning with Kernels. MIT Press, Cambridge, MA.
12. SCHÖLKOPF, B., SMOLA, A. and MÜLLER, K. R. (1998). Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10 1299-1319.
13. SCOTT, G. and LONGUET-HIGGINS, H. (1990). Feature grouping by relocalisation of eigenvectors of proximity matrix. In Proceedings of British Machine Vision Conference 103-108. Oxford, UK.
14. SHI, J. and MALIK, J. (2000). Normalized cuts and image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 22 888-905.
15. SHI, T., BELKIN, M. and YU, B. (2008). Data spectroscopy: Learning mixture models using eigenspaces of convolution operators. In Proceedings of the 25th Annual International Conference on Machine Learning (ICML 2008) (A. McCallum and S. Roweis, eds.) 936-943. Omnipress, Madison, WI.
16. VAPNIK, V. (1995). The Nature of Statistical Learning. Springer, New York. MR1367965
17. VERMA, D. and MEILA, M. (2001). A comparison of spectral clustering algorithms. Technical report, Univ. Washington Computer Science and Engineering.
18. VON LUXBURG, U. (2007). A turorial on spectral clustering. Stat. Comput. 17 395-416. MR2409803
19. VON LUXBURG, U., BELKIN, M. and BOUSQUET, O. (2008). Consistency of spectral clustering. Ann. Statist. 36 555-586. MR2396807
20. WEISS, Y. (1999). Segmentation using eigenvectors: A unifying view. In Proceedings of the Seventh IEEE International Conference on Computer Vision 975-982. IEEE, Los Alamitos, CA.
21. WILLIAMS, C. K. and SEEGER, M. (2000). The effect of the input density distribution on kernel-based classifiers. In Proceedings of the 17th International Conference on Machine Learning (P. Langley, ed.) 1159-1166. Morgan Kaufmann, San Francisco, CA.
22. ZHU, H., WILLIAMS, C., ROHWER, R. and MORCINIE, M. (1998). Gaussian regression and optimal finite-dimensional linear models. In Neural Networks and Machine Learning (C. Bishop, ed.) 167-184. Springer, Berlin.