Semi-supervised eigenvectors for large-scale locally-biased learning

Journal of Machine Learning Research

Volumn 15, Issue , 2015, Pages 3691-3734

  Hansen, Toke J ^a   Mahoney, Michael W ^b  

^a TECHNICAL UNIVERSITY OF DENMARK   (Denmark)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Kernel methods; Large scale machine learning; Local spectral methods; Locally biased learning; Semi supervised learning; Spectral clustering

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLUSTERING ALGORITHMS; INFORMATION ANALYSIS; LAPLACE TRANSFORMS; LEARNING ALGORITHMS; SUPERVISED LEARNING;

KERNEL METHODS; LARGE-SCALE MACHINE LEARNING; LOCALLY-BIASED LEARNING; SEMI- SUPERVISED LEARNING; SPECTRAL CLUSTERING; SPECTRAL METHODS;

EIGENVALUES AND EIGENFUNCTIONS;

EID: 84919754604     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (53)

1. R. Andersen and K. Lang. Communities from seed sets. In WWW '06: Proceedings of the 15th International Conference on World Wide Web, pages 223-232, 2006.
2. R. Andersen, F.R.K. Chung, and K. Lang. Local graph partitioning using PageRank vectors. In FOCS '06: Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science, pages 475-486, 2006.
3. M. Belkin and P. Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15(6):1373-1396, 2003.
4. M. Belkin and P. Niyogi. Semi-supervised learning on Riemannian manifolds. Machine Learning, 56:209-239, 2004.
5. M. Belkin and P. Niyogi. Towards a theoretical foundation for Laplacian-based manifold methods. Journal of Computer and System Sciences, 74(8):1289-1308, 2008.
6. M. Belkin, P. Niyogi, and V. Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 7:2399-2434, 2006.
7. P. Boldi and S. Vigna. The push algorithm for spectral ranking. Technical report, 2011. Preprint: arXiv:1109.4680 (2011).
8. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, Cambridge, UK, 2004.
9. O. Chapelle, J. Weston, and B. Schölkopf. Cluster kernels for semi-supervised learning. In Advances in Neural Information Processing Systems 15: Proceedings of the 2002 Conference, pages 585-592, 2003.
10. F.R.K. Chung. Spectral graph theory, volume 92 of CBMS Regional Conference Series in Mathematics. American Mathematical Society, 1997.
11. R.R. Coifman, S. Lafon, A.B. Lee, M. Maggioni, B. Nadler, F. Warner, and S.W. Zucker. Geometric diffusions as a tool for harmonic analysis and structure definition in data: Diffusion maps. Proceedings of the National Academy of Sciences of the United States of America, 102(21):7426-7431, 2005.
12. M. Cucuringu and M. W. Mahoney. Localization on low-order eigenvectors of data matrices. Technical report, 2011. Preprint: arXiv:1109.1355 (2011).
13. P. Drineas and M. W. Mahoney. On the Nyström method for approximating a Gram matrix for improved kernel-based learning. Journal of Machine Learning Research, 6:2153-2175, 2005.
14. A. P. Eriksson, C. Olsson, and F. Kahl. Normalized cuts revisited: A reformulation for segmentation with linear grouping constraints. In Proceedings of the 11th International Conference on Computer Vision, pages 1-8, 2007.
15. W. Gander, G. H. Golub, and U. von Matt. A constrained eigenvalue problem. Linear Algebra and its Applications, 114/115:815-839, 1989.
16. A. Gittens and M. W. Mahoney. Revisiting the Nyström method for improved large-scale machine learning. Technical report, 2012. Preprint arXiv:1303.1849 (2012).
17. J. Ham, D.D. Lee, S. Mika, and B. Schölkopf. A kernel view of the dimensionality reduction of manifolds. In Proceedings of the 21st International Conference on Machine Learning, pages 000-000, 2004.
18. T.H. Haveliwala. Topic-sensitive PageRank: A context-sensitive ranking algorithm for web search. IEEE Transactions on Knowledge and Data Engineering, 15(4):784-796, 2003.
19. G. Jeh and J. Widom. Scaling personalized web search. In WWW '03: Proceedings of the 12th International Conference on World Wide Web, pages 271-279, 2003.
20. T. Joachims. Transductive learning via spectral graph partitioning. In Proceedings of the 20th International Conference on Machine Learning, pages 290-297, 2003.
21. R. I. Kondor and J. Lafferty. Diffusion kernels on graphs and other discrete structures. In Proceedings of the 19th International Conference on Machine Learning, pages 315-322, 2002.
22. Y. Lecun and C. Cortes. The MNIST database of handwritten digits. URL http://yann.lecun.com/exdb/mnist/. http://yann.lecun.com/exdb/mnist/.
23. J. Leskovec, K.J. Lang, A. Dasgupta, and M.W. Mahoney. Statistical properties of community structure in large social and information networks. In WWW '08: Proceedings of the 17th International Conference on World Wide Web, pages 695-704, 2008.
24. M. W. Mahoney and L. Orecchia. Implementing regularization implicitly via approximate eigenvector computation. In Proceedings of the 28th International Conference on Machine Learning, pages 121-128, 2011.
25. M. W. Mahoney, L. Orecchia, and N. K. Vishnoi. A local spectral method for graphs: with applications to improving graph partitions and exploring data graphs locally. Journal of Machine Learning Research, 13:2339-2365, 2012.
26. S. Maji, N. K. Vishnoi, and J. Malik. Biased normalized cuts. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2057-2064, 2011.
27. P.J. Mucha, T. Richardson, K. Macon, M.A. Porter, and J.P. Onnela. Community structure in time-dependent, multiscale, and multiplex networks. Science, 328(5980):876-878, 2010.
28. A.Y. Ng, M.I. Jordan, and Y. Weiss. On spectral clustering: Analysis and an algorithm. In Advances in Neural Information Processing Systems 15: Proceedings of the 2001 Conference, 2001.
29. K. A. Norman, S. M. Polyn, G. J. Detre, and J. V. Haxby. Beyond mind-reading: multivoxel pattern analysis of fMRI data. Trends in Cognitive Sciences, 10(9):424-430, 2006.
30. L. Page, S. Brin, R. Motwani, and T. Winograd. The PageRank citation ranking: Bringing order to the web. Technical report, Stanford InfoLab, 1999.
31. B. Paolo and V. Sebastiano. The WebGraph framework I: Compression techniques. In Proc. of the Thirteenth International World Wide Web Conference (WWW 2004), pages 595-601, Manhattan, USA, 2004. ACM Press.
32. B. Paolo, C. Bruno, S. Massimo, and V. Sebastiano. Ubicrawler: A scalable fully distributed web crawler. Software: Practice & Experience, 34(8):711-726, 2004.
33. B. Paolo, R. Marco, S. Massimo, and V. Sebastiano. Layered label propagation: A multiresolution coordinate-free ordering for compressing social networks. In Proceedings of the 20th International Conference on World Wide Web. ACM Press, 2011.
34. P. O. Perry and M. W. Mahoney. Regularized Laplacian estimation and fast eigenvector approximation. In Advances in Neural Information Processing Systems 25: Proceedings of the 2011 Conference, 2011.
35. K.T. Poole. The decline and rise of party polarization in congress during the twentieth century. Extensions: A Journal of the Carl Albert Congressional Research and Studies Center, Fall 2005.
36. K.T. Poole and H. Rosenthal. Patterns of congressional voting. American Journal of Political Science, 35:228-278, 1991.
37. A. Pothen, H.D. Simon, and K.-P. Liou. Partitioning sparse matrices with eigenvectors of graphs. SIAM Journal on Matrix Analysis and Applications, 11(3):430-452, 1990.
38. S.T. Roweis and L.K. Saul. Nonlinear dimensionality reduction by local linear embedding. Science, 290:2323-2326, 2000.
39. L. K. Saul, K. Q. Weinberger, J. H. Ham, F. Sha, and D. D. Lee. Spectral methods for dimensionality reduction. In O. Chapelle, B. Schoelkopf, and A. Zien, editors, Semisu-pervised Learning, pages 293-308. MIT Press, 2006.
40. B. Schölkopf and A. J. Smola. Learning with Kernels: Support Vector Machines, Regular-ization, Optimization, and Beyond. MIT Press, Cambridge, MA, USA, 2001.
41. J. Shi and J. Malik. Normalized cuts and image segmentation. IEEE Transcations of Pattern Analysis and Machine Intelligence, 22(8):888-905, 2000.
42. D.A. Spielman and S.-H. Teng. Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems. In STOC '04: Proceedings of the 36th Annual ACM Symposium on Theory of Computing, pages 81-90, 2004.
43. M. Szummer and T. Jaakkola. Partially labeled classification with Markov random walks. In Advances in Neural Information Processing Systems 14: Proceedings of the 2001 Con-ference, pages 945-952, 2002.
44. A. Talwalkar and A. Rostamizadeh. Matrix coherence and the Nyström method. In Proceedings of the 26th Conference in Uncertainty in Artificial Intelligence, 2010.
45. J.B. Tenenbaum, V. de Silva, and J.C. Langford. A global geometric framework for nonlinear dimensionality reduction. Science, 290:2319-2323, 2000.
46. S.-H. Teng. The Laplacian paradigm: Emerging algorithms for massive graphs. In Proceedings of the 7th Annual Conference on Theory and Applications of Models of Computation, pages 2-14, 2010.
47. S. Vigna. Spectral ranking. Technical report, 2009. Preprint: arXiv:0912.0238 (2009).
48. D.J. Watts and S.H. Strogatz. Collective dynamics of small-world networks. Nature, 393:440-442, 1998.
49. A. S. Waugh, L. Pei, J. H. Fowler, P. J. Mucha, and M. A. Porter. Party polarization in congress: A network science approach. Technical report, 2009. Preprint: arXiv:0907.3509 (2009).
50. C.K.I. Williams and M. Seeger. The effect of the input density distribution on kernel-based classifiers. In Proceedings of the 17th International Conference on Machine Learning, pages 1159-1166, 2000.
51. C.K.I. Williams and M. Seeger. Using the Nyström method to speed up kernel machines. In Advances in Neural Information Processing Systems 13: Proceedings of the 2000 Conference, pages 682-688, 2001.
52. S. X. Yu and J. Shi. Grouping with bias. In Advances in Neural Information Processing Systems 14: Proceedings of the 2001 Conference, pages 1327-1334, 2002.
53. D. Zhou, O. Bousquet, T. N. Lal, J. Weston, and B. Schölkopf. Learning with local and global consistency. In Advances in Neural Information Processing Systems 16: Proceed-ings of the 2003 Conference, pages 321-328, 2004.