Scaling up graph-based semisupervised learning via prototype vector machines

IEEE Transactions on Neural Networks and Learning Systems

Volumn 26, Issue 3, 2015, Pages 444-457

  Zhang, Kai ^a   Lan, Liang ^b   Kwok, James T ^c   Vucetic, Slobodan ^d   Parvin, Bahram ^e  

^a NEC LABORATORIES AMERICA   (United States)

^b HUAWEI NOAH S ARK LAB   (China)

^c HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

^d Temple University   (United States)

^e LAWRENCE BERKELEY NATIONAL LABORATORY   (United States)

Author keywords

Graph based methods; large data sets; low rank approximation; manifold regularization; semisupervised learning.

Indexed keywords

APPROXIMATION THEORY; SPEECH RECOGNITION; SUPERVISED LEARNING; VIRTUAL REALITY;

GRAPH-BASED METHODS; LARGE DATASETS; LOW RANK APPROXIMATIONS; MANIFOLD REGULARIZATIONS; SEMI- SUPERVISED LEARNING;

GRAPHIC METHODS;

AUTOMATED PATTERN RECOGNITION; INFORMATION PROCESSING; PROCEDURES; SUPERVISED MACHINE LEARNING; SUPPORT VECTOR MACHINE; TRENDS;

DATASETS AS TOPIC; PATTERN RECOGNITION, AUTOMATED; SUPERVISED MACHINE LEARNING; SUPPORT VECTOR MACHINE;

EID: 84923874457     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2014.2315526     Document Type: Article

References (49)

1. F. Bach and M. Jordan, "Predictive low-rank decomposition for kernel methods," in Proc. 22nd Int. Conf. Mach. Learn., Bonn, Germany, Aug. 2005, pp. 33-40.
2. S. Baluja, "Probabilistic modeling for face orientation discrimination: Learning from labeled and unlabeled data," in Proc. Adv. NIPS, vol. 11. 1999, pp. 854-860.
3. M. Belkin and P. Niyogi, "Laplacian eigenmaps and spectral techniques for embedding and clustering," in Proc. Adv. NIPS, vol. 14. 2002, pp. 585-591.
4. M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold regularization: A geometric framework for learning from labeled and unlabeled examples," J. Mach. Learn. Res., vol. 7, no. 11, pp. 2399-2434, 2006.
5. S. Ben-David, T. Lu, and D. Pal, "Does unlabeled data provably help? Worst-case analysis of the sample complexity of semi-supervised learning," in Proc. 21st Annu. Conf. Learn. Theory, 2008, pp. 33-44.
6. A. Blum and S. Chawla, "Learning from labeled and unlabeled data using graph mincuts," in Proc. 18th Int. Conf. Mach. Learn., San Francisco, CA, USA, 2001, pp. 19-26.
7. A. Blum and T. Mitchell, "Combining labeled and unlabeled data with co-training," in Proc. 11th Annu. Conf. Comput. Learn. Theory, New York, NY, USA, Jul. 1998, pp. 209-214.
8. G. Camps-Valls, T. V. B. Marsheva, and D. Zhou, "Semi-supervised graph-based hyperspectral image classification," IEEE Trans. Geosci. Remote Sens. E, vol. 45, no. 10, pp. 3044-3054, Oct. 2007.
9. V. Castelli and T. Cover, "On the exponential value of labeled samples," Pattern Recognit. Lett., vol. 16, no. 1, pp. 105-111, 1995.
10. O. Chapelle, B. Schölkopf, and A. Zien, Semi-Supervised Learning. Cambridge, MA, USA: MIT Press, 2006.
11. O. Chapelle and A. Zien, "Semi-supervised classification by low density separation," in Proc. 10th Int. Workshop Artif. Intell. Statist., Jan. 2005.
12. A. Coates and A. Y. Ng, "The importance of encoding versus training with sparse coding and vector quantization," in Proc. 28th Int. Conf. Mach. Learn., Bellevue, WA, USA, Jun. 2011, pp. 921-928.
13. O. Delalleau, Y. Bengio, and N. L. Roux, "Efficient non-parametric function induction in semi-supervised learning," in Proc. 10th Int. Workshop Artif. Intell. Statist., 2005, pp. 96-103.
14. P. Drineas, R. Kannan, and M. Mahoney, "Fast Monte Carlo algorithms for matrices II: Computing a low rank approximation to a matrix," SIAM J. Comput., vol. 36, no. 1, pp. 158-183, 2006.
15. P. Drineas and M. W. Mahoney, "On the Nyström method for approximating a Gram matrix for improved kernel-based learning," J. Mach. Learn. Res., vol. 6, pp. 2153-2175, Dec. 2005.
16. C. Fowlkes, S. Belongie, F. Chung, and J. Malik, "Spectral grouping using the Nyström method," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 2, pp. 214-225, Feb. 2004.
17. A. Frieze, R. Kannan, and S. Vempala, "Fast Monte-Carlo algorithms for finding low-rank approximations," in Proc. 39th Annu. Symp. Found. Comput. Sci., Palo Alto, CA, USA, 1998, pp. 370-378.
18. J. Goldberger and S. Roweis, "Hierarchical clustering of a mixture model," in Proc. Adv. NIPS, vol. 17. 2005, pp. 505-512.
19. T. Joachims, "Transductive inference for text classification using support vector machines," in Proc. 16th Int. Conf. Mach. Learn., San Francisco, CA, USA, 1999, pp. 200-209.
20. J. Lafferty, X. Zhu, and Y. Liu, "Kernel conditional random fields: Representation and clique selection," in Proc. 21st Int. Conf. Mach. Learn., Banff, AB, Canada, Jul. 2004, p. 64.
21. C.-H. Lee, S. Wang, F. Jiao, D. Schuurmans, and D. Greiner, "Learning to model spatial dependency: Semi-supervised discriminative random fields," in Proc. Adv. NIPS, vol. 19. Cambridge, MA, USA, 2007.
22. L. Chen, I. W. Tsang, and D. Xu, "Laplacian embedded regression for scalable manifold regularization," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 6, pp. 902-915, Jun. 2012.
23. W. Liu, J. He, and S. F. Chang, "Large graph construction for scalable semi-supervised learning," in Proc. 27th Int. Conf. Mach. Learn., Haifa, Israel, Jun. 2010, pp. 679-686.
24. S. Melacci and M. Belkin, "Laplacian support vector machines trained in the primal," J. Mach. Learn. Res., vol. 12, no. 3, pp. 1149-1184, 2011.
25. A. Y. Ng, M. I. Jordan, and Y. Weiss, "On spectral clustering: Analysis and an algorithm," in Proc. Adv. NIPS, vol. 14. 2001, pp. 849-856.
26. Y. Huang, D. Xu, and F. Nie, "Semi-supervised dimension reduction using trace ratio criterion," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 3, pp. 519-526, Mar. 2012.
27. K. Nigam, A. McCallum, S. Thrun, and T. Mitchell, "Text classification from labeled and unlabeled documents using EM," Mach. Learn., vol. 39, nos. 2-3, pp. 103-134, 2000.
28. P. Rigollet, "Generalization error bounds in semi-supervised classification under the cluster assumption," J. Mach. Learn. Res., vol. 8, pp. 1369-1392, Dec. 2007.
29. S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol. 290, no. 5500, pp. 2323-2326, 2000.
30. J. R. Shewchuk, "An introduction to the conjugate gradient method without the agonizing pain," School Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, USA, Tech. Rep., 1994.
31. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 888-905, Aug. 2000.
32. A. Singh, R. D. Nowak, and X. Zhu, "Unlabeled data: Now it helps, now it doesn't," in Proc. Adv. NIPS, 2008, pp. 1513-1520.
33. R. G. F. Soares, H. Chen, and X. Yao, "Semisupervised classification with cluster regularization," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 11, pp. 1779-1792, Nov. 2012.
34. J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science, vol. 290, no. 5500, pp. 2319-2323, 2000.
35. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Roy. Statist., Soc. B, vol. 58, no. 1, pp. 267-288, 1996.
36. I. W. Tsang, J. T. Kwok, and P.-M. Cheung, "Core vector machines: Fast SVM training on very large data sets," J. Mach. Learn. Res., vol. 6, pp. 363-392, Dec. 2005.
37. C. Williams and M. Seeger, "Using the Nyström method to speed up kernel machines," in Proc. Adv. NIPS, vol. 13. 2001, pp. 682-688.
38. Z. Xu, I. King, M. R.-T. Lyu, and R. Jin, "Discriminative semisupervised feature selection via manifold regularization," IEEE Trans. Neural Netw., vol. 21, no. 7, pp. 1033-1047, Jul. 2010.
39. D. Yarowsky, "Unsupervised word-sense disambiguation rivaling supervised methods," in Proc. 33rd Annu. Meet. Assoc. Comput. Linguistics, Stroudsburg, PA, USA, 1995, pp. 189-196.
40. K. Zhang and J. T. Kwok, "Improved Nyström low rank approximation and error analysis," in Proc. 25th Int. Conf. Mach. Learn., Helsinki, Finland, Jun. 2008, pp. 1232-1239.
41. K. Zhang and J. T. Kwok, "Clustered Nyström method for large scale manifold learning and dimension reduction," IEEE Trans. Neural Netw., vol. 21, no. 10, pp. 1576-1587, Oct. 2010.
42. K. Zhang, J. T. Kwok, and B. Parvin, "Prototype vector machine for large scale semi-supervised learning," in Proc. 26th Int. Conf. Mach. Learn., Montreal, QC, Canada, Jun. 2009, pp. 1233-1240.
43. K. Zhang, L. Lan, Z. Wang, and F. Moerchen, "Scaling up kernel SVM on limited resources: A low-rank linearization approach," in Proc. Int. Conf. Artif. Intell. Statist., 2012, pp. 1425-1434.
44. K. Zhang, V. Zheng, Q. Wang, J. Kwok, Q. Yang, and I. Marsic, "Covariate shift in Hilbert space: A solution via surrogate kernels," in Proc. 30th Int. Conf. Mach. Learn., Atlanta, GA, USA, Jun. 2013, pp. 388-395.
45. D. Zhou, O. Bousquet, T. N. Lal, J.Weston, and B. Schölkopf, "Learning with local and global consistency," in Proc. Adv. NIPS, vol. 16. 2003, pp. 321-328.
46. Y. Wang, S. Chen, and Z.-H. Zhou, "New semi-supervised classification method based on modified cluster assumption," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 5, pp. 689-702, May 2012.
47. X. Zhu, "Semi-supervised learning literature survey," Dept. Comput. Sci., Univ. Wisconsin-Madison, Madison, WI, USA, Tech. Rep. 1530, 2008.
48. X. Zhu, Z. Ghahramani, and J. Lafferty, "Semi-supervised learning using Gaussian fields and harmonic functions," in Proc. 20th Int. Conf. Mach. Learn., Washington, DC, USA, Aug. 2003, pp. 912-919.
49. X. Zhu and J. Lafferty, "Harmonic mixtures: Combining mixture models and graph-based methods for inductive and scalable semi-supervised learning," in Proc. 22nd Int. Conf. Mach. Learn., Bonn, Germany, Aug. 2005, pp. 1052-1059.