On efficient large margin semisupervised learning: Method and theory

Journal of Machine Learning Research

Volumn 10, Issue , 2009, Pages 719-742

  Wang, Junhui ^a   Shen, Xiaotong ^b   Pan, Wei ^b  

^a Och Spine at New York Presbyterian Hospitals   (United States)

^b UNIVERSITY OF MINNESOTA   (United States)

Author keywords

Classification; Difference convex programming; Nonconvex minimization; Regulanzation; Support vectors

Indexed keywords

CLASSIFICATION; DIFFERENCE CONVEX PROGRAMMING; NONCONVEX MINIMIZATION; REGULANZATION; SUPPORT VECTORS;

EID: 64149104410     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (46)

1. S. Abney. Understanding the Yarowsky algorithm. Computat. Linguistics, 30: 365-395, 2004.
2. R. A. Adams. Sobolev Spaces. Academic Press, New York, 1975.
3. M. Amini, and P. Gallinari. Semi-supervised learning with an explicit label-error model for misclassified data. In IJCAI, 2003.
4. L. An and P. Tao. Solving a class of linearly constrained indefinite quadratic problems by D.C. algorithms. J. of Global Optimization, 11:253-285, 1997.
5. R. Ando and T. Zhang. A framework for learning predictive structures from multiple tasks and unlabeled data. J. Mach. Learn. Res., 6:1817-1853, 2005.
6. M. Balcan, A. Blum, P. Choi, J. Lafferty, B. Pantano, M. Rwebangira and X. Zhu. Person identification in webcam images: an application of semi-supervised learning. In ICML, 2005.
7. C. L. Blake and C. J. Merz. UCI repository of machine learning databases. University of California, Irvine, Department of Information and Computer Science, 1998.
8. A. Blum and T. Mitchell. Combining labeled and unlabeled data with co-training. In Proc. 11th Ann. Conf. on Computat. Learn. Theory, 1998.
9. O. Chapelle, B. Schölkopf, and A. Zien. Semi-supervised Learning. MIT press, Cambridge, 2006.
10. O. Chapelle and A. Zien. Semi-supervised classification by low density separation. In Proc. Int. Workshop on Artif. Intell. and Statist., pages 57-64, 2005.
11. M. Collins and Y Singer. Unsupervised models for named entity classification. In Proc. Joint SIG- DAT Conf. on Empirical Methods in Natural Language Processing and Very Large Corpora, pages 100-110, 1999.
12. R. A. Fisher. A system of scoring linkage data, with special reference to the pied factors in mice. Amen Nat., 80:568-578, 1946.
13. C. Gu. Multidimension smoothing with splines. In, M. G. Shimek, (ed.), Smoothing and Regression: Approaches, Computation and Application, 2000.
14. T. Hughes, M. Marton, A. Jones, C. Roberts, R. Stoughton, C. Armour, H. Bennett, E. Coffey, H. Dai, Y He, M. Kidd, A. King, M. Meyer, D. Slade, P. Lum, S. Stepaniants, D. Shoemaker, D. Gachotte, K. Chakraburtty, J. Simon, M. Bard and S. Friend. Functional discovery via a compendium of expression profiles. Cell, 102:109-126, 2000.
15. D. Hunter and K. Lange. Quantile regression via an MM algorithm. J. Computat. & Graph. Statist., 9:60-77, 2000.
16. T. Jaakkola, M. Diekhans and D. Haussler. Using the Fisher kernel method to detect remote protein homologies. In Proc. Int. Conf. on Intelligent Systems for Molecular Biology, pages 149-158, 1999.
17. A. N. Kolmogorov and V. M. Tihomirov. e-entropy and e-capacity of sets in function spaces. Uspekhi Mat. Nauk., 14:3-86, 1959. [In Russian. English translation, Ameri. Math. Soc. Transl., 14:277-364, 1961.
18. Y. Lin. Support vector machines and the Bayes rule in classification. Data Mining and Knowledge Discovery, 6:259-275, 2002.
19. S. Liu, X. Shen and W. Wong. Computational development of ψ-learning. In Proc. SIAM 2005 Int. Data Mining Conf., pages 1-12, 2005.
20. Y. Liu and X. Shen. Multicategory ψ-learning. J. Amer. Statist. Assoc., 101:500-509, 2006.
21. P. Mason, L. Baxter, J. Bartlett and M. Frean. Boosting algorithms as gradient descent. In Advances in Neural Information Processing Systems, 12:512-518. MIT Press, Cambridge, 2000.
22. P. McCullagh and J. Nelder. Generalized Linear Models, 2nd edition. Chapman and Hall/CRC, 1983.
23. H. Mewes, K. Albermann, K. Heumann, S. Liebl and F. Pfeiffer. MIPS: a database for protein sequences, homology data and yeast genome information. Nucleic Acids Res., 25:28-30, 2002.
24. K. Nigam, A, McCallum, S. Thrun and T. Mitchell . Text classification from labeled and unlabeled documents using EM. Mach. Learn., 39:103-134, 1998.
25. J. Platt. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In Advances in Large Margin Classifiers, pages 61-74, MIT press, Cambridge, 1999.
26. P. Rigollet. Generalization Error Bounds in Semi-supervised Classification Under the Cluster Assumption. J. Mach. Learn. Res., 8:1369-1392, 2007.
27. B. Schölkopf, A. Smola, R. Williamson and P. Bartlett. New support vector algorithms. Neural Computation, 12:1207-1245, 2000.
28. X. Shen. On method of sieves and penalization. Ann. Statist., 25:2555-2591, 1997.
29. X. Shen, G. C. Tseng, X. Zhang and W. Wong. On psi-learning. J. Amer. Statist. Assoc., 98:724-734, 2003.
30. X. Shen and L, Wang. Generalization error for multi-class margin classification. Electronic J. of Statist., 1:307-330,2007.
31. X. Shen and W. Wong. Convergence rate of sieve estimates. Ann. Statist., 22:580-615, 1994.
32. A. Singh, R. Nowak and X. Zhu. Unlabeled data: Now it helps, now it doesn't. In NIPS, 2008.
33. A. Tsybakov, Optimal aggregation of classifiers in statistical learning, Ann. Statist., 32:135-166, 2004.
34. V. Vapnik. Statistical Learning Theory. Wiley, New York, 1998.
35. G. Wahba. Spline models for observational data. Series in Applied Mathematics, Vol. 59, SIAM, Philadelphia, 1990.
36. G. Wahba. Support vector machines, reproducing kernel Hilbert spaces and the randomized GACV. In Advances in Kernel Methods: Support Vector Learning, edited by B. Schoelkopf, C. Burges and A. Smola, MIT Press, Cambridge, 1998.
37. J. Wang andX. Shen. Large margin semi-supervised learning. J. Mach. Learn. Res., 8:1867-1891, 2007.
38. J. Wang, X. Shen and Y. Liu. Probability estimation for large margin classifiers. Biometrika, 95:149- 167, 2008.
39. J. Wang, X. Shen and W Pan. On transductive support vector machine. Contemp. Math., 43:7-19, 2007.
40. G. Xiao and W. Pan, Gene function prediction by a combined analysis of gene expression data and protein-protein interaction data. J. Bioinformatics and Computat. Biol., 3:1371-1390, 2005.
41. D. Yarowsky. Unsupervised word sense disambiguation rivaling supervised methods. In Proc. of the 33rd Annual Meeting of the Association for Computational Linguistics, pages 189-196, 1995.
42. T. Zhang and F. Oles. A probability analysis on the value of unlabeled data for classification problems. In ICML, 2000.
43. X. Zhou, M. Kao and W. Wong. Transitive functional annotation by shortest-path analysis of gene expression data. Proc. Nat. Acad. Sci., 99:12783-12788, 2000.
44. J. Zhu and T. Hastie. Kernel logistic regression and the import vector machine. J. Computat. Graph. Statist., 14:185-205,2005.
45. X. Zhu, Z. Ghahramani and J. Lafferty. Semi-supervised learning using Gaussian fields and harmonic functions. In ICML, 2003.
46. X. Zhu. Semi-supervised learning literature survey. Technical Report 1530, University of Wisconsin, Madison, 2005.