Semisupervised learning using Bayesian interpretation: Application to LS-SVM

IEEE Transactions on Neural Networks

Volumn 22, Issue 4, 2011, Pages 513-524

  Adankon, Mathias M ^a   Cheriet, Mohamed ^a   Biem, Alain ^b  

^a ÉCOLE DE TECHNOLOGIE SUPÉRIEURE   (Canada)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

Bayesian inference; kernel machine; least square support vector machine (SVM); semisupervised learning; SVM

Indexed keywords

BAYESIAN INFERENCE; KERNEL MACHINE; LEAST-SQUARE SUPPORT VECTOR MACHINES; SEMI-SUPERVISED LEARNING; SVM;

BAYESIAN NETWORKS; INFERENCE ENGINES; PATTERN RECOGNITION; SUPPORT VECTOR MACHINES;

LEARNING ALGORITHMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; BAYES THEOREM; COMPUTER SIMULATION; HUMAN; NORMAL DISTRIBUTION; REGRESSION ANALYSIS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BAYES THEOREM; COMPUTER SIMULATION; HUMANS; LEAST-SQUARES ANALYSIS; NORMAL DISTRIBUTION; PATTERN RECOGNITION, AUTOMATED;

EID: 79953812916     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2011.2105888     Document Type: Article

References (50)

1. A. Biem, "Minimum classification error training for online handwriting recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 7, pp. 1041-1051, Jul. 2006.
2. T. Mitchell, "The role of unlabeled data in supervised learning," in Proc. 6th Int. Colloq. Cognitive Sci., San Sebastian, Spain, 1999.
3. X. Zhu, "Semi-supervised learning literature survey," Dept. Comput. Sci., Univ. Wisconsin, Madison, Tech. Rep. 1530, 2007.
4. M. Seeger, "Learning with labeled and unlabeled data," Inst. Adapt. Neural Comput., Univ. Edinburgh, Edinburgh, U.K., Tech. Rep., Feb. 2001.
5. O. Chapelle, B. Schölkopf, and A. Zien, Semi-Supervised Learning. Cambridge, MA: MIT Press, 2006.
6. C. Rosenberg, M. Hebert, and H. Schneiderman, "Semi-supervised self-training of object detection models," in Proc. 7th IEEE Workshop Appl. Comput. Vis., vol. 1. Breckenridge, CO, Jan. 2005, pp. 29-36.
7. E. Riloff, J. Wiebe, and T. Wilson, "Learning subjective nouns using extraction pattern bootstrapping," in Proc. 7th Conf. Natural Lang. Learn., 2003, pp. 25-32.
8. A. Blum and T. Mitchell, "Combining labeled and unlabeled data with co-training," in Proc. 11th Annu. Conf. Comput. Learn. Theory, San Francisco, CA, 1998, pp. 92-100.
9. A. Blum and S. Chawla, "Learning from labeled and unlabeled data using graph mincuts," in Proc.18th Int. Conf. Mach. Learn., San Francisco, CA, 2001, pp. 19-26.
10. T. Joachims, "Transductive learning via spectral graph partitioning," in Proc. 20th Int. Conf. Mach. Learn., 2003, pp. 290-297.
11. O. Chapelle and A. Zien, "Semi-supervised classification by low density separation," in Proc. 10th Int. Workshop Artif. Intell. Stat., 2005, pp. 1-8.
12. J. Luts, J. A. K. Suykens, and S. Van Huffel, "Semi-supervised learning: Avoiding zero label assumptions in kernel based classifiers," ESATSISTA, Katholieke Univ. Leuven, Leuven, Belgium, Tech. Rep. 07-122,2007.
13. N. D. Lawrence and M. I. Jordan, "Semi-supervised learning via Gaussian processes," in Proc. Adv. Neural Inf. Process. Syst. 17, 2004, pp. 1-8.
14. V. Sindhwani, W. Chu, and S. S. Keerthi, "Semi-supervised Gaussian process classifiers," in Proc. Int. Joint Conf. Artif. Intell., 2004, pp. 1-6.
15. X. J. Zhu, J. Lafferty, and Z. Ghahramani, "Semi-supervised learning: From Gaussian fields to Gaussian processes," Dept. Comput. Sci., Univ. Carnegie Mellon, Pittsburgh, PA, Tech. Rep. CMU-CS-03-175, 2003.
16. A. Kapoor, Y. Qi, H. Ahn, and R. Picard, "Hyperparameter and kernel learning for graph based semisupervised classification," in Proc. Adv. Neural Inf. Process. Syst. 18, 2005, pp. 627-634.
17. A. Patel, S. Sundararajan, and S. Shevade, "Semisupervised classification using sparse Gaussian process regression," in Proc. 21st Int. Joint Conf. Artif. Intell., 2009, pp. 1193-1198.
18. O. Chapelle, V. Sindhwani, and S. S. Keerthi, "Optimization techniques for semi-supervised support vector machines," J.Mach. Learn. Res., vol. 9, pp. 203-233, Feb. 2008. (Pubitemid 351469022)
19. C. M. Bishop, Neural Networks for Pattern Recognition. London, U.K.: Oxford Univ. Press, 1995.
20. B. E. Boser, I. M. Guyon, and V. N. Vapnik, "A training algorithm for optimal margin classifiers," in Proc. 5th Annu. Workshop Comput. Learn. Theory, 1992, pp. 144-152.
21. B. Schölkopf and A. J. Smola, Learning with Kernels. Cambridge, MA: MIT Press, 2002.
22. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines. Cambridge, U.K.: Cambridge Univ. Press, 2000.
23. K. Bennett and A. Demiriz, Semi-Supervised Support Vector Machines, S. A. Solla, and D. A. Cohn, Eds. Cambridge, MA: MIT Press, 1998, pp. 368-374.
24. T. Joachims, "Transductive inference for text classification using support vector machines," in Proc. 16th Int. Conf. Mach. Learn., Bled, Slovenia, 1999, pp. 200-209.
25. G. Fung and O. Mangasarian, "Semi-supervised support vector machines for unlabeled data classification," Optim. Methods Softw., vol. 15, no. 1, pp. 29-44, 2001. (Pubitemid 33817502)
26. J. A. K. Suykens, T. V. Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle, Least Squares Support Vector Machines. Singapore: World Scientific, 2002.
27. T. Van Gestel, J. A. K. Suykens, B. Baesens, S. Viaene, J. Vanthienen, G. Dedene, B. De Moor, and J. Vandewalle, "Benchmarking least squares support vector machine classifiers," Mach. Learn., vol. 54, no. 1, pp. 5-32, Jan. 2004.
28. K. S. Chua, "Efficient computations for large least square support vector machine classifiers," Pattern Recognit. Lett., vol. 24, nos. 1-3, pp. 75-80, 2003. (Pubitemid 36080758)
29. W. Chu, C. J. Ong, and S. S. Keerthi, "An improved conjugate gradient scheme to the solution of least squares SVM," IEEE Trans. Neural Netw., vol. 16, no. 2, pp. 498-501, Mar. 2005. (Pubitemid 40390833)
30. G. C. Cawley and N. L. C. Talbot, "Fast exact leave-one-out cross-validation of sparse least-squares support vector machines," Neural Netw., vol. 17, no. 10, pp. 1467-1475, Dec. 2004. (Pubitemid 39487142)
31. G. C. Cawley, "Leave-one-out cross-validation based model selection criteria for weighted LS-SVMs," in Proc. Int. Joint Conf. Neural Netw., Vancouver, BC, Canada, Jul. 2006, pp. 1661-1668. (Pubitemid 351369522)
32. M. M. Adankon and M. Cheriet, "Model selection for LS-SVM. Application to handwriting recognition," Pattern Recognit., vol. 42, no. 12, pp. 3264-3270, Dec. 2009.
33. J. A. K. Suykens, L. Lukas, and L. Vandewalle, "Sparse least squares support vector machine classifiers," in Proc. Eur. Symp. Artif. Neural Netw., 2000, pp. 37-42.
34. G. C. Cawley and N. L. C. Talbot, "Improved sparse least-squares support vector machines," Neurocomputing, vol. 48, nos. 1-4, pp. 1025-1031, Oct. 2002. (Pubitemid 36221542)
35. B. J. de Kruif and T. J. de Vries, "Pruning error minimization in least squares support vector machines," IEEE Trans. Neural Netw., vol. 14, no. 3, pp. 696-702, May 2003.
36. L. Jiao, L. Bo, and L. Wang, "Fast sparse approximation for least square support vector machine," IEEE Trans. Neural Netw., vol. 18, no. 3, pp. 685-697, May 2007. (Pubitemid 46773736)
37. J. I. Marden, "Hypothesis testing: From p values to Bayes factors," J. Amer. Stat. Assoc., vol. 95, no. 452, pp. 1316-1320, Dec. 2000.
38. H. Chen, P. Tino, and X. Yao, "Probabilistic classification vector machines," IEEE Trans. Neural Netw., vol. 20, no. 6, pp. 901-914, Jun. 2009.
39. I. Psorakis, T. Damoulas, and M. A. Girolami, "Multiclass relevance vector machines: Sparsity and accuracy," IEEE Trans. Neural Netw., vol. 21, no. 10, pp. 1588-1598, Oct. 2010.
40. V. N. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
41. P. Sollich, "Probabilistic methods for support vector machines," in Proc. Conf. 12th Adv. Neural Inf., 2000, pp. 349-355.
42. O. Chapelle, V. Sindhwani, and S. S. Keerthi, "Branch and bound for semi-supervised support vector machines," in Proc. Neural Inf. Syst., 2006, pp. 217-224.
43. J. A. K. Suykens and J. Vandewalle, "Least squares support vector machine classifiers," Neural Process. Lett., vol. 9, no. 3, pp. 293-300, Jun. 1999.
44. T. V. Gestel, J. Suykens, G. Lanckriet, A. Lambrechts, B. De Moor, and J. Vandewalle, "Bayesian framework for least squares support vector machine classifiers, Gaussian processes and kernel fisher discriminant analysis," Neural Comput., vol. 14, no. 5, pp. 1115-1147, May 2002.
45. Using the CPLEX Callable Library and CPLEX Mixed Integer Library. (1995). CPLEX Optimization Inc., Incline Village, NV [Online]. Available: http://www.ilog.com/products/cplex/
46. Y. Grandvalet and Y. Bengio, "Semi-supervised learning by entropy minimization," in Advances in Neural Information Processing Systems 17, L. K. Saul, Y. Weiss, and L. Bottou, Eds. Cambridge, MA: MIT Press, 2004, pp. 529-536.
47. M. Szummer and T. Jaakkola, "Partially labeled classification with Markov random walks," in Proc. Adv. Neural Inf. Process. Syst., vol. 14. 2001, pp. 1-8.
48. M. M. Adankon and M. Cheriet, "Genetic algorithm based training for semi-supervised SVM," Neural Comput. Appl., vol. 19, no. 8, pp. 1197-1206, Nov. 2010.
49. R. Collobert, F. Sinz, J. Weston, and L. Bottou, "Large scale transductive SVM," J. Mach. Learn. Res., vol. 7, pp. 1687-1712, Aug. 2006.
50. C. Gold, A. Holub, and P. Sollich, "Bayesian approach to feature selection and parameter tuning for support vector machine classifiers," Neural Netw., vol. 18, nos. 5-6, pp. 693-701, Jun. 2005. (Pubitemid 43186590)