Mixing linear SVMs for nonlinear classification

IEEE Transactions on Neural Networks

Volumn 21, Issue 12, 2010, Pages 1963-1975

  Fu, Zhouyu ^a,b   Robles Kelly, Antonio ^b,c   Zhou, Jun ^b,c  

^a MONASH UNIVERSITY   (Australia)

^b AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^c UNIVERSITY OF NEW SOUTH WALES   (Australia)

Author keywords

Classification; expectation maximization algorithm; mixture of experts; model selection; support vector machines

Indexed keywords

CLASSIFICATION; EXPECTATION-MAXIMIZATION ALGORITHMS; MIXTURE OF EXPERTS; MODEL SELECTION; SUPPORT VECTOR;

CLASSIFICATION (OF INFORMATION); FACE RECOGNITION; IMAGE SEGMENTATION; MAXIMUM PRINCIPLE; MIXTURES; OPTIMIZATION; PARAMETER ESTIMATION; SUPPORT VECTOR MACHINES;

ALGORITHMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; CLASSIFICATION; COMPUTER SIMULATION; METHODOLOGY; THEORETICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; CLASSIFICATION; COMPUTER SIMULATION; MODELS, THEORETICAL; PATTERN RECOGNITION, AUTOMATED;

EID: 78650056777     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2080319     Document Type: Article

References (37)

1. B. Schölkopf and A. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. Cambridge, MA: MIT Press, 2001.
2. J. Rissanen, Stochastic Complexity in Statistical Inquiry. Singapore: World Scientific, 1989.
3. A. P. Dempster, N. M. Laird, and D. B. Rubin, "Maximum-likelihood from incomplete data via the EM algorithm," J. Royal Statist. Soc. B, vol. 39, no. 1, pp. 1-38, 1977.
4. R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, "Adaptive mixtures of local experts," Neural Comput., vol. 3, no. 1, pp. 79-87, 1991.
5. C.-W. Hsu and C.-J. Lin, "A comparison of methods for multiclass support vector machines," IEEE Trans. Neural Netw., vol. 13, no. 2, pp. 415-425, Mar. 2002.
6. L. González, C. Angulo, F. Velasco, and A. Catalí, "Dual unification of bi-class support vector machine formulations," Pattern Recognit., vol. 39, no. 7, pp. 1325-1332, Jul. 2006.
7. T. Joachims, "Training linear SVMs in linear time," in Proc. 12th ACM Int. Conf. Knowl. Discovery Data Mining, Philadelphia, PA, 2006, pp. 217-226.
8. K.-W. Chang, C.-J. Hsieh, C.-J. Lin, S. S. Keerthi, and S. Sundararajan, "A dual coordinate descent method for large-scale linear SVM," in Proc. 25th Int. Conf. Mach. Learn., 2008, pp. 1-12.
9. S. R. Waterhouse and A. J. Robinson, "Constructive algorithms for hierarchical mixtures of experts," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 1996.
10. J. Fritsch, M. Finke, and A. Waibel, "Adaptively growing hierarchical mixtures of experts," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 1997.
11. L. Xu, M. I. Jordan, and G. E. Hinton, "An alternative model for mixtures of experts," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 1995.
12. W. H. Press, S. A. Teukolsky, W. T. Vetterlin, and B. P. Flannery, Numerical Recipes in C. Cambridge, U.K.: Cambridge Univ. Press, 1992.
13. D. Liu and J. Nocedal, "On the limited memory method for large scale optimization," Math. Program. B, vol. 45, no. 3, pp. 503-528, 1989.
14. M. A. F. Figueiredo and A. K. Jain, "Unsupervised learning of finite mixture models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 3, pp. 381-396, Mar. 2002.
15. M. I. Jordan and R. A. Jacobs, "Hierarchical mixtures of experts and the EM algorithm," Neural Comput., vol. 6, no. 2, pp. 181-214, Mar. 1994.
16. J. T.-Y. Kwok, "Support vector mixture for classification and regression problems," in Proc. 14th Int. Conf. Pattern Recognit., Brisbane, Australia, Aug. 1998, pp. 255-258.
17. R. Collobert, S. Bengio, and Y. Bengio, "A parallel mixture of SVMs for very large scale problems," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2002.
18. S. E. Kruger, M. Schaffoner, M. Katz, E. Andelic, and A. Wendemuth, "Mixture of support vector machines for HMM based speech recognition," in Proc. 18th Int. Conf. Pattern Recognit., Hong Kong, China, 2006, pp. 326-329.
19. K. Saito and R. Nakano, "A constructive learning algorithm for an HME," in Proc. IEEE Int. Conf. Neural Netw., Washington D.C., Jun. 1996, pp. 1268-1273.
20. V. Ramamurti and J. Ghosh, "Structural adaptation in mixture of experts," in Proc. 13th Int. Conf. Pattern Recognit., Vienna, Austria, Aug. 1996, pp. 704-708.
21. Z. Fu and A. Robles-Kelly, "On mixtures of linear SVMs for nonlinear classification," in Proc. Int. Workshop Structural, Syntactic, Statist. Pattern Recognit., Orlando, FL, 2008, pp. 489-499.
22. S. Fine and K. Scheinberg, "Efficient SVM training using low-rank kernel representations," J. Mach. Learn. Res., vol. 2, pp. 243-264, Mar. 2001.
23. F. R. Bach and M. I. Jordan, "Predictive low-rank decomposition for kernel methods," in Proc. 22nd Int. Conf. Mach. Learn., 2005, pp. 33-40.
24. C. K. I. Williams and M. Seeger, "Using the Nyström method to speed up kernel machines," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2001.
25. A. J. Smola and B. Schölkopf, "Sparse greedy matrix approximation for machine learning," in Advances in Neural Information Processing Systems. San Mateo, CA: Morgan Kaufmann, 2000.
26. A. Rahimi and B. Recht, "Random features for large-scale kernel machines," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2007.
27. L. Breiman, "Bagging predictors," Mach. Learn., vol. 24, no. 2, pp. 123-140, Aug. 1996.
28. Y. Freund and R. E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting," J. Comput. Syst. Sci., vol. 55, no. 1, pp. 119-139, Aug. 1997.
29. L. Breiman, "Random forests," Mach. Learn., vol. 45, no. 1, pp. 5-32, 2001.
30. R.-E. Fan, P.-H. Chen, and C.-J. Lin, "Working set selection using the second order information for training support vector machines," J. Mach. Learn. Res., vol. 6, pp. 1889-1918, Dec. 2005.
31. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
32. J. A. Blackard and D. J. Dean, "Comparative accuracies of artificial neural networks and discriminant analysis in predicting forest cover types from cartographic variables," Comput. Electron. Agriculture, vol. 24, no. 3, pp. 131-151, Dec. 1999.
33. Z. Fu, A. Robles-Kelly, R. T. Tan, and T. Caelli, "Invariant object material identification via discriminant learning on absorption features," in Proc. Comput. Vis. Pattern Recognit. Workshop, Jun. 2006, p. 140.
34. G. R. G. Lanckriet, N. Cristianini, P. Bartlett, L. E. Ghaoui, and M. I. Jordan, "Learning the kernel matrix with semidefinite programming," J. Mach. Learn. Res., vol. 5, pp. 27-72, Dec. 2004.
35. M. Hu, Y. Chen, and J. T.-Y. Kwok, "Building sparse multiple-kernel SVM classifiers," IEEE Trans. Neural Netw., vol. 20, no. 5, pp. 827-839, May 2009.
36. P. L. Bartlett, S. Boucheron, and G. Lugosi, "Model selection and error estimation," Mach. Learn., vol. 48, nos. 1-3, pp. 85-113, 2002.
37. S. Decherchi, S. Ridella, R. Zunino, P. Gastaldo, and D. Anguita, "Using unsupervised analysis to constrain generalization bounds for support vector classifiers," IEEE Trans. Neural Netw., vol. 21, no. 3, pp. 424-438, Mar. 2010.