Efficient sparse generalized multiple kernel learning

IEEE Transactions on Neural Networks

Volumn 22, Issue 3, 2011, Pages 433-446

  Yang, Haiqin ^a   Xu, Zenglin ^b,c,d   Ye, Jieping ^e   King, Irwin ^a   Lyu, Michael R ^a  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^b SAARLAND UNIVERSITY   (Germany)

^c MAX PLANCK INSTITUTE FOR INFORMATICS   (Germany)

^d Purdue University   (United States)

^e Arizona State University   (United States)

Author keywords

Grouping effect; Kernel methods; level method; multiple Kernel learning

Indexed keywords

COMBINATION WEIGHTS; CONVEX OPTIMIZATION PROBLEMS; DATA SIMILARITY; GENERALIZATION PERFORMANCE; GLOBAL OPTIMAL SOLUTIONS; GROUPING EFFECT; KERNEL METHODS; KERNEL REPRESENTATION; KERNEL WEIGHT; LEVEL METHOD; LINEAR COMBINATIONS; LOCAL SOLUTION; MULTIPLE KERNEL LEARNING; NON-SMOOTH; OPTIMIZATION PROBLEMS; ORTHOGONAL BASE; PERFORMANCE MEASURE; REAL-WORLD DATASETS; REDUNDANT INFORMATIONS; SPARSE SOLUTIONS; TYPE CONSTRAINTS;

CONVEX OPTIMIZATION;

OPTIMIZATION;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; BIOLOGICAL MODEL; METHODOLOGY; TEACHING;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; TEACHING;

EID: 79952183228     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2103571     Document Type: Article

References (50)

1. K. Huang, H. Yang, I. King, and M. R. Lyu, "Maxi-min margin machine: Learning large margin classifiers locally and globally," IEEE Trans. Neural Netw., vol. 19, no. 2, pp. 260-272, Feb. 2008. (Pubitemid 351359291)
2. K.-Z. Huang, H.-Q. Yang, I. King, and M. Lyu, "Machine learning: Modeling data locally and globally," in Advanced Topics in Science and Tecnology in China, 1st ed. New York: Springer-Verlag, Apr. 2008.
3. B. Schölkopf and A. Smola, Learning with Kernels. Cambridge, MA: MIT Press, 2002.
4. J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis. Cambridge, U.K.: Cambridge Univ. Press, 2004.
5. G. R. G. Lanckriet, T. D. Bie, N. Cristianini, M. I. Jordan, andW. S. Noble, "A statistical framework for genomic data fusion," Bioinformatics, vol. 20, no. 16, pp. 2626-2635, 2004. (Pubitemid 39530149)
6. J. Weston, A. Elisseeff, B. Schölkopf, and M. Tipping, "Use of the zero norm with linear models and kernel methods," J. Mach. Learn. Res., vol. 3, pp. 1439-1461, Mar. 2003.
7. Y. Chen, M. R. Gupta, and B. Recht, "Learning kernels from indefinite similarities," in Proc. 26th Int. Conf. Mach. Learn., Montreal, QC Canada, 2009, pp. 1-8.
8. M. Gönen and E. Alpaydin, "Localized multiple kernel learning," in Proc. Int. Conf. Mach. Learn., Helsinki, Finland, 2008, pp. 352-359.
9. M. Hu, Y. Chen, and J. T.-Y. Kwok, "Building sparse multiple-kernel SVM classifiers," IEEE Trans. Neural Netw., vol. 20, no. 5, pp. 827-839, May 2009.
10. J. Liu, J. Chen, S. Chen, and J. Ye, "Learning the optimal neighborhood kernel for classification," in Proc. Int. Joint Conf. Artif. Intell., 2009, pp. 1144-1149.
11. I. W.-H. Tsang and J. T.-Y. Kwok, "Efficient hyperkernel learning using second-order cone programming," IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 48-58, Jan. 2006. (Pubitemid 43263939)
12. G. R. G. Lanckriet, N. Cristianini, P. Bartlett, L. E. Ghaoui, and M. I. Jordan, "Learning the kernel matrix with semidefinite programming," J. Mach. Learn. Res., vol. 5, pp. 27-72, Jan. 2004.
13. F. R. Bach, G. R. G. Lanckriet, and M. I. Jordan, "Multiple kernel learning, conic duality, and the SMO algorithm," in Proc. 21st Int. Conf. Mach. Learn., Banff, AB, Canada, 2004, pp. 41-48. (Pubitemid 40290790)
14. N. Cristianini, J. Shawe-Taylor, A. Elisseeff, and J. Kandola, "On kernel-target alignment," in Advances in Neural Information Processing Systems 14. Cambridge, MA: MIT Press, 2001, pp. 367-373.
15. J. Ye, J. Chen, and S. Ji, "Discriminant kernel and regularization parameter learning via semidefinite programming," in Proc. Int. Conf. Mach. Learn., 2007, pp. 1095-1102.
16. C. Cortes, M. Mohri, and A. Rostamizadeh, "L2 regularization for learning kernels," in Proc. 25th Conf. Uncertainty Artif. Intell., 2009, pp. 1-8.
17. M. Kloft, U. Brefeld, P. Laskov, and S. Sonnenburg, "Non-sparse multiple kernel learning," in Proc. NIPS Workshop Kernel Learn.: Autom. Sel. Optimal Kernels, Whistler, BC, Canada, 2008, pp. 1-4.
18. M. Kloft, U. Brefeld, S. Sonnenburg, P. Laskov, K.-R. Müller, and A. Zien, "Efficient and accurate lp -norm multiple kernel learning," in Proc. Adv. Neural Inf. Process. Syst. 22, Vancouver, BC, Canada, 2010, pp. 997-1005.
19. S. Sonnenburg, G. Rätsch, and C. Schäfer, "Learning interpretable SVMs for biological sequence classification," in Research in Computational Molecular Biology (LNBI), vol. 3500. Berlin, Germany: Springer- Verlag, 2005, pp. 389-407. (Pubitemid 41420679)
20. Z. Harchaoui and F. Bach, "Image classification with segmentation graph kernels," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Minneapolis, MN, Jun. 2007, pp. 1-8.
21. M. Yuan and Y. Lin, "Model selection and estimation in regression with grouped variables," J. Royal Stat. Soc., Ser. B, vol. 68, no. 1, pp. 49-67, 2006. (Pubitemid 43415335)
22. R. Tibshirani, M. Saunders, S. Rosset, J. Zhu, and K. Knight, "Sparsity and smoothness via the fused lasso," J. Royal Stat. Soc., Ser. B, vol. 67, no. 1, pp. 91-108, 2005. (Pubitemid 40167043)
23. F. R. Bach, "Consistency of the group lasso and multiple kernel learning," J. Mach. Learn. Res., vol. 9, pp. 1179-1225, Jun. 2008.
24. M. Szafranski, Y. Grandvalet, and A. Rakotomamonjy, "Composite kernel learning," in Proc. 25th Int. Conf. Mach. Learn., Helsinki, Finland, 2008, pp. 1040-1047.
25. A. Rakotomamonjy, F. R. Bach, S. Canu, and Y. Grandvalet, "SimpleMKL," J. Mach. Learn. Res., vol. 9, pp. 2491-2521, Nov. 2008.
26. V. Vapnik, The Nature of Statistical Learning Theory, 2nd ed. New York: Springer-Verlag, 1999.
27. C. A. Micchelli and M. Pontil, "Learning the kernel function via regularization," J. Mach. Learn. Res., vol. 6, pp. 1099-1125, Jul. 2005.
28. A. Zien and C. S. Ong, "Multiclass multiple kernel learning," in Proc. 24th Int. Conf. Mach. Learn., Corvallis, OR, 2007, pp. 1191-1198.
29. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
30. S. Sonnenburg, G. Rätsch, C. Schäfer, and B. Schölkopf, "Large scale multiple kernel learning," J. Mach. Learn. Res., vol. 7, pp. 1531-1565, Jul. 2006. (Pubitemid 44373694)
31. Z. Xu, R. Jin, I. King, and M. R. Lyu, "An extended level method for efficient multiple kernel learning," in Advances in Neural Information Processing Systems 21, D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, Eds. 2009, pp. 1825-1832.
32. Z. Xu, R. Jin, H. Yang, I. King, and M. R. Lyu, "Simple and efficient multiple kernel learning by group lasso," in Proc. Int. Conf. Mach. Learn., Haifa, Israel, 2010, pp. 1-8.
33. M. Kowalski, M. Szafranski, and L. Ralaivola, "Multiple indefinite kernel learning with mixed norm regularization," in Proc. 26th Int. Conf. Mach. Learn., Montreal, QC, Canada, 2009, pp. 545-552.
34. S. N. Jagarlapudi, D. Govindaraj, R. S. C. Bhattacharyya, A. Ben- Tal, and K. R. Ramakrishnan, "On the algorithmics and applications of a mixed-norm based kernel learning formulation," in Proc. Adv. Neural Inf. Process. Syst. 22, Y. Bengio, D. Schuurmans, J. Lafferty, C. K. I. Williams, and A. Culotta, Eds. Vancouver, BC, Canada, 2010, pp. 844-852.
35. R. Tomioka and T. Suzuki. (2010). Sparsity-accuracy trade-off in MKL [Online]. Available: http://arxiv.org/abs/1001.2615v1
36. M. Kloft, U. Rückert, and P. Bartlett. (2010). A unifying view of multiple kernel learning. in Proc. Eur. Conf. Mach. Learn. [Online]. Available: http://arxiv.org/abs/1005.0437
37. C. Longworth and M. J. F. Gales, "Combining derivative and parametric kernels for speaker verification," IEEE Trans. Audio, Speech, Lang. Process., vol. 17, no. 4, pp. 748-757, May 2009.
38. J. Shawe-Taylor, "Kernel learning for novelty detection," in Proc. NIPS Workshop Kernel Learn.: Autom. Sel. Optimal Kernels, Dec. 2008, pp. 1-45.
39. H. Zou and T. Hastie, "Regularization and variable selection via the elastic net," J. Royal Stat. Soc., Ser. B, vol. 67, no. 2, pp. 301-320, 2005. (Pubitemid 40465877)
40. I. Steinwart and A. Christmann, Support Vector Machines. New York: Springer-Verlag, 2008.
41. L. Bottou and C.-J. Lin, "Support vector machine solvers," in Large Scale Kernel Machines, L. Bottou, O. Chapelle, D. DeCoste, and J. Weston, Eds. Cambridge, MA: MIT Press, 2007, pp. 301-320.
42. P.-H. Chen, R.-E. Fan, and C.-J. Lin, "A study on SMO-type decomposition methods for support vector machines," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 893-908, Jul. 2006. (Pubitemid 44194150)
43. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Royal Stat. Soc. Ser. B, vol. 58, no. 1, pp. 267-288, 1996.
44. M. Varma and B. R. Babu, "More generality in efficient multiple kernel learning," in Proc. Int. Conf. Mach. Learn., vol. 382. Montreal, QC, Canada, 2009, pp. 1-8.
45. A. Nemirovski, "Efficient methods in convex programming," Faculty Ind. Eng. Manage., Technion - Israel Inst. Technol., Haifa, Israel, Lecture Notes, 1994.
46. A. J. Smola, S. V. N. Vishwanathan, and Q. V. Le, "Bundle methods for machine learning," in Advances in Neural Information Processing Systems 20. Cambridge, MA: MIT Press, 2007, pp. 1-8.
47. J. Neumann and O. Morgenstern, Theory of Games and Economic Behavior, 3rd ed. Princeton, NJ: Princeton Univ. Press, 1953.
48. K. Koh, S.-J. Kim, and S. Boyd, "An interior-point method for largescale l1-regularized logistic regression," J. Mach. Learn. Res., vol. 8, pp. 1519-1555, Jul. 2007. (Pubitemid 47210273)
49. C. Blake and C. Merz. (1998). UCI Repository of Machine Learning Databases [Online]. Available: http://archive.ics.uci.edu/ml/
50. W. Härdle, M. Müller, S. Sperlich, and A. Werwatz, Nonparametric and Semiparametric Models. New York: Springer-Verlag, 2004.