Non-sparse multiple kernel fisher discriminant analysis

Journal of Machine Learning Research

Volumn 13, Issue , 2012, Pages 607-642

  Yan, Fei ^a   Kittler, Josef ^a   Mikolajczyk, Krystian ^a   Tahir, Atif ^a  

^a UNIVERSITY OF SURREY   (United Kingdom)

Author keywords

Kernel fisher discriminant analysis; Multiple kernel learning; Regularised least squares; Support vector machines

Indexed keywords

APPLICATION AREA; DATA SETS; KERNEL FISHER DISCRIMINANT ANALYSIS; KERNEL MACHINE; KERNEL WEIGHT; LEAST SQUARE; MULTIPLE KERNEL LEARNING; MULTIPLE KERNELS; OPTIMISATIONS; REGULARISATION; SEMI-INFINITE PROGRAMS; SUPPORT VECTOR MACHINE (SVM); UNIFIED FRAMEWORK;

OPTIMIZATION;

SUPPORT VECTOR MACHINES;

EID: 84859457072     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (73)

1. F. Bach and G. Lanckriet. Multiple kernel learning, conic duality, and the smo algorithm. In International Conference on Machine Learning, 2004.
2. G. Baudat and F. Anouar. Generalized discriminant analysis using a kernel approach. Neural Computation, 12:2385-2404, 2000.
3. O. Bousquet and D. Herrmann. On the complexity of learning the kernel matrix. In Advances in Neural Information Processing Systems, 2003.
4. M. Braun, J. Buhmann, and K. Müller. On relevant dimensions in kernel feature spaces. Journal of Machine Learning Research, 9:1875-1908, 2008.
5. D. Cai, X. He, and J. Han. Efficient kernel discriminant analysis via spectral regression. In International Conference on Data Mining, 2007.
6. H. Cai, K. Mikolajczyk, and J. Matas. Learning linear discriminant projections for dimensionality reduction of image descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(2):338-352, 2011.
7. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee. Choosing multiple parameters for support vector machines. Machine Learning, 46:131-159, 2002. (Pubitemid 34129966)
8. C. Cortes, M. Mohri, and A. Rostamizadeh. L2 regularization for learning kernels. In Uncertainty in Artificial Intelligence, 2009.
9. N. Cristianini, J. Shawe-Taylor, A. Elisseeff, and J. Kandola. On kernel-target alignment. In Advances in Neural Information Processing Systems, 2002.
10. G. Csurka, C. Dance, L. Fan, J. Willamowski, and C Bray. Visual categorization with bags of keypoints. In ECCV workshop on Statistical Learning in Computer Vision, 2004.
11. R. Duda, P. Hart, and D. Stork. Pattern Classification. Wiley, 2000.
12. M. Everingham, L. van Gool, C. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results. http://www.pascal-network.org/challenges/VOC/voc2007/workshop/index.html, 2007.
13. M. Everingham, L. van Gool, C. Williams, J. Winn, and A. Zisserman. The PASCAL Visual Object Classes Challenge 2008 (VOC2008) Results. http://www.pascal-network.org/challenges/VOC/voc2008/workshop/index.html, 2008.
14. T. Evgeniou, M. Pontil, and T. Poggio. Regularization networks and support vector machines. Advances in Computational Mathematics, 13:1-50, 2000.
15. L. Fei-Fei, R. Fergus, and P. Perona. One-shot learning of object categories. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(4):594-611, 2006. (Pubitemid 43264657)
16. R. Fisher. The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7:179-188, 1936.
17. G. Fung and O. L. Mangasarian. Proximal support vector machine classifier. In International Conference on Knowledge Discovery and Data Mining, 2001.
18. P. Gehler and S. Nowozin. On feature combination for multiclass object classification. In International Conference on Computer Vision, 2009.
19. J. Gemert, J. Geusebroek, C. Veenman, and A. Smeulders. Kernel codebooks for scene categorization. In European Conference on Computer Vision, 2008.
20. T. Gestel, J. Suykens, G. Lanckriet, A. Lambrechts, B. Moor, and J. Vandewalle. Bayesian framework for least-squares support vector machine classifiers, gaussian processes, and kernel fisher discriminant analysis. Machine Learning, 14(5):1115-1147, 2002.
21. F. Girosi, M. Jones, and T. Poggio. Regularization theory and neural networks architectures. Neural Computation, 7:219-269, 1995.
22. G. Golub and C. van Loan. Matrix Computations. John Hopkins University Press, third edition, 1996.
23. K. Grauman and T. Darrell. The pyramid match kernel: Efficient learning with sets of features. Journal of Machine Learning Research, 8:725-760, 2007. (Pubitemid 46677049)
24. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning. Springer, second edition, 2002.
25. R. Herbrich, T. Graeple, and C. Campbell. Bayes point machines. Journal of Machine Learning Research, 1:245-279, 2001. (Pubitemid 33687204)
26. R. Hettich and K. Kortanek. Semi-infinite programming: Theory, methods, and applications. SIAM Review, 35(3):380-429, 1993. (Pubitemid 23712350)
27. T. Joachims. Making Large-Scale Support Vector Machine Learning Practical. MIT Press, Cambridge, MA 1988.
28. S. Keerthi and S. Shevade. Smo algorithm for least squares svm formulations. Neural Computation, 15(2):487-507, 2003. (Pubitemid 37049831)
29. S. Kim, A. Magnani, and S. Boyd. Optimal kernel selection in kernel fisher discriminant analysis. In International Conference on Machine Learning, 2006.
30. M. Kloft, U. Brefeld, P. Laskov, and S. Sonnenburg. Non-sparse multiple kernel learning. In NIPS Workshop on Kernel Learning: Automatic Selection of Optimal Kernels, 2008.
31. M. Kloft, U. Brefeld, S. Sonnenburg, and A. Zien. Efficient and accurate lp-norm mkl. In Advances in Neural Information Processing Systems, 2009.
32. M. Kloft, U. Brefeld, S. Sonnenburg, and A. Zien. Lp norm multiple kernel learning. Journal of Machine Learning Research, 12:953-997, 2011.
33. G. Lanckriet, N. Cristianini, P. Bartlett, L. E. Ghaoui, and M. Jordan. Learning teh kernel matrix with semi-definite programming. In International Conference on Machine Learning, 2002.
34. G. Lanckriet, N. Cristianini, P. Bartlett, L. E. Ghaoui, and M. Jordan. Learning the kernel matrix with semidefinite programming. Journal of Machine Learning Research, 5:27-72, 2004.
35. S. Lazebnik, C. Schmid, and J. Ponce. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In International Conference on Computer Vision and Pattern Recognition, 2006.
36. D. Liu and J. Nocedal. On the limited memory method for large scale optimization. Mathematical Programming B, 45(3):503-528, 1989. (Pubitemid 20660315)
37. J. Lopez and J. Suykens. First and second order smo algorithms for ls-svm classifiers. Neural Processing Letters, 33(1):31-44, 2011.
38. D. Lowe. Distincetive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2):91-110, 2004.
39. S. Mika. Kernel fisher discriminants. PhD Thesis, University of Technology, Berlin, Germany, 2002.
40. S. Mika, G. Rätsch, J. Weston, B. Schölkopf, and K. Müller. Fisher discriminant analysis with kernels. In IEEE Signal Processing Society Workshop: Neural Networks for Signal Processing, 1999.
41. K. Mikolajczyk and C. Schmid. A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(10):1615-1630, 2005. (Pubitemid 41555406)
42. M. Momma, K. Hatano, and H. Nakayama. Ellipsoidal support vector machines. In Asian Conference on Machine Learning, 2010.
43. S. Nakajima, A. Binder, C. Muller, W. Wojcikiewicz, M. Kloft, U. Brefeld, K. Müller, and M. Kawanabe. Multiple kernel learning for object classification. Technical Report on Information-Based Induction Sciences, 2009.
44. M. Nilsback and A. Zisserman. Automated flower classification over a large number of classes. In Indian Conference on Computer Vision, Graphics and Image Processing, 2008.
45. C. Ong and A. Zien. An automated combination of kernels for predicting protein subcellular localization. In Workshop on Algorithms in Bioinformatics, 2008.
46. C. Ong, A. Smola, and R. C. Williamson. Hyperkernels. In Advances in Neural Information Processing Systems, 2003.
47. F. Orabona and L. Jie. Ultra-fast optimization algorithm for sparse multi kernel learning. In International Conference on Machine Learning, 2011.
48. F. Orabona, L. Jie, and B. Caputo. Online-batch strongly convex multi kerenl learning. In International Conference on Computer Vision and Pattern Recognition, 2010.
49. T. Poggio, S. Mukherjee, R. Rifkin, A. Rakhlin, and A. Verri. B. In Conference on Uncertainty in Geometric Computations, 2004.
50. A. Rakotomamonjy, F. Bach, Y. Grandvalet, and S. Canu. Simplemkl. Journal of Machine Learning Research, 9:2491-2521, 2008.
51. G. Rätsch. Robust boosting via convex optimization. PhD Thesis, University of Potsdam, Potsdam, Germany, 2001.
52. R. Rifkin. Everything old is new again: a fresh look at historical approaches in machine learning. PhD Thesis, Massachusetts Institute of Technology, Boston, USA, 2002.
53. P. Rujan. Playing billiard in version space. Neural Computation, 9:99-122, 1997.
54. K. Sande, T. Gevers, and C. Snoek. Evaluation of color descriptors for object and scene recognition. In International Conference on Computer Vision and Pattern Recognition, 2008.
55. C. Saunders, A. Gammerman, and V. Vovk. Ridge regression learning algorithm in dual variables. In International Conference on Machine Learning, 1998.
56. B. Schölkopf and A. Smola. Learning with Kernels. MIT Press, 2002.
57. B. Schölkopf, A. Smola, and K. Müller. Kernel principal component analysis. Advances in Kernel Methods: Support Vector Learning, pages 327-352, 1999.
58. J. Shawe-Taylor and N. Cristianini. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.
59. C. Snoek, M. Worring, J. Gemert, J. Geusebroek, and A. Smeulders. The challenge problem for automated detection of 101 semantic concepts in multimedia. In ACM Multimedia Conference, 2006.
60. S. Sonnenburg, G. Rätsch, C. Schafer, and B. Schölkopf. Large scale multiple kernel learning. Journal of Machine Learning Research, 7:1531-1565, 2006. (Pubitemid 44373694)
61. S. Sonnenburg, G. Rätsch, S. Henschel, C. Widmer, J. Behr, A. Zien adn F. Bona, A. Binder, C. Gehl, and V. Franc. The shogun machine learning toolbox. Journal of Machine Learning Research, 11: 1799-1802, 2010.
62. J. Suykens and J. Vandewalle. Least squares support vector machine classifiers. Neural Processing Letters, 9:293-300, 1999.
63. M. Szafranski, Y. Grandvalet, and A. Rakotomamonjy. Composite kernel learning. In International Conference on Machine Learning, 2008.
64. A. Tahir, J. Kittler, K. Mikolajczyk, F. Yan, K. Sande, and T. Gevers. Visual category recognition using spectral regression and kernel discriminant analysis. In International Workshop on Subspace Methods, 2009.
65. A. Tikhonov and V. Arsenin. Solutions of Ill-Posed Problems. Winston, Washington DC, 1977.
66. V. Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, 1999.
67. S. Vishwanathan, Z. Sun, and N. Theera-Ampornpunt. Multiple kernel learning and the smo algorithm. In Advances in Neural Information Processing Systems, 2010.
68. F. Yan, J. Kittler, K. Mikolajczyk, and A. Tahir. Non-sparse multiple kernel learning for fisher discriminant analysis. In International Conference on Data Mining, 2009a.
69. F. Yan, K. Mikolajczyk, J. Kittler, and A. Tahir. A comparison of l1 norm and l2 norm multiple kernel svms in image and video classification. In International Workshop on Content-Based Multimedia Indexing, 2009b.
70. F. Yan, K. Mikolajczyk, M. Barnard, H. Cai, and J. Kittler. Lp norm multiple kernel fisher discriminant analysis for object and image categorisation. In International Conference on Computer Vision and Pattern Recognition, 2010.
71. J. Ye, S. Ji, and J. Chen. Multi-class discriminant kernel learning via convex programming. Journal of Machine Learning Research, 9:719-758, 2008.
72. J. Zhang, M. Marszalek, S. Lazebnik, and C. Schmid. Local features and kernels for classification of texture and object categories: A comprehensive study. International Journal of Computer Vision, 73(2):213-238, 2007. (Pubitemid 46181625)
73. A. Zien and C. Ong. Multiclass multiple kernel learning. In International Conference on Machine Learning, 2007.