Learning with support vector machines

Synthesis Lectures on Artificial Intelligence and Machine Learning

Volumn 10, Issue , 2011, Pages 1-95

  Campbell, Colin ^a   Ying, Yiming ^b  

^a UNIVERSITY OF BRISTOL   (United Kingdom)

^b UNIVERSITY OF EXETER   (United Kingdom)

Author keywords

classification; data integration; generalization; kernels; learning; optimization; prediction; regression; Support Vector Machine

Indexed keywords

CLASSIFICATION; DATA INTEGRATION; GENERALIZATION; KERNELS; LEARNING; PREDICTION; REGRESSION;

BIOINFORMATICS; CHARACTER RECOGNITION; CLASSIFICATION (OF INFORMATION); DATA HANDLING; FORECASTING; GEARS; OPTIMIZATION; VECTORS;

SUPPORT VECTOR MACHINES;

EID: 79951822517     PISSN: 19394608     EISSN: 19394616     Source Type: Book Series    
DOI: 10.2200/S00324ED1V01Y201102AIM010     Document Type: Article

References (103)

1. E. L. Allwein, R. E. Schapire, and Y. Singer. Reducing multiclass to binary: a unifying approach for margin classifiers. Journal of Machine Learning Research, 1:133-141, 2000. http://www.jmlr.org 8
2. E. D.Anderson and A. D. Anderson. The MOSEK interior point optimizer for linear programming: An implementation of the homogeneous algorithm. InT.Terlaky H. Frenk, C. Roos and S. Zhang, editors, High Performance Optimization, pages 197-232. Kluwer Academic Publishers, 2000. 60
3. F. Bach. Consistency of the group lasso and multiple kernel learning. Journal of Machine Learning Research, 9:1179-1225, 2008. 57
4. F. Bach, G. R. G. Lanckriet, and M. I. Jordan. Multiple kernel learning, conic duality and the SMO algorithm. In Proceedings of theTwenty-first International Conference on Machine Learning (ICML), 2004. DOI: 10.1145/1015330.1015424 57, 60
5. G. Bakir, B. Taskar, T. Hofmann, B. Schölkopf, A. Smola, and S. V. N Viswanathan. Predicting structured data. MIT Press, 2007. 43
6. G. Baudat and F. Anouar. Generalised discriminant analysis using a kernel approach. Neural Computation, 12:2385-2404, 2000. DOI: 10.1162/ 089976600300014980 28
7. M. S. Bazarra,H.D.Sherali, and C. M. Shetty. Nonlinear Programming,Theory and Algorithms.Wiley, 2006. 67
8. K. P. Bennett. Combining support vector and mathematical programming methods for induction. In Advances in Kernel Methods, pages 307-326. MIT Press, 1999. 28
9. D. P. Bertsekas. Nonlinear Programming. Athena Scientific, 2004. 67
10. J. F. Bonnans and A. Shapiro. Optimization problems with perturbation: a guided tour. SIAM Review, 40:202-227, 1998. DOI: 10.1137/S0036144596302644 62
11. K. M. Borgwardt, C. S. Ong, S. Schönauer, and S. V. N. Vishwanathan. Protein function prediction via graph kernels. In Proce (Pubitemid 41794474)
12. L. Bottou,O. Chapelle,D.DeCoste,and J. Weston. Large-Scale Kernel Machines.Neural Information Processing Series, The MIT Press, 2007. 15, 17
13. P. S. Bradley andO.L. Mangasarian. Massive data discrimination via linear support vector machines. Optimization Methods and Software, 13:1-10, 2000. DOI: 10.1080/10556780008805771 28
14. C. Burges.Atutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery, 2:121-167, 1998. DOI: 10.1023/A: 1009715923555 33 (Pubitemid 128126769)
15. C. Campbell and K. P. Bennett. A linear programming approach to novelty detection. In Advances in Neural Information Processing Systems, 14, pages 395-401. MIT Press, 2001. 33
16. C. Campbell, N. Cristianini, and A. Smola. Query learning with large margin classifiers. In Proceedings of ICML2000, pages 111-118, 2000. 25
17. O. Chapelle and V. Vapnik. Model selection for support vector machines. In Advances in Neural Information Processing Systems, 12, pages 673-680. MIT Press, 2000. 50
18. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukerjhee. Choosing multiple parameters for SVM. Machine Learning, 46:131-159, 2002. DOI: 10.1023/A:1012450327387 62 (Pubitemid 34129966)
19. C. Cortes and V. Vapnik. Support vector networks. Machine Learning, 20:273-297, 1995. DOI: 10.1023/A:1022627411411 14
20. N. Cristianini, C. Campbell, and J. Shawe-Taylor. Dynamically adapting kernels in support vector machines. In Advances in Neural Information Processing Systems, 11, pages 204-210. MIT Press, Cambridge, MA, 1999. 50
21. F. Cucker and D. X. Zhou. Learning Theory: An Approximation Theory Viewpoint. Cambridge University Press, 2007. 8
22. T. Damoulas and M. Girolami. Probabilistic multi-class multi-kernel learning: on protein fold recognition and remote homology detection. Bioinformatics, 24:1264-1270, 2008. DOI: 10.1093/bioinformatics/btn112 64 (Pubitemid 351659623)
23. T. G. Dietterich and G. Bakiri. Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence, 2:263-286, 1995. DOI: 10.1613/jair.105 8
24. K.-B.Duan and S. S. Keerthi.Which is the bestmulticlassSVMmethod? an empirical study. Lecture Notes in Computer Science, 3541:278-285, 2006. DOI: 10.1007/11494683-28 8 (Pubitemid 41422626)
25. T. Gartner, P. Flach, and S. Wrobel. On graph kernels: Hardness results and efficient alternatives. In Proceedings Annual Conference Computational Learning Theory (COLT), pages 129-143. Springer, 2003. 56 (Pubitemid 37053200)
26. T. Graepel, R. Herbrich, P. Bollmann-Sdorra, and K. Obermayer. Classification on pairwise proximity data. In Advances in Neural Information Processing Systems, 11, pages 438-444. MIT Press, 1998. 47
27. T. Graepel,R. Herbrich, B. Schölkopf, A. J. Smola, P. L. Bartlett, K. Muller,K.Obermayer, and R. C. Williamson. Classification on proximity data with LP-machines. In Ninth International Conference on Artificial Neural Networks, volume 470, pages 304-309, 1999. DOI: 10.1049/cp:19991126 28 (Pubitemid 30519956)
28. I. Guyon and A. Elisseeff. An introduction to variable and feature selection. Journal of Machine Learning Research, 3:1157-1182, 2003. http://www.jmlr.org 21
29. I. Guyon, J. Weston, S. Barnhill, andV.Vapnik. Gene selection for cancer classification using support vector machines. Machine Learning, 46:389-422, 2002. DOI: 10.1023/A:1012487302797 21 (Pubitemid 34129977)
30. E. Harrington, R. Herbrich, J. Kivinen, J. C. Platt, and R. C. Williamson. Online Bayes point machines. In Proceedings of the Seventh Pacific-Asia Conference on Knowledge Discovery and Data Mining. Springer-Verlag, 2003. DOI: 10.1007/3-540-36175-8-24 28
31. T. Hastie and R. Tibshirani. Classification by pairwise coupling. The Annals of Statistics, 26:451-471, 1998. DOI: 10.1214/aos/1028144844 8
32. R. Herbrich, T. Graepel, and C. Campbell. Bayes point machines. Journal of Machine Learning Research, 1:245-279, 2001. http://www.jmlr.org 28 (Pubitemid 33687204)
33. R. Hettich and K. O. Kortanek. Semi-infinite programming: Theory, methods and applications. SIAM Review, 3:380-429, 1993. DOI: 10.1137/1035089 62, 67
34. A. E. Hoerl and R. Kennard. Ridge regression: biased estimation for nonorthogonal problems. Technometrics, 12:55-67, 1970. DOI: 10.2307/1267352 40
35. T. Joachims. Estimating the generalization performance of an SVM efficiently. In Proceedings of the Seventeenth International Conference on Machine Learning, pages 431-438. Morgan Kaufmann, 2000. 50
36. H. Kashima, K. Tsuda, and A. Inokuchi. Kernels on graphs. In Kernels and Bioinformatics, pages 155-170. MIT Press, Cambridge, MA, 2004. 56
37. S. Keerthi, S. Shevade, C. Bhattacharyya, and K. Murthy. Improvements to platt's SMO algorithm for SVM classifier design. Neural Computation, 13:637-649, 2001. DOI: 10.1162/089976601300014493 17 (Pubitemid 33595014)
38. M. Kloft, U. Brefeld, S. Sonnenburg, P. Laskov, K.-R. Müller, and A. Zien. Efficient and accurate LP-norm multiple kernel learning. In Advances in Neural Information Processing Systems, 22, pages 997-1005. MIT Press, 2009. 64
39. I. R. Kondor and J. D. Lafferty. Diffusion kernels on graphs and other discrete structures. In Proceedings of the International Conference on Machine Learning, pages 315-322. Morgan Kaufmann, San Francisco, CA, 2002. 56
40. G. R. G. Lanckriet, N. Cristianini, P. Bartlett, L. El Ghaoui, and M. I. Jordan. Learning the kernel matrix with semidefinite programming. Journal of Machine Learning Research, 5:27-72, 2004a. 57, 58, 64
41. G. R. G. Lanckriet,T. De Bie,N.Cristianini, M. I. Jordan, andW.S.Noble. A statistical framework for genomic data fusion. Bioinformatics, 2004b. DOI: 10.1093/bioinformatics/bth294 64 (Pubitemid 39530149)
42. Y. Lee, Y. Lin, and G. Wahba. Multicategory support vector machines, 2001. DOI: 10.1198/016214504000000098 8
43. C. Lemaréchal and C. Sagastizabal. Practical aspects of moreau-yosida regularization : theoretical preliminaries. SIAM Journal of Optimization, 7:867-895, 1997. DOI: 10.1137/S1052623494267127 62
44. C. Leslie and R. Kuang. Fast kernels for inexact string matching. Lecture Notes in Computer Science, 2777:114-128, 2003. DOI: 10.1007/978-3-540-45167-9- 10 53 (Pubitemid 37053199)
45. H. Lodhi, C. Saunders, J. Shawe-Taylor, N. Cristianini, and C. Watkins. Text classification using string kernels. Journal of Machine Learning Research, 2:419-444, 2002. http://www.jmlr.org 53 (Pubitemid 135712568)
46. D. G. Luenberger. Linear and Nonlinear Programming. AddisonWesley, 2003. 67
47. R. Luss and A. d'Aspremont. Support vector machine classification with indefinite kernels. In Advances in Neural Information Processing Systems, 20, pages 953-960. MIT Press, 2008. 47
48. A. M. Malyscheff and T. B. Trafalis. An analytic center machine for regression. Central European Journal of Operational Research, 10:297-334, 2002. 28
49. O. L. Mangasarian. Linear and nonlinear separation of patterns by linear programming. Operations Research, 13:444-452, 1965. DOI: 10.1287/opre.13.3.444 28
50. C. A. Micchelli and M. Pontil. Learning the kernel function via regularization. Journal of Machine Learning Research, 9:1049-1125, 2005. 57
51. S. Mika, G. Ratsch, J. Weston, B. Schölkopf, and K.-R. Muller. Fisher discriminant analysis with kernels. In Neural networks for signal processing IX, pages 41-48, 1999. DOI: 10.1109/NNSP.1999.788121 28 (Pubitemid 30510358)
52. T. Minka. A family of algorithms for approximate Bayesian inference. PhD thesis, MIT, 2001. 28
53. Y. Nesterov. Introductory Lectures on Convex Optimization: A Basic Course. Springer, 2003. 62
54. Y. Nesterov and A. Nemirovsky. Interior Point Polynomial Methods in Convex Programming: Theory and Algorithms. SIAM, 1994. 60
55. E. Pekalska, P. Paclik, and R. P.W. Duin. A generalized kernel approach to dissimilarity based classification. Journal of Machine Learning Research, 2:175-211, 2002. http://www.jmlr.org 47
56. J. Platt. Fast training of SVMs using sequential minimal optimization. In C. BurgesB. Schölkopf and A. Smola, editors, Advances in Kernel Methods: Support Vector Learning. MIT press, Cambridge, MA, 1999a. 17
57. J. Platt. Probabilistic outputs for support vector machines and comparison to regularised likelihood methods. In Advances in large margin classifiers, pages 61-74. MIT Press, 1999b. 8, 29
58. J. Platt, N. Cristianini, and J. Shawe-Taylor. Large margin DAGS for multiclass classification. In Advances in Neural Information Processing Systems, 12, pages 547-553. MIT Press, Cambridge, MA, 2000. 8
59. A. Rakotomamonjy, F. Bach, S. Canu, and Y. Grandvalet. SimpleMKL. Journal of Machine Learning Research, 9:2491-2521, 2008. 57, 62
60. C. E. Rasmussen and C. Williams. Gaussian Processes for Machine Learning. The MIT Press, 2006. 30
61. V. Roth and V. Steinhage. Nonlinear discriminant analysis using kernel functions. In Advances in Neural Information Processing Systems, 12, pages 568-574. MIT Press, 2000. 28
62. V. Roth, J. Laub, M. Kawanabe, and J. M. Buhmann. Optimal cluster preserving embedding of nonmetric proximity data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25: 1540-1551, 2003. DOI: 10.1109/TPAMI.2003.1251147 47
63. P. Rujan and M. Marchand. Computing the Bayes kernel classifier. In Advances in Large Margin Classifiers, pages 329-347. MIT Press, 2000. 28
64. C. Saunders, A. Gammermann, and V. Vovk. Ridge regression learning algorithm in dual variables. In J. Shavlik, editor, Proceedings of the Fifteenth International Conference in Machine Learning (ICML). Morgan Kaufmann, 1998. 40
65. B. Schölkopf and A. Smola. Learning with Kernels. The MIT Press, Cambridge, MA, 2002a. 17, 28, 29, 30, 33
66. B. Schölkopf and A. J. Smola. Learning with Kernels. The MIT Press, Cambridge,MA,USA, 2002b. 40
67. B. Schölkopf, C. Burges, and V. Vapnik. Extracting support data for a given task. In U. M. Fayyad and R. Uthurusamy, editors, Proceedings: First International Conference on Knowledge Discovery and Data Mining. AAAI Press, Menlo park, CA, 1995. 33
68. B. Schölkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola, and R. C.Williamson. Estimating the support of a high-dimensional distribution. In Microsoft Research Corporation Technical Report MSR-TR-99-87, 1999. DOI: 10.1162/089976601750264965 33
69. B. Schölkopf,A. Smola,R.C.Williamson, and P. L. Bartlett. New support vector algorithms. Neural Computation, 12:1207-1245, 2000. DOI: 10.1162/089976600300015565 14
70. P. Schölkopf, B. and Bartlett, A. Smola, and R. Williamson. Support vector regression with automatic accuracy control. In M. Boden L. Niklasson and T. Ziemke, editors, Proceedings of the 8th International Conference on Artificial Neural Networks, Perspectives in Neural Computing. Springer Verlag, Berlin, 1998. 40
71. J. Shawe-Taylor and N. Cristianini. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004. 29, 53
72. S. K. Shevade, S. S. Keerthi, C. Bhattacharyya, and K. R. Krishna Murthy. Improvements to the SMO algorithm for SVM regression. IEEE Transactions on Neural Networks, 11:1188-1194, 2000. DOI: 10.1109/72.870050 17
73. A. J. Smola and I. R. Kondor. Kernels and regularization on graphs. In Lecture Notes in Computer Science, pages 144-158. Springer-Verlag, Heidelberg, Germany, 2003. DOI: 10.1007/978-3-540-45167-9-12 56 (Pubitemid 37053201)
74. A. J. Smola and B. Schölkopf. A tutorial on support vector regression. Statistics and Computing, 14: 199-222, 2004. DOI: 10.1023/B:STCO.0000035301.49549.88 40 (Pubitemid 39063488)
75. P. Sollich. Bayesian methods for support vector machines: Evidence and predictive class probabilities. Machine Learning, 46:21-52, 2002. DOI: 10.1023/A:1012489924661 50 (Pubitemid 34129962)
76. S. Sonnenburg,G. Rätsch, C. Schäfer, and B. Schölkopf. Large scalemultiple kernel learning. Journal of Machine Learning Research, 7:1531-1565, 2006. 57, 62
77. S. Sonnenburg, G Ratsch, and K. Rieck. Large-Scale Learning with String Kernels, pages 73-103. MIT Press, 2007. 54
78. I. Steinwart and A. Christmann. Support Vector Machines. Springer, 2008. 8
79. J. F. Sturm. Using SEDUMI 1.02a, a matlab toolbox for optimization over symmetric cones. Optimization Methods and Software, pages 625-653, 1999. DOI: 10.1080/10556789908805766 60
80. J. A.K. Suykens and J. Vandewalle. Least squares support vector machine classifiers. Neural Processing Letters, 9:293-300, 1999. DOI: 10.1023/A:1018628609742 28
81. B. Taskar, V. Chatalbashev, D. Koller, and C. Guestrin. Learning structured prediction models: a large margin approach. In Proceedings of the 22nd International conference on machine learning, 2005. DOI: 10.1145/1102351.1102464 43
82. D. Tax andR.Duin. Data domain description by support vectors. In M. Verleysen, editor,Proceedings of ESANN99, pages 251-256. D. Facto Press, Brussels, 1999. 33
83. D. Tax, A. Ypma,andR.Duin. Support vector data description applied to machine vibration analysis. In M. Boasson, J. Kaandorp, J. Tonino, and M. Vosselman, editors, Proc. 5th Annual Conference of the Advanced School for Computing and Imaging, pages 398-405. Heijen, NL, 1999. 33
84. S. Tong andD. Koller. Support vector machine active learning with applications to text classification. Journal of Machine Learning Research, 2:45-66, 2001. http://www.jmlr.org 25
85. T. B. Trafalis and A. M. Malyscheff. An analytic center machine. Machine Learning, 46:203 - 224, 2002. DOI: 10.1023/A:1012458531022 28 (Pubitemid 34129969)
86. I. Tsochantaridis,T. Joachims,T. Hofmann, and Y. Altun. Large margin methods for structured and independent output variables. Journal of machine learning research, 6:1453-1584, 2005. 43 UCI Machine learning Repository. http://www.ics.uci.edu/∼mlearn/mlrepository.html. 10
87. L. Vandenberghe and S. Boyd. Semidefinite programming. SIAM Review, 38, 1996. DOI: 10.1137/1038003 60, 62, 67
88. V. Vapnik. The Nature of Statistical Learning Theory. Springer, New York, 1995. 40
89. V. Vapnik. Statistical Learning Theory. Wiley, 1998. 7, 40
90. V. Vapnik and O. Chapelle. Bounds on error expectation for support vector machines. Neural Computation, 12:2013-2036, 2000. DOI: 10.1162/ 089976600300015042 50
91. K. Veropoulos, C. Campbell, and N. Cristianini. Controlling the sensitivity of support vector machines. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 1999. 14
92. S. Vishwanathan and A Smola. Fast kernels for string and tree matching. In Advances in Neural Information Processing Systems, 15, pages 569-576. MIT Press, 2003. 53
93. S. V. N. Vishwanathan, K. M. Borgwardt, I. R. Kondor, and N. N. Schraudolph. Graph kernels. Journal of Machine Learning Research, 9:1-41, 2008. 56
94. M. K.Warmuth, J. Liao,G. Ratsch, M. Mathieson, S. Putta, and C. Lemmen. Active learning with support vector machine in the drug discovery process. J. Chem. Inf. Comput. Sci., 43:667-673, 2003. DOI: 10.1021/ci025620t 25
95. J. Weston and C. Watkins. Multi-class support vector machines. InMVerleysen, editor, Proceedings of ESANN99, pages 219-224. D. Facto Press, Brussels, 1999. 8
96. J. Weston, A. Gammerman, M. Stitson,V.Vapnik,V.Vovk, and C. Watkins. Support vector density estimation. In C. Burges B. Schölkopf and A. Smola, editors, Advances in Kernel Methods: Support Vector Machines, pages 293-306. MIT Press, Cambridge, MA, 1998. 40
97. R. D. Williams, S. N. Hing, B. T. Greer, Craig C. Whiteford, J. S. Wei, R. Natrajan, A. Kelsey, S. Rogers, C. Campbell, K. Pritchard-Jones, and J. Khan. Prognostic classification of relapsing favourable histology wilms tumour using cdna microarray expression profiling and support vector machines. Genes, Chromosomes and Cancer, 41:65 - 79, 2004. DOI: 10.1002/gcc.20060 21 (Pubitemid 38988739)
98. Z. Xu, R. Jin, I. King, and M. R. Lyu. An extended level method for multiple kernel learning. In Advances in Neural Information Processing Systems, 22.MIT Press, 2008. 62
99. Y. Ying and C. Campbell. Generalization bounds for learning the kernel. In Proceedings of 22nd Annual Conference on Learning Theory (COLT), 2009. 57
100. Y. Ying andD.X. Zhou. Learnability of gaussians with flexible variances. Journal of Machine Learning Research, 8:249-276, 2007. 57
101. Y. Ying, C. Campbell, and M. Girolami. Analysis of SVM with indefinite kernels. In Advances in Neural Information Processing Systems, 22, pages 2205-2213. MIT Press, 2009a. 47, 62
102. Y. Ying, K. Huang, and C. Campbell. Enhanced protein fold recognition through a novel data integration approach. BMC Bioinformatics, 10:267-285, 2009b.DOI: 10.1186/1471-2105-10-267 62, 64
103. A. L. Yuille and A. Rangarajan. The concave convex procedure. Neural Computation, 15:915-936, 2003. DOI: 10.1162/08997660360581958 43 (Pubitemid 37049812)