SVM regression through variational methods and its sequential implementation

Neurocomputing

Volumn 55, Issue 1-2, 2003, Pages 151-167

  Gao, J B ^a   Gunn, S R ^b   Harris, C J ^b  

^a UNIVERSITY OF NEW ENGLAND   (Australia)

^b UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

Author keywords

Bayesian inference; Bayesian variational algorithm; EM algorithm; Support vector machine

Indexed keywords

MATHEMATICAL MODELS; PARAMETER ESTIMATION; REGRESSION ANALYSIS;

KERNEL METHODS;

NEURAL NETWORKS;

ARTICLE; CONTROLLED STUDY; DATA BASE; LEARNING; MATHEMATICAL MODEL; METHODOLOGY; NORMAL DISTRIBUTION; PARAMETER; PREDICTION; PRIORITY JOURNAL; REGRESSION ANALYSIS;

EID: 0242383459     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0925-2312(03)00365-5     Document Type: Article

References (37)

1. H. Akaike, A new look at the statistical model identification, IEEE Trans. Automatic Control 19 (6) (1974) 716-723.
2. D. Barber, C. Williams, Gaussian processes for Bayesian classification via hybrid Monte Carlo, in: M. Mozer, M. Jordan, T. Petsche (Eds.), Neural Information Processing Systems, Vol. 9, MIT Press, Cambridge, MA, 1997, pp. 340-346.
3. G. Cauwenberghs, T. Poggio, Incremental and decremental support vector machine learning, in: L. Todd, T.G. Dietterich, V. Tresp (Eds.), Proceeding of NIPS, Vol. 13, MIT Press, Cambridge, MA, 2001.
4. L. Csató, M. Opper, Sparse online Gaussian processes, Neural Computation 14 (3) (2002) 641-668.
5. A. Dempster, N. Laird, D. Rubin, Maximum-likelihood from incomplete data via the EM algorithm, J. Roy. Statist. Soc. Set. B 39 (1977) 1-38.
6. T. Evgeniou, M. Pontil, T. Poggio, A unified framework for regularization networks and support vector machines, A.I. Memo 1654, AI Lab, MIT, MA, 1999.
7. J. Gao, S. Gunn, C. Harris, M. Brown, Regression with input-dependent noise: a relevance vector machine treatment, Research report, ISIS Research Group, Department of Electronics and Computer Science, University of Southampton, May 2000.
8. J. Gao, S. Gunn, C. Harris, M. Brown, A probabilistic framework for SVM regression and error bar estimation, Machine Learn. 46 (2002) 71-89.
9. M. Gibbs, D. MayKay, Variational Gaussian process classifiers. Technical report, Cavendish Laboratory, University of Cambridge, Manuscript, 1997.
10. F. Girosi, M. Jones, T. Poggio, Regularization theory and neural networks architectures, Neural Comput. 7 (1995) 219-269.
11. S. Gunn, Support vector machines for classification and regression, Technical report, ISIS, Department of Electronics and Computer Science, University of Southampton, 1998.
12. D. Heckerman, D. Geiger, D. Chickering, Learning Bayesian networks: the combination of knowledge and statistical data, Machine Learn. 20 (3) (1995) 197-201.
13. T. Jaakkola, D. Haussler, Probabilistic kernel regression models, in: Proceedings of The Seventh International Workshop on Artificial Intelligence and Statistics, 1998.
14. T. Jaakkola, M. Jordan, Variational probabilistic inference and the qmr-dt database, J. Artificial Intelligence Res. 10 (1999) 291-322.
15. T. Jaakkola, M. Jordan, Bayesian parameter estimation through variational methods, Statistics and Computing 10 (2000) 25-37.
16. T. Joachims, Making large-scale SVM learning practical, in: B. Schölkopf, C. Burges, A. Smola (Eds.), Advances in Kernel Methods - Support Vector Learning, MIT Press, Cambridge, MA, 1999, pp. 169-184.
17. Y. Li, Y. Lee, G. Wahba, Support Vector Machines for classification in nonstandard situations, Technical Report 1016, Department of Statistics, University of Wisconsin, Madison, Wisconsin, 2000.
18. D. MacKay, Gaussian processes, A replacement for neural networks, NIPS tutorial 1997, Cambridge University, 1997.
19. N. Murata, S. Yoshizawa, S. Amari, Network information criterion-determining the number of hidden units for artificial neural network models, IEEE Trans. Neural Networks 5 (1994) 865-872.
20. R. Neal, Monte Carlo implementation of Gaussian process models for Bayesian regression and classification, Technical Report CRG-TR-97-2, Dept. of Computer Science, University of Toronto, 1997.
21. M. Opper, O. Winther, Gaussian processes for classification: mean field algorithms, Neural Computation 12 (2000) 2655-2684.
22. E. Osuna, R. Freund, F. Girosi, An improved training algorithm for support vector machines, in: J. Principe, L. Gile, N. Morgan, E. Wilson (Eds.), Neural Networks for Signal Processing VII -Proceedings of the 1997 IEEE Workshop, IEEE, New York, 1997, pp. 276-285.
23. J. Platt, Fast training of support vector machines using sequential minimal optimization, in: B. Schölkopf, C. Burges, A. Smola (Eds.), Advances in Kernel Methods - Support Vector Learning, MIT Press, Cambridge, MA, 1999, pp. 185-208.
24. M. Pontil, S. Mukherjee, F. Girosi, On the noise model of support vector machine regression, A.I. Memo 1651, AI Laboratory, MIT, 1998.
25. R. Redner, H. Walker, Mixture densities, miximum likelihood and the EM algorithm, SIAM Rev. 26 (2) (1984) 195-235.
26. A. Smola, Learning with Kernels, Ph.D. Thesis, Technischen Universität Berlin, Berlin, Germany, 1998.
27. P. Sollich, Approximate learning curves for Gaussian processes, in: ICANN99: Ninth International Conference on Artificial Neural Networks, London, The Institution of Electrical Engineers, 1999, pp. 437-442.
28. P. Sollich, Probabilistic interpretations and Bayesian methods for support vector machines, in: CANN99 - Ninth International Conference on Artificial Neural Networks, London, The Institution of Electrical Engineers, 1999, pp. 91-96.
29. A. Tikhonov, V. Arsenin, Solution of I11-posed Problems, W.H. Winston, Washington, DC, 1977.
30. R. Vanderbei, LOGO: an interior point code for quadratic programming, TR SOR-94-15, Statistics and Operations Research, Princeton University, NJ, 1994.
31. V. Vapnik, The Nature of Statistical Learning Theory, Springer, New York, 1995.
32. V. Vapnik, Statistical Learning Theory, Wiley, New York, 1998.
33. G. Wahba, Splines Models for Observational Data, in: Series in Applied Mathematics, Vol. 59, SIAM Press, Philadelphia, 1990.
34. C. Williams, Computing with infinite networks, in: M. Mozer, M. Jordan, T. Petsche (Eds.), Neural Information Processing Systems, Vol. 9, MIT Press, Cambridge, MA, 1997, pp. 295-301.
35. C. Williams, Prediction with Gaussian processes: from linear regression to linear prediction and beyond, in: M. Jordan (Ed.), Learning in Graphical Models, MIT Press, Cambridge, MA, 1998, pp. 599-621.
36. C. Williams, D. Barber, Bayesian classification with Gaussian processes, IEEE Trans. Pattern Anal. Machine Intelligence 20 (1999) 1342-1351.
37. C. Williams, C. Rasmuseen, Gaussian processes for regression, in: D. Touretzky, M. Mozer, M. Hasselmo (Eds.), Neural Information Processing Systems, Vol. 8, MIT Press, Cambridge, MA, 1997, pp. 514-520.