Parsimonious least squares support vector regression using orthogonal forward selection with the generalised kernel model

International Journal of Modelling, Identification and Control

Volumn 1, Issue 4, 2006, Pages 245-256

  Wang, Xunxian ^a   Lowe, David ^a   Chen, Sheng ^b   Harris, Chris J ^b  

^a ASTON UNIVERSITY   (United Kingdom)

^b UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

Author keywords

Generalised kernel model; Least squares support vector machine; Orthogonal least squares forward selection; Regression; Sparse modelling

Indexed keywords

COVARIANCE MATRIX; DOMAIN DECOMPOSITION METHODS; HILBERT SPACES; LAGRANGE MULTIPLIERS; LEAST SQUARES APPROXIMATIONS; MATHEMATICAL MODELS; VECTORS;

GENERALISED KERNEL MODEL; ORTHOGONAL LEAST SQUARES FORWARD SELECTION;

REGRESSION ANALYSIS;

EID: 34248656230     PISSN: 17466172     EISSN: 17466180     Source Type: Journal    
DOI: 10.1504/IJMIC.2006.012612     Document Type: Article

References (37)

1. Akaike, H. (1974) 'A new look at the statistical model identification', IEEE Transactions on Automatic Control, Vol. AC-19, pp.716-723.
2. Aronszajn, N. (1950) 'Theory of reproducing kernels', Transactions of the American Mathematical Society, Vol. 68, pp.337-404.
3. Billings, S.A., Chen, S. and Backhouse, R.J. (1989) 'The identification of linear and non-linear models of a turbocharged automotive diesel engine', Mechanical Systems and Signal Processing, Vol. 3, No. 2, pp. 123-142.
4. Breiman, L. (1999) 'Prediction games and arcing algorithms', Neural Computation, Vol. 11, No. 7, pp.1493-1518.
5. Chapelle, O., Vapnik, V., Bousquet, O. and Mukherjee, S. (2002) 'Choosing multiple parameters for support vector machines', Machine Learning, Vol. 46, Nos. 1-3, pp. 131-159.
6. Chen, S., Billings, S.A. and Luo, W. (1989) 'Orthogonal least squares methods and their application to non-linear system identification', International Journal of Control, Vol. 50, No. 5, pp.1873-1896.
7. Chen, S., Cowan, C.F.N. and Grant, P.M. (1991) 'Orthogonal least squares learning algorithm for radial basis function networks', IEEE Transactions on Neural Networks, Vol. 2, No. 2, pp.302-309.
8. Chen, S. and Luk, B.L. (1999) 'Adaptive simulated annealing for optimization in signal processing applications', Signal Processing, Vol. 79, No. 1, pp.117-128.
9. Chen, S., Wu, Y. and Luk, B.L. (1999) 'Combined genetic algorithm optimisation and regularised orthogonal least squares learning for radial basis function networks', IEEE Transactions on Neural Networks, Vol. 10, No. 5, pp.1239-1243.
10. Chen, S., Hong, X. and Harris, C.J. (2003) 'Sparse kernel regression modelling using combined locally regularized orthogonal least squares and D-optimality experimental design', IEEE Transactions on Automatic Control, Vol. 48, No. 6, pp.1029-1036.
11. Chen, S., Hong, X., Harris, C.J. and Sharkey, P.M. (2004) 'Sparse modelling using orthogonal forward regression with PRESS statistic and regularization', IEEE Transactions on Systems, Man and Cybernetics, Part B, Vol. 34, No. 2, pp.898-911.
12. Chen, S., Wang, X.X. and Harris, C.J. (2005) 'Experiments with repeating weighted boosting search for optimization in signal processing applications', IEEE Transactions on Systems, Man and Cybernetics, Part B, Vol. 35, No. 4, pp.682-693.
13. Cristianini, N. and Shawe-Taylor, J. (2000) An Introduction to Support Vector Machines: And Other Kernel-Based Learning Methods, Cambridge, UK: Cambridge University Press.
14. de Kruif, B.J. and de Vries, T.J.A. (2003) 'Pruning error minimization in least squares support vector machines', IEEE Transactions on Neural Networks, Vol. 14, No. 3, pp.696-702.
15. Duan, K., Keerthi, S.S. and Poo, A.N. (2003) 'Evaluation of simple performance measures for tuning SVM hyperparameters', Neurocomputing, Vol. 51, pp.41-59.
16. Freund, Y. and Schapire, R.E. (1997) 'A decision-theoretic generalization of on-line learning and an application to boosting', Journal of Computer and System Sciences, Vol. 55, No. 1, pp.119-139.
17. Goldberg, D.E. (1989) Genetic Algorithms in Search, Optimization and Machine Learning, Reading, MA: Addison Wesley.
18. Gunn, S. (1998) 'Support vector machines for classification and regression', Technical Report, ISIS Research Group, Department of Electronics and Computer Science, University of Southampton, UK, May 1998.
19. Hong, X. and Harris, C.J. (2002) 'Nonlinear model structure design and construction using orthogonal least squares and D-optimality design', IEEE Transactions on Neural Networks, Vol. 13, No. 5, pp.1245-1250.
20. Hong, X., Harris, C-J., Chen, S. and Sharkey, P.M. (2003) 'Robust nonlinear model identification methods using forward regression', IEEE Transactions on Systems, Man and Cybernetics, Part A, Vol. 33, No. 4, pp.514-523.
21. Ingber, L. (1993) 'Simulated annealing: practice versus theory', Mathematical and Computer Modeling, Vol. 18, No. 11, pp.29-57.
22. Lanckriet, R.G., Cristianini, N., Bartlett, P., Ghaoui, L.E. and Jordan, M.I. (2004) 'Learning the kernel matrix with semidefinite programming', Journal of Machine Learning Research, Vol. 5, pp.27-72.
23. Leontaritis, I.J. and Billings, S.A. (1987) 'Model selection and validation methods for non-linear systems', International Journal of Control, Vol. 45, No. 1, pp.311-341.
24. Man, K.F., Tang, K.S. and Kwong, S. (1998) Genetic Algorithms: Concepts and Design, London: Springer-Verlag.
25. Meir, R. and Rätsch, G. (2003) 'An introduction to boosting and leveraging', in S. Mendelson and A. Smola (Eds). Advanced Lectures in Machine Learning. Springer Verlag, pp.119-184.
26. Murphy, P.M. and Aha, D.W. (1992) UCI Repository of Machine Learning Databases Available at: http://www.ics.uci. edu/~mlearn/ MLRepository.html.
27. Ong, C.S., Smola, A.J. and Williamson, R.C. (2005) 'Learning the kernel with hyperkernels', Journal of Machine Learning Research, Vol. 6, pp.1043-1071.
28. Schapire, R.E. (1990) 'The strength of weak learnability', Machine Learning, Vol. 5, No. 2, pp.197-227.
29. Schölkopf, B., Sung, K.K., Burges, C.J.C., Girosi, F., Niyogi, P., Poggio, T. and Vapnik, V. (1997) 'Comparing support vector machines with Gaussian kernels to radial basis function classifiers', IEEE Transactions on Signal Processing, Vol. 45, No. 11, pp.2758-2765.
30. Schölkopf, B., Smola, A.J., Williamson, R.C. and Bartlett, P.L. (2000) 'New support vector algorithms', Neural Computation, Vol. 12, No. 5, pp.1207-1245.
31. Schölkopf, B. and Smola, A.J. (2002) Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, Cambridge, MA: MIT Press.
32. Suykens, J.A.K. and Vandewalle, J. (1999) 'Least squares support vector machine classifiers', Neural Processing Letters, Vol. 9, No. 3, pp.293-300.
33. Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B. and Vandewalle, J. (2002) Least Squares Support Vector Machines, Singapore: World Scientific.
34. Van Gestel, T., Suykens, J.A.K., Baestaens, D-E., Lambrechts, A., Lanckriet, G., Vandaele, B., De Moor, B. and Vandewalle, J. (2001) 'Financial time series prediction using least squares support vector machines within the evidence framework', IEEE Transactions on Neural Networks, Vol. 12, No. 4, pp.809-821.
35. Vapnik, V. (1995) The Nature of Statistical Learning Theory, New York: Springer-Verlag.
36. Vapnik, V., Golowich, S. and Smola, A. (1997) 'Support vector method for function approximation, regression estimation and signal processing', in M.C. Mozer, M.I. Jordan and T. Petsche (Eds). Advances in Neural Information Processing Systems 9, Cambridge, MA: MIT Press, pp.281-287.
37. Vincent, P. and Bengio, Y. (2002) 'Kernel matching pursuit', Machine Learning, Vol. 48, No. 1, pp. 165-187.