Weighted solution path algorithm of support vector regression based on heuristic weight-setting optimization

Neurocomputing

Volumn 73, Issue 1-3, 2009, Pages 495-505

  Mao, Wentao ^a   Yan, Guirong ^a   Dong, Longlei ^a  

^a XI'AN JIAOTONG UNIVERSITY   (China)

Author keywords

Keywords: Support vector machines; Particle swarm optimization; Solution path; Weight setting

Indexed keywords

ARC-TANGENTS; FUNCTION REGRESSION; GENERALIZATION ABILITY; KEYWORDS: SUPPORT VECTOR MACHINES; ON TIME; OPTIMAL WEIGHT; OPTIMIZATION ALGORITHMS; PENALTY PARAMETERS; SOLUTION PATH; SUPPORT VECTOR REGRESSIONS; TIME SERIES PREDICTION; TRAINING SAMPLE; WEIGHT-SETTING;

ALGORITHMS; HEURISTIC ALGORITHMS; HEURISTIC METHODS; REGRESSION ANALYSIS; SUPPORT VECTOR MACHINES; TIME SERIES; VECTORS;

PARTICLE SWARM OPTIMIZATION (PSO);

ARTICLE; LEARNING ALGORITHM; MEASUREMENT ERROR; PRIORITY JOURNAL; REGRESSION ANALYSIS; STATISTICAL MODEL; SUPPORT VECTOR MACHINE; TIME SERIES ANALYSIS;

EID: 70350714263     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2009.06.008     Document Type: Article

References (41)

1. G.E.P. Box, G.M. Jenkins, Time Series Analysis Forecasting and Control, rev. ed., Holden-Day, San Francisco, 1976.
2. C.C. Change, C.J. Lin, LIBSVM: a library for support vector machine, software available at: 〈http://www.csie.ntu.edu.tw/~cjlin/libsvm〉.
3. Chapelle O., Vapnik V.N., Bousquet O., and Mukherjee S. Choosing multiple parameters for support vector machines. J. Mach. Learn. Res. 46 1 (2002) 131-159
4. Cherkassky V., and Ma Y. Practical selection of SVM parameters and noise estimation for SVM regression. Neural Networks 17 (2004) 113-126
5. Clerc M., and Kennedy J. The particle swarm-explosion, stability, and convergence in amultidimensional complex space. IEEE Trans. Evol. Comput. 6 1 (2002) 58-73
6. Craven P., and Wahba G. Smoothing noisy data with spline function. Numer. Math. 31 4 (1979) 377-403
7. H.J. Escalante, M. Montes, L.E. Sucar, Particle swarm full model selection, J. Mach. Learn. Res., 10 (2009) 405-440.
8. Friedman J.H. Multivariate adaptive regression splines. Ann. Statist. 19 1 (1991) 1-67
9. Friedrichs F., and Igel C. Evolutionary tuning of multiple SVM parameters. Neurocomputing 64 (2005) 107-117
10. Gu H., and Wang H.W. Fuzzy prediction of chaotic time series based on singular value decomposition. Appl. Math. Comput. 185 2 (2007) 1171-1185
11. Gunter L., and Zhu J. Efficient computation and model selection for the support vector regression. Neural Comput. 19 (2007) 1633-1655
12. Guo X.C., Yang J.H., Wu C.G., et al. A novel LS-SVMs hyper-parameter selection based on particle swarm optimization. Neurocomputing 71 (2008) 3211-3215
13. Harpham C., and Dawson C.W. The effect of different basis functions on a radial basis function network for time series prediction: a comparative study. Neurocomputing 69 (2006) 2161-2170
14. Hastie T., Rosset S., Tibshirani R., and Zhu J. The entire regularization path for the support vector machine. J. Mach. Learn. Res. 5 (2004) 1391-1415
15. Keerthi S. Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms. IEEE Trans. Neural Networks 13 5 (2002) 1225-1229
16. J. Kennedy, R. Eberhart, Particle swarm optimization, in: Proceedings of IEEE International Conference on Neural Networks, vol. 4, Perth, Australia, IEEE Service Center, Piscataway, NJ, 1995, pp. 1942-1948.
17. J.T. Kwok, Linear dependency between epsilon and the input noise in epsilon-support vector regression, in: Proceedings of the International Conference on Artificial Neural Networks, Lecture Notes in Computer Science, vol. 2130, Springer, London, UK, 2001, pp. 405-410.
18. Lau K.W., and Wu Q.H. Local prediction of non-linear time series using support vector regression. Pattern Recognition 41 5 (2008) 1539-1547
19. Lin C.F., and Wang S.D. Fuzzy support vector machines. IEEE Trans. Neural Networks 13 2 (2002) 464-471
20. Liu H.F., Dai Z.H., Li W.F., Gong X., and Yu Z.H. Noise robust estimates of the largest Lyapunov exponent. Phys. Lett. A 341 (2005) 119-127
21. G. Loosli, G. Gasso, S. Canu, Regularization paths for ν-SVM and ν-SVR, in: D. Liu, S. Fei, Z.G. Hou, H. Zhang, C.Y. Sun (Eds.), Advances in Neural Networks-ISNN, Lecture Notes in Computer Science, vol. 4493, Springer, Berlin, 2007, pp. 486-496.
22. Lorenz E. Deterministic nonperiodic flow. J. Atmos. Sci. 20 (1963) 130
23. S. Mukherjee, E. Osuna, F. Girosi, Nonlinear prediction of chaotic time series using support vector machines, in: Proceedings of the IEEE Workshop on Neural Network for Signal Processing (NNSP'97), IEEE Press, Amelia Island, FL, 1997, pp. 511-520.
24. K.R. Müller, A.J. Smola, G. Rätsch, B. Schölkopf et al., Predicting time series with support vector machines, in: Proceedings of International Conference on Artificial Neural Networks, 1997, Lausanne, Switzerland, pp. 999-1004.
25. J.S. Pei, E.C. Mai, J.P. Wright, A.W. Smyth, Neural network initialization with prototypes-function approximation in engineering mechanics applications, in: Proceedings of 2007 International Joint Conference on Neural Networks (IJCNN 2007), IEEE Press, Orlando, FL, 2007, pp. 2110-2116.
26. Rosset S., and Zhu J. Piecewise linear regularized solution paths. Ann. Statist. 35 3 (2007) 1012-1030
27. Schölkopf B., Smola A.J., Williamson R.C., and Bartlett P.L. New support vector algorithms. Neural Comput. 12 (2000) 1207-1245
28. Y.H. Shi, R.Eberhart, A modified particle swarm optimizer, in: IEEE World Congress on Computer Intelligence, vol. 1, Alaska, ALTEC, 1998, pp. 69-73.
29. A.J. Smola, N. Murata, B. Schölkopf, K.R. Müller, Asymptotically optimal choice of ε-loss for support vector machines, in: Proceedings of the 8th International Conference on Artificial Neural Networks (ICANN 98), Perspectives in Neural Computing, Springer, Berlin, 1998, pp. 105-110.
30. A.J. Smola, B. Schölkopf, A tutorial on support vector regression, NeuroCOLT Technical Report TR, Royal Holloway College, London, UK, 1998.
31. Stein C. Estimation of the mean of a multivariate normal distribution. Ann. Statist. 9 6 (1981) 1135-1151
32. Suykens J.A.K., Brabanter J.D., Lukas L., and Vandewalle J. Weighted least squares support vector machines: robustness and sparse approximation. Neurocomputing 48 (2002) 85-105
33. Tay F.E.H., and Cao L.J. Modified support vector machines in financial time series forecasting. Neurocomputing 48 (2002) 847-861
34. Vapnik V.N. The Nature of Statistical Learning Theory (1995), Springer, New York
35. Vapnik V.N., Golowich S., and Smola A.J. Support vector method for function approximation, regression estimation, and signal processing. In: Mozer M., Jordan M., and Petsche T. (Eds). Advances in Neural Information Processing Systems vol. 9 (1997), MIT Press, Cambridge 281-287
36. G. Wang, D.Y. Yeung, F.H. Lochovsky, A new solution path algorithm in support vector regression, IEEE Trans. Neural Networks, 19 (10) (2008) 1753-1767
37. G. Wang, D.Y. Yeung, F.H. Lochovsky, Two dimensional solution path for support vector regression, in: Proceedings of the 23rd International Conference on Machine Learning (ICML 2006), ACM International Conference Proceeding Series, Pittsburgh, Pennsylvania, 2006, pp. 993-1000.
38. Wen W., Hao Z.F., and Yang X.W. A heuristic weight-setting strategy and iteratively updating algorithm for weighted least-squares support vector regression. Neurocomputing 71 (2008) 3096-3103
39. Wiesenfeldt M., Parlitz U., and Lauterborn W. Mixed state analysis of multivariate time series. International Journal of Bifurcation and Chaos 11 8 (2001) 2217-2226
40. Zhang J.S., and Gao G. Reweighted robust support vector regression method. Chin. J. Comput. Sci. 28 7 (2005) 1171-1177
41. Zhu J., Rosset S., Hastie T., and Tibshirani R. 1-norm support vector machines. In: Thrun S., Saul L.K., and Schölkopf B. (Eds). Advances in Neural Information Processing Systems vol. 16 (2004), MIT Press, Cambridge 49-56