Approximate confidence and prediction intervals for least squares support vector regression

IEEE Transactions on Neural Networks

Volumn 22, Issue 1, 2011, Pages 110-120

  De Brabanter, Kris ^a   De Brabanter, Jos ^b   Suykens, Johan A K ^a   De Moor, Bart ^a  

^a Katholieke Universiteit Leuven   (Belgium)

^b Associatie KULeuven   (Belgium)

Author keywords

Bias; confidence interval; heteroscedasticity; homoscedasticity; kernel based regression; variance

Indexed keywords

BIAS; CONFIDENCE INTERVAL; HETEROSCEDASTICITY; HOMOSCEDASTICITY; KERNEL-BASED REGRESSION; VARIANCE;

ESTIMATION; REGRESSION ANALYSIS;

FINANCIAL DATA PROCESSING;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; COMPUTER PROGRAM; EPIDEMIOLOGY; NONLINEAR SYSTEM; STANDARD;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BIAS (EPIDEMIOLOGY); NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; SOFTWARE; SOFTWARE VALIDATION;

EID: 78651267218     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2087769     Document Type: Article

References (42)

1. C.-J. Ong, S. Shao, and J. Yang, "An improved algorithm for the solution of the regularization path of support vector machine," IEEE Trans. Neural Netw., vol. 21, no. 3, pp. 451-462, Mar. 2010.
2. Z. Sun, Z. Zhang, H. Wang, and M. Jiang, "Cutting plane method for continuously constrained kernel-based regression," IEEE Trans. Neural Netw., vol. 21, no. 2, pp. 238-247, Feb. 2010.
3. A. B. Tsybakov, Introduction to Nonparametric Estimation. New York: Springer-Verlag, 2009.
4. P. Hall, "On bootstrap confidence intervals in nonparametric regression," Ann. Statist., vol. 20, no. 2, pp. 695-711, Jun. 1992.
5. G. Chryssoloiuris, M. Lee, and A. Ramsey, "Confidence interval prediction for neural network models," IEEE Trans. Neural Netw., vol. 7, no. 1, pp. 229-232, Jan. 1996.
6. G. Papadopoulos, P. J. Edwards, and A. F. Murray, "Confidence estimation methods for neural networks: A practical comparison," IEEE Trans. Neural Netw., vol. 12, no. 6, pp. 1278-1287, Nov. 2001.
7. I. Rivals and L. Personnaz, "Construction of confidence intervals for neural networks based on least squares estimation," Neural Netw., vol. 13, nos. 4-5, pp. 463-484, Jun. 2000.
8. C. M. Bishop and C. S. Qazaz, "Regression with input-dependent noise: A Bayesian treatment," in Advances in Neural Information Processing Systems 9. Cambridge, MA: MIT Press, 1997, pp. 347-353.
9. P. W. Goldberg, C. K. Williams, and C. M. Bishop, "Regression with input-dependent noise: A Gaussian process treatment," in Advances in Neural Information Processing Systems 10. Cambridge, MA: MIT Press, 1998.
10. K. Kersting, C. Plagemann, P. Pfaff, and W. Burgard, "Most likely heteroscedastic Gaussian process regression," in Proc. 24th ICML, Corvalis, OR, 2007, pp. 393-400.
11. J. M. Bernardo and A. F. M. Smith, Bayesian Theory. New York: Wiley, 2000.
12. G. C. Cawley, N. L. C. Talbot, R. J. Foxall, S. R. Dorling, and D. P. Mandic, "Heteroscedastic kernel ridge regression," Neurocomputing, vol. 57, pp. 105-124, Mar. 2004.
13. G. C. Cawley, N. L. C. Talbot, and O. Chapelle, "Estimating predictive variances with kernel ridge regression," in Machine Learning Challenges Workshop. Berlin, Germany: Springer-Verlag, 2006, pp. 56-77.
14. R. L. Eubank and P. L. Speckman, "Confidence bands in nonparametric regression," J. Amer. Statist. Assoc., vol. 88, no. 424, pp. 1287-1301, Dec. 1993.
15. P. J. Bickel and M. Rosenblatt, "On some global measures of the deviations of density function estimates," Ann. Statist., vol. 1, no. 6, pp. 1071-1095, 1973.
16. P. Hall and D. M. Titterington, "On confidence bands in nonparametric density estimation and regression," J. Multivariate Anal., vol. 27, no. 1, pp. 228-254, Oct. 1998.
17. W. Härdle, "Asymptotic maximal deviation of M-smoothers," J. Multivariate Anal., vol. 29, no. 2, pp. 163-179, May 1989.
18. G. J. Johnstone, "Probabilities of maximal deviations for nonparametric regression function estimates," J. Multivariate Anal., vol. 12, no. 3, pp. 402-414, Sep. 1982.
19. J. Fan and I. Gijbels, "Data-driven bandwidth selection in local polynomial fitting: Variable bandwidth and spatial adaptation," J. R. Statist. Soc. B, vol. 57, no. 2, pp. 371-394, 1995.
20. J. Fan and I. Gijbels, Local Polynomial Modelling and its Applications. London, U.K.: Chapman & Hall, 1996.
21. R. Isermann, Fault-Diagnosis Systems: An Introduction from Fault Detection to Fault Tolerance. New York: Springer-Verlag, 2005.
22. M. Witczak, Modelling and Estimation Strategies for Fault Diagnosis of Non-Linear Systems: From Analytical to Soft Computing Approaches (Lecture Notes in Control and Information Sciences). New York: Springer-Verlag, 2007.
23. J. A. K. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle, Least Squares Support Vector Machines. Singapore: World Scientific, 2002.
24. V. N. Vapnik, Statistical Learning Theory. New York: Wiley, 1999.
25. W. Härdle, P. Hall, and J. S. Marron, "Regression smoothing parameters that are not far from their optimum," J. Amer. Statist. Assoc., vol. 87, no. 417, pp. 227-233, Mar. 1992.
26. W. Härdle and A. Tsybakov, "Local polynomial estimators of the volatility function in nonparametric autoregression," J. Econometrics, vol. 81, no. 1, pp. 223-242, Nov. 1997.
27. J. Fan and Q. Yao, "Efficient estimation of conditional variance functions in stochastic regression," Biometrika, vol. 85, no. 3, pp. 645-660, Sep. 1998.
28. J. A. Rice, Mathematical Statistics and Data Analysis, 2nd ed. Pacific Grove, CA: Duxbury Press, 1995.
29. A. N. Shiryaev, Probability, 2nd ed. New York: Springer, 1996.
30. A. E. Hoerl and R. W. Kennard, "Ridge regression: Biased estimation for nonorthogonal problems," Technometrics, vol. 12, no. 1, pp. 55-67, Feb. 1970.
31. D. Ruppert, M. P. Wand, and R. J. Carroll, Semiparametric Regression. Cambridge, U.K.: Cambridge Univ. Press, 2003.
32. C. Loader, Local Regression and Likelihood. New York: Springer-Verlag, 1999.
33. S. O. Rice, "The distribution of the maxima of a random curve," Amer. J. Math., vol. 61, no. 2, pp. 409-416, Apr. 1939.
34. J. Sun and C. R. Loader, "Simultaneous confidence bands for linear regression and smoothing," Ann. Statist., vol. 22, no. 3, pp. 1328-1345, Sep. 1994.
35. G. Knafl, J. Sacks, and D. Ylvisaker, "Confidence bands for regression functions," J. Amer. Statist. Assoc., vol. 80, no. 391, pp. 683-691, Sep. 1985.
36. W. Härdle, "Resampling for inference from curves," in Proc. 47th Session Int. Statist. Inst., Bonn, Germany, 1989, pp. 59-69.
37. B. Efron, "Bootstrap methods: Another look at the jackknife," Ann. Statist., vol. 7, no. 1, pp. 1-26, 1979.
38. R. Y. Liu, "Bootstrap procedure under some non-I.I.D. models," Ann. Statist., vol. 16, no. 4, pp. 1696-1708, Dec. 1988.
39. C. F. J. Wu, "Jackknife, bootstrap and other resampling methods in regression analysis," Ann. Statist., vol. 14, no. 4, pp. 1261-1295, 1986.
40. C. F. J. Wu, "Jackknife bootstrap and other resampling methods in regression analysis," Ann. Statist., vol. 14, no. 4, pp. 1295-1298, 1986.
41. E. Mammen, "Bootstrap and wild bootstrap for high dimensional linear models," Ann. Statist., vol. 21, no. 1, pp. 255-285, Mar. 1993.
42. J. Davidson, A. Monticini, and D. Peel, "Implementing the wild bootstrap using a two-point distribution," Econ. Lett., vol. 96, no. 3, pp. 309-315, Sep. 2007.