An orthogonal forward regression technique for sparse kernel density estimation

Neurocomputing

Volumn 71, Issue 4-6, 2008, Pages 931-943

  Chen, S ^a   Hong, X ^b   Harris, C J ^a  

^a UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

^b UNIVERSITY OF READING   (United Kingdom)

Author keywords

Cross validation; Multiplicative nonnegative quadratic programming; Orthogonal forward regression; Parzen window estimate; Probability density function; Regularisation; Sparse kernel modelling

Indexed keywords

MATHEMATICAL MODELS; OPTIMIZATION; PROBABILITY DENSITY FUNCTION; QUADRATIC PROGRAMMING;

KERNEL DENSITY ESTIMATION; MULTIPLICATIVE NONNEGATIVE QUADRATIC PROGRAMMING; ORTHOGONAL FORWARD REGRESSION; PARZEN WINDOW; REGULARIZATION METHOD; SPARSE KERNEL DENSITY;

REGRESSION ANALYSIS;

ACCURACY; ALGORITHM; ANALYTICAL ERROR; ARTICLE; BAYES THEOREM; INTERMETHOD COMPARISON; KERNEL METHOD; MATHEMATICAL MODEL; NORMAL DISTRIBUTION; PRIORITY JOURNAL; PROCESS OPTIMIZATION; REGRESSION ANALYSIS;

EID: 38649088632     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2007.02.008     Document Type: Article

References (25)

1. H. Akaike, A new look at the statistical model identification, IEEE Trans. Autom. Control AC-19 (1974) 716-723.
2. Bishop C.M. Neural Networks for Pattern Recognition (1995), Oxford University Press, Oxford, UK
3. Chen S. Local regularization assisted orthogonal least squares regression. Neurocomputing 69 4-6 (2006) 559-585
4. Chen S., Billings S.A., and Luo W. Orthogonal least squares methods and their application to non-linear system identification. Int. J. Control 50 5 (1989) 1873-1896
5. Chen S., Hong X., and Harris C.J. Sparse kernel regression modeling using combined locally regularized orthogonal least squares and D-optimality experimental design. IEEE Trans. Autom. Control 48 6 (2003) 1029-1036
6. Chen S., Hong X., and Harris C.J. Sparse kernel density construction using orthogonal forward regression with leave-one-out test score and local regularization. IEEE Trans. Syst. Man Cybern. Part B 34 4 (2004) 1708-1717
7. Chen S., Hong X., Harris C.J., and Sharkey P.M. Sparse modeling using orthogonal forward regression with PRESS statistic and regularization. IEEE Trans. Syst. Man Cybern. Part B 34 2 (2004) 898-911
8. Choudhury A. Fast Machine Learning Algorithms for Large Data (2002), PhD Thesis, Computational Engineering and Design Center, School of Engineering Sciences, University of Southampton
9. Girolami M., and He C. Probability density estimation from optimally condensed data samples. IEEE Trans. Pattern Analy. Mach. Intell. 25 10 (2003) 1253-1264
10. Hansen L.K., and Larsen J. Linear unlearning for cross-validation. Adv. Comput. Math. 5 (1996) 269-280
11. Hansen M.H., and Yu B. Model selection and the principle of minimum description length. J. Am. Statist. Assoc. 96 454 (2001) 746-774
12. Hong X., Sharkey P.M., and Warwick K. Automatic nonlinear predictive model construction algorithm using forward regression and the PRESS statistic. IEE Proc. Control Theory Appl. 150 3 (2003) 245-254
13. MacKay D.J.C. Bayesian interpolation. Neural Comput. 4 3 (1992) 415-447
14. McLachlan G., and Peel D. Finite Mixture Models (2000), Wiley, New York
15. Monari G., and Dreyfus G. Local overfitting control via leverages. Neural Comput. 14 (2002) 1481-1506
16. Mukherjee S., and Vapnik V. Support vector method for multivariate density estimation (1999), Technical Report, A.I. Memo No. 1653, MIT AI Lab
17. Myers R.H. Classical and Modern Regression with Applications. Second Ed. (1990), PWS-KENT, Boston
18. Parzen E. On estimation of a probability density function and mode. Ann. Math. Statist. 33 (1962) 1066-1076
19. Ripley B.D. Pattern Recognition and Neural Networks (1996), Cambridge University Press, Cambridge
20. Sha F., Saul L.K., and Lee D.D. Multiplicative updates for nonnegative quadratic programming in support vector machines (2002), Technical Report, MS-CIS-02-19, University of Pennsylvania, USA
21. Silverman B.W. Density Estimation (1996), Chapman Hall, London
22. Stone M. Cross validation choice and assessment of statistical predictions. J. R. Statist. Soc. Ser. B 36 (1974) 111-147
23. Tipping M.E. Sparse Bayesian learning and the relevance vector machine. J. Mach. Learn. Res. 1 (2001) 211-244
24. V. Vapnik and S. Mukherjee, Support vector method for multivariate density estimation, in: S. Solla, T. Leen, K.R. Müller (Eds.), Advances in Neural Information Processing Systems, MIT Press, Cambridge, MA, 2000, pp. 659-665.
25. J. Weston, A. Gammerman, M.O. Stitson, V. Vapnik, V. Vovk, C. Watkins, Support vector density estimation, in: B. Schölkopf, C. Burges, A.J. Smola (Eds), Advances in Kernel Methods-Support Vector Learning, MIT Press, Cambridge MA, 1999, pp. 293-306.