Sparse kernel density construction using orthogonal forward regression with leave-one-out test score and local regularization

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 34, Issue 4, 2004, Pages 1708-1717

  Chen, Sheng ^a   Hong, Xia ^b   Harris, Chris J ^a  

^a UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

^b UNIVERSITY OF READING   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ESTIMATION; LEARNING ALGORITHMS; LEAST SQUARES APPROXIMATIONS; MATHEMATICAL MODELS; MATRIX ALGEBRA; OPTIMIZATION; PROBABILITY DENSITY FUNCTION; REGRESSION ANALYSIS;

LEAVE ONE OUT TEST SCORE; ORTHOGONAL FORWARD REGRESSION; PARZEN WINDOW ESTIMATE; SPARSE KERNEL MODELING;

LEARNING SYSTEMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; COMPARATIVE STUDY; COMPUTER SIMULATION; EVALUATION; REGRESSION ANALYSIS; STATISTICAL DISTRIBUTION; STATISTICAL MODEL; VALIDATION STUDY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; LEAST-SQUARES ANALYSIS; MODELS, STATISTICAL; REGRESSION ANALYSIS; STATISTICAL DISTRIBUTIONS;

EID: 3442881906     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2004.828199     Document Type: Article

References (24)

1. C. M. Bishop, Neural Networks for Pattern Recognition, Oxford, U.K.: Oxford Univ. Press, 1995.
2. B. W. Silverman, Density Estimation, London, U.K.: Chapman & Hall, 1996.
3. H. Wang, "Robust control of the output probability density functions for multivariable stochastic systems with guaranteed stability," IEEE Trans. Automat. Contr., vol. 44, pp. 2103-2107, Nov. 1999.
4. S. Chen, A. K. Samingan, B. Mulgrew, and L. Hanzo, "Adaptive minimum-BER linear multiuser detection for DS-CDMA signals in multi-path channels," IEEE Trans. Signal Processing, vol. 49, pp. 1240-1247, June 2001.
5. E. Parzen, "On estimation of a probability density function and mode," Ann. Math. Statist., vol. 33, pp. 1066-1076, 1962.
6. J. Weston, A. Gammerman, M. O. Stitson, V. Vapnik, V. Vovk, and C. Watkins, "Support vector density estimation," in Advances in Kernel Methods - Support Vector Learning, B. Schölkopf, C. Burges, and A. J. Smola, Eds. Cambridge, MA: MIT Press, 1999, pp. 293-306.
7. S. Mukherjee and V. Vapnik, "Support Vector Method for Multivariate Density Estimation," MIT AI Lab., Tech. Rep., A.I. Memo no. 1653, 1999.
8. M. E. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol. 1, pp. 211-244, 2001.
9. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM Rev., vol. 43, no. 1, pp. 129-159, 2001.
10. A. Choudhury, Fast Machine Learning Algorithms for Large Data. Southhampton, U.K.: Comput. Eng. Design Center, School Eng. Sciences, Univ. Southampton, 2002.
11. P. B. Nair, A. Choudhury, and A. J. Keane, "Some greedy learning algorithms for sparse regression and classification with Mercer kernels," J. Mach. Learn. Res., vol. 3, pp. 781-801, 2002.
12. M. H. Hansen and B. Yu, "Model selection and the principle of minimum description length," J. Amer. Stat. Assoc., vol. 96, no. 454, pp. 746-774, 2001.
13. H. Akaike, "A new look at the statistical model identification," IEEE Trans. Automat. Contr., vol. AC-19, pp. 716-723, 1974.
14. S. Chen, "Locally regularised orthogonal least squares algorithm for the construction of sparse kernel regression models," in Proc. 6th Int. Conf. Signal Processing, Beijing, China, Aug. 26-30, 2002, pp. 1229-1232.
15. S. Chen, X. Hong, and C. J. Harris, "Sparse kernel regression modeling using combined locally regularized orthogonal least squares and D-optimality experimental design," IEEE Trans. Automat. Contr., vol. 48, pp. 1029-1036, June 2003.
16. S. Chen, X. Hong, C. J. Harris, and P. M. Sharkey, "Sparse modeling using orthogonal forward regression with PRESS statistic and regularization," IEEE Trans. Syst., Man, Cybern. B, vol. 34, pp. 898-911, Apr. 2004.
17. R. H. Myers, Classical and Modern Regression with Applications, 2nd ed. Boston, MA: PWS-KENT, 1990.
18. L. K. Hansen and J. Larsen, "Linear unlearning for cross-validation," Adv. Computat. Math., vol. 5, pp. 269-280, 1996.
19. G. Monari and G. Dreyfus, "Local overfitting control via leverages," Neural Comput., vol. 14, pp. 1481-1506, 2002.
20. X. Hong, P. M. Sharkey, and K. Warwick, "Automatic nonlinear predictive model construction algorithm using forward regression and the PRESS statistic," Insitut. Elec. Eng. Proc. Contr. Theory Applicat., vol. 150, no. 3, pp. 245-254, 2003.
21. S. Chen, S. A. Billings, and W. Luo, "Orthogonal least squares methods and their application to nonlinear system identification," Int. J. Contr., vol. 50, no. 5, pp. 1873-1896, 1989.
22. S. Chen, C. F. N. Cowan, and P. M. Grant, "Orthogonal least squares learning algorithm for radial basis function networks," IEEE Trans. Neural Networks, vol. 2, pp. 302-309, Mar. 1991.
23. D. J. C. MacKay, "Bayesian interpolation," Neural Computat., vol. 4, no. 3, pp. 415-447, 1992.
24. B. D. Ripley, Pattern Recognition and Neural Networks, Cambridge, U.K.: Cambridge Univ. Press, 1996.