Backward elimination model construction for regression and classification using leave-one-out criteria

International Journal of Systems Science

Volumn 38, Issue 2, 2007, Pages 101-113

  Hong, X ^a   Mitchell, R J ^a  

^a UNIVERSITY OF READING   (United Kingdom)

Author keywords

Backward elimination; Classification; Cross validation; Forward regression; System identification

Indexed keywords

ALGORITHMS; CLASSIFICATION (OF INFORMATION); ERROR ANALYSIS; IDENTIFICATION (CONTROL SYSTEMS); OPTIMIZATION; REGRESSION ANALYSIS;

BACKWARD ELIMINATION; CROSS VALIDATION; FORWARD REGRESSION;

NONLINEAR SYSTEMS;

EID: 33847074971     PISSN: 00207721     EISSN: 14645319     Source Type: Journal    
DOI: 10.1080/00207720601051463     Document Type: Article

References (32)

1. D.M. Allen, "The relationship between variable selection and dat augumentation and a method for prediction", Technometrics, 16, pp. 125-127, 1974.
2. P. L. Barlett, "The sample complexity of pattern classification with neural networks: the size of the weights is more important than the size of the network", IEEE Trans. Inform. Theory, 44, pp. 525-536, 1998.
3. Brown and C.J. Harris, Neurofuziy Adaptive Modeling and Control, Hemel Hempstead: Prentice Hall, 1994.
4. S. Chen, S.A. Billings and W. Luo, "Orthogonal least squares methods and their applications to non-linear system identification", Int. J. Conm, 50, pp. 1873-1896, 1989.
5. S.S. Chen, D.L. Donoho and M.A. Saunders, "Atomic decomposition by basis pursuit", SIAM Rev., 43, pp. 129-159, 2001.
6. S. Chen, X. Hong, C.J. Harris and P.M. Sharkey, "Sparse modeling using orthogonal forward regression with PRESS statistic and regularisation", IEEE Trans. Syn., Man Cybernet., Part B: Cybernet., 34, pp. 898-911, 2004a.
7. S. Chen, X.X. Wang, X. Hong and C.J. Harris, "Kernel classifier construction using orthogonal forward selection and boosting with Fisher ratio class separability", IEEE Trans. Neural Netw., in press (2006).
8. S. Chen, Y. Wu and B.L. Luk, "Combined genetic algorithm optimization and regularized orthogonal least squares learning for radial basis function networks", IEEE Trans. Neural Netw., 10, pp. 1239-1243, 1999.
9. B. Efron, I. Johnstone, T. Hastie and R. Tibshirani, "Least angle regression", Ann. Stat., 32, pp. 407-451, 2004.
10. P.J. Green, and B.W. Silverman, Nonparametric Regression and Generalized Linear Models, London: Chapman and Hall, 1994.
11. C.J. Harris, X. Hong and Q. Gan, Adaptive Modeling, Estimation and Fusion from Data: A Neurofuzzy Approach, Berlin: Springer-Verlag, 2002.
12. T.J. Hastie, R.J. Tibshirani and J. Friedman, The Elements of Statistical Learning - Data Mining, Inference and Prediction, New York: Springer Verlag, 2001.
13. Hong and C.J. Harris, "A neurofuzzy network knowledge extraction and extended Gram-Schmidt algorithm for model subspace decomposition", IEEE Trans. Fuzzy Sys., 11, pp. 528-541, 2003.
14. Hong, and S.A. Billings, "Givens rotation based fast backward elimination algorithm for RBF neural networks pruning", IEE Proc. - Contr. Theory Appl., 144, pp. 381-384, 1997.
15. Hong, C.J. Harris, M. Brown and S. Chen, "Backward elimination methods for associative memory network pruning", Int. J. Hybrid Intellig. Sys., 1, pp. 90-98, 2004.
16. Hong, P.M. Sharkey and K. Warwick, "Automatic nonlinear predictive model construction using forward regression and the PRESS statistic", IEE Proc. - Contr. Theory Appl., 150, pp. 245-254, 2003.
17. Hong, S. Chen and C.J. Harris, "A fast kernel classifier construction algorithm using orthogonal forward selection to minimize leave-one-out misclassification rate", Int. J. Sys. Sci., submitted for publication, 2005.
18. Keerthi, S.S. http://guppy.mpe.nus.edu.sg/mpessk/sparselogreg.shtml, 2006.
19. K.Z. Mao, "RBF neural network center selection based on Fisher ratio class separability measure", IEEE Trans. Neural Netw., 13, pp. 1211-1217, 2002.
20. K.R. Müller, S. Mika, G. Rätsch, K. Tsuda and B. Schölkopf, "An introduction to kernel based learning algorithms", IEEE Trans. Neural Netw., 12, pp. 181-201, 2001.
21. A.J. Miller, Subset Selection in Regression, New York: Chapman and Hall, 1990.
22. W.T. Miller, R.S. Button and P.J. Werbos (Eds) Neural Networks for Control, Cambridge, MA: MIT Press, 1990.
23. R. Murray-Smith and T.A. Johansen, Multiple Model Approaches to Modeling and Control, London: Taylor and Francis, 1997.
24. M.J.L. Orr, "Regularisation in the selection of radial basis function centers", Near. Comput., 7, pp. 954-975, 1995.
25. G. Ratsch, http://ida.first.fraunhofer.de/projects/bench/benchmarks.htm, 2006.
26. G. Ratsch, T. Onoda and K.R. Müller, "Soft margins for AdaBoost", Mac. Learn., 42, pp. 287-320, 2001.
27. S.K. Shevade, and S.S. Keerthi, "A simple and efficient algorithm for gencselection using sparse logistic regression", Bioinformatics, 19, pp. 2246-2253, 2003.
28. Stone, "Cross validatory choice and assessment of statistical predictions", J. R. Stast. Soc., Ser. B, 36, pp. 117-147, 1974.
29. R. Tibshirani, "Regression shrinkage and selection via the LASSO", J. R. Stat. Soc. Ser. B, 58, pp. 267-288, 1996.
30. M.E. Tipping, "Sparse Bayesian learning and the relevance vector machine", J. Mac. Learn. Res., 1, pp. 211-244, 2001.
31. Vapnik, Statistical Learning Theory, New York: J. Wiley, 1998.
32. G. Wahba, Spline Models for Observational Data, Philadelphia, PA: Society for Industrial and Applied Mathematics, 1990.