Simultaneous input variable and basis function selection for RBF networks

Neurocomputing

Volumn 72, Issue 10-12, 2009, Pages 2649-2658

  Tikka, Jarkko ^a  

^a AALTO UNIVERSITY   (Finland)

Author keywords

Basis function selection; Feature selection; Feedforward neural network; Function approximation; Input selection; Radial basis function network; RBF; Regression

Indexed keywords

BASIS FUNCTION SELECTION; FEATURE SELECTION; FUNCTION APPROXIMATION; INPUT SELECTION; RBF; REGRESSION;

ATTITUDE CONTROL; CONSTRAINED OPTIMIZATION; PROBLEM SOLVING; RADIAL BASIS FUNCTION NETWORKS; REGRESSION ANALYSIS;

NEURAL NETWORKS;

ACCURACY; ALGORITHM; ARTICLE; MATHEMATICAL ANALYSIS; MEASUREMENT; MODEL; PREDICTION; PRIORITY JOURNAL; PROBLEM SOLVING; QUALITY CONTROL; RADIAL BASED FUNCTION; SIMULATION; SUPPORT VECTOR MACHINE; TIME;

EID: 67349221521     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2008.10.003     Document Type: Article

References (52)

1. Blum A.L., and Langley P. Selection of relevant features and examples in machine learning. Artificial Intelligence 97 (1997) 245-271
2. Kohavi R., and John G.H. Wrappers for feature subset selection. Artificial Intelligence 97 (1997) 273-324
3. Guyon I., and Elisseeff A. An introduction to variable and feature selection. Journal of Machine Learning Research 3 (2003) 1157-1182
4. Bi J., Bennett K.P., Embrechts M., Breneman C.M., and Song M. Dimensionality reduction via sparse support vector machines. Journal of Machine Learning Research 3 (2003) 1229-1243
5. Tikka J., and Hollmén J. Sequential input selection algorithm for long-term prediction of time series. Neurocomputing 71 (2008) 2604-2615
6. Li K., and Peng J.-X. Neural input selection-A fast model-based approach. Neurocomputing 70 (2007) 762-769
7. François D., Rossi F., Wertz V., and Verleysen M. Resampling methods for parameter-free and robust feature selection with mutual information. Neurocomputing 70 (2007) 1276-1288
8. J. Tikka, J. Hollmén, Selection of important inputs for RBF network using partial derivatives, in: Proc. The 16th European Symposium on Artificial Neural Networks (ESANN 2008), 167-172.
9. T. Van Gestel, J.A.K. Suykens, B. De Moor, J. Vandewalle, Automatic relevance determination for least squares support vector machine regression, in: Proc. International Joint Conference on Neural Networks (IJCNN 2001), vol. 4, IEEE, 2416-2421.
10. Chapados N., and Bengio Y. Input decay: Simple and effective soft variable selection. Proc. International Joint Conference on Neural Networks (IJCNN 2001) vol. 2 (2001), IEEE, New York 1233-1237
11. J. Tikka, Input selection for radial basis function networks by constrained optimization, in: Proc. The 17th International Conference on Artificial Neural Networks (ICANN 2007), Lecture Notes in Computer Science, Vol. 4668, Springer, 239-248.
12. Park J., and Sandberg I.W. Universal approximation using radial-basis-function networks. Neural Computation 3 (1991) 246-257
13. Anders U., and Korn O. Model selection in neural networks. Neural Networks 12 (1999) 309-323
14. M.J. Orr, Introduction to radial basis functions networks, Technical Report, Edinburgh University, Edinburgh, Scotland, UK, April 1996.
15. Moody J., and Darken C.J. Fast learning in networks of locally-tuned processing units. Neural Computation 1 (1989) 218-294
16. Bishop C.M. Neural networks for pattern recognition (1995), Oxford University Press
17. Benoudjit N., and Verleysen M. On the kernel widths in radial-basis function networks. Neural Processing Letters 18 (2003) 139-154
18. Orr M.J. Regularization in the selection of radial basis function centers. Neural Computation 7 (1995) 606-623
19. Smola A.J., and Schölkopf B. A tutorial on support vector regression. Statistics and Computing 14 (2004) 199-222
20. C. Saunders, A. Gammerman, V. Vovk, Ridge regression learning algorithm in dual variables, in: Proc. The 15th International Conference on Machine Learning (ICML 1998), 515-521.
21. Suykens J.A.K., Van Gestel T., De Brabanter J., De Moor B., and Vandewalle J. Least squares support vector machines (2002), World Scientifc Publishing
22. Hoerl A.E., and Kennard R.W. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12 (1970) 55-67
23. Suykens J.A.K., De Brabanter J., Lukas L., and Vadewalle J. Weighted least squares support vector machines: robustness and sparse approximation. Neurocomputing 48 (2002) 85-105
24. Cawley G.C., and Talbot N.L.C. Reduced rank kernel ridge regression. Neural Processing Letters 16 (2002) 293-302
25. Baudat G., and Anouar F. Kernel-based methods and function approximation. Proc. International Joint Conference on Neural Networks IJCNN vol. 2 (2001), IEEE 1244-1249
26. Cawley G.C., and Talbot N.L.C. Fast exact leave-one-out cross-validation of sparse least-squares support vector machines. Neural Networks 17 (2002) 1467-1475
27. Efron B., and Tibshirani R.J. An introduction to the bootstrap (1993), Chapman & Hall/CRC
28. Platt J. A resource-allocating network for function interpolation. Neural Computation 3 (1991) 213-225
29. Kadirkamanathan V., and Niranjan M. A function estimation approach to sequential learning with neural networks. Neural Computation 5 (1993) 954-975
30. Yingwei L., Sundararajan N., and Saratchandran P. A sequential learning scheme for function approximation using minimal radial basis function neural networks. Neural Computation 9 (1997) 461-478
31. Huang G.-B., Saratchandran P., and Sundararajan N. A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation. IEEE Transactions on Neural Networks 16 (2005) 57-67
32. Huang G.-B., Zhu Q.-Y., and Siew C.-K. Extreme learning machine: Theory and applications. Neurocomputing 70 (2006) 489-501
33. Reunanen J. Overfitting in making comparisons between variable selection methods. Journal of Machine Learning Research 3 (2003) 1371-1382
34. Miller A.J. Subset selection in regression (1990), Chapman and Hall
35. Ruck D.W., Rogers S.K., and Kabrisky M. Feature selection using a multilayer perceptron. Journal of Neural Network Computing 2 (1990) 40-48
36. B. Dorizzi, G. Pellieux, F. Jacquet, T. Czernichow, A. Munoz, Variable selection using generalized RBF networks: Application to the forecast of the French T-bonds, in: Proc. IEEE-IMACS Multiconference on Computational Engineering in Systems Applications (CESA 1996), 1996.
37. F. Rossi, Attribute suppression with multi-layer perceptron, in: Proc. IEEE-IMACS Multiconference on Computational Engineering in Systems Applications (CESA 1996), 1996.
38. Leray P., and Gallinari P. Feature selection with neural networks. Behaviormetrika 26 (1999) 145-166
39. A.N. Refenes, A. Zapranis, J. Utans, Neural model identification, variable selection, and model adequacy, in: Proc. Conference on Neural Networks in the Capital Markets (NNCM 1996), 1996.
40. Villman Th., Schleif F., and Hammer B. Comparison of relevance learning vector quantization with other metric adaptive classification methods. Neural Networks 19 (2006) 610-622
41. Poggio T., and Girosi F. Networks for approximation and learning. Proceedings of the IEEE 78 (1990) 1481-1497
42. Tibshirani R. Regression shrinkage and selection via the LASSO. Journal of the Royal Statistical Society, Series B (Methodological) 58 (1996) 267-288
43. Similä T., and Tikka J. Input selection and shrinkage in multiresponse linear regression. Computational Statistics & Data Analysis 52 (2007) 406-422
44. Bazaraa M.S., Sherali H.D., and Shetty C.M. Nonlinear programming theory and algorithms (1979), Wiley
45. M. Schmidt, G. Fung, R. Rosales, Fast optimization methods for L1 regularization: A comparative study and two new approaches, in: Proc. The 18th European Conference on Machine Learning (ECML 2007), Lecture Notes in Computer Science, vol. 4701, Springer, 286-297.
46. J.C. Bezdek, R.J. Hathaway, Some notes on alternating optimization, in: Proc. The 2002 AFSS International Conference on Fuzzy Systems (AFSS 2002), Lecture Notes in Computer Science, vol. 2275, Springer, 288-300.
47. Osborne M.R., Presnell B., and Turlach B.A. A new approach to variable selection in least squares problems. IMA Journal of Numerical Analysis 20 (2000) 389-404
48. Efron B., Hastie T., Johnstone I., and Tibshirani R. Least angle regression. The Annals of Statistics 32 (2004) 407-499
49. Bezdek J.C., Hathaway R.J., Howard R.E., Wilson C.A., and Windham M.P. Local convergence analysis of grouped variable version of coordinate descent. Journal of Optimization Theory and Applications 54 (1987) 471-477
50. Hathaway R.J., and Bezdek J.C. Grouped coordinate minimization using Newton's method for inexact minimization in one vector coordinate. Journal of Optimization Theory and Applications 71 (1991) 503-516
51. Friedman J.H. Multivariate adaptive regression splines. The Annals of Statistics 19 (1991) 1-141
52. Nørgaard M., Ravn 0., Poulsen N.K., and Hansen L.K. Neural networks for modelling and control of dynamic systems (2003), Springer