Variable selection with neural networks

Neurocomputing

Volumn 12, Issue 2-3, 1996, Pages 223-248

  Cibas, Tautvydas ^a   Soulié, Françoise Fogelman ^b   Gallinari, Patrick ^c   Raudys, Sarunas ^d  

^a UFR 919 Laboratoire d'Informatique Pour la Mécanique et les Sciences de l'Ingénieur   (France)

^b SLIGOS   (France)

^c UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

^d Institute of Mathematics   (Lithuania)

Author keywords

Dimensionality reduction; Neural network pruning; Regularization; Variable selection

Indexed keywords

CONSTRAINT THEORY; FUNCTIONS; LEARNING SYSTEMS; SELECTION; STATISTICAL METHODS;

BAYESIAN FRAMEWORK; COST FUNCTION; DIMENSIONALITY REDUCTION; NEURAL NETWORK PRUNING; OPTIMAL CELL DAMAGE; REGULARIZATION THEORY; VARIABLE SELECTION;

NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; BAYES THEOREM; CONTROLLED STUDY; PRIORITY JOURNAL; STATISTICAL ANALYSIS; THEORY;

EID: 0030608136     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/0925-2312(95)00121-2     Document Type: Article

References (37)

1. [1] H. Akaike, A new look at the statistical model identification, IEEE Trans. Auto. Control 19 (1974) 716-723.
2. [2] R. Battiti, Using mutual information for selecting features in supervised neural net learning, IEEE Trans. NN 5 (4) (1994).
3. [3] C. Bishop, Exact calculation of the Hessian matrix for the multilayer perceptron, Neural Comp. 4 (1992) 494-501.
4. [4] M. Bollivier de, P. Gallinari and S. Thiria, Cooperation of neural nets and task decomposition, IJCNN'91, Seattle, vol. II (1991) 573-576.
5. [5] G.E.P. Box and G.C. Tiao, Bayesian Inference in Statistical Analysis (Addison Wesley, 1973).
6. [6] L. Breiman, J. Freidman, R. Olshen and C. Stone, Classification and Regression Trees (Wadsworth Int. Group 1984).
7. [7] W.L. Buntine and A.S. Weigend, Bayesian back-propagation, Complex systems 5 (1991).
8. [8] Y. Chauvin, A back-propagation algorithm with optimal use of hidden units, in: Neural Information Processing Systems, NIPS'88, D. Touretzky ed., vol. 1 (Morgan Kaufmann, 1989) 519-526.
9. [9] Y. Chauvin, Dynamic behavior of constrained back-propagation networks, in: Neural Information Processing Systems, Denver 1989, D. Touretzky ed. (Morgan Kaufmann, 1990) vol. 2, 643-649.
10. [10] T. Cibas, F. Fogelman Soulie, P. Gallinari and S. Raudys, Variable Selection with Neural Networks, ICANN'94, Proceedings, M. Marinaro and P.G. Morasso eds (Springer-Verlag, 1994) vol. 2, 1464-1469.
11. [11] T. Cibas, F. Fogelman Soulie, P. Gallinari and S. Raudys, Variable Selection with Optimal Cell Damage. ICANN'94, Proceedings, M. Marinaro and P.G. Morasso eds (Springer-Verlag, 1994) vol. 1, 727-730.
12. [12] J.S. Denker and Y. Le Cun, Transforming neural net output levels to probability distributions, in Advances in Neural Information Processing Systems, R. Lippmann, J. Moody and D. Touretzky eds., vol. 3 (Morgan Kaufmann, 1991) 853-859.
13. [13] R.O. Duda and P.E. Hart, Pattern Recognition and Scene Analysis (Wiley, 1973).
14. [14] W. Finnoff, F. Hergert and H.G. Zimmermann, Improving model selection by nonconvergent methods, Neural Networks 6 (6) (1993) 771-783.
15. [15] K. Fukunaga, Statistical Pattern Recognition, 2nd ed. (Academic Press, 1990).
16. [16] J.D.F. Habema and J. Hermans, Selection of variables in discriminant analysis by F-statistic and error rate, Technometrics 19 (4) (1977).
17. [17] S.J. Hanson and L.Y. Pratt, Comparing biases for minimal network construction with back-propagation in: Neural Information Processing Systems, NIPS'89, D.S. Touretzky ed., (Morgan Kaufmann, 1989) vol. 1, 177-185.
18. [18] B. Hassibi and D.G. Stork, Second order derivatives for network pruning: Optimal brain surgeon, In Neural Information Processing Systems, NIPS'92, S.J. Hanson, J.D. Cowan and C.L. Giles eds. (Morgan Kaufmann, 1993) vol. 5, 164-171.
19. [19] P.A. Devijver and J. Kittler, Pattern Recognition: A Statistical Approach (Prentice-Hall, 1982).
20. [20] E.D. Karnin, A simple procedure for pruning back-propagation trained nural networks, IEEE Trans. NN 1 (2) (1990) 239-242.
21. [21] A. Kouam, Approches connexionnistes pour la prévision des séries temporelles, Thèse, Université de Paris-Sud, 1993.
22. [22] Y. Le Cun, J.S. Denker and S.A. Solla, Optimal brain damage, in: Neural Information Processing Systems, NIPS'89, D. Touretzky ed. (Morgan Kaufmann, 1990) vol. 2, 598-605.
23. [23] D.J.C. McKay, A practical bayesian framework for backpropagation networks, Neural Computation 4 (1992) 448-472.
24. [24] D.J.C. McKay, Bayesian interpolation, Neural Computation 4 (1992) 415-447.
25. [25] D.J.C. McKay, Bayesian non-linear modeling for the energy prediction competition, Tech. Rep., University of Cambridge, 1993.
26. [26] A.J. Miller, Subset Selection in Regression (Chapman and Halt, 1990).
27. [27] T.J. Mitchell and J.J. Beauchamp, Bayesian variable selection in linear regression, JASA 83, (1988) 1023-1036.
28. [28] M. Moller, Efficient training of feed-forward neural networks, Thesis, Univ. Aarhus, 1993.
29. [29] M.C. Mozer and P. Smolensky, Skeletonization: a technique for trimming the fat from a network via relevance assesment, NIPS 1 (1989) 107-115.
30. [30] S.J. Nowlan and G.E. Hinton, Simplifying neural networks by soft weight-sharing, Neural Computation 4 (1992) 473-493.
31. [31] F. Girosi, M. Jones and T. Poggio, Regularization theory and neural networks architectures, Neural Computation 1 (2) (1995) 219-269.
32. [32] D.E. Rumelhart, G.E. Hinton and R.J. Williams, Learning internal representations by error propagation in Parallel Distributed Processing. D.E. Rumelhart and J.L. McClelland eds., (MIT Press, 1986) vol. 1, 318-362.
33. [33] SAS/Stat User's Guide, version 6, 4th ed.
34. [34] G. Schwarz, Estimating the dimension of a model. Annals of Statistics, 6-2, (1978) 461-464.
35. [35] H.H. Thodberg, Ace of Bayes; application of neural networks with pruning, Tech. Rep. n°1132E, Danish Meat Res. Inst., 1993.
36. [36] M.L. Thompson, Selection of variables in multiple regression: Part 1 & 2, Int. Stat. Rev, 46 (1978) 1-19 & 129-146.
37. [37] A.S. Weigend, D.E. Rumelhart and B.A. Huberman, Generalization by weight elimination with application to forecasting, In: Neural Information Processing Systems, NIPS'90. R.P. Lippmann, J.E. Moody and D.S. Touretzky eds., (Morgan Kaufmann, 1991) vol. 3, 875-882.