Integrated feature and architecture selection

IEEE Transactions on Neural Networks

Volumn 7, Issue 4, 1996, Pages 1007-1014

  Steppe, Jean M ^a   Bauer Jr , Kenneth W ^a   Rogers, Steven K ^a  

^a AIR FORCE INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; BACKPROPAGATION; COMPUTER ARCHITECTURE; HEURISTIC METHODS; MATHEMATICAL MODELS; MATRIX ALGEBRA; NONLINEAR EQUATIONS; OBJECT RECOGNITION; REGRESSION ANALYSIS; STATISTICAL METHODS; VECTORS;

ARCHITECTURE SELECTION; MODEL SELECTION; NONLINEAR REGRESSION;

FEEDFORWARD NEURAL NETWORKS;

EID: 0030190724     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.508942     Document Type: Review

References (36)

1. A. Barron and R. Baron, "Statistical learning networks: A unifying view," in Proc. 20th Symp. Interface, 1988, pp. 192-203.
2. E. B. Baum and D. Haussler, "What sized net gives valid generalization?" Neur. Comput., vol. 1, no. 1, pp. 151-160, Spring 1989.
3. M. G. Bello, "Enhanced training algorithms, and integrated training/architecture selection for multilayer perceptron networks," IEEE Trans. Neural Networks, vol. 3, no. 6, Nov. 1992.
4. L. M. Belue, J. M. Steppe, and K. W. Bauer, "Multivariate statistical techniques for determining neural network architecture and data requirements," AISB96 Wrkshp. Tutorial Programme, Brighton, U.K., April 1996.
5. L. M. Belue and K. W. Bauer, "Methods of determining input features for multilayer perceptrons," Neur. Comput., vol. 7, no. 2, 1995.
6. T. M. Cover, "Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition," IEEE Trans. Elec. Comp., vol. EC-14, pp. 326-334, June 1965.
7. G. Cybenko, "Continuous valued neural networks with two hidden layers are sufficient," Dep. Computer Sci., Tufts Univ., Tech. Rep., Mar. 1988.
8. M. de la Maza, "SPLITnet dynamically adjusting the number of hidden units in a neural network," Artificial Neural Networks, T. Kohonen, K. Mäkisara, O. Simula, and J. Kangas, Eds. Amsterdam: North Holland, 1991, pp. 647-651.
9. D. B. Fogel, "An information criterion for optimal neural network selection," IEEE Trans. Neural Networks, vol. 2, no. 5, Sept. 1991.
10. A. R. Gallant, Nonlinear Statistical Models. New York: Wiley, 1987.
11. J. W. Gutierrez and R. Grondin, "Estimating hidden unit number for two-layer perceptrons," in IEEE Proc. Int. Joint Conf. Neural Networks, Washington, DC, 1989, pp. 677-681.
12. R. Hecht-Nielsen, "Theory of the backpropagation neural network," in Proc. International Joint Conference on Neural Networks, I, New York, IEEE Press, June 1989, pp. 593-611.
13. Y. Hirose, K. Yamashita, and S. Hijiya, "Back-propagation algorithm which varies the number of hidden units," Neural Networks, vol. 4, pp. 61-66, 1991.
14. K. Hornik, M. Stinchcombe, and H. White, "Multilayer feedforward networks are universal approximators," Dep. Economics, Univ. California, San Diego, manuscript, June 1988.
15. S. Huang and Y. Huang, "Bounds on the number of hidden neurons in multilayer perceptrons," IEEE Trans. Neural Networks, vol. 2, no. 1, pp. 47-55, Jan. 1991.
16. C. M. Kocur, S. K. Rogers, L. R. Myers, T. Burns, J. W. Hoffmeister, K. W. Bauer, and J. M. Steppe, "Neural network wavelet feature selection for breast cancer diagnosis," IEEE Trans. Biomed. Eng., to appear.
17. S. Y. Kung and J. N. Hwang, "An algebraic analysis for optimal hidden units size and learning rates in back-propagation learning," in Int. Conf. Neural Networks, 1, 363370, San Diego, CA, July 24-27, 1988.
18. Y. Le Cun et al., "Optimal brain damage," in Neur. Infor. Proc. Syst. II, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990, pp. 598-605.
19. N. Mantel, "Why stepdown procedures in variable selection," Technometrics, vol. 12, no. 2, pp. 621-625, Aug. 1970.
20. M. C. Mozer and P. Smolensky, "Skeletonization: A technique for trimming the fat from a network via relevance assessment," in Advanced Neural Information Systems I, David S. Touretzky, Ed. San Mateo, CA: Morgan Kaufman.
21. R. H. Myers, Classical and Modern Regression with Applications, 2nd ed. Boston: PWSKENT, 1990.
22. K. L. Priddy et al., "Bayesian selection of important features for feedforward neural networks," Neurocomputing, vol. 5, no. 2 and 3, 1993.
23. N. J. Redding et al., "Higher order separability and minimal hidden-unit fan-in," in Artificial Neural Networks, T. Kohonen, K. Mäkisara, O. Simula, and J. Kangas, Eds. Ansterdam: North Holland, 1991, pp. 25-30.
24. M. C. Roggemann, "Multiple sensor fusion for detecting targets in FLIR and range images," Ph.D. dissertation, Air Force Inst. Technol., Dec. 1989.
25. M. C. Roggemann et al., "An approach to multiple sensor target detection," in Proc. SPIE Conf. Sens. Fus., 1989, vol. 1100.
26. _, "Multiple sensor tactical target detection in FLIR and range images," in Proc. Tri-Serv. Data Fus. Symp., May 1989.
27. D. W. Ruck et al., "Feature selection using a multilayer perceptron," J. Neur. Net. Comp., vol. 2, no. 2, pp. 40-48, Fall 1990.
28. M. A. Sartori and P. J. Antsaklis, "A simple method to derive bounds on the size and to train multilayer neural networks," IEEE Trans. Neural Networks, vol. 2, pp. 407-471, July 1991.
29. G. A. F. Seber and C. J. Wild, Nonlinear Regression. New York: Wiley, 1989.
30. E. D. Sontag, "Feedforward nets for interpolation and classification," J. Comp. Syst. Sci., vol. 45, pp. 20-48, 1992.
31. J. M. Steppe, "Feature and model selection in feedforward neural networks," Ph.D. dissertation, Air Force Inst. Technol., June 1994.
32. J. M. Steppe and K. W. Bauer, "Improved feature screening in feedforward neural networks," Neurocomputing, to appear.
33. V. N. Vapnik and A. Y. Chervonenkis, "On the convergence of relative frequencies of events to their probabilities," Theory Probab. Its Appl., vol. 16, no. 2, pp. 264-280, 1971.
34. H. White, Artificial Neural Networks Approximation and Learning Theory. Cambridge, U.K.: Blackwell, 1993.
35. _, "An additional hidden unit test for neglected nonlinearity in multilayer feedforward networks," in IEEE INNS Int. J. Conf. Neur. Net. II, Washington, DC, June 18-22, 1989, pp. 451-455.
36. _, "Learning in artificial neural networks: A statistical perspective," Neur. Comput. 1, pp. 425-464, 1989.