An integrated growing-pruning method for feedforward network training

Neurocomputing

Volumn 71, Issue 13-15, 2008, Pages 2831-2847

  Narasimha, Pramond L ^a   Delashmit, Walter H ^b   Manry, Michael T ^a   Li, Jaing ^c   Maldonado, Francisco ^d  

^a The University of Texas at Arlington   (United States)

^b LOCKHEED MARTIN   (United States)

^c Old Dominion University   (United States)

^d Williams Pyro Inc   (United States)

Author keywords

Back propagation; Cascade correlation; Growing; Output weight optimization Hidden weight optimization; Pruning

Indexed keywords

BACKPROPAGATION; CULTIVATION; DEEP NEURAL NETWORKS; ERRORS; OPTIMIZATION;

CASCADE CORRELATION; COMPLEXITY OPTIMIZATION; FEED-FORWARD NETWORK; GROWING AND PRUNING; ORTHOGONAL LEAST SQUARES; OUTPUT WEIGHT OPTIMIZATION-HIDDEN WEIGHT OPTIMIZATION; PRUNING; REDUCED SENSITIVITY;

ITERATIVE METHODS;

ALGORITHM; ANALYTICAL ERROR; ARTICLE; ARTIFICIAL NEURAL NETWORK; BACK PROPAGATION; DATA ANALYSIS; POSITIVE FEEDBACK; PRIORITY JOURNAL; PROCESS OPTIMIZATION; REGRESSION ANALYSIS; VALIDATION PROCESS;

EID: 56449086760     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2007.08.026     Document Type: Article

References (34)

1. H. Amin, K.M. Curtis, B.R.H. Gill, Dynamically pruning output weights in an expanding multilayer perceptron neural network, in: 13th International Conference on DSP, vol. 2, 1997.
2. P. Arena, R. Caponetto, L. Fortuna, M.G. Xibilia, Genetic algorithms to select optimal neural network topology, in: Proceedings of the 35th Midwest Symposium on Circuits and Systems, vol. 2, 1992.
3. C.M. Bishop, Neural Networks and Machine Learning, vol. 168, Springer, Berlin, 1998.
4. G. Castellano, A.M. Fanelli, M. Pelillo, An iterative pruning algorithm for feedforward neural networks, IEEE Trans. Neural Networks 8 (3) (1997) 519-531.
5. H.H. Chen, M.T. Manry, H. Chandrasekaran, A neural network training algorithm utilizing multiple sets of linear equations, Neurocomputing 25 (1-3) (1999) 55-72.
6. S. Chen, C.F.N. Cowan, P.M. Grant, Orthogonal least squares learning algorithm for radial basis function networks, IEEE Trans. Neural Networks 2 (2) (1991) 302-309.
7. F.L. Chung, T. Lee, Network-growth approach to design of feedforward neural networks, IEE Proce. Control Theory 142 (5) (1995) 486-492.
8. J. Davila, Genetic optimization of NN topologies for the task of natural language processing, in: Proceedings of International Joint Conference on Neural Networks, vol. 2, 1999.
9. M.S. Dawson, A.K. Fung, M.T. Manry, Surface parameter retrival using fast learning neural networks, Remote Sensing Rev 7 (1993).
10. M.S. Dawson, J. Olvera, A.K. Fung, M.T. Manry, Inversion of surface parameters using fast learning neural networks, in: Proceedings of IGARSS, vol. 2, 1992.
11. W.H. Delashmit, Multilayer perceptron structured initialization and separating mean processing, Dissertation, University of Texas, Arlington, December 2003.
12. S.E. Fahlman, C. Lebiere, The cascade-correlation learning architecture, in: Advances in Neural Information Processing Systems 2, Morgan Kaufmann, Los Altos, CA, 1990.
13. M.H. Fun, M.T. Hagan, Levenberg-marquardt training for modular networks, in: IEEE International Conference on Neural Networks, vol. 1, 1996.
14. A.K. Fung, Z. Li, K.S. Chen, Back scattering from a randomly rough dielectric surface, IEEE Trans. Geosci. Remote Sensing 30 (2) (1992) 356-369.
15. S.-U. Guan, S. Li, Parallel growing and training of neural networks using output parallelism, IEEE Trans. Neural Networks 13 (3) (2002) 542-550.
16. Y. Hirose, K. Yamashita, S. Hijiya, Back-propagation algorithm that varies the number of hidden units, Neural Networks 4 (1991).
17. G.B. Huang, P. Saratchandran, N. Sundararajan, A generalized growing and pruning RBF neural network for function approximation, IEEE Trans. Neural Networks 16 (1) (2005) 57-67.
18. W. Kaminski, P. Strumillo, Kernel orthonormalization in radial basis function neural networks, IEEE Trans. Neural Networks 8 (5) (1997) 1177-1183.
19. M. Lehtokangas, Modelling with constructive backpropagation, Neural Networks 12 (1999) 707-716.
20. X. Liang, L. Ma, A study of removing hidden neurons in cascade correlation neural networks, in: IEEE International Joint Conference on Neural Networks, vol. 2, 2004.
21. F.J. Maldonado, T.H.K.M.T. Manry, Finding optimal neural network basis function subsets using the schmidt procedure, in: Proceedings of the International Joint Conference on Neural Networks, vol. 1, 2003, pp. 20-24.
22. V. Maniezzo, Genetic evolution of the topology and weight distribution of neural networks, IEEE Trans. Neural Networks 5 (1) (1994).
23. M.T. Manry, S.J. Apollo, L.S. Allen, W.D. Lyle, W. Gong, M.S. Dawson, A.K. Fung, Fast training of neural networks for remote sensing, Remote Sensing Rev. 9 (1994).
24. P.V.S. Ponnapalli, K.C. Ho, M. Thomson, A formal selection and pruning algorithm for feedforward artificial neural network optimization, IEEE Trans. Neural Networks 10 (4) (1999) 964-968.
25. R. Reed, Pruning algorithms-a survey, IEEE Trans. Neural Networks 4 (5) (1993) 740-747.
26. D.E. Rumelhart, J.L. McClelland, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vol. 1, MIT Press, Cambridge, MA, 1986.
27. I.I. Sakhnini, M.T. Manry, H. Chandrasekaran, Iterative improvement of trigonometric networks, in: Proceedings of the International Joint Conference on Neural Networks, vol. 2, 1999.
28. Y.K. Sarabandi, F.T. Ulaby, An empirical model and an inversion technique for radar scattering from bare soil surfaces, IEEE Trans. Geosci. Remote Sensing 30 (2) (1992) 370-381.
29. M.A. Sartori, P.J. Antsaklis, A simple method to derive bounds on the size and to train multilayer neural networks, IEEE Trans. Neural Networks 2 (4) (1991) 467-471.
30. R.S. Scalero, N. Tepedelenlioglu, A fast new algorithm for training feedforward neural networks, IEEE Trans. Signal Process. 40 (1992) 202-210.
31. R. Sentiono, Feedforward neural network construction using cross validation, Neural Computation 13 (2001) 2865-2877.
32. G. Strang, Linear Algebra and its Application, Harcourt Brace, New York, 1988.
33. University of Texas at Arlington, Training datasets, (http://www-ee.uta.edu/eeweb/IP/training_data_files.htm).
34. C. Yu, M.T. Manry, J. Li, P.L. Narasimha, An efficient hidden layer training method for the multilayer perceptron, Neurocomputing 70 (2006) 525-535.