The combined statistical stepwise and iterative neural network pruning algorithm

Intelligent Automation and Soft Computing

Volumn 15, Issue 4, 2009, Pages 573-589

  Fnaiech, Nader ^a   Fnaiech, Farhat ^a   Jervis, B W ^b   Cheriet, M ^c  

^a UNIVERSITY OF TUNIS   (Tunisia)

^b Artificial Intelligence ^*  (United Kingdom)

^c Laboratoire d'Imagerie de Vision et d'Intelligence Artificielle (LIVIA)   (Canada)

Author keywords

Feed forward neural network (NN); Iterative Pruning algorithm (IP); Pruning algorithm; SSM Iterative Pruning algorithm (SSIP); Statistical Stepwise Method (SSM)

Indexed keywords

EID: 69249117844     PISSN: 10798587     EISSN: 2326005X     Source Type: Journal    
DOI: 10.1080/10798587.2009.10643050     Document Type: Article

References (35)

1. Y. Girard M. Mangeas, M. Cottrell, B. Girard, and C. Muller. “Neural Modelling for Time Series: Statistical Stepwise Method for Weight Elimination.” IEEE Transactions on Neural Networks, Vol 6, N6, 1995.
2. N. Fnaiech, F. Fnaiech, and M. Najim. “A New Feed-forward Neural Network Hidden Layer's Neurons Pruning Algorithm.” IEEE International Conference on Acoustic Speech and Signal Processing, (ICASSP’2001): Salt-lake-City, USA.
3. A.P. Engelbrecht. “A New Pruning Heuristic based on Variance Analysis of Sensitivity Information.” IEEE Transactions on, Neural Networks, Volume: 12 No. 6, pp: 1386-1399, 2001.
4. G. Castellano and A. M. Fanelli. “An Iterative Pruning Algorithm for Feed-forward Neural Networks.” IEEE Transactions on Neural Networks, Vol. 8, No. 3, 1997.
5. F. Fnaiech, M. Sayadi, and M. Najim. “Texture Characterization Based on Two Dimensional Lattice Coefficients.” IEEE ICASSP'98, Seattle, Washington, USA and IEEE ICASSP'99, Phoenix, Arizona, USA.
6. M. Sayadi, B. W. Jervis, and F. Fnaiech. “Classification of Brain Conditions using Multilayer Perceptrons Trained by the Recursive Least Squares Algorithm.” Proceeding of the 2nd International Conference on Neural Network and expert system in medicine and healthcare, Playmouth, UK, pp. 5-13, 28-30, 1996.
7. A. Dumitras and F. Kossentini. “Feed-forward Neural Network Design with Tridiagonal Symmetry Constraints.” IEEE Transactions on signal Processing, Vol. 48, No. 5.
8. A. M. Fanelli, G. Castellano, and M. Pelillo. “An Empirical comparison of node pruning methods for layered feed-forward Neural Networks.” Proceedings of1993 International Joint Conference on Neural Networks.
9. B. Girard, M. Cottrell, and Y. Girard. “Réseaux de Neurones et Series Temporelles.” XXIV Journes de Statistiques, ASU, Bruxelles, 1992.
10. B. Hassibi and D.G. Stork. “Second Order Derivative for Network Pruning: Optimal Brain Surgeon.” in Advances in neural Information Processing systems 5, 1993.
11. C. C. Teng and B. W. Wah. “Automated Learning for Reducing the Configuration of a FeedForward Neural Network.” IEEE Transaction on Neural Network, Vol. 7, No 5.
12. D.C. Psichogios and L.H Ungar. “SVD net: An algorithm that automatically selects network structures.” IEEE Tranc. On Neural Networks, Vol 5, N3.
13. E. D. Karnin. “A simple Procedure for Pruning Back-Propagation Trained Neural Network.” IEEE Transaction on Neural Network, Vol. 1. No. 2.
14. F. Fnaiech, S. Abid, and M. Najim. “A Fast Feed-Forward Training Algorithm Using A Modified Form of Standard Back-Propagation Algorithm.” IEEE Transactions on Neural Network, 2001.
15. G. H. Young, J. Sum, C. S. Leung, and W. K. Kan. “On the Kalman Filtering Method in Neural Network Training and Pruning.” IEEE Transaction Neural Networks, Vol. 10.
16. G. Liu and K. G. Ramakrishnan. “A Prune: An Algorithm for Finding K Shortest Paths Subject to Multiple Constraints.” IEEEINFOCOM, 2001.
17. H. Sakai, Y. Hgunl, and H. Tokumary. “Real-Time Learning Algorithm for a Multilayered Neural Network Based on the Extended Kalman Filter.” IEEE Transaction Signal Processing, Vol 40.
18. J. E. Moody. “Prediction Risk and Architecture Selection for Neural Networks.” In Form Statistic to neural networks: Theory and pattern Recognition Application, 1993.
19. J. Hu, W. Wan, K. Hirasawa, and J. Murata. “Relation between weight initialization of neural networks and pruning algorithms case study on Mackey-glass time series.” 2001 IEEE.
20. J.-G. Juang. “Trajectory Synthesis based on different Fuzzy modelling Network Pruning Algorithms.” Proceeding of the 200 IEEE, International conference on control applications, Anchorage, Alaska, USA, 2000.
21. K.-W. Wong, S.-J. Chang, J. Sun, and C. S. Leung. “Adaptive training and pruning in feedforward networks.” ELECTRONICS LETTERS, 2001.
22. L. Fletcher, A. P. Engelbrecht, and I. Cloete. “Variance Analysis of Sensitivity Information for Pruning Feed-forward Neural Networks.” in IEEE Int. Joint Conf. Neural Networks, Washington, DC, 1999.
23. L. Prechelt. “Connection Pruning with Static and Adaptative Pruning Schedules.” Neurocomput, Vol. 16, no. 1, 1997.
24. M. Hagiwara. “Removal of Hidden Units and Weights for Back-Propagation Networks.” In Proc. 1993 Int. Joint Conf. Neural networks, vol. 1.
25. M. Jones, F. Girosi and T. Poggio. “Regularization Theory and Neural Network Architectures.” Neural Comput, vol. 7.
26. N. Fnaiech, B. W. Jervis, F. Fnaiech, M. Sayadi, and M. Cheriet. “A pruned Multilayer Neural Network for brain diseases classification.” Conference NNESMED 2003, 2003.
27. N. Fnaiech, F. Fnaiech, and M. Cheriet. “A New Feed-forward Neural Network Pruning Algorithm: Iterative-SSM Pruning.” IEEE International Conference on Systems, Man and Cybernetics, 2002.
28. R. H. Brown, T. L. Ruchti, and J. J. Garside. “Estimation of Artificial Neural Network Parameters for Non-Linear System Identification.” Proceeding of 31st conference on Decision and control Tucson, Arizona, 1992.
29. R. Reed. “Pruning Algorithms-a Survey.” IEEE Transactions on Neural Networks, Volume: 4 Issue: 5.
30. R. Setiono and H. Liu. “Neural-Network Feature Selector.” IEEE Transactions on Neural Networks, Vol.8, N3, 1997.
31. S. K. Rogers, D. W. Ruck, and M. Kabrisky. “Feature Selection using a Multilayer Perceptron.” Neural Network Computing, Vol. 2 No. 2, 1990.
32. S. Yasui. “Convergence Suppression and Divergence Facilitation: Minimum and joint use of Hidden Unit by Multiple Output.” Neural Networks, vol. 10, no. 2.
33. Y. le Cun, J.S. Denker, and S.A. Solla. “Optimal Brain Damage.” In D.Touretzky, editor, Advances in Neural Information Processing Systems 2, Morgan Kaufmann. 1990.
34. Y. Hu, Q. Xue, and W. J. Tompkins. “Analyses of the Hidden Units of Backpropagation Model.” in Proc. Int. Joint Conf. Neural Networks, Vol. 1 1990.
35. Z. Huang, S. Mei, and K. Fang. “A Neural Network Controller Based on Genetic Algorithms.” International Conference on intelligent Processing systems, October 28-31.