Neural network initialization with prototypes - A case study in function approximation

Proceedings of the International Joint Conference on Neural Networks

Volumn 3, Issue , 2005, Pages 1377-1382

  Pei, Jin Song ^a   Wright, Joseph P ^b   Smyth, Andrew W ^c  

^a University of Oklahoma   (United States)

^b WEIDLINGER ASSOCIATES INC   (United States)

^c Department of Earth and Environmental Sciences   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACTIVATION FUNCTIONS; ENGINEERING PROBLEMS; PROTOTYPE-BASED INITIALIZATION;

APPROXIMATION THEORY; COMPUTER SIMULATION; HEURISTIC METHODS; NONLINEAR PROGRAMMING; PROBLEM SOLVING;

NEURAL NETWORKS;

EID: 33750115049     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2005.1556075     Document Type: Conference Paper

References (36)

1. K. A. Al-Mashouq and S. R. Irving, "Includig Hints in Training Neural Nets," Neural Computation, MIT, 1991, vol. 3, pp. 418-427.
2. CM. Bishop. Neural Networks for Pattern Recognition, Oxford University Press. 1995, pp. 482.
3. T. L. Burrows and M. Niranjan, "Feed-Forward and Recurrent Neural Networks for System Identification," Cambridge University Engineering Department, CUED/F-INFENG/TR158, 1993.
4. P. Costa and P. Larzabal, "Initialization of Supervised Training for Parametric Estimation," Neural Processing Letters. Kluwer Academic Publishers, Printed in the Netherlands. 1999, vol. 9, pp. 53-61.
5. G. Cybenko, "Approximation by Superpositions of Sigmoidal Function," Mathematics of Control, Signals, and Systems, 1989. vol. 2, pp. 303-314.
6. T. Denoeux and R. Lengelle, "Initializing Back Propagation Networks with Prototypes," Neural Networks, 1993, vol. 6, pp. 351-363.
7. G. P. Drago and S. Ridella, "Statistically Controlled Activation Weight Initialization (SCAWI)," IEEE Transactions on Neural Networks, 1992, vol. 3, no. 3, pp. 627-631.
8. J. Ghaboussi and X. Wu, "Soft Computing with Neural Networks for Engineering Applications: Fundamental Issues and Adaptive Approaches," Structural Engineering Mechanics, 1998, vol. 6, no. 8, pp. 955-969.
9. K. Hornik, M. Stinchcombe and H. White, "Multilayer Feedforward Networks are Universal Approximators," Neural Networks, 1989, vol. 2, pp. 359-366.
10. W. Y. Huang and R. P. Lippmann, "Neural Net and Traditional Classifiers," Neural Information Processing Systems, D. Anderson (ed.), American Institute of Physics, New York. 1988, pp. 387-396.
11. L. K. Jones, "Constructive Approximations for Neural Networks by Sigmoidal Functions," Proceedings of the IEEE, 1990, vol. 78, no. 10, pp. 1586-1589.
12. L. S. Kim, "Initializing Weights to a Hidden Layer of a Multilayer Neural Network by Linear Programming," Proceedings of 1993 International Joint Conference on Neural Networks, 1993. vol. 2, pp. 1701-1704.
13. M.-Y. Kim and C.-H. Choi, "A New Weight Initialization Method for the MLP with the BP in Multiclass Classification Problems, Neuaral Processing Letters, 1997, vol. 6, pp. 11-23.
14. M. Lentokangas, J. Saarinen and K. Kaski, "Initializing Weights of a Multilayer Perceptron Network by Using the Orthogonal Least Squares Algorithm," Neural Computation. 1995, vol. 7, pp. 982-999.
15. M. Lehtokangas, "Fast Initialization for Cascade-Corrolation Learning," IEEE Transactions on Neural Networks, 1999, vol. 10, no. 2, pp. 410-414.
16. N. B. Karayiannis, "Accelerating the Training of Feedforward Neural Networks Using Generalized Hebbian Rules for Initializing the Internal Representations," IEEE Transactions on Neural Networks, 1996, vol. 7, no. 2, pp. 419-426.
17. G. Li and H. Alnuweiri and Y. Wu and H. Li, "Acceleration of Back Propagations through Iniitial Weights Pre-Training with Delta Rule," Proc. Int. Joint Conf. Neural Networks, pp. 580-585, 1993, vol. 1.
18. L. Ma and K. Khorasani, "New Training Strategies for Constructive Neural Networks with Application to Regression Problems," Neual Networks, 2004, vol. 17, pp. 589-609.
19. S. F. Masri and T. K. Caughey, "A Nonparametric Identification Technique for Nonlinear Dynamic Problems," Journal of Applied Mechanics, 1979, vol. 46, pp. 433-447.
20. S. F. Masri, J. P. Caffrey, T. K. Caughey, A. W. Smyth and A. G. Chassiakos, "Identification of the State Equation in Complex Non-Linear Systems," International Journal of Non-Linear Mechanics, 2004, vol. 39, pp. 1111-1127.
21. A. J. Meade, Jr., "Regularization of a Programmed Recurrent Artificial Neural Network," Journal of Guidance, Control, and Dynamics, 2003.
22. H. N. Mhaskar, "Neural Networks for Optimal Approximation of Smooth and Analytic Functions," Neural Computation. 1995, vol. 8, pp. 164-177.
23. O. Nelles, Nonlinear System Identification: From Classical Approaches to Neural Networks and Fuzzy Models, Springer Verlag, 2000.
24. D. Nguyen and B. Widrow, "Improving the Learning Speed of 2-Layer Neural Networks by Choosing Initial Values of the Adaptive Weights," Proceedings of the IJCNN, pp. 21-26, 1990, vol.III.
25. S. Osowski, "New Approach to Selection of Initial Values of Weights in Neural Function Approximation," Electronic Letters, 1993, vol. 29, no. 3, pp. 313-315.
26. J. S. Pei, "Parametric and Nonparametric Identification of Nonlinear Systems," Columbia University, 2001, Ph.D. Dissertation.
27. J. S. Pei, A. W. Smyth, A.W. and E. B. Kosmatopoulos, "Analysis and Modification of Volterra/Wiener Neural Networks for Identification of Nonlinear Hysteretic Dynamic Systems," Journal of Sound and Vibration, vol. 275, no. 3-5, pp. 693-718.
28. J. S. Pei and A. W. Smyth, "A New Approach to Design Multilayer Feedforward Neural Network Architecture in Modeling Nonlinear Restoring Forces: Part I - Formulation," ASCE Journal of Engineering Mechanics, 2004, accepted for publication.
29. J. S. Pei and A. W. Smyth, "A New Approach to Design Multilayer Feedforward Neural Network Architecture in Modeling Nonlinear Restoring Forces: Part II - Applications," ASCE Journal of Engineering Mechanics, 2004, accepted for publication.
30. J. S. Pei, J. P. Wright, and A. W. Smyth, "Mapping Polynomial Fitting into Feedforward Neural Networks for Modeling Nonlinear Dynamic Systems and Beyond," Computer Methods in Applied Mechanics and Engineering, 2004, accepted for publication.
31. W. F. Schmidt and S. Raudys and M. A. Kraaijveld and M. Skurikhina, and R. P. W. Duin, "Initializations, Back-Propagation and Generalization of Feed-Forward Classifiers," Proc. Int. Joint Conf. Neural Networks, pp. 598-604, 1993.
32. Y. K. Wen, "Methods of Random Vibration for Inelastic Structures," Appl. Mech. Rev., ASME, 1989, vol. 42, no. 2, pp. 39-52.
33. L. F. A. Wessels and E. Barnard, "Avoiding False Local Minima by Proper Initialization of Connections," IEEE Transactions on Neural Networks, 1992, vol. 3, no. 6, pp. 899-905.
34. K. Worden and G. R. Tomlinson, Nonlinearity in Structural Dynamics: Detection, Identification and Modelling. Institute of Physics Pub. 2001.
35. J. Y. F. Yam and T. W. S. Chow, "Feedforward Networks Training Speed Enhancement by Optimal Initialization of the Synaptic Coefficients," IEEE Transactions on Neural Networks, 2001, vol. 12, no. 2, pp. 430-434.
36. S. Zhao and T. S. Dillon, "Incorporating Prior Knowledge in the Form of Production Rules into Neural Networks Using Boolean-Like Neurons," Applied Intelligence. 1997, vol. 7, pp. 275-285.