Alleviating the problem of local minima in Backpropagation through competitive learning

Neurocomputing

Volumn 94, Issue , 2012, Pages 152-158

  Dai, Qun ^a   Liu, Ningzhong ^a  

^a NANJING UNIVERSITY OF AERONAUTICS AND ASTRONAUTICS   (China)

Author keywords

Backpropagation (BP); Classification; Competitive learning; Feedforward neural networks (FNNs); Local minima

Indexed keywords

BENCHMARK CLASSIFICATION; BP ALGORITHM; CLASSIFICATION PERFORMANCE; COMPETITIVE LEARNING; CONVERGENCE RATES; DATA SETS; GENERALIZATION CAPABILITY; LOCAL MINIMUMS; NETWORK WEIGHTS; POSSIBLE SOLUTIONS; STEEPEST DESCENT TECHNIQUES; SUBOPTIMAL SOLUTION; TRAINING PROCEDURES;

CLASSIFICATION (OF INFORMATION); FEEDFORWARD NEURAL NETWORKS;

BACKPROPAGATION ALGORITHMS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; BACK PROPAGATION; COMPETITIVE LEARNING; EXPERIMENTAL STUDY; FEED FORWARD NEURAL NETWORK; LEARNING; LEARNING ALGORITHM; MATHEMATICAL COMPUTING; PERFORMANCE MEASUREMENT SYSTEM; POSITIVE FEEDBACK; PRIORITY JOURNAL; PROBLEM SOLVING; QUALITY CONTROL; STANDARDIZATION; VALIDATION PROCESS;

EID: 84863476251     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2012.03.011     Document Type: Article

References (18)

1. Rumelhart D.E., Hinton G.E., Williams R.J. Learning Internal Representations by Back-Propagating Errors 1986, vol. 1. MIT Press, Cambridge, MA.
2. Blum E.K., Li L.K. Approximation theory and feedforward networks. Neural Net. 1991, 4:511-515.
3. Gori M., Tesi A. On the problem of local minima in back-propagation. IEEE Trans. Pattern Anal. Mach. Intell. 1992, 14:76-86.
4. S.E. Fahlman, Fast learning variations on back-propagation: An empirical study, presented at Proceedings of the 1988 Connectionist Models Summer School, California, 1989.
5. M. Riedmiller, H. Braun, A direct adaptive method for faster back-propagation learning: The RPROP Algorithm, presented at Proceedings of International Conference on Neural Networks 1993.
6. Treadgold N.K., Gedeon T.D. Simulated Annealing and weight decay in adaptive learning: the SARPROP algorithm. IEEE Trans. Neural Networks 1998, 9:662-668.
7. Bard Y. Non-Linear Parameter Estimation 1974, Academic Press, New York.
8. Wang X.G., Tang Z., Tamura H., Ishii M. A modified error function for the backpropagation algorithm. Neurocomputing 2004, 57:477-484.
9. Lee Y., Oh S.H., Kim M.W. An analysis of premature saturation in back propagation learning. Neural Net. 1993, 6:719-728.
10. Stager F., Agarwal M. Three methods to speed up the training of feedforward and feedback perceptrons. Neural Net. 1997, 10:1435-1443.
11. L.F.A. Wessels, E. Barnard, E.V. Rooyen, The physical correlates of local minima, presented at Proceedings of the International Neural Network Conference, Paris, 1990.
12. Vitela J.E., Reifman J. Premature saturation in backpropagation networks: mechanism and necessary conditions. Neural Net. 1997, 10:721-735.
13. Atakulreka A., Sutivong D. Avoiding local minima in feedforward neural networks by simultaneous learning. AI 2007: Adv. Artif. Intell., Lect. Notes Comput. Sci. 2007, 4830:100-109.
14. V.P. Plagianakos, G.D. Magoulas, G.S. Androulakis, M.N. Vrahatis, presented at Proceedings of the 3rd IEEE-IMACS World Multiconference on Circuits, Systems, Communications and Computers, 1999.
15. Weijters A. The BP-SOM architecture and learning rule. Neural Process. Lett. 1995, 2:13-16.
16. A. Weijters, V.D. Bosch, H.J.V.D. Herik, Intelligible Neural Networks with BP-SOM, Marcke and Daelemans, 1997.
17. J. Eggermont, Rule-extraction and Learning in the BP-SOM Architecture, Computer Science Department of Leiden University, 1998.
18. C. Blake, E. Keogh, C.J. Merz, UCI Repository of Machine Learning Databases, 1998.