Merging back-propagation and Hebbian learning rules for robust classifications

Neural Networks

Volumn 9, Issue 7, 1996, Pages 1213-1222

  Dong Gyu, Jeong ^a   Soo Young, Lee ^a  

^a Korea Advanced Institute of Science and Technology (KAIST)   (South Korea)

Author keywords

Classifier networks; Error back propagation; Generalization; Hebbian learning; Hidden neuron saturation; Hybrid learning; Mapping sensitivity; Robustness

Indexed keywords

BACKPROPAGATION; COMPUTATIONAL METHODS; COMPUTER SIMULATION; CONVERGENCE OF NUMERICAL METHODS; LEARNING ALGORITHMS; OPTIMIZATION; PATTERN RECOGNITION;

CLASSIFIER NETWORKS; ERROR BACKPROPAGATION; GRADIENT DESCENT METHOD; HEBBIAN LEARNING RULES; HIDDEN NEURON SATURATION; MAPPING SENSITIVITY; SIGMOID FUNCTIONS; STEEPEST DESCENT METHOD;

NEURAL NETWORKS;

ARTICLE; CONTROLLED STUDY; LEARNING; MATHEMATICAL ANALYSIS; PATTERN RECOGNITION; PRIORITY JOURNAL;

EID: 0030273576     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/0893-6080(96)00042-1     Document Type: Article

References (25)

1. Baum E., Haussler D. What size net gives valid generalization? Neural Computation. 1:1989;151-160.
2. Bishop C.M. Curvature-driven smoothing: a learning algorithm for feedforward networks. IEEE Transactions on Neural Networks. 4:1993;882-884.
3. Drucker H., Le Cun Y. Improving generalization performance using double backpropagation. IEEE Transactions on Neural Networks. 3:1992;991-997.
4. Fukushima K. Neocognitron: A hierarchical neural network capable of visual pattern recognition. Neural Networks. 1:1989;119-130.
5. Fukushima, K. (1993). Improved generalization ability using constrained neural network architecture. Proceedings of the International Joint Conference on Neural Networks (IJCNN'93) (pp. 2049-2054). Nagoya, Japan.
6. Hanson, S. J., & Pratt, L. Y. (1989). Comparing biases for minimal network construction with back-propagation. In D. Touretzky (Ed.), Advances in neural information processing systems 1 (pp. 177-185). San Matero, CA: Morgan Kaufmann.
7. Hartman E.J., Keeler J.D., Kowalski J.M. Layered neural networks with Gaussian hidden units with universal approximations. Neural Computation. 2:1990;210-215.
8. Ishikawa, M. (1990). A structural learning algorithm with forgetting of link weights. (ETL Report TR-90-7), Electrotechnical Laboratory, Tsukuba, Japan.
9. Ishikawa, M. (1994). Structural learning in neural networks. Proceedings of the 3rd International Conference on Fuzzy Logic, Neural Nets, and Soft Computing (pp. 37-44). Iizuka, Japan.
10. Karnin E.D. A simple procedure for pruning back-propagation trained neural networks. IEEE Transactions on Neural Networks. 1:1990;239-242.
11. Koh, S. H., Lee, S. Y., Jang, J. S., & Shin, S. Y. (1990). Merging Hebbian learning rule and least-mean-square error algorithm for two layer neural networks. Proceedings of the International Joint Conference on Neural Networks (I, pp. 647-650). Washington DC., USA.
12. Krogh, A., & Hertz, J. A. (1992). A simple weight decay can improve generalization. In D. Touretzky (Ed.), Advances in neural information processing systems 4 (pp. 950-957). San Mateo, CA: Morgan Kaufmann.
13. LeCun Y., Boser B., Denker J.S., Henderson D., Howard R.E., Hubbard W., Jackel L.D. Back propagation applied to handwritten zip code recognition. Neural Computation. 1:1989;541-551.
14. LeCun, Y., Denker, J. S., & Solla, S. A. (1990). Optimal brain damage, In D. Touretzky (Ed.), Advances in neural information processing systems 2 (pp. 598-605). San Mateo, CA: Morgan Kaufmann.
15. Lee S., Kil R.M. A Gaussian potential function network with hierarchically self-organizing learning. Neural Networks. 4:1991;207-224.
16. Lee, S. Y., & Jeong, D. G. (1994). Error minimization, generalization, and hardware implementability of supervised learning. Proceedings of World Congress on Neural Network (III, pp. 325-330). San Diego, CA.
17. Mozer, M. C., & Smolensky, P. (1989). Skeletonization: a technique for trimming the fat from a network via relevance assessment. In D. Touretzky (Ed.), Advances in neural information processing systems 1 (pp. 107-115). San Mateo, CA: Morgan Kaufmann.
18. Nowlan S.J., Hinton G.E. Simplifying neural networks by soft weight sharing. Neural Computation. 4:1992;473-493.
19. Oh, S.H., Lee, Y. Sensitivity analysis of single hidden-layer neural networks with threshold functions. IEEE Transactions on Neural Networks, 6 1005-1007 (1995).
20. Reed R. Pruning algorithms - a survey. IEEE Transactions on Neural Networks. 4:1993;740-747.
21. Sietsma J., Dow R.J.F. Creating artificial neural networks that generalize. Neural Networks. 4:1991;67-79.
22. Stevenson M., Winter R., Widrow B. Sensitivity of feedforward neural networks to weight errors. IEEE Transactions on Neural Networks. 1:1990;71-80.
23. Von Lehmann, A. et al. (1988). Factors influencing learning by back propagation. Proceedings of the IEEE International Conference on Neural Networks (pp. 335-341). San Diego, CA.
24. Waibel A., Hanazawa T., Hinton G., Shikano K., Lang K. Phoneme recognition using time-delay neural networks. IEEE Transactions on Acoustics, Speech, Signal Processing. 37:1989;328-339.
25. Weigend, A. S., Rumelhart, D. E., & Huberman, B. A. (1990). Back-propagation, weight elimination and time-series prediction. Proceedings of the 1990 Connectionist Models Summer School (pp. 65-80). San Mateo, CA: Morgan Kaufmann.