Accelerating the training of feedforward neural networks using generalized Hebbian rules for initializing the internal representations

IEEE Transactions on Neural Networks

Volumn 7, Issue 2, 1996, Pages 419-426

  Karayiannis, Nicolaos B ^a,b  

^a IEEE

^b UNIVERSITY OF HOUSTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BACKPROPAGATION; COMPUTATIONAL COMPLEXITY; CONVERGENCE OF NUMERICAL METHODS; ERRORS; LEARNING ALGORITHMS; LEARNING SYSTEMS; MATHEMATICAL MODELS; OPTIMIZATION; VECTORS;

ERROR BACKPROPAGATION ALGORITHM; FAST BACKPROPAGATION ALGORITHM; GRADIENT DESCENT METHOD; QUADRATIC ERROR CRITERION; QUICK BACKPROPAGATION ALGORITHM; RANDOMLY SELECTED SYNAPTIC WEIGHT; SIGMOID NONLINEARITIES; SINGULAR VALUE DECOMPOSITION;

FEEDFORWARD NEURAL NETWORKS;

EID: 0030104509     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.485677     Document Type: Article

References (19)

1. E. B. Baum and F. Wilczek, "Supervised learning of probability distributions by neural networks," in Neural Information Processing Systems, D. Z. Anderson, Ed. New York: Amer. Inst. Physics, 1988, pp. 52-61.
2. T. Denoeux, R. Lengelle, and S. Canu, "Initialization of weights in a feedforward neural network using prototypes," in Artificial Neural Networks, T. Kohonen, K. Makisara, O. Simuba, and J. Kangas, Eds. Amsterdam, The Netherlands: Elsevier, 1991, pp. 623-628.
3. S. E. Fahlman and C. Lebiere, "The cascaded-correlation learning architecture," Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. CMU-CS-90-100, 1990.
4. H. Haario and P. Jokinen, "Increasing the learning speed of back-propagation algorithm by linearization," in Artificial Neural Networks, T. Kohonen, Ed. Amsterdam, The Netherlands: Elsevier, 1991, pp. 629-634.
5. D. O. Hebb, The Organization of Behavior. New York: Wiley, 1949.
6. I. Hertz, A. Krogh, and R. G. Palmer, Introduction to the Theory of Neural Computation. Redwood City, CA: Addison-Wesley, 1991.
7. J. J. Hopfield, "Learning algorithms and probability distributions in feedforward and feedback networks," in Proc. Nat. Academy Sci., vol. 84, 1987, pp. 8429-8433.
8. N. B. Karayiannis, "Artificial neural networks: Learning algorithms, performance evaluation, and applications," Ph.D. dissertation, Univ. Toronto, Canada, 1991.
9. N. B. Karayiannis and A. N. Venetsanopoulos, "Fast learning algorithms for neural networks," IEEE Trans. Circuits Syst., vol. 39, pp. 453-474, 1992.
10. _, "Efficient learning algorithms for neural networks (ELEANNE)," IEEE Trans. Syst., Man, Cybern., vol. 23, pp. 1372-1383, 1993.
11. _, Artificial Neural Networks: Learning Algorithms, Performance Evaluation, and Applications. Boston, MA: Kluwer, 1993.
12. J. Karhunen and J. Joutsensalo, "Representation and separation of signals using nonlinear PCA type learning," Neural Network, vol. 7, pp. 113-127, 1994.
13. E. Oja, "A simplified neuron model as a principal component analyzer," J. Math. Biol., vol. 15, pp. 267-273, 1982.
14. _, "Principal components, minor components, and linear neural networks," Neural Networks, vol. 5, pp. 927-935, 1992.
15. E. Oja, H. Ogawa, and J. Wangviwattana, "Learning in nonlinear constrained Hebbian networks," in Artificial Neural Networks, T. Kohonen, K. Makisara, O. Simula, and J. Kangas, Eds. Amsterdam, The Netherlands: Elsevier, 1991, pp. 385-390.
16. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propagation," in Parallel Distributed Processing, D. E. Rumelhart and J. L. McClelland, Eds., vol. 1. Cambridge, MA: MIT Press, 1986, pp. 318-362.
17. T. D. Sanger, "Optimal unsupervised learning in a single-layer linear feedforward neural network," Neural Networks, vol. 2, pp. 459-473, 1989.
18. S. A. Solla, E. Levin, and M. Fleisher, "Accelerated learning in layered neural networks," Complex Syst., vol. 2, pp. 625-640, 1988.
19. D. Whitley and N. Karunanithi, "Generalization in feedforward neural networks," in Proc. Int. Joint Conf. Neural Networks (IJCNN '91), Seattle, WA, July 8-12, 1991, pp. 77-82.