Fast Learning Algorithms for Neural Networks

IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing

Volumn 39, Issue 7, 1992, Pages 453-474

  Karayiannis, Nicolaos B ^a   Venetsanopoulos, Anastasios N ^b  

^a UNIVERSITY OF HOUSTON   (United States)

^b UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

LEARNING SYSTEMS; OPTIMIZATION;

BACK PROPAGATION ALGORITHM; DELTA RULE ALGORITHM; ELEANNE ALGORITHMS; FEEDFORWARD NEURAL NETWORKS;

NEURAL NETWORKS;

EID: 0026898850     PISSN: 10577130     EISSN: None     Source Type: Journal    
DOI: 10.1109/82.160170     Document Type: Article

References (30)

1. B. Widrow and M. A. Lehr, “30 Years of adaptive neural networks: Perceptron, madaline, and backpropagation,” Proc. IEEE, vol. 78, pp. 1415–1442, Sept. 1990.
2. P. Werbos, “Beyond Regression: New tools for prediction and analysis in behavioral sciences,” Ph.D. dissertation, Harvard Univ., Cambridge, MA, Aug. 1974.
3. D. E. Rumelhart and J. L. McClelland, Parallel Distributed Processing. MIT Press, Cambridge, MA: Vols. I and II, 1986.
4. L. F. Abbott and T. B. Kepler, “Optimal learning in neural network memories,” J. Physics A, vol. 22, pp. L711-L717, 1989.
5. J. K. Anlauf and M. Biehl, “The AdaTron: An adaptive perceptron algorithm,” Europhysics Lett., vol. 10, pp. 687–692, 1989.
6. R. A. Jacobs, “Increased rate of convergence through learning rate adaptation,” Neural Networks, vol. 1, pp. 295–307, 1988.
7. F. M. Silva and L. B. Almeida, “Acceleration techniques for the backpropagation algorithm,” in Proc. EURASIP Workshop on Neural Networks, L. B. Almeida and C. J. Wellekens, Eds. New York: Springer-Verlag, 1990.
8. R. Battiti, “Accelerated backpropagation learning: Two optimization methods,” Complex Systems, vol. 3, pp. 331–342, 1989.
9. T. P. Vogl, J. K. Mangis, A. K. Rigler, W. T. Zink, and D. L. Alton, “Accelerating the convergence of the back-propagation method,” Bio. Cybem. vol. 59, pp. 257–263, 1988.
10. S. E. Fahlman, “Fast-learning variations on back-propagation: An empirical study,” in Proc. 1988 Connectionist Models Summer School, D. Touretzky, G. Hinton, and T. Sejnowski, Eds. San Mateo, CA: Morgan Kaufmann, pp. 38–51, 1989.
11. D. B. Parker, “Optimal algorithms for adaptive networks: Second order back propagation, second order direct propagation, and second order Hebbian learning,” in Proc. Int. Conf. Neural Networks, vol. II, pp. 593–600, 1987.
12. S. Kollias and D. Anastassiou, “An adaptive least squares algorithm for the efficient training of artificial neural networks,” IEEE Trans. Circuits Syst., vol. 36, pp. 1092–1101, 1989.
13. S. Becker and Y. le Cun, “Improving the convergence of backpropagation learning with second order methods,” in Proc. 1988 Connectionist Models Summer School, D. Touretzky, G. Hinton, and T. Sejnowski, Eds. San Mateo, CA: Morgan Kaufmann, pp. 29–37, 1989.
14. S. Singhal and L. Wu, “Training multilayer perceptrons with extended Kalman filter,” in Advances in Neural Information Processing. San Mateo, Morgan Kaufmann, CA: pp. 133–140, 1989.
15. R. S. Scalero and N. Tepedelenlioglu, “A fast new algorithm for training feedforward neural networks,” IEEE Trans. Signal Processing, vol. 40, pp. 202–210, 1992.
16. J. Hertz, A. Krogh, and R. G. Palmer, Introduction to the Theory of Neural Computation. Redwood City, CA: Addison-Wesley, 1991.
17. N. B. Karayiannis, “Artificial neural networks: Learning algorithms, performance evaluation, and applications,” Ph.D. dissertation, Univ. Toronto, Apr. 1991.
18. N. B. Karayiannis and A. N. Venetsanopoulos, “Recursive least squares learning algorithms for single-layered neural networks,” in Proc. of LASTED Int. Conf. Modelling, Simulation, and Optimization, pp. 162–165, May 1990.
19. —, “Efficient learning algorithms for single-layered neural networks,” Parallel Processing in Neural Systems and Computers, R. Eckmiller, G. Hartmann, and G. Hauske, Eds. North Holland: Elsevier Publishers, pp. 173–176, 1990.
20. — “Efficient learning algorithms for neural networks,” submitted to IEEE Trans. Syst., Man, Cybem.
21. S. A. Solla E. Levin, and M. Fleisher, “Accelerated learning layered neural networks,” Complex Systems, vol. 2, pp. 625–640, 1988.
22. E. B. Baum and F. Wilczek, “Supervised learning of probability distributions by neural networks,” in Neural Information Processing Systems, D. Z. Anderson, Ed. New York: American Institute of Physics, pp. 52–61, 1988.
23. J. J. Hopfield, “Learning algorithms and probability distributions in feed-forward and feed-back networks,” in Proc. National Academy of Sciences, USA, vol. 84, pp. 8429–8433, 1987.
24. N. B. Karayiannis and A. N. Venetsanopoulos, “Fast learning algorithms for neural networks,” in Artificial Neural Networks, T. Kohonen et. al., Eds. North Holland: Elsevier Publishers, pp. 1141–1144, 1991.
25. D. W. Ruck, S. K. Rogers, M. Kabrisky, M. E. Oxley, and B. W. Suter, “The multilayer perceptron as an approximation to Bayes optimal discriminant function,” IEEE Trans. Neural Networks, vol. 1, pp. 296–298, 1990.
26. N. B. Karayiannis and A. N. Venetsanopoulos, “On the dynamics of neural networks realizing associative memories of first and higher order,” Network: Computation in Neural Systems, vol. 1, pp. 345–364, July 1990.
27. B. Noble and J. W. Daniel, Applied Linear Algebra. Englewood Cliffs, NJ: Prentice-Hall, 1977.
28. G. W. Cottrell, P. Munro, and D. Zipser, “Learning internal representations from gray-scale images: An example of extensional programming,” in Proc. Ninth Annual Conf. Cognitive Science Society, pp. 462–473, 1987.
29. H. Boulard and Y. Kamp, “Auto-association by multilayer perceptrons and singular value decomposition,” Bio. Cybem., vol. 59, pp. 291–294, 1988.
30. P. Baldi and K. Hornik, “Neural networks and principal component analysis: Learning from examples without local minima,” Neural Networks, vol. 2, pp. 53–58, 1989.