A successive overrelaxation backpropagation algorithm for neural-network training

IEEE Transactions on Neural Networks

Volumn 9, Issue 3, 1998, Pages 381-388

  De Leone, Renato ^a   Capparuccia, Rosario ^a   Merelli, Emanuela ^a,b  

^a UNIVERSITY OF CAMERINO   (Italy)

^b UNIVERSITÀ POLITECNICA DELLE MARCHE   (Italy)

Author keywords

Backpropagation; Successive overrelaxation

Indexed keywords

BACKPROPAGATION; CONVERGENCE OF NUMERICAL METHODS; LEARNING ALGORITHMS; PERTURBATION TECHNIQUES;

LINEAR COMPLEMENTARY PROBLEMS; LINEAR EQUATIONS; SUCCESSIVE OVERRELAXATION (SOR) ALGORITHM;

NEURAL NETWORKS;

EID: 0032072175     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.668881     Document Type: Article

References (23)

1. O. L. Mangasarian and M. V. Solodov, "Serial and parallel back-propagation convergence via nonmonotone perturbed minimization," Optimization Methods and Software, vol. 4, no. 2, pp. 103-116, 1994.
2. K. Fukushima, S. Miyake, and T. Ito, "Neocognitron: A neural-network model for a mechanism of visual pattern recognition," IEEE Trans. Syst., Man, Cybern., vol. 13, 1983, reprinted in [23].
3. T. J. Sejnowski and C. R. Rosenberg, "Parallel networks that learn to pronounce English text," Complex Syst., vol. 1, pp. 145-168, 1987.
4. N. Qian and T. J. Sejnowski, "Predicting the secondary structure of globular proteins using neural-network models," J. Molecular Biol., vol. 202, pp. 865-884, 1988.
5. R. P. Lippmann, "Review of neural networks for speech recognition," Neural Computa., vol. 1, pp. 1-38, 1989.
6. W. S. McCulloch and W. Pitts, "A logical calculus of ideas immanent in nervous activity," Bull. Math. Biophys., vol. 5, 1943, reprinted in [23].
7. F. Rosenblatt, Principles of Neurodynamics. New York: Spartan, 1962.
8. A. Lapedes and R. Farber, "How neural nets work," in Neural Information Processing Systems, D. Z. Anderson, Ed. New York: Amer. Inst. Phys., pp. 442-456, 1987, 1988.
9. J. Denker, D. Schwartz, B. Wittner, S. Solla, R. Howard, L. Jackel, and J. Hopfield, "Large automatic learning, rule extraction, and generalization," Complex Syst., vol. 1, pp. 877-922, 1987.
10. T. Masters, Practical Neural Networks Recipes in C++. New York: Academic, 1993.
11. P. Werbos, "Beyond regression: New tools for prediction and analysis in the behavioral sciences," Ph.D. dissertation, Harvard Univ., Cambridge, MA, 1974.
12. H. White, "Some asymptotic results for learning in single hidden-layer feedforward network models," J. Amer. Statist. Assoc., vol. 84, no. 408, pp. 1003-1013, 1989.
13. L. Grippo, "A class of unconstrained minimization methods for neural-network training," Optimization Methods and Software, vol. 4, no. 2, pp. 135-150, 1994.
14. Z.-A. Luo and P. Tseng, "Analysis of an approximate gradient projection method with application to the backpropagation algorithm," Optimization Methods and Software, vol. 4, no. 2, pp. 85-102, 1994.
15. D. M. Young, Iterative Solution of Large Linear Systems. New York: Academic, 1971.
16. O. L. Mangasarian, "Solution of symmetric linear complementarity problems by iterative methods," J. Optimization Theory and Applicat., vol. 22, no. 4, pp. 465-485, Aug. 1977.
17. A. Herz, B. Sulzer, R. Kühn, and J. L. van Hemmen, "Hebbian learning reconsidered: Representation of static and dynamic objects in associative neural nets," Biol. Cybern., vol. 60, pp. 457-467, 1989.
18. L. Prechelt, "A set of neural-network benchmark problems and benchmarking rules," Fakultät für Informatik, Univ. Karlsruhe, Germany, Tech. Rep. 2194, 1994.
19. R. A. Jacobs, "Increased rates of convergence through learning rate adaptation," Neural Networks, vol. 1, pp. 295-307, 1988.
20. S. E. Fahlman and C. Lebiere, "The cascade-correlation learning architecture," in Advances in Neural Information Processing Systems, D. S. Touretzky, Ed., vol. 2. San Mateo, CA: Morgan Kaufmann, 1989, pp. 524-532.
21. O. L. Mangasarian, R. Setiono, and W. H. Wolberg, "Pattern recognition via linear programming: Theory and application to medical diagnosis," in Large-Scale Numerical Optimization, T. F. Coleman and Y. Li, Eds. Philadelphia, PA: SIAM, Oct. 19-20, 1989, pp. 22-31.
22. O. L. Mangasarian and W. H. Wolberg, "Cancer diagnosis via linear programming," SIAM News, vol. 23, pp. 1 and 18, 1990.
23. J. A. Anderson and E. Rosenfeld, Eds., Neurocomputing: Foundations of Research. Cambridge, MA: MIT Press, 1988.