A generalized learning paradigm exploiting the structure of feedforward neural networks

IEEE Transactions on Neural Networks

Volumn 7, Issue 6, 1996, Pages 1450-1460

  Parisi, Raffaele ^b   Di Claudio, Elio D ^b   Orlandi, Gianni ^a,b   Rao, Bhaskar D ^a,c  

^a IEEE

^b SAPIENZA UNIVERSITY OF ROME   (Italy)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0000014434     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.548172     Document Type: Article

References (28)

1. C. Bishop, "Exact calculation of the Hessian matrix for the multilayer perceptron," Neural Computa., vol. 4, pp. 494-501, 1992.
2. W. L. Buntine and A. S. Weigend, "Computing second derivatives in feedforward networks: A review," IEEE Trans. Neural Networks, vol. 5, May 1994.
3. S. Saarinen, R. Bramley, and G. Cybenko, "Ill conditioning in neural-network training problems," SIAM J. Sci. Comput., vol. 14, no. 3, pp. 693-714, May 1993.
4. R. Battiti, "First- and second-order methods for learning: Between steepest descent and Newton's method," Neural Computa., vol. 4, pp. 141-166, 1992.
5. D. G. Luenberger, Linear and Nonlinear Programming, 2nd ed. Reading, MA: Addison Wesley, 1989.
6. [61 R. Fletcher, Practical Methods of Optimization, 2nd ed. New York: Wiley, 1987.
7. D. E. Rumelhart, G. E. Hinton, and R. G. Williams, "Learning internal representations by error propagation," in Parallel Distributed Processing: Exploration in the Microstructure of Cognition, vol. 1, D. E. Rumelhart and J. L. McLelland, Eds. Cambridge, MA: MIT Press/Bradford, 1986.
8. S. Haykin, Adaptive Filter Theory, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1991.
9. _, Neural Networks - A Comprehensive Foundation. Piscataway, NJ: IEEE Press, 1994.
10. R. A. Jacobs, "Increased rates of convergence through learning rate adaptation," Neural Networks, vol. 1, pp. 295-307, 1988.
11. T. P. Vogl, J. K. Mangis, A. K. Rigler, W. T. Zink, and D. L. Alkon, "Accelerating the convergence of the backpropagation method," Biol. Cybern., vol. 59, pp. 257-263, 1988.
12. R. Watrous, "Learning algorithms for connectionist networks: Applied gradient methods of nonlinear optimization," Univ. Pennsylvania, Tech. Rep. MS-CIS-87-51.
13. G. H. Golub and C. F. van Loan, Matrix Computations, 2nd ed. Baltimore, MD: John Hopkins Univ. Press, 1989.
14. P. E. Gill and W. Murray, "Algorithms for the solution of the nonlinear least-squares problem," SIAM J. Numer. Anal., vol. 15, no. 5, pp. 977-992, Oct. 1978.
15. J. E. Dennis, Jr., D. M. Gay, and R. E. Welsch, "An adaptive nonlinear least-squares algorithm," ACM Trans. Math. Software, vol. 7, no. 3, pp. 348-368, Sept. 1981.
16. M. G. Bello, "Enhanced training algorithms, and integrated training/architecture selection for multilayer perceptrons networks," IEEE Trans. Neural Networks, vol. 3, Nov. 1992.
17. P. L. Joint, "On large scale nonlinear least squares calculations," SIAM J. Sci. Stat. Comput., vol. 8, no. 3, May 1987, pp. 416-435.
18. Y. Le Cua, "Second-order properties of error surfaces: Learning time and generalization," in Advances in Neural Information Processing Systems 3, 1991.
19. S. E. Fahlman, "Faster-learning variations on backpropagation: An empirical study," in Proc. 1988 Connectionist Models Summer School, Carnegie Mellon Univ, Pittsburgh, PA.
20. S. Kollias and D. Anastassiou, "An adaptive least squares algorithm for the efficient training of artificial neural networks," IEEE Trans. Circuits Syst., vol. 36, Aug. 1989.
21. R. S. Scalero and N. Tepedelenlioglu, "A fast new algorithm for training feedforward neural networks," IEEE Trans. Signal Processing, vol. 40, Jan. 1992.
22. M. R. Azimi-Sadjadi and R. J. Liou, "Fast learning process of multilayer neural networks using recursive least squares method," IEEE Trans. Signal Processing, vol. 40, Feb. 1992.
23. S. Shah, F. Palmieri, and M. Datum, "Optimal filtering algorithms for fast learning in feedforward neural networks," Neural Networks, vol. 5, pp. 779-787, 1992.
24. E. D. Di Claudio, R. Parisi, and G. Orlandi, "LS-based training algorithm for neural networks," in Proc. 1993 IEEE Wkshp. Neural Networks Signal Processing (NNSP'93), Baltimore, MD, Sept. 7 9, 1993.
25. G. Golub and V. Pereyra, "The differentiation of pseudoinverses and nonlinear least squares problems whose variables are separate," SIAM J. Numerical Analysis, vol. 10, no. 2, pp. 413-432, 1973.
26. W. H. Lawton and E. A. Sylvestre, "Elimination of linear parameters in nonlinear regression," Technometrics, vol. 13, no. 3, pp. 461-467, 1971.
27. I. Guttman, V. Pereyra, and H. D. Scolnik, "Least squares estimation for a class of nonlinear models," Technometrics, vol. 15, no. 2, pp. 209-218, 1973.
28. P. J. Huber, Robust Statistics. New York: Wiley, 1981.