Efficient calculation of the Gauss-Newton approximation of the Hessian matrix in neural networks

Neural Computation

Volumn 24, Issue 3, 2012, Pages 607-610

  Fairbank, Michael ^a   Alonso, Eduardo ^a  

^a CITY UNIVERSITY   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTIFICIAL NEURAL NETWORK; NOTE;

ALGORITHMS; NEURAL NETWORKS (COMPUTER);

EID: 84861119838     PISSN: 08997667     EISSN: 1530888X     Source Type: Journal    
DOI: 10.1162/NECO_a_00248     Document Type: Note

References (8)

1. Bishop, C. M. (1995). Neural networks for pattern recognition. New York: Oxford University Press.
2. Huss-Lederman, S., Jacobson, E. M., Tsao, A., Turnbull, T., & Johnson, J. R. (1996). Implementation of Strassen's algorithm for matrix multiplication. In Proceedings of the 1996 ACM/IEEE Conference on Supercomputing (CDROM), Supercomputing '96. Washington, DC: IEEE Computer Society.
3. Levenberg, K. (1944). A method for the solution of certain non-linear problems in least squares. Quart. Appl. Math., 2, 164-168.
4. Marquardt, D. W. (1963). An algorithm for least-squares estimation of nonlinear parameters. Journal of the Society for Industrial and Applied Mathematics, 11, 431-441.
5. Rumelhart, D., Hinton, G., &Williams, R. (1986). Learning representations by backpropagating errors. Nature, 6088, 533-536.
6. Schraudolph, N. N. (2002). Fast curvature matrix-vector products for second-order gradient descent. Neural Computation, 14(7), 1723-1738.
7. Ranga Suri, N.N.R., Deodhare, D., & Nagabhushan, P. (2002). Parallel Levenberg- Marquardt-based neural network training on Linux clusters-a case study. In Linux Clusters, ICVGIP 2002, 3rd Indian Conference on Computer Vision, Graphics and Image Processing. N.P.: Allied Publishers.
8. Werbos, P. J. (1974). Beyond regression: New tools for prediction and analysis in the behavioral sciences. Unpublished doctoral dissertation, Harvard University.