Towards stochastic conjugate gradient methods

ICONIP 2002 - Proceedings of the 9th International Conference on Neural Information Processing: Computational Intelligence for the E-Age

Volumn 2, Issue , 2002, Pages 853-856

  Schraudolph, N N ^a   Graepel, T ^a  

^a ETH ZURICH   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; ARTIFICIAL INTELLIGENCE; INFORMATION SCIENCE; STOCHASTIC SYSTEMS;

BENCHMARK EXPERIMENTS; CURVATURE INFORMATION; DETERMINISTIC SYSTEMS; HESSIAN-VECTOR PRODUCTS; STOCHASTIC APPROXIMATIONS; STOCHASTIC CONJUGATE GRADIENTS; STOCHASTIC GRADIENT DESCENT; STOCHASTIC SETTINGS;

CONJUGATE GRADIENT METHOD;

EID: 84964506875     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICONIP.2002.1198180     Document Type: Conference Paper

References (11)

1. K. Levenberg. A method for the solution of certain non-linear problems in least squares. Quarterly Journal of Applied Mathematics, II(2):164-168, 1944.
2. D. Marquardt. An algorithm for least-squares estimation of non-linear parameters. Journal of the Society of Industrial and Applied Mathematics, 11(2):431-441, 1963.
3. M. R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 49:409-436, 1952.
4. N. N. Schraudolph. Local gain adaptation in stochastic gradient descent. In Proceedings of the 9th International Conference on Artificial Neural Networks, pages 569-574, Edinburgh, Scotland, 1999. IEE, London, http://www.inf.ethz.ch/~schraudo/pubs/smd.ps.gz.
5. N. N. Schraudolph. Fast curvature matrix-vector products for second-order gradient descent. Neural Computation, 14(7):1723-1738, 2002.
6. B. A. Pearlmutter. Fast exact multiplication by the Hessian. Neural Computation, 6(1):147-160, 1994.
7. M. F. Møller. A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks, 6(4): 525-533, 1993.
8. G. B. Orr. Dynamics and Algorithms for Stochastic Learning. PhD thesis, Department of Computer Science and Engineering, Oregon Graduate Institute, Beaverton, OR 97006, 1995. ftp://neural.cse.ogi.edu/pub/neural/papers/orrPhDch1-5.ps.Z, orrPhDch6-9.ps.Z.
9. T. Graepel and N. N. Schraudolph. Stable adaptive momentum for rapid online learning in nonlinear systems. In Dorronsoro [11]. http://www.inf.ethz.ch/~schraudo/pubs/sam.ps.gz.
10. N. N. Schraudolph and T. Graepel. Conjugate directions for stochastic gradient descent. In Dorronsoro [11]. http://www.inf.ethz.ch/~schraudo/pubs/hg.ps.gz.
11. J. R. Dorronsoro, editor. Proceedings of the International Conference on Artificial Neural Networks, volume 2415 of Lecture Notes in Computer Science, Madrid, Spain, 2002. Springer Verlag, Berlin.