A fast natural Newton method

ICML 2010 - Proceedings, 27th International Conference on Machine Learning

Volumn , Issue , 2010, Pages 623-630

  Le Roux, Nicolas ^a   Fitzgibbon, Andrew ^a  

^a MICROSOFT RESEARCH   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

FASTER CONVERGENCE; LANGUAGE MODELING; LEARNING PROBLEM; NEWTON METHODS; OPTIMIZATION ALGORITHMS; ROBUST LEARNING ALGORITHM; SPECIAL STRUCTURE; TWO DIRECTIONS;

COMPUTATIONAL LINGUISTICS; LEARNING SYSTEMS; NEWTON-RAPHSON METHOD; OBJECT RECOGNITION; OPTIMIZATION;

LEARNING ALGORITHMS;

EID: 77956525316     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (6)

1. Amari, Shun-ichi. Natural gradient works efficiently in learning. Neural Computation, 10(2):251-276, 1998.
2. Bordes, Antoine, Bottou, Léon, and Gallinari, Patrick. SGD-QN: Careful quasi-newton stochastic gradient descent. Journal of Machine Learning Research, 10: 1737-1754, July 2009.
3. Bottou, Léon and Bousquet, Olivier. The tradeoffs of large scale learning. In Platt, J.C., Koller, D., Singer, Y., and Roweis, S. (eds.), Advances in Neural Information Processing Systems, volume 20, pp. 161-168. 2008.
4. Le Roux, Nicolas, Manzagol, Pierre-Antoine, and Bengio, Yoshua. Topmoumoute online natural gradient algorithm. In Advances in Neural Information Processing Systems 20, pp. 849-856. MIT Press, Cambridge, MA, 2008.
5. Nocedal, J. and Wright, S. J. Numerical Optimization, Second Edition. Springer Verlag, New York, 2006.
6. Schraudolph, Nicol N., Yu, Jin, and Günter, Simon. A stochastic quasi-newton method for online convex optimization. In Proceedings of AISTATS 2007, Puerto Rico. 2007.