Variable metric stochastic approximation theory

Journal of Machine Learning Research

Volumn 5, Issue , 2009, Pages 560-566

  Sunehag, Peter ^a,b   Trumpf, Jochen ^b   Vishwanathan, S V N ^c   Schraudolph, Nicol N ^d  

^a CSIRO   (Australia)

^b AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^c PURDUE UNIVERSITY   (United States)

^d Adaptive Tools AG   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

BFGS ALGORITHM; CONVERGENCE THEORY; EXPERT ADVICE; ON-LINE VARIABLES; ONLINE VERSIONS; STOCHASTIC APPROXIMATIONS;

ARTIFICIAL INTELLIGENCE;

APPROXIMATION THEORY;

EID: 84862282880     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (17)

1. S. Amari and H. Nagaoka. Methods of Information Geometry. Oxford University Press, 1993.
2. N. N. Schraudolph, J. Yu, and S. Günter. A stochastic quasiNewton method for online convex optimization. In Proc. 11th Intl. Conf. Artificial Intelligence and Statistics (AIstats), San Juan, Puerto Rico, 2007. Society for Artificial Intelligence and Statistics.
3. L. Bottou and Y. LeCun. Online learning for very large datasets. Applied Stochastic Models in Busi ness and Industry, 21(2):137-151, 2005.
4. S. Amari. Natural gradient works efficiently in learning. Neural Computation, 10(2):251-276, 1998.
5. N. Murata. A statistical study of online learning. On line learning in neural networks, pages 63-92, 1998.
6. J. Kivinen and M. K. Warmuth. Exponentiated gradient versus gradient descent for linear predictors. Information and Computation, 132(1):164, 1997.
7. N. N. Schraudolph. Fast curvature matrix-vector products for second-order gradient descent. Neural Computation, 14(7):1723-1738, 2002.
8. K. Azoury and M. K. Warmuth. Relative loss bounds for online density estimation with the exponential family of distributions. Machine Learning, 43 (3):211-246, 2001. Special issue on Theoretical Advances in Online Learning, Game Theory and Boosting.
9. S. ShalevShwartz, Y. Singer, and N. Srebro. Pegasos: Primal estimated sub-gradient solver for svm. In Proc. Intl. Conf. Machine Learning, 2007.
10. E. Hazan, A. Agarwal, and S. Kale. Logarithmic regret algorithms for online convex optimization. Machine Learning, 69(23):169-192, 2007.
11. M. Zinkevich. Online convex programming and generalised infinitesimal gradient ascent. In Proc. Intl. Conf. Machine Learning, pages 928-936, 2003.
12. H. E. Robbins and S. Monro. A stochastic approximation method. Annals of Mathematical Statistics, 22: 400-407, 1951.
13. J. Blum. Multidimensional stochastic approximation methods. Annals of Mathematical Statistics, 25:737-744, 1954.
14. H. E. Robbins and D. O. Siegmund. A convergence theorem for non negative almost supermartingales and some applications. In Proc. Sympos. Optimiz ing Methods in Statistics, pages 233-257, Ohio State Univ., Columbus, Ohio, 1971. Academic Press, New York.
15. L. Bottou. Online algorithms and stochastic approximations. In D. Saad, editor, Online Learning and Neural Networks. Cambridge University Press, Cambridge, UK, 1998.
16. J. Nocedal and S. J. Wright. Numerical Optimization. Springer Series in Operations Research. Springer, 1999.
17. T. Kailath, A. Sayed, and B. Hassibi. Linear Estimation. Prentice Hall, 2000.