Accelerated randomized stochastic optimization

Annals of Statistics

Volumn 31, Issue 4, 2003, Pages 1260-1281

  Dippon, Júrgen ^a  

^a UNIVERSITY OF STUTTGART   (Germany)

Author keywords

Asymptotic normality; Gradient estimation; Optimal rates of convergence; Randomization; Stochastic approximation; Stochastic optimization

Indexed keywords

EID: 0042859914     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/aos/1059655913     Document Type: Article

References (20)

1. BERGER, E. (1986). Asymptotic behaviour of a class of stochastic approximation procedures. Probab. Theory Related Fields 71 517-552.
2. BLUM, J. R. (1954). Multidimensional stochastic approximation methods. Ann. Math. Statist. 25 737-744.
3. CHEN, H. (1988). Lower rate of convergence for locating a maximum of a function. Ann. Statist. 16 1330-1334.
4. CHEN, H. F., DUNCAN, T. E. and PASIK-DUNCAN, B. (1999). A Kiefer-Wolfowitz algorithm with randomized differences. IEEE Trans. Automat. Control 44 442-453.
5. DIPPON, J. and RENZ, J. (1997). Weighted means in stochastic approximation of minima. SIAM J. Control Optim. 35 1811-1827.
6. FABIAN, V. (1967). Stochastic approximation of minima with improved asymptotic speed. Ann. Math. Statist. 38 191-200.
7. FABIAN, V. (1968). On asymptotic normality in stochastic approximation. Ann. Math. Statist. 39 1327-1332.
8. FABIAN, V. (1971). Stochastic approximation. In Optimizing Methods in Statistics (J. S. Rustagi, ed.) 439-470. Academic Press, New York.
9. GERENCSÉR, L. (1999). Convergence rate of moments in stochastic approximation with simultaneous perturbation gradient approximation and resetting. IEEE Trans. Automat. Control 44 894-905.
10. KIEFER, J. and WOLFOWITZ, J. (1952). Stochastic estimation of the maximum of a regression function. Ann. Math. Statist. 23 462-466.
11. KUSHNER, H. J. and CLARK, D. S. (1978). Stochastic Approximation Methods for Constrained and Unconstrained Systems. Springer, New York.
12. KUSHNER, H. J. and YIN, G. G. (1997). Stochastic Approximation Algorithms and Applications. Springer, New York.
13. POLYAK, B. T. and JUDITSKY, A. B. (1992). Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30 838-855.
14. POLYAK, B. T. and TSYBAKOV, A. B. (1990). Optimal orders of accuracy for search algorithms of stochastic optimization. Problems Inform. Transmission 26 126-133.
15. ROBBINS, H. and SIEGMUND, D. (1971). A convergence theorem for nonnegative almost supermartingales and some applications. In Optimizing Methods in Statistics (J. S. Rustagi, ed.) 233-257. Academic Press, New York.
16. RUPPERT, D. (1991). Stochastic approximation. In Handbook of Sequential Analysis (B. K. Ghosh and P. K. Sen, eds.) 503-529. Dekker, New York.
17. SPALL, J. C. (1988). A stochastic approximation algorithm for large-dimensional systems in the Kiefer-Wolfowitz setting. In Proc. Conference on Decision and Control 1544-1548. IEEE, New York.
18. SPALL, J. C. (1992). Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans. Automat. Control. 37 332-341.
19. SPALL, J. C. (1997). A one-measurement form of simultaneous perturbation stochastic approximation. Automatica J. IFAC 33 109-112.
20. WALK, H. (1988). Limit behaviour of stochastic approximation processes. Statist. Decisions 6 109-128.