Stochastic approximation: A survey

Wiley Interdisciplinary Reviews: Computational Statistics

Volumn 2, Issue 1, 2010, Pages 87-96

  Kushner, Harold ^a  

^a BROWN UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

A-STABILITY; ASYMPTOTIC PROPERTIES; BASIC IDEA; CONTINUOUS TIME; CONVERGENCE WITH PROBABILITY ONE; HIGH-DIMENSIONAL PROBLEMS; MULTIPLE TIME SCALE; NOISE PROCESS; RATE OF CONVERGENCE; RECURSIVE ALGORITHMS; STATE-DEPENDENT NOISE; STOCHASTIC APPROXIMATIONS; TYPE ARGUMENTS; WEAK CONVERGENCE;

APPROXIMATION THEORY; ASYMPTOTIC ANALYSIS; ORDINARY DIFFERENTIAL EQUATIONS; STOCHASTIC SYSTEMS;

APPROXIMATION ALGORITHMS;

EID: 78651592863     PISSN: 19395108     EISSN: 19390068     Source Type: Journal    
DOI: 10.1002/wics.57     Document Type: Review

References (74)

1. Robbins H, Monro S. A stochastic approximation method. Ann Math Stat 1951, 22:400-407.
2. Wasan MT. Stochastic Approximation. Cambridge: Cambridge University Press; 1969.
3. Blum JR. Multidimensional stochastic approximation. Ann Math Stat 1954, 9:737-744.
4. Dvoretzky A. On stochastic approximation. Proceedings of the Third Berkeley Symposium on Mathematical Statistics and Probability, 1956, 17:39-55.
5. Schmetterer L. Stochastic approximation. Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability. Berkeley: University of California;1960, 587-609.
6. Dupač V. On the Kiefer-Wolfowitz approximation method. Casopis Pest Mat 1957, 82:47-75.
7. Dupač V. Stochastic approximation. Kybernetica (Prague) Suppl 1981, 17.
8. Gladyshev EG. On stochastic approximation. Theory Probab Appl 1965, 10:275-278.
9. Robbins H, Siegmund D. A convergence theorem for some nonnegative almost supermartingales and some applications. In: Rustagi JS, ed. Optimizing Methods in Statistics. New York: Academic Press; 1971, 233-257.
10. Sakrison DJ. A continuous Kiefer-Wolfowitz procedure for random processes. Ann Math Stat 1964, 35:590-599.
11. Sacks J. Asymptotic distribution of stochastic approximation processes. Ann Math Stat 1958, 29:373-405.
12. Kushner HJ, Yin G. Stochastic Approximation and Recursive Algorithms and Applications. Berlin and New York: Springer-Verlag; 2003.
13. Kiefer J, Wolfowitz J. Stochastic estimation of the maximum of a regression function. Ann Math Stat 1952, 23:462-466.
14. Nevelson MB, Khasminskii RZ. Stochastic Approximation and Recursive Estimation, vol.47, Translation of Mathematical Monographs. Providence: American Mathematical Society; 1976.
15. Tsypkin YaZ. Adaptation and Learning in Automatic Systems. New York: Academic Press; 1971.
16. Lai TL. Stochastic approximation. Ann Stat 2003, 31:391-406.
17. Schmetterer L. Multidimensional stochastic approximation. In: Krishnaiah PR, ed. Multivariate Analysis II. New York: Academic Press; 1969, 443-460.
18. Ruppert D. Stochastic approximation. In: Ghosh BK, Sen PK, ed. Handbook in Sequential Analysis. New York: Marcel Dekker; 1991, 503-529.
19. Fabian V. Stochastic approximation. In: Rustagi JS, ed. Optimizing Methods in Statistics. New York: Academic Press; 1971.
20. Benveniste A, Metivier M, Priouret P. Adaptive Algorithms and Stochastic Approximation. Berlin and New York: Springer-Verlag; 1990.
21. Kushner HJ. Approximation and Weak Convergence Methods for Random Processes with Applications to Stochastic Systems Theory. Cambridge:MIT Press; 1984.
22. Solo V, Kong X. Adaptive Signal Processing Algorithms. Englewood Cliffs: Prentice-Hall; 1995.
23. Borkar VS, Meyn SP. The o.d.e. method for convergence of stochastic approximation and reinforcement learning. SIAM J Control Optim 2000, 38:447-469.
24. Ljung L. Analysis of recursive stochastic algorithms. IEEE Trans Automat Control 1977, 22:551-575.
25. Kushner HJ, Clark DS. Stochastic Approximation for Constrained and Unconstrained Systems. Berlin andNew York: Springer-Verlag; 1978.
26. Kushner HJ, Shwartz A. An invariant measure approach to the convergence of stochastic approximations with state dependent noise. SIAM J Control Optim 1984, 22:13-27.
27. Chen H-F. Stochastic Approximation and its Applications. Boston: Kluwer Academic Publishers; 2002.
28. Chen H-F, Zhu YM. Stochastic approximation procedure with randomly varying truncations. Sci Sin Ser., A 1986, 29:914-926.
29. Wang IJ, Chong EKP, Kulkarni SR. Equivalent necessary and sufficient conditions on noise sequences for stochastic approximation algorithms. Adv Appl Probab 1996, 28:784-801.
30. Benaïm M. A dynamical systems approach to stochastic approximations. SIAM J Control Optim 1996, 34: 437-472.
31. Benaïm M. Dynamics of stochastic approximation algorithms. Séminaire de robabilities, Lecture Notes in Mathematics: 1769. Berlin and New York: Springer-Verlag; 1999, 1-69.
32. Benaïm M. Dynamics of stochastic approximation algorithms. Séminaire de Probabilités, vol.XXXIII, Lecture Notes in Mathematics, 1709. Berlin and New York: Springer; 1999, 1-68.
33. Guckenheimer J, Holmes P. Nonlinear Oscillations, Dynamical Systems, and Bifurcations of Vector Fields. Berlin and New York: Springer-Verlag; 1983.
34. Kushner HJ, Dupuis P. Numerical Methods for Stochastic Control Problems in Continuous Time. Berlin and New York: Springer-Verlag; 1992, (2nd edition, 2001).
35. Dupuis P, Kushner HJ. Stochastic approximation and large deviations: upper bounds and w.p.1 convergence. SIAM J Control Optim 1989, 27:1108-1135.
36. Billingsley P. Convergence of Probability Measures, 2nd edn. New York: John Wiley & Sons; 1999.
37. Ethier SN, Kurtz TG. Markov Processes: Characterization and Convergence. New York: John Wiley & Sons; 1986.
38. Bhatnagar S, Borkar VS. A two-time-scale stochastic approximation scheme for imulation based parametric optimization. Probab Eng Inform Sci 1998, 12: 519-531. (Pubitemid 128544961)
39. Kushner HJ. Diffusion approximations to output processes of nonlinear systems with wide-band inputs, with applications. IEEE Trans Inf Theory 1980, 26: 715-725.
40. Sadegh P, Spall JC. Optimal random perturbations for stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans Autom Control 1998, 43:1480-1484.
41. Spall JC. Multivariate stochastic approximation using a simultaneous perturbation gradient approximation. IEEE Trans Autom Control 1992, 37:331-341.
42. Spall JC. Adaptive stochastic approximation by the simultaneous perturbation method. IEEE Trans Autom Control 2000, 45:1839-1853.
43. Delyon B, Juditsky A. Asymptotical study of parameter tracking algorithms. SIAM J Control Optim 1995, 33:323-345.
44. Gunnarsson S, Ljung L. Frequency domain tracking characteristics of adaptive algorithms. IEEE Trans Acoust Speech Signal Process 1989, 37:1072-1089.
45. Guo L. Stability of recursive tracking algorithms. SIAM J Control Optim 1994, 32:1195-1225.
46. Guo L, Ljung L. Performance analysis of general tracking algorithms. IEEE Trans Autom Control 1995, AC- 40:1388-1402.
47. Guo L, Ljung L, Priouret P. Tracking performance analyses of the forgetting factor RLS algorithm. In Proceedings of the 31st Conference on Decision and Control, Tucson, Arizona, pages 688-693, New York, 1992. IEEE.
48. Ljung L. System Identification Theory for the User. Englewood Cliffs: Prentice-Hall; 1986.
49. Ljung L, Söderström T. Theory and Practice of Recursive Identification. Cambridge: IT Press; 1983.
50. Solo V. The limit behavior of LMS. IEEE Trans Acoust Speech Signal Process 1989, 37:1909-1922.
51. Widrow B, Stearns SD. Adaptive Signal Processing. Englewood Cliffs: Prentice-Hall; 1985.
52. J-M Brossier. Egalization Adaptive et Estimation de Phase: Application aux Communications Sous-Marines. PhD thesis, Institut National Polytechnique de Grenoble, 1992.
53. Kushner HJ, Yang J. Analysis of adaptive step size SA algorithms for parameter tracking. IEEE Trans Autom Control 1995, 40:1403-1410.
54. Buche R, Kushner HJ. Adaptive optimization of least squares tracking algorithms: with applications to adaptive antennas arrays for randomly time-varying mobile communications systems. IEEE Trans Autom Control 2005, 50:1749-1760.
55. Benveniste A, GoursatM, Ruget G. Robust identification of a nonminimum phase system: blind adjustment of a linear equalizer in data communications. IEEE Trans Autom Control 1980, AC-25:385-399.
56. Juditsky A. A stochastic estimation algorithm with observation averaging. IEEE Trans Autom Control 1993, 38:794-798.
57. Polyak BT, Juditsky AB. Acceleration of stochastic approximation by averaging. SIAM J Control Optim 1992, 30:838-855.
58. Chen H-F. Asymptotically efficient stochastic approximation. Stochastics Stochastic Rep 1993, 45:1-16.
59. Delyon B, Juditsky A. Stochastic optimization with averaging of trajectories. Stochastics Stochastic Rep 1992, 39:107-118.
60. Kushner HJ, Yang J. Stochastic approximation with averaging of the iterates: optimal asymptotic rates of convergence for general processes. SIAM J Control Optim 1993, 31:1045-1062.
61. Kushner HJ, Yang J. Stochastic approximation with averaging and feedback: faster convergence. In: Goodwin GC, Aström K, Kumar PR, eds. IMA Volumes in Mathematics and Applications: Adaptive Control, Filtering and Signal Processing, Volume 74, The IMA Series. Berlin and New York: Springer-Verlag; 1995, 205-228.
62. KushnerHJ, Yang J. Stochastic approximation with averaging and feedback: rapidly onvergent "on line" algorithms.IEEE Trans Autom Control 1995, 40:24-34.
63. Le Breton A. Averaging with feedback in Gaussian schemes in stochastic approximation. Math Methods Stat 1997, 6:313-331.
64. Yin G. On extensions of Polyak's averaging approach to stochastic approximation. Stochastics Stochastic Rep 1991, 36:245-264.
65. Yin G. Stochastic approximation via averaging: Polyak's approach revisited. In: Pflug , Dieter U, eds. Lecture Notes in Economics and Mathematical Systems 374. Berlin and New York: Springer-Verlag; 1992, 119-134.
66. Kushner HJ. Rates of convergence for sequential montecarlo optimization methods. SIAM J Control Optim 1978, 16:150-168.
67. Kushner HJ, Huang H. Rates of convergence for stochasticapproximation type algorithms. SIAM J Control Optim 1979, 17:607-617.
68. Buche R, Kushner HJ. Rate of convergence for constrained stochastic approximation algorithms. SIAM J Control Optim 2001, 40:1011-1041.
69. Gerencsér L. Convergence rate of moments in stochastic approximation with simultaneous perturbation gradient approximation and resetting. IEEE Trans Autom Control 1999, 44:894-905.
70. Joslin JA, Heunis AJ. Law of the iterated logarithm for constant-gain linear stochastic gradient algorithm. SIAM J Control Optim 2000, 39:533-570.
71. Pezeshki-Esfanahani H, Heunis AJ. Strong diffusion approximations for recursive stochastic algorithms. IEEE Trans Inform Theory 1997, 43:312-323.
72. Kushner HJ, Vázquez-Abad FJ. Stochastic approximation algorithms for systems over an infinite horizon. SIAM J Control Optim 1996, 34:712-756.
73. Dupuis P, Kushner HJ. Stochastic approximation via large deviations: asymptotic properties. SIAM J Control Optim 1985, 23:675-696.
74. Dupuis P, Kushner HJ. Asymptotic behavior of constrained stochastic approximations via the theory of large deviations. Probab Theory Related Fields 1987, 75:223-244.