How does a stochastic optimization/approximation algorithm adapt to a randomly evolving optimum/root with jump Markov sample paths

Mathematical Programming

Volumn 120, Issue 1 SPEC. ISS., 2009, Pages 67-99

  Yin, G ^a   Ion, C ^a   Krishnamurthy, V ^b  

^a WAYNE STATE UNIVERSITY   (United States)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Convergence and rate of convergence; Nonsmooth jump component; Stochastic approximation; Stochastic optimization; Two time scale

Indexed keywords

CONVERGENCE AND RATE OF CONVERGENCE; NONSMOOTH-JUMP COMPONENT; STOCHASTIC APPROXIMATION; STOCHASTIC OPTIMIZATION; TWO-TIME SCALE;

APPROXIMATION THEORY; ASYMPTOTIC ANALYSIS; CONVERGENCE OF NUMERICAL METHODS; DIFFERENTIAL EQUATIONS; MARKOV PROCESSES; OPTIMIZATION; RECURSIVE FUNCTIONS; STOCHASTIC CONTROL SYSTEMS;

APPROXIMATION ALGORITHMS;

EID: 65249174988     PISSN: 00255610     EISSN: 14364646     Source Type: Journal    
DOI: 10.1007/s10107-007-0145-1     Document Type: Article

References (32)

1. Benveniste A., Metivier M. and Priouret P. (1990). Adaptive algorithms and stochastic approximations. Springer, New York
2. Billingsley P. (1968). Convergence of probability measures. Wiley, New York
3. R. Buche H.J. Kushner 2000 Stochastic approximation and user adaptation in a competitive resource sharing system IEEE Trans. Autom. Control 45 844 853
4. Chen H.-F. (2002). Stochastic approximation and its applications. Kluwer Academic, Dordrecht
5. Chin, D.C., Maryak, J.T.: A cautionary note in iterate averaging in stochastic approximation. In: INFORMS annual meeting pp. 5-8 (1996)
6. K.L. Chung 1954 On a stochastic approximation method Ann. Math. Stat. 25 463 483
7. J. Dippon J. Renz 1997 Weighted means in stochastic approximation of minima SIAM J. Control Optim. 35 1811 1827
8. Y. Ephraim N. Merhav 2002 Hidden Markov processes IEEE Trans. Inform. Theory 48 1518 1569
9. S. Haykin 2005 Cognitive radio: brain empowered wireless communications IEEE J. Sel. Areas Commun. 23 201 220
10. V. Krishnamurthy T. Rydén 1998 Consistent estimation of linear and non-linear autoregressive models with Markov regime J. Time Ser. Anal. 19 291 308
11. V. Krishnamurthy G. Yin 2002 Recursive algorithms for estimation of hidden Markov models and autoregressive models with Markov regime IEEE Trans. Inform. Theory 48 2 458 476
12. V. Krishnamurthy G. Yin 2006 Controlled hidden Markov models for dynamically adapting patch clamp experiment to estimate Nernst potential of single-ion channels IEEE Trans. Nanobiosci. 5 115 125
13. V. Krishnamurthy X. Wang G. Yin 2004 Spreading code optimization and adaptation in CDMA via discrete stochastic approximation IEEE Trans. Inform. Theory 50 1927 1949
14. Kushner H.J. (1984). Approximation and weak convergence methods for random processes, with applications to stochastic systems theory. MIT Press, Cambridge
15. H.J. Kushner H. Huang 1981 Asymptotic properties of stochastic approximations with constant coefficients SIAM J. Control Optim. 19 87 105
16. H.J. Kushner J. Yang 1993 Stochastic approximation with averaging of iterates: optimal asymptotic rate of convergence for general processes SIAM J. Control Optim. 31 1045 1062
17. Kushner H.J. and Yin G. (2003). Stochastic approximation and recursive algorithms and applications. 2nd edn. Springer, New York
18. Nevel'son, M.B., Khasminskii, R.Z.: Stochastic approximation and recursive estimation. Translation of Math. Monographs, vol. 47. AMS, Providence (1976)
19. B.T. Polyak 1990 New method of stochastic approximation type Autom. Remote Control 51 937 946
20. B.T. Polyak A.B. Juditsky 1992 Acceleration of stochastic approximation by averaging SIAM J. Control Optim. 30 838 855
21. Ruppert D. (1991). Stochastic approximation. In: Ghosh, B.K. and Sen, P.K. (eds) Handbook in sequential analysis., pp 503-529. Marcel Dekker, New York
22. R. Schwabe 1993 Stability results for smoothed stochastic approximation procedures Z. Angew. Math. Mech. 73 639 644
23. J.C. Spall 1992 Multivariate stochastic approximation using a simultaneous perturbation gradient approximation IEEE Trans. Autom. Control AC-37 332 341
24. Spall J.C. (2003). Introduction to stochastic search and optimization: estimation, simulation and control. Wiley, New York
25. A. Tarighat A.H. Sayed 2004 Least mean-phase adaptive filters with application to communications systems IEEE Signal Process. Lett. 11 220 223
26. G. Yin 1990 A stopping rule for the Robbins-Monro method J. Optim. Theory Appl. 67 151 173
27. G. Yin 1991 On extensions of Polyak's averaging approach to stochastic approximation Stoch. Stoch. Rep. 36 245 264
28. G. Yin 1999 Rates of convergence for a class of global stochastic optimization algorithms SIAM J. Optim. 10 99 120
29. G. Yin V. Krishnamurthy 2005 Analysis of LMS algorithm for slowly varying Markovian parameters-tracking slow hidden Markov models and adaptive multiuser detection in DS/CDMA IEEE Trans. Inform. Theory 51 2475 2490
30. G. Yin V. Krishnamurthy 2005 Least mean square algorithms with Markov regime switching limit IEEE Trans. Autom. Control 50 577 593
31. Yin G. and Zhang Q. (2005). Discrete-time Markov chains: two-time-scale methods and applications. Springer, New York
32. G. Yin V. Krishnamurthy C. Ion 2004 Regime switching stochastic approximation algorithms with application to adaptive discrete stochastic optimization SIAM J. Optim. 14 1187 1215