Trajectory averaging for stochastic approximation MCMC algorithms

Annals of Statistics

Volumn 38, Issue 5, 2010, Pages 2823-2856

  Liang, Faming ^a  

^a TEXAS A AND M UNIVERSITY   (United States)

Author keywords

Asymptotic efficiency; Convergence; Markov chain Monte Carlo; Stochastic approximation Monte Carlo; Trajectory averaging

Indexed keywords

EID: 77957597582     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/10-AOS807     Document Type: Article

References (43)

1. ANDRIEU, C. and MOULINES, É. (2006). On the ergodicity properties of some adaptive MCMC algorithms. Ann. Appl. Probab. 16 1462-1505. MR2260070 (Pubitemid 44662199)
2. ANDRIEU, C., MOULINES, É. and PRIOURET, P. (2005). Stability of stochastic approximation under verifiable conditions. SIAM J. Control Optim. 44 283-312. MR2177157
3. ATCHADÉ, Y. F. and LIU, J. S. (2010). TheWang-Landau algorithm in general state spaces: Applications and convergence analysis. Statist. Sinica 20 209-233.
4. BENVENISTE, A., MÉTIVIER, M. and PRIOURET, P. (1990). Adaptive Algorithms and Stochastic Approximations. Springer, New York. MR1082341
5. CHANDRA, T. K. and GOSWAMI, A. (2006). Ces?ro a-integrability and laws of large numbers. II. J. Theoret. Probab. 19 789-816. MR2279604
6. CHEN, H. F. (1993). Asymptotically efficient stochastic approximation. Stochastics Stochastics Rep. 45 1-16. MR1277359
7. CHEN, H. F. (2002). Stochastic Approximation and Its Applications. Kluwer Academic, Dordrecht. MR1942427
8. CHEN, H. F., GUO, L. and GAO, A. (1988). Convergence and robustness of the Robbins-Monro algorithm truncated at randomly varying bounds. Stochastic Process. Appl. 27 217-231. MR0931029
9. CHEN, H. F. and ZHU, Y. M. (1986). Stochastic approximation procedures with randomly varying truncations. Sci. Sinica Ser. A 29 914-926. MR0869196
10. CHEON, S. and LIANG, F. (2007). Phylogenetic tree reconstruction using sequential stochastic approximation Monte Carlo. BioSystems 91 94-107.
11. CHEON, S. and LIANG, F. (2009). Bayesian phylogeny analysis via stochastic approximationMonte Carlo. Mol. Phylog. Evol. 53 394-403.
12. DELYON, B., LAVIELLE, M. andMOULINES, E. (1999). Convergence of a stochastic approximation version of the EM algorithm. Ann. Statist. 27 94-128. MR1701103
13. DIPPON, J. and RENZ, J. (1997). Weighted means in stochastic approximation of minima. SIAM J. Control Optim. 35 1811-1827. MR1466929
14. DUFLO, M. (1997). Random Iterative Models. Springer, Berlin. MR1485774
15. GEYER, C. J. (1991). Markov chain Monte Carlo maximum likelihood. In Computing Science and Statistics: Proceedings of the 23rd Symposium on the Interface (E. M. Keramigas, ed.) 156-163. Interface Foundation, Fairfax, VA.
16. GEYER, C. J. and THOMPSON, E. A. (1995). Annealing Markov chain Monte Carlo with applications to ancestral inference. J. Amer. Statist. Assoc. 90 909-920.
17. GU, M. G. and KONG, F. H. (1998). A stochastic approximation algorithm with Markov chain Monte Carlo method for incomplete data estimation problems. Proc. Natl. Acad. Sci. USA 95 7270-7274. MR1630899
18. GU, M. G. and ZHU, H. T. (2001). Maximum likelihood estimation for spatial models by Markov chain Monte Carlo stochastic approximation. J. R. Stat. Soc. Ser. B Stat. Methodol. 63 339-355. MR1841419
19. HASTINGS, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57 97-109.
20. KUSHNER, H. J. and YANG, J. (1993). Stochastic approximation with averaging of the iterates: Optimal asymptotic rate of convergence for general processes. SIAM J. Control Optim. 31 1045-1062. MR1227546
21. KUSHNER, H. J. and YANG, J. (1995). Stochastic approximation with averaging and feedback: Rapidly convergent "on-line" algorithms. IEEE Trans. Automat. Control 40 24-34. MR1344315
22. KUSHNER, H. J. and YIN, G. G. (2003). Stochastic Approximation and Recursive Algorithms and Applications, 2nd ed. Springer, New York. MR1993642
23. LIANG, F. (2005). Generalized Wang-Landau algorithm for Monte Carlo computation. J. Amer. Statist. Assoc. 100 1311-1327. MR2236444
24. LIANG, F. (2007a). Continuous contour Monte Carlo for marginal density estimation with an application to a spatial statistical model. J. Comp. Graph. Statist. 16 608-632. MR2351082
25. LIANG, F. (2007b). Annealing stochastic approximation Monte Carlo for neural network training. Mach. Learn. 68 201-233.
26. LIANG, F. (2009). Improving SAMC using smoothing methods: Theory and applications to Bayesian model selection problems. Ann. Statist. 37 2626-2654. MR2541441
27. LIANG, F., LIU, C. and CARROLL, R. J. (2007). Stochastic approximation in Monte Carlo computation. J. Amer. Statist. Assoc. 102 305-320. MR2345544
28. LIANG, F. and ZHANG, J. (2009). Learning Bayesian networks for discrete data. Comput. Statist. Data Anal. 53 865-876.
29. MARINARI, E. and PARISI, G. (1992). Simulated tempering: A new Monte Carlo scheme. Europhys. Lett. 19 451-458.
30. METROPOLIS, N., ROSENBLUTH, A.W., ROSENBLUTH, M. N., TELLER, A. H. and TELLER, E. (1953). Equation of state calculations by fast computing machines. J. Chem. Phys. 21 1087-1091.
31. MOYEED, R. A. and BADDELEY, A. J. (1991). Stochastic approximation of the MLE for a spatial point pattern. Scand. J. Statist. 18 39-50. MR1115181
32. PELLETIER, M. (2000). Asymptotic almost sure efficiency of averaged stochastic algorithms. SIAM J. Control Optim. 39 49-72. MR1780908
33. POLYAK, B. T. (1990). New stochastic approximation type procedures. Avtomat. i Telemekh. 7 98-107 (in Russian). MR1071220
34. POLYAK, B. T. and JUDITSKY, A. B. (1992). Acceleration of stochastic approximation by averaging. SIAM J. Control Optim. 30 838-855. MR1167814
35. ROBBINS, H. and MONRO, S. (1951). A stochastic approximation method. Ann. Math. Statist. 22 400-407. MR0042668
36. RUPPERT, D. (1988). Efficient estimators from a slowly convergent Robbins-Monro procedure. Technical Report 781, School of Operations Research and Industrial Engineering, Cornell Univ.
37. TADÍC , V. (1997). Convergence of stochastic approximation under general noise and stability conditions. In: Proceedings of the 36th IEEE Conference on Decision and Control 3 2281-2286. IEEE Systems Society, San Diego, CA.
38. TANG, Q. Y., L'ECUYER, P. and CHEN, H. F. (1999). Asymptotic efficiency of perturbationanalysis-based stochastic approximation with averaging. SIAM J. Control Optim. 37 1822-1847. MR1720140
39. WANG, F. and LANDAU, D. P. (2001). Efficient, multiple-range random walk algorithm to calculate the density of states. Phys. Rev. Lett. 86 2050-2053.
40. WANG, I.-J., CHONG, E. K. P. and KULKARNI, S. R. (1997). Weighted averaging and stochastic approximation. Math. Control Signals Systems 10 41-60. MR1462279
41. YOUNES, L. (1989). Parametric inference for imperfectly observed Gibbsian fields. Probab. Theory Related Fields 82 625-645. MR1002904
42. YOUNES, L. (1999). On the convergence ofMarkovian stochastic algorithms with rapidly decreasing ergodicity rates. Stochastics Stochastics Rep. 65 177-228. MR1687636
43. YU, K. and LIANG, F. (2009). Efficient P -value evaluation for resampling-based tests. Technical report, Dept. Statistics, Texas A&M Univ., College Station, TX.