Stochastic approximation and Newton's estimate of a mixing distribution

Statistical Science

Volumn 23, Issue 3, 2008, Pages 365-382

  Martin, Ryan ^a   Ghosh, Jayanta K ^a,b  

^a PURDUE UNIVERSITY   (United States)

^b INDIAN STATISTICAL INSTITUTE   (India)

Author keywords

Empirical bayes; Lyapunov functions; Mixture models; Stochastic approximation

Indexed keywords

EID: 59349089701     PISSN: 08834237     EISSN: None     Source Type: Journal    
DOI: 10.1214/08-STS265     Document Type: Article

References (39)

1. ALLISON, D., GADBURY, G., HEO, M., FERNÁNDEZ, J., LEE, C., PROLLA, T. and WEINDRUCH, R. (2002). A mixture model approach for the analysis of microarray gene expression data. Comput. Statist. Data Anal. 39 1-20. MR1895555
2. ANDRIEU, C., MOULINES, E. and PRIOURET, P. (2005). Stability of stochastic approximation under verifiable conditions. SIAM J. Control Optim. 44 283-312. MR2177157
3. BARRON, A., SCHERVISH, M. and WASSERMAN, L. (1999). The consistency of posterior distributions in nonparametric problems. Ann. Statist. 27 536-561. MR1714718
4. CIPRA, B. (1987). Introduction to the Ising model. Amer. Math. Monthly 94 937-959. MR0936054
5. CSISZÁR, I. (1975). I-divergence geometry of probability distributions and minimization problems. Ann. Probab. 3 146-158. MR0365798
6. DELYON, B., LAVIELLE, M. and MOULINES, E. (1999). Convergence of a stochastic approximation version of the EM algorithm. Ann. Statist. 27 98-128. MR1701103
7. DEMPSTER, A., LAIRD, N. and RUBIN, D. (1977). Maximum-likelihood from incomplete data via the EM algorithm. J. Roy. Statist. Soc. Ser. B 39 1-38. MR0501537
8. EFRON, B. and TIBSHIRANI, R. (2002). Empirical Bayes methods and false discovery rates for microarrays. Genet. Epidemiol. 23 70-86.
9. EFRON, B., TIBSHIRANI, R., STOREY, J. and TUSHER, V. (2001). Empirical Bayes analysis of a microarray experiment. J. Amer. Statist. Assoc. 96 1151-1160. MR1946571
10. FAN, J. (1991). On the optimal rates of convergence for nonparametric deconvolution problems. Ann. Statist. 19 1257-1272. MR1126324
11. GHOSAL, S., GHOSH, J. K. and RAMAMOORTHI, R. V. (1999). Posterior consistency of Dirichlet mixtures in density estimation. Ann. Statist. 27 143-158. MR1701105
12. GHOSAL, S. and VAN DER VAART, A. (2001). Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Ann. Statist. 29 1233-1263. MR1873329
13. GHOSH, J. K. and RAMAMOORTHI, R. V. (2003). Bayesian Nonparametrics. Springer, New York. MR1992245
14. GHOSH, J. and TOKDAR, S. (2006). Convergence and consistency of Newton's algorithm for estimating a mixing distribution. In Frontiers in Statistics 429-443. Imp. Coll. Press, London. MR2326012
15. GILKS, W., ROBERTS, G. and SAHU, S. (1998). Adaptive Markov chain Monte Carlo through, regeneration. J. Amer. Statist. Assoc. 93 1045-1054. MR1649199
16. HAARIO, H., SAKSMAN, E. and TAMMINEN, J. (2001). An adaptive Metropolis algorithm. Bernoulli 7 223-242. MR1828504
17. ISAACSON, D. L. and MADSEN, R. W. (1976). Markov Chains: Theory and Applications. Wiley, New York. MR0407991
18. KOU, S. C., ZHOU, Q. and WONG, W. H. (2006). Equi-energy sampler with applications in statistical inference and statistical mechanics. Ann. Statist. 34 1581-1619. MR2283711
19. KUSHNER, H. J. and YIN, G. G. (2003). Stochastic Approximation and Recursive Algorithms and Applications, 2nd ed. Springer, New York. MR1993642
20. LASALLE, J. and LEFSCHETZ, S. (1961). Stability by Liapunov's Direct Method with Applications. Academic Press, New York. MR0132876
21. LIANG, F., LIU, C. and CARROLL, R. J. (2007). Stochastic approximation in Monte Carlo computation. J. Amer. Statist. Assoc. 102 305-320. MR2345544
22. LINDSAY, B. (1995). Mixture Models: Theory, Geometry and Applications. IMS, Haywood, CA.
23. LIU, J. S. (.1996). Nonparametric hierarchical Bayes via sequential imputations. Ann. Statist. 24 911-930. MR1401830
24. MCLACHLAN, G., BEAN, R. and PEEL, D. (2002). A mixture model-based approach to the clustering of microarray expression data. Bioinformatics 18 413-422.
25. NEVEL'SON, M. B. and HAS'MINSKII, R. Z. (1973). Stochastic Approximation and Recursive Estimation. Amer. Math. Soc., Providence, RI. Translations of Mathematical Monographs, Vol. 47. MR0423714
26. NEWTON, M. A. (2002). A nonparametric recursive estimator of the mixing distribution. Sankhyā Ser. A 64 306-322. MR.1981761
27. NEWTON, M. A., QUINTANA, F. A. and ZHANG, Y. (1998). Nonparametric Bayes methods using predictive updating. In Practical Nonparametric and Semiparametric Bayesian Statistics (D. Dey, P. Muller and D. Sinha, eds.) 45-61. Springer, New York. MR1630075
28. NEWTON, M. A. and ZHANG, Y. (1999). A recursive algorithm for nonparametric analysis with missing data. Biometrika 86 15-26. MR1688068
29. QUINTANA, F. A. and NEWTON, M. A. (2000). Computational aspects of nonparametric Bayesian analysis with applications to the modeling of multiple binary sequences. J. Comput. Graph. Statist. 9 711-737. MR1821814
30. ROBBINS, H. (1964). The empirical Bayes approach to statistical decision problems. Ann. Math. Statist. 35 1-20. MR0163407
31. ROBBINS, H. and MONRO, S. (1951). A stochastic approximation method. Ann. Math. Statist. 22 400-407. MR0042668
32. ROBERT, C. P. and CASELLA, G. (2004). Monte Carlo Statistical Methods, 2nd ed. Springer, New York. MR2080278
33. SAN MARTIN, E. and QUINTANA, F. (2002). Consistency and identifiability revisited. Braz. J. Probab. Stat. 16 99-106. MR1962497
34. SCOTT, J. G. and BERGER, J. O. (2006). An exploration of aspects of Bayesian multiple testing. J. Statist. Plann. Inference 136 2144-2162. MR2235051
35. 0 projections and its applications in statistics. Technical Report 96-24, Dept. Statistics, Purdue Univ., West Lafayette, IN.
36. TANG, Y., GHOSAL, S. and ROY, A. (2007). Nonparametric Bayesian estimation of positive false discovery rates. Biometrics 63 1126-1134.
37. TEICHER, H. (1963). Identifiability of finite mixtures. Ann. Math. Statist. 34 1265-1269. MR0155376
38. TOKDAR, S. T., MARTIN, R. and GHOSH, J. K. (2008). Consistency of a recursive estimate of mixing distributions. Ann. Statist. To appear.
39. WEI, G. C. G. and TANNER, M. A. (1990). A Monte Carlo implementation of the EM algorithm and the Poor Man's data augmentation algorithm. J. Amer. Statist. Assoc. 85 699-704.