Implementing and diagnosing the stochastic approximation EM algorithm

Journal of Computational and Graphical Statistics

Volumn 15, Issue 4, 2006, Pages 803-829

  Jank, Wolfgang ^a  

^a UNIVERSITY OF MARYLAND   (United States)

Author keywords

Convergence diagnostics; Geostatistical model; Likelihood ascent; Monte Carlo EM; Online retailing; Rate of convergence; Spatial data; Step size

Indexed keywords

EID: 33846182940     PISSN: 10618600     EISSN: None     Source Type: Journal    
DOI: 10.1198/106186006X157469     Document Type: Article

References (28)

1. Booth, J. G., and Hobert, J. P. (1999), "Maximizing Generalized Linear Mixed Model Likelihoods With an Automated Monte Carlo EM Algorithm," Journal of the Royal Statistical Society, Series B, 61, 265-285.
2. Booth, J. G., Robert, J. P., and Jank, W. S. (2001), "A Survey of Monte Carlo Algorithms for Maximizing the Likelihood Of A Two-Stage Hierarchical Model," Statistical Modelling, 1, 333-349.
3. Boyles, R. A. (1983), "On the Convergence of the EM Algorithm," Journal of the Royal Statistical Society, Series B, 45, 47-50.
4. Caffo, B. S., Jank, W. S., and Jones, G. L. (2005), "Ascent-Based Monte Carlo EM," Journal of the Royal Statistical Society, Series B, 67, 235-252.
5. Delyon, B., Lavielle, M., and Moulines, E. (1999), "Convergence of a Stochastic Approximation Version of the EM Algorithm," The Annals of Statistics, 27, 94-128.
6. Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977), "Maximum Likelihood From Incomplete Data Via the EM Algorithm," Journal of the Royal Statistical Society, Series B, 39, 1-22.
7. Diggle, P. J., Tawn, J. A., and Moyeed, R. A. (1998), "Model-Based Geostatistics," Journal of the Royal Statistical Society, Series A, 47, 299-350.
8. Gu, M. G., and Zhu, H.-T. (2001), "Maximum Likelihood Estimation for Spatial Models by Markov Chain Monte Carlo Stochastic Approximation," Journal of the Royal Statistical Society, Series B, 63, 339-355.
9. Jank, W. S. (2004), "Quasi-Monte Carlo Sampling to Improve the Efficiency of Monte Carlo EM," Computational Statistics and Data Analysis, 48, 685-701.
10. Jank, W. S., and Booth, J. G. (2003), "Efficiency of Monte Carlo EM and Simulated Maximum Likelihood in Two-Stage Hierarchical Models," Journal of Computational and Graphical Statistics, 12, 214-229.
11. Resten, H. (1958), "Accelerated Stochastic Approximation," The Annals of Mathematical Statistics, 29, 41-59.
12. Levine, R., and Fan, J. (2003), "An Automated (Markov Chain) Monte Carlo EM Algorithm," Journal of Statistical Computation and Simulation, forthcoming»update?«.
13. Levine, R. A., and Casella, G. (2001), ''Implementations of the Monte Carlo EM Algorithm," Journal of Computational and Graphical Statistics, 10, 422-439.
14. Louis, T. A. (1982), "Finding the Observed Information Matrix When Using the EM Algorithm," Journal of the Royal Statistical Society, Series B, 44, 226-233.
15. Mallat, S. G. (1989), "A Theory for Multiresolution Signal Decomposition: The Wavelet Representation," IEEE Transactions on Pattern Analysis and Machine Intelligence, 11, 674-693.
16. McCulloch, C. E. (1997), "Maximum Likelihood Algorithms for Generalized Linear Mixed Models," Journal of the American Statistical Association, 92, 162-170.
17. McLachlan, G. J., and Peel, D. (2001), Finite Mixture Models, New York: Wiley.
18. Meng, X.-L. (1994), "On The Rate of Convergence of the ECM Algorithm," The Annals of Statistics, 22, 326-339.
19. Meng, X.-L., and Rubin, D. B. (1991), "Using EM to Obtain Asymptotic Variance-Covariance Matrices: The SEM Algorithm," Journal of the American Statistical Association, 86, 899-909.
20. _ (1993), "Maximum Likelihood Estimation via the ECM Algorithm: A General Framework," Biometrika, 80, 267-278.
21. _ (1994), "On the Global and Componentwise Rates of Convergence of the EM Algorithm," Linear Algebra and its Applications, 199, 413-425.
22. Orchard, T., and Woodhury, M. A. (1972), "A Missing Information Principle: Theory and Applications," in Proceedings of the Sixth Rerkeley Symposium on Mathematical Statistics and Probability, Vol. I: Theory of Statistics, Berkeley, CA: Univ, California Press, pp. 697-715.
23. Pena, J. M., Lozano, J. A., and Larranaga, P. (2002), "Learning Recursive Bayesian Multinets for Data Clustering by Means of Constructive Induction," Machine Learning, 47, 63-89.
24. Polyak, B. T., and Juditsky, A. B. (1992), "Acceleration of Stochastic Approximation by Averaging," SIAM Journal of Control and Optimization, 30, 838-855.
25. Robbins, H., and Monro, S. (1951), "A Stochastic Approximation Method," The Annals of Mathematical Statistics, 22, 400-407.
26. Wei, G. C. G., and Tanner, M. A. (1990), "A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms," Journal of the American Statistical Association, 85, 699-704.
27. Wu, C. F. J. (1983), "On the Convergence Properties of the EM Algorithm," The Annals of Statistics, 11, 95-103.
28. Zhang, H. (2002), "On Estimation and Prediction for Spatial Generalized Linear Mixed Models," Biometrics, 58, 129-136.