Maximum likelihood from spatial random effects models via the stochastic approximation expectation maximization algorithm

Statistics and Computing

Volumn 17, Issue 2, 2007, Pages 163-177

  Zhu, Hongtu ^a   Gu, Minggao ^b   Peterson, Bradley ^c  

^a University of North Carolina at Chapel Hill   (United States)

^b NEW YORK STATE PSYCHIATRIC INSTITUTE   (United States)

^c CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Expectation maximization; Markov chain Monte Carlo; Markov random fields; Spatial random effects models; Stochastic approximation

Indexed keywords

EID: 34249667588     PISSN: 09603174     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11222-006-9012-9     Document Type: Article

References (59)

1. Aitkin M. 1996. A general maximum likelihood analysis of overdispersion in generalized linear models. Statistics and Computing 6: 251-262.
2. Benveniste A., Métivier M., and Priouret P. 1990. Adaptive Algorithms and Stochastic Approximations. Springer-Verlag, New York.
3. Besag J.E. 1974. Spatial interaction and the statistical analysis of lattice systems (with discussion). Journal of Royal Statistical Society, Series B 36: 192-236.
4. Besag J.E. 1986. On the statistical analysis of dirty pictures (with discussion). Journal of Royal Statistical Society, Series B 48: 259-302.
5. Besag J.E., Green P., Higdon D., and Mengersen K. 1995. Bayesian computation and stochastic systems (with discussion). Statistical Science 10: 3-66.
6. Breslow N.E. and Clayton D.G. 1993. Approximate inference in generalized linear mixed models. Journal of American Statistical Association 88: 9-25.
7. Chan K.S. and Ledolter J. 1995. Monte Carlo EM estimation for time series models involving counts. Journal of American Statistical Association 90: 242-252.
8. Christensen O.F. and Waagepetersen R.P. 2002. Bayesian prediction of spatial count data using generalized linear mixed models. Biometrics 58: 280-286.
9. Delyon B., Lavielle E., and Moulines E. 1999. Convergence of a stochastic approximation version of the EM algorithm. Annals of Statistics 27: 94-128.
10. Derin H. and Elliott H. 1987. Modeling and segmentation of noisy and textured images using Gibbs random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 9: 39-55.
11. Diggle P.J., Tawn J.A., and Moyeed R.A. 1998. Model-based geostatistics (with discussion). Applied Statistics 47: 299-350.
12. Durbin J. and Koopman S.J. 1997. Monte Carlo maximum likelihood estimation for non-Gaussian state space models. Biometrika 84: 669-684.
13. Durbin J. and Koopman S.J. 2000. Time series analysis of non-Gaussian observations based on state space models from both classical and Bayesian perspectives (with discussion). Journal of Royal Statistical Society, Series B 62: 3-56.
14. Gelman A. and Meng X.L. 1998. Simulating normalizing constants: from importance sampling to bridge sampling to path sampling. Statistical Science 13: 163-185.
15. Geman S. and Geman D. 1984. Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence 6: 721-741.
16. Geyer C.J. and Thompson E.A. 1992. Constrained Monte Carlo maximum likelihood for dependent data (with discussion). Journal of Royal Statistical Society, Series B 54: 657-699.
17. Gu M.G. and Kong F.H. 1998. A stochastic approximation algorithm with Markov chain Monte Carlo method for incomplete data estimation problems. In: Proceeding of National Academic Science of USA 95: 7270-7274.
18. Gu M.G. and Zhu H.T. 2001. Maximum likelihood estimation for spatial models by Markov chain Monte Carlo stochastic approximation. Journal of Royal Statistical Society, Series B 63: 339-355.
19. Higdon D.M. 1998. Auxiliary variable methods for Markov chain Monte Carlo with applications. Journal of American Statistical Association 93: 585-595.
20. Horton N.J. and Laird N.M. 1998. Maximum likelihood analysis of generalized linear models with missing covariates. Statistical Methods in Medical Research 8: 37-50.
21. Huang F. and Ogata Y. 2001. Comparison of two methods for calculating the partition functions of various spatial statistical models. The Australian and New Zealand Journal of Statistics 43: 47-65.
22. Huffer F.W. and Wu H.L. 1998. Markov chain Monte Carlo for autologistic regression models with application to the distribution of plant species. Biometrics 54: 509-524.
23. Jens L.J. and Niels V.P. 1999. Asymptotic normality of the maximum likelihood estimator in state space models. Annals of Statistics 27: 514-535.
24. Karcher P. and Wang Y. 2001. Generalized nonparametric mixed effects models. Journal of Computational and Graphical Statistics 10: 641-655.
25. Kwan R.K.S., Evans A.C., and Pike G.B. 1999. MRI simulation-based evaluation of image-processing and classification methods. IEEE Transactions on Medical Imaging 18: 1085-1097.
26. Lai T.L. 2003. Stochastic approximation. Annals of Statistics 31: 391-406.
27. Lange K. 1995. A gradient algorithm locally equivalent to the EM algorithm. Journal of Royal Statistical Society, Series B 55: 425-437.
28. Lee Y. and Neider J.A. 1996. Hierarchical generalized linear models (with discussion). Journal of Royal Statistical Society, Series B 58: 619-678.
29. Li S.Z. 2001. Markov Random Field Modeling in Image Analysis. Springer-Verlag, Tokyo.
30. Liu J. 2001. Monte Carlo Strategies in Scientific Computing. Springer, New York.
31. Louis T.A. 1982. Finding the observed information matrix when using the EM algorithm. Journal of Royal Statistical Society, Series B 44: 190-200.
32. Marroquin J.L., Santana E.A., and Botello S. 2003. Hidden Markov measure field models for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 25: 1380-1397.
33. McCullagh P. and Nelder J.A. 1989. Generalized Linear Models (2nd edn.). Chapman and Hall, London.
34. Møller J. 1999. Markov chain Monte Carlo and spatial point processes. In W.S. Kendall, O.E. Barndorff-Nielsen, and M.C. van Lieshout (Eds.), Stochastic Geometry: Likelihood and Computation, Chapman and Hall, London.
35. Moyeed R.A. and Baddeley A.J. 1991. Stochastic approximation of the maximum likelihood estimate for a spatial point pattern. Scandinavian Journal of Statistics 18: 39-50.
36. Ortega J.M. 1990. Numerical Analysis: A Second Course. Society for Industrial and Academic Press, Philadelphia.
37. Penttinen A. 1984. Modelling interaction in spatial point patterns: parameter estimation by the maximum likelihood method. Jy. Stud. Comput. Sci. Econometr. Statist. 7.
38. Pettitt A.N., Friel N., and Reeves R. 2003. Efficient calculation of the normalisation constant of the autologistic model on the lattice. Journal of Royal Statistical Society, Series B 65: 235-247.
39. Polyak B.T. 1990. New stochastic approximation type procedures. Autom. Telem. pp. 98-107. (English translation in Automat. Remote Contr. 51).
40. Polyak B.T. and Juditski A.B. 1992. Acceleration of stochastic approximation by averaging. SIAM Journal of Control and Optimization 30: 838-855.
41. Qian W. and Titterington D.M. 1991. Estimation of parameters in hidden Markov models. Philosophical Transactions of the Royal Society of London, Series A 337: 407-428.
42. Rajapakse J.C., Giedd J.N., and Rapoport J.L. 1997. Statistical approach to segmentation of single-channel cerebral MR images. IEEE Transactions on Medical Imaging 16: 176-186.
43. Robbins H. and Monro S. 1951. A stochastic approximation method. Annals of Mathematical Statistics 22: 400-407.
44. Robert C.P. and Casella G. 1999. Monte Carlo Statistical Methods. Springer-Verlag, New York.
45. Rydén T. 1997. On recursive estimation for hidden Markov models. Stochastic Processes and their Applications 66: 79-96.
46. Saquib S.S., Bouman C.A., and Sauer K. 1998. ML parameter estimation for Markov random fields with applications to Bayesian tomography. Transactions on Image Processing 7: 1029-1044.
47. Stoer J. and Bulisch R. 1980. Introduction to Numerical Analysis. Springer-Verlag, New York.
48. Swendsen R.H. and Wang J.S. 1987. Nonuniversal critical dynamics in Monte Carlo simulation. Physics Review Letters 58: 86-88.
49. Wang N., Lin X., Gutierrez R.G., and Carroll R.J. 1998. Bias analysis and SIMEX approach in generalized linear mixed measurement error models. Journal of American Statistical Association 93: 249-261.
50. Wei G.C.G. and Tanner M.A. 1990. A Monte Carlo implementation of the EM algorithm and the Poor man's data augmentation algorithm. Journal of American Statistical Association 85: 699-704.
51. Winkler G. 1995. Image Analysis, Random Fields and Dynamic Monte Carlo Methods: A Mathematical Introduction. Springer-Verlag, Berlin Heidelberg.
52. Younes L. 1989. Parameter estimation for imperfectly observed Gibbsian fields. Probability Theory Related Feilds 82: 625-645.
53. Zeger S.L. 1988. A regression model for time series of counts. Biometrika 75: 621-629.
54. Zeger S.L., Liang K.Y., and Albert P.S. 1988. Models for longitudinal data: a generalized estimating equation approach. Biometrics 44: 1049-1060.
55. Zhang H. 2002. On estimation and prediction for spatial generalized linear mixed models. Biometrics 56: 129-136.
56. Zhang Y., Brady M., and Smith S. 2001. Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm. IEEE Transactions on Medical Imaging 15: 45-57.
57. Zhu H.T. and Gu M.G. 2005. Maximum likelihood from spatial random effects models via the stochastic approximation expectation maximization algorithm (supplement). Technical report, Department of Biostatistics, University of North Carolina at Chapel Hill.
58. Zhu H.T. and Lee S.Y. 2002. Analysis of generalized linear mixed models via a stochastic approximation algorithm with Markov chain Monte Carlo method. Statistics and Computing 12: 175-183.
59. Zhu H.T. and Zhang H.P. 2004. Hypothesis testing in a class of mixture regression models. Journal of Royal Statistical Society, Series B: 66: 3-16.