A stochastic approximation algorithm for maximum-likelihood estimation with incomplete data

Canadian Journal of Statistics

Volumn 26, Issue 4, 1998, Pages 567-582

  Gu, Ming Gao ^a   Li, Shaolin ^b  

^a MCGILL UNIVERSITY   (Canada)

^b Chase Manhattan Corporation   (United States)

Author keywords

EM algorithm; Incomplete data; Maximum likelihood; Missing data; Stochastic approximation

Indexed keywords

EID: 0032350101     PISSN: 03195724     EISSN: None     Source Type: Journal    
DOI: 10.2307/3315718     Document Type: Article

References (39)

1. Blum, J.R. (1954) Multidimensional stochastic approximation methods. Ann. Math. Statist., 25, 382-386.
2. Carroll, R.J., Spiegelman, C.H., Lean, K.K.G., Bailey, K.T., and Abbott, R.D. (1984). On error-in-variables for binary regression models. Biometrika, 71, 19-25.
3. Carroll, R.J., Ruppert, D., and Stefanski, L.A. (1995). Measurement Error in Nonlinear Models. Chapman and Hall. New York.
4. Chung, K.L. (1954). On a stochastic approximation method. Ann. Math. Statist., 25, 463-483.
5. Dempster, A.P., Laird, N.M., and Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm. J. Roy. Statist. Soc. Ser. B. 39, 1-38.
6. Dvoretzky, A. (1977). Asymptotic normality for sums of dependent random vectors. Multivariate Analysis Volume 4. (P.R. Krishnaiah, ed.), North-Holland, Amsterdam, 23-34.
7. Gelfand, A.E., and Carlin, B.P. (1993). Maximum-likelihood estimation for constrained-or missing-data models. Canad. J. Statist., 21, 303-311.
8. Gelfand, A.E., and Smith, A.F.M. (1990). Sampling based approaches to calculating marginal densities. J. Amer. Statist. Assoc., 85, 398-409.
9. Geman, S., and Geman. D. (1984). Stochastic relaxation, Gibbs distributions and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell., 6, 721-741.
10. Geyer, C.J., and Thompson, E.A. (1992). Constrained Monte Carlo maximum likelihood for dependent data (with discussion). J. Roy. Statist. Soc. Ser. B, 54, 657-699.
11. Green, P.J., (1990). On use of EM algorithm for penalized likelihood estimation. J. Roy. Statist. Soc. Ser. B, 52, 443-452.
12. Hastings, W.K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika, 57, 97-109.
13. Huber, P.J. (1973). Robust regression: Asymptotics, conjecture and Monte Carlo. Ann. Statist., 1, 799-821.
14. Lai, T.L. (1985). Stochastic approximation and sequential search for optimum. Proc. Berkeley Conf. in Honor of J. Neyman and J. Kiefer. Volume II. Wadsworth, Hayward, Calif.
15. Lai, T.L., and Ying, Z. (1995). A missing information principle and M-estimators in regression analysis with censored and truncated data. Ann. Statist., 22, 1222-1255.
16. Lange, K. (1995a). A gradient algorithm locally equivalent to the EM algorithm. J. Roy. Statist. Soc. Ser. B, 57, 425-137.
17. Lange, K. (1995b). A quasi-Newton acceleration of the EM algorithm. Statist. Sinica, 5, 1-18.
18. Lindstrom, M.J., and Bates, D.M. (1990). Nonlinear mixed effects models for repeated measures data. Biometrics, 46, 673-687.
19. Little, R.J.A., and Rubin, D.B. (1987). Statistical Analysis with Missing Data. Wiley, New York.
20. Liu, C., and Rubin, D.B. (1994). The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence. Biometrika, 81, 633-648.
21. Louis, T.A. (1982). Finding observed information using the EM algorithm. J. Roy. Statist. Soc. Ser. B, 44, 98-130.
22. Meilijson, I. (1989). A faster improvement to the EM algorithm on its own terms. J. Roy. Statist. Soc. Ser. B, 51, 127-138.
23. Meng, X.L., and Rubin, D.B. (1993). Maximum likelihood estimation via the ECM algorithm: A general framework. Biometrika, 80, 267-278.
24. Metroplis, N., Rosenbluth, A.W., Rosenbluth, M.N., Teller, A.M., and Teller, E. (1953). Equations of state calculations by fast computing machines. J. Chem. Phys., 21, 1087-1091.
25. Nevel'son, M.B., and Has'minski, R.Z. (1973a). Stochastic Approximation and Recursive Estimation. Transl. Math. Monogr. 47. American Mathematical Society. Providence.
26. Nevel'son, M.B., and Has'minski, R.Z. (1973b). An adaptive Robbins-Monro procedure. Automat. Remote Control, 34, 1594-1607.
27. Rao, C.R. (1973). Linear Statistical Inference and Applications. New York: Wiley.
28. Robbins, H., and Monro, S. (1951). A stochastic approximation method. Ann. Math. Statist., 22. 400-407.
29. Sachs, J. (1958). Asymptotic distribution of stochastic approximation procedures. Ann. Math. Statist., 29, 373-405.
30. Schafer, D.W. (1987). Covariate measurement error in generalized linear models. Biometrika, 74, 385-391.
31. Schmee, J., and Hahn, G.L. (1979). A simple method for regression analysis with censored data. Technometrics, 21, 417-432.
32. Stefanski, L.A., and Carroll, R.J. (1987). Conditional scores and optimal scores for generalized linear measurement error models. Biometrika, 74, 703-716.
33. Stiratelli, R., Laird, N., and Ware, J.H. (1984). Random-effects models for serial observation. Biometrics, 40, 961-971.
34. Tanner, M.A. (1993). Tools for Statistical Inference. Second Edition. Springer-Verlag, New York.
35. Thisted, R.A. (1988). Elements of Statistical Computing - Numerical Computation. Chapman and Hall, New York.
36. Titterington, D.M. (1984). Recursive parameter estimation using incomplete data. J. Roy. Statist. Soc. Ser. B, 46, 257-267.
37. Wei, C.Z. (1987). Multivariate adaptive stochastic approximation. Ann. Statist., 15, 1115-1130.
38. Wei, G.C.G., and Tanner, M.A. (1990). A Monte Carlo implementation of the EM algorithm and the poor man's data augmentation algorithm. J. Amer. Statist. Assoc., 85, 699-704.
39. Zeger, S.L., Liang, K.-Y., and Albert, P. (1988). Models for longitudinal data: A generalized estimating equation approach. Biometrics, 44, 1049-1060.