Acceleration of the EM algorithm by using quasi-Newton methods

Journal of the Royal Statistical Society. Series B: Statistical Methodology

Volumn 59, Issue 3, 1997, Pages 569-587

  Jamshidian, Mortaza ^a   Jennrich, Robert I ^b  

^a UNIVERSITY OF CENTRAL FLORIDA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Aitken Acceleration; Conjugate Gradient; Covariance Estimation; Incomplete Data; Multivariate Normal Mixtures; Observed Information Matrix; Poisson Mixtures

Indexed keywords

EID: 0009568899     PISSN: 13697412     EISSN: None     Source Type: Journal    
DOI: 10.1111/1467-9868.00083     Document Type: Article

References (27)

1. Baker, S. G. (1992) A simple method for computing the observed information matrix when using the EM algorithm with categorical data. J. Comput. Graph. Statist., 1, 63-76.
2. Broyden, C. G. (1965) A class of methods for solving nonlinear simultaneous equations. Math. Comput., 19, 577-593.
3. Conn, A. R., Gould, N. I. M. and Toint, P. (1991) Convergence of quasi-Newton matrices generated by the symmetric rank one update. Math. Progrmmng, 50, 177-195.
4. Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977) Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. R. Statist. Soc. B, 39, 1-38.
5. Dennis, J. E. and Schnabel, R. B. (1983) Numerical Methods for Unconstrained Optimization and Nonlinear Equations. Englewood Cliffs: Prentice Hall.
6. Fisher, R. A (1925) Theory of statistical estimation. Proc. Camb. Phil. Soc., 22, 700-725.
7. Fletcher, R. (1987) Practical Methods of Optimization, 2nd edn. New York: Wiley.
8. Hasselblad, V. (1969) Estimation of finite mixtures of distributions from the exponential family. J. Am. Statist. Ass., 64, 1459-1471.
9. Horng, S. C. (1987) Examples of sublinear convergence of the EM algorithm. Proc. Statist. Comput. Sect. Am. Statist. Ass., 266-271.
10. Jamshidian, M. and Jennrich, R. I. (1993) Conjugate gradient acceleration of the EM algorithm. J. Am. Statist. Ass., 88, 221-228.
11. _(1994) Conjugate gradient methods in confirmatory factor analysis. Comput. Statist. Data Anal., 17, 247-263.
12. Khalfan, H. F., Byrd, R. H. and Schnabel, R. B. (1993) A theoretical and experimental study of the symmetric rank-one update. SIAM J. Optimzn, 3, 1-24.
13. Laird, N. M., Lange, N. and Stram, D. O. (1987) Maximum likelihood computations with repeated measures: application of the EM algorithm. J. Am. Statist. Ass., 82, 97-105.
14. Lange, K. (1995a) A quasi Newton acceleration of the EM algorithm. Statist. Sin., 5, 1-18.
15. _(1995b) A gradient algorithm locally equivalent to the EM algorithm. J. R. Statist. Soc. B, 57, 425-437.
16. Lansky, D. and Casella, G. (1990) Improving the EM algorithm. In Computing Science and Statistics: Proc. 22nd Symp. Interface, pp. 420-424. New York: Springer.
17. Lindstram, M. J. and Bates, D. M. (1988) Newton-Raphson and EM algorithms for linear mixed-effects models for repeated-measures data. J. Am. Statist. Ass., 83, 1014-1022.
18. Louis, T. A. (1982) Finding the observed information matrix when using the EM algorithm. J. R. Statist. Soc. B, 44, 226-233.
19. Luenberger, D. G. (1984) Linear and Nonlinear Programming, 2nd edn. Reading: Addison-Wesley.
20. Meilijson, I. (1989) A fast improvement to the EM algorithm on its own terms. J. R. Statist. Soc. B, 51, 127-138.
21. Meng, X. L. and Rubin, D. B. (1991) Using EM to obtain asymptotic variance-covariance matrices: the SEM algorithm. J. Am. Statist. Ass., 86, 899-909.
22. Nazareth, L. (1980) Some recent approaches to solving large residual nonlinear least squares problems. SIAM Rev., 22, 1-11.
23. Redner, R. A. and Walker, H. F. (1984) Mixture densities, maximum likelihood, and the EM algorithm. SIAM Rev., 26, 195-239.
24. Spiegel, J. S., Spiegel, T. M., Ward, N. B., Paulus, H. E., Leake, B. and Kane, R. L. (1986) Rehabilitation for rheumatoid arthritis patients, a controlled trial. Arth. Rheum., 29, 628-637.
25. Thisted, R. A. (1988) Elements of Statistical Computing: Numerical Computation. New York: Chapman and Hall.
26. Titterington, D. M., Smith, A. F. M. and Makov, U. E. (1985) Statistical Analysis of Finite Mixture Distributions. New York: Wiley.
27. Wu, C. F. J. (1983) On the convergence properties of the EM algorithm. Ann. Statist., 11, 95-103.