Asymptotic convergence properties of the EM algorithm with respect to the overlap in the mixture

Neurocomputing

Volumn 68, Issue 1-4, 2005, Pages 105-129

  Ma, Jinwen ^a,b   Xu, Lei ^a  

^a CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^b PEKING UNIVERSITY   (China)

Author keywords

Asymptotic convergence rate; EM algorithm; Gaussian mixtures; Maximum likelihood; Mixture of densities from exponential families

Indexed keywords

ASYMPTOTIC STABILITY; CONVERGENCE OF NUMERICAL METHODS; GRADIENT METHODS; MATHEMATICAL MODELS; MAXIMUM LIKELIHOOD ESTIMATION; PROBABILITY DISTRIBUTIONS;

ASYMPTOTIC CONVERGENCE RATE; EXPECTATION-MAXIMIZATION (EM) ALGORITHM; EXPONENTIAL FAMILIES; GAUSSIAN MIXTURES;

ALGORITHMS;

ALGORITHM; ARTICLE; DENSITY; LINEAR SYSTEM; MATHEMATICAL ANALYSIS; MATHEMATICAL MODEL; MAXIMUM LIKELIHOOD METHOD; NORMAL DISTRIBUTION; PRIORITY JOURNAL; SAMPLING; THEORETICAL STUDY;

EID: 24144442632     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2004.12.009     Document Type: Article

References (20)

1. O.E. Barndorf-Nielsen Information and Exponential Families in Statistical Theory 1978 Wiley New York
2. B. Delyon M. Lavielle E. Moulines Convergence of a stochastic approximation version of the EM algorithm Ann. Statist. 27 1999 94-128
3. A.P. Dempster N.M. Laird D.B. Rubin Maximum likelihood from incomplete data via the EM algorithm J. R. Statist. Soc. B 39 1977 1-38
4. S.C. Horng Examples of sublinear convergence of the EM algorithm in: Proceeding of the Statistical Computing Section 1987 American Statistical Association 266-271
5. M. Jamshidian R.I. Jennrich Conjugate gradient acceleration of the EM algorithm J. Am. Statist. Assoc. 88 1993 221-228
6. M. Jamshidian R.I. Jennrich Acceleration of the EM algorithm by using Quasi-Newton methods J. R. Statist. Soc. B 59 1997 569-587
7. M.I. Jordan R.A. Jacobs Hierarchical mixtures of experts and the EM algorithm Neural Comput. 6 1994 181-214
8. M.I. Jordan L. Xu Convergence results for the EP approach to mixtures of experts architectures Neural Networks 8 1995 1409-1431
9. N. Laird N. Lange D. Stram Maximizing likelihood computations with repeated measures: Application of the EM algorithm J. Am. Statist. Assoc. 82 1987 97-105
10. K. Lange A quasi-Newtonian acceleration of the EM algorithm Statist. Sin. 5 1995 1-18
11. E.L. Lehmann Theory of Point Estimation 1985 Wiley New York
12. C. Liu D.B. Rubin The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence Biometrika 81 1994 633-648
13. C. Liu D.B. Rubin Y. Wu Parameter expansion to accelerate EM: The PX-EM algorithm Biometrika 85 1998 755-770
14. J. Ma L. Xu M.I. Jordan Asymptotic convergence rate of the EM algorithm for Gaussian mixtures Neural Comput. 12 2000 2881-2907
15. X.L. Meng On the rate of convergence of the ECM algorithm Ann. Statist. 22 1994 326-339
16. X.L. Meng D. van Dyk The EM algorithm - an old folk-song sung to a fast new tune J. R. Statist. Soc. B 59 1997 511-567
17. L.R. Rabiner A tutorial on Hidden Markov models and selected applications in speech recognition Proc. IEEE 77 1989 257-286
18. R.A. Redner H.F. Walker Mixture densities, maximum likelihood, and the EM algorithm SIAM R. 26 1984 195-239
19. C.F.J. Wu On the convergence properties of the EM algorithm Ann. Statist. 11 1983 95-103
20. L. Xu M.I. Jordan On convergence properties of the EM algorithm for Gaussian mixtures Neural Comput. 8 1996 129-151