Optimization with EM and Expectation-Conjugate-Gradient

Proceedings, Twentieth International Conference on Machine Learning

Volumn 2, Issue , 2003, Pages 672-679

  Salakhutdinov, Ruslan ^a   Roweis, Sam ^a   Ghahramani, Zoubin ^b  

^a UNIVERSITY OF TORONTO   (Canada)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; CONVERGENCE OF NUMERICAL METHODS; FUNCTIONS; MATHEMATICAL MODELS; MAXIMUM LIKELIHOOD ESTIMATION; OPTIMIZATION; PATTERN RECOGNITION; PROBABILISTIC LOGICS;

EXPECTATION-CONJUGATE-GRADIENT (ECG) ALGORITHM; EXPECTATION-MAXIMIZATION (EM) ALGORITHM;

LEARNING SYSTEMS;

EID: 1942420675     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (19)

1. S.E. Atkinson. The performance of standard and hybrid EM algorithms for ML estimates of the normal mixture model with censoring. J. of Stat. Computation and Simulation, 44, 1992.
2. Stephen Della Pietra, Vincent J. Della Pietra, and John D. Lafferty. Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(4):380-393, 1997.
3. A. P. Dempster, N. M. Laird, and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. J. of the RS Society series B, 39:1-38, 1977.
4. Chris Fraley. On computing the largest fraction of missing information for the EM algorithm and the worst linear function for data augmentation. Technical report, University of Washington
5. GENIE gene data set. LBNL and UC Santa Cruz, http://www.fruitfly.org/ sequence.
6. Mortaza Jamshidian and Robert I. Jennrich. Conjugate gradient acceleration of the EM algorithm. Journal of the American Statistical Association, 88(421):221-228, March 1993.
7. Mortaza Jamshidian and Robert I. Jennrich. Acceleration of the EM algorithm by using quasi-newton methods. J. of the RS Society series B, 49, 1997.
8. Meng X. L. and van Dyk D. Fast EM-type implementations for mixed effects models. J. of the Royal Statistical Society series B, 60:559-578, 1998.
9. G. J. McLachlan and T. Krishnan. The EM Algorithm and Extensions. Wiley, 1997.
10. R. M. Neal and G. E. Hinton. A view of the EM algorithm that justifies incremental, sparse and other variants. In M. I. Jordan, editor, Learning in Graphical Models, pages 355-368. Kluwer Academic Press, 1998.
11. Richard A. Redner and Homer F. Walker. Mixture densities, maximum likelihood and the EM algorithm. SIAM Review, 26(2):195-239, April 1984.
12. S. T. Roweis. EM algorthms for PCA and SPCA. In Advances in neural information processing systems, volume 10, pages 626-632, Cambridge, MA, 1998.
13. Ruslan Salakhutdinov, Sam Roweis, and Zoubin Ghahramani. Relationship between gradient and EM steps in latent variable models. http://www.cs.toronto. edu/~rsalakhu/ecg.
14. Lawrence Saul and Fernando Pereira. Aggregate and mixed-order Markov models for statistical language processing. In Proceedings of the Second Conference on Empirical Methods in Natural Language Processing, pages 81-89. 1997.
15. M. E. Tipping and C. M. Bishop. Mixtures of probabilistic principal component analysers. Neural Computation, 11(2):443-482, 1999.
16. Naonori Ueda, Ryohei Nakano, Zoubin Ghabramani, and Geoffrey E. Hinton. SMEM algorithm for mixture models. Neural Computation, 12(9):2109-2128, 2000.
17. J. H. van Hateren and A. van der Schaaf. Independent component filters of natural images compared with simple cells in primary visual cortex. In Proceedings of the Royal Society of London, pages 359-366, 1998.
18. L. Xu and M. I. Jordan. On convergence properties of the EM algorithm for Gaussian mixtures. Neural Computation, 8(1):129-151, 1996.
19. Alan Yuille and Anand Rangarajan. The convex-concave computational procedure (CCCP). In Advances in NIPS, volume 13, 2001.