Online expectation maximisation

Mixtures: Estimation and Applications

Volumn , Issue , 2011, Pages 31-53

  Cappé, Olivier ^a  

^a INSTITUT TELECOM   (France)

Author keywords

Application to finite mixtures; EM algorithm and limiting EM recursion; Limitations of batch EM for long data records; Model and assumptions; Motivation behind association of two groups of words 'online (estimation)' and 'expectation maximisation (algorithm)'; Online expectation maximisation; Online expectation maximization the algorithm; Probabilistic principal component analysis (PCA) model of Tipping and Bishop (1999); Recursive, ubiquitous in computer science ambiguous, and not recommended; Traditional applications of online algorithms situations, in which data cannot be stored

Indexed keywords

MAXIMUM PRINCIPLE;

EXPECTATION-MAXIMISATION; FINITE MIXTURES; ON-LINE ALGORITHMS; ONLINE EXPECTATION MAXIMIZATIONS; PROBABILISTIC PRINCIPAL COMPONENT ANALYSIS; RECURSIONS; RECURSIVE, UBIQUITOUS IN COMPUTER SCIENCE - AMBIGUOUS, AND NOT RECOMMENDED;

PRINCIPAL COMPONENT ANALYSIS;

EID: 84908477733     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781119995678.ch2     Document Type: Chapter

References (31)

1. Benveniste, A., Métivier, M. and Priouret, P. (1990) Adaptive Algorithms and Stochastic Approximations, Vol. 22. Springer, Berlin. Translated from the French by Stephen S. S. Wilson.
2. Bickel, P. J. and Doksum, K. A. (2000) Mathematical Statistics. Basic Ideas and Selected Topics, 2nd edn. Prentice-Hall.
3. Cappé, O. (2009a) Online EM algorithm for hidden Markov models. Preprint.
4. Cappé, O. (2009b) Online sequential Monte Carlo EM algorithm. In IEEE Workshop on Statistical Signal Process (SSP), Cardiff, Wales, UK.
5. Cappé, O. and Moulines, E. (2009) On-line expectation-maximization algorithm for latent data models. Journal of the Royal Statistical Society, Series B, 71, 593-613.
6. Cappé, O., Charbit, M. and Moulines, E. (2006) Recursive EM algorithm with applications to DOA estimation. In Proceedings of the IEEE International Conference on Acoustics Speech, and Signal Processing, Toulouse, France.
7. Cappé, O., Douc, R., Guillin, A., Marin, J. M. and Robert, C. P. (2008) Adaptive importance sampling in general mixture classes. Statistics and Computing, 18, 447-459.
8. Césa-Bianchi, N. and Lugosi, G. (2006) Prediction, Learning, and Games. Cambridge University Press, Cambridge.
9. Chopin, N. (2002) A sequential particle filter method for static models. Biometrika, 89, 539-552.
10. Del Moral, P., Doucet, A. and Singh, S. S. (2009) Forward smoothing using sequential Monte Carlo. Technical Report CUED/F-INFENG/TR 638, Cambridge University Engineering Department.
11. Dempster,A. P., Laird, N. M. and Rubin, D. B. (1977) Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 39, 1-38 (with discussion).
12. Kantas, N., Doucet, A., Singh, S. and Maciejowski, J. (2009) An overview of sequential Monte Carlo methods for parameter estimation in general state-space models. In Proceedings of the IFAC Symposium on System Identification (SYSID).
13. Kushner, H. J. and Yin, G. G. (2003) Stochastic Approximation and Recursive Algorithms and Applications, Vol. 35, 2nd edn. Springer, New York.
14. Lai, T. L. (2003) Stochastic approximation. Annals of Statistics, 31, 391-406.
15. Lange, K. (1995) A gradient algorithm locally equivalent to the EM algorithm. Journal of the Royal Statistical Society, Series B, 57, 425-437.
16. Lehmann, E. L. and Casella, G. (1998) Theory of Point Estimation, 2nd edn. Springer, New York.
17. Liang, P. and Klein, D. (2009) Online EM for unsupervised models. In Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pp. 611-619. Association for Computational Linguistics, Boulder, Colorado.
18. Liu, Z., Almhana, J., Choulakian, V. and McGorman, R. (2006) Online EM algorithm for mixture with application to internet traffic modeling. Computational Statistics and Data Analysis, 50, 1052-1071.
19. Ljung, L. and Söderström, T. (1983) Theory and Practice of Recursive Identification. MIT Press, Cambridge, Massachusetts.
20. Meng, X. L. and Van Dyk, D. (1997) The EM algorithm - an old folk song sung to a fast new tune. Journal of the Royal Statistical Society, Series B, 59, 511-567.
21. Mongillo, G. and Denève, S. (2008) Online learning with hidden Markov models. Neural Computation, 20(7), 1706-1716.
22. Neal, R. M. and Hinton, G. E. (1999) A view of the EM algorithm that justifies incremental, sparse, and other variants. In Learning in Graphical Models (ed. M. I. Jordan) pp. 355-368. MIT Press, Cambridge, Massachusetts.
23. Nesterov,Y. (2003) Introductory Lectures on Convex Optimization: A Basic Course. Kluwer.
24. Polyak, B. T. and Juditsky, A. B. (1992) Acceleration of stochastic approximation by averaging. SIAM Journal on Control and Optimization, 30, 838-855.
25. Robert, C. P. (2001) The Bayesian Choice, 2nd edn. Springer, New York.
26. Ruppert, D. (1988) Efficient estimation from a slowly convergent Robbins-Monro process. Technical Report 781, Cornell University, School of Operations Research and Industrial Engineering.
27. Sato, M. (2000) Convergence of on-line EM algorithm. In Prooceedings of the International Conference on Neural Information Processing, vol. 1, pp. 476-481.
28. Sato, M. and Ishii, S. (2000) On-line EM algorithm for the normalized Gaussian network. Neural Computation, 12, 407-432.
29. Slonim, N. and Weiss, Y. (2003) Maximum likelihood and the information bottleneck. In Advances in Neural Information Processing Systems (NIPS), Vol. 15, pp. 335-342.
30. Tipping, M. and Bishop, C. (1999) Probabilistic principal component analysis. Journal of the Royal Statistical Society, Series B, 6, 611-622.
31. Titterington, D. M. (1984) Recursive parameter estimation using incomplete data. Journal of the Royal Statistical Society, Series B, 46, 257-267.