The MM alternative to EM

Statistical Science

Volumn 25, Issue 4, 2010, Pages 492-505

  Wu, Tong Tong ^a   Lange, Kenneth ^b  

^a UNIVERSITY OF MARYLAND   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Inequalities; Iterative majorization; Maximum likelihood; Penalization

Indexed keywords

EID: 77956665039     PISSN: 08834237     EISSN: None     Source Type: Journal    
DOI: 10.1214/08-STS264     Document Type: Article

References (63)

1. ANDERSON, G. D., VAMANAMURTHY, M. K. and VUORINEN, M. (2007). Generalized convexity and inequalities. J. Math. Anal. Appl. 335 1294-1308.
2. ASUNCION, A. and NEWMAN, D. J. (2007). UCI Machine Learning Repository. Available at http://www.ics.uci.edu/~mlearn/MLRepository.html.
3. BECKER, M. P., YANG, I. and LANGE, K. (1997). EM algorithms without missing data. Stat. Methods Med. Res. 6 38-54.
4. BERGSTROM, T. C. and BAGNOLI, M. (2005). Log-concave probability and its applications. Econom. Theory 26 445-469.
5. BIJLEVELD, C. C. J. H. and DE LEEUW, J. (1991). Fitting longitudinal reduced-rank regression models by alternating least squares. Psychometrika 56 433-447.
6. BIOUCAS-DIAS, J. M., FIGUEIREDO, M. A. T. and OLIVEIRA, J. P. (2006). Total variation-based image deconvolution: a majorization-minimization approach. In IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2006 Proceedings 861-864.
7. BLITZSTEIN, J., CHATTERJEE, S. and DIACONIS, P. (2008). A new algorithm for high dimensional maximum likelihood estimation. Technical report.
8. BOHNING, D. and LINDSAY, B. G. (1988). Monotonicity of quadratic approximation algorithms. Ann. Inst. Statist. Math. 40 641-663.
9. BORG, I. and GROENEN, P. (1997). Modern Multidimensional Scaling: Theory and Applications. Springer, New York.
10. BOYD, S. and VANDENBERGHE, L. (2004). Convex Optimization. Cambridge Univ. Press.
11. BOYLES, R. A. (1983). On the convergence of the EM algorithm. J. Roy. Statist. Soc. Ser. B 45 47-50.
12. DE LEEUW, J. (1977). Applications of convex analysis to multidimensional scaling. In Recent Developments in Statistics (J. R. Barra, F. Brodeau, G. Romie and B. Van Cutsem, eds.) 133-145. North-Holland, Amsterdam.
13. DE LEEUW, J. (1994). Block relaxation algorithms in statistics. In Information Systems and Data Analysis (H.-H. Bock,W. Lenski and M. M. Richter, eds.). Springer, Berlin.
14. DE LEEUW, J. and HEISER, W. J. (1977). Convergence of correction matrix algorithms for multidimensional scaling. In Geometric Representations of Relational Data (J. C. Lingoes, E. Roskam and I. Borg, eds.). Mathesis Press, Ann Arbor, MI.
15. DE LEEUW, J. and HEISER, W. J. (1980). Multidimensional scaling with restriction on the configuration. In Multivariate Analysis-V: Proceeding of the Fifth International Symposium on Multivariate Analysis (P. R. Krishnaiah ed.) 501-522. North-Holland, Amsterdam.
16. DE LEEUW, J. and LANGE, K. (2009). Sharp quadratic majorization in one dimension. Comput. Statist. Data Anal. 53 2471-2484.
17. DE PIERRO, A. R. (1995). A modified expectation maximization algorithm for penalized likelihood estimation in emission tomography. IEEE Trans. Med. Imaging 14 132-137.
18. DEMPSTER, A. P., LAIRD, N. M. and RUBIN, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. Roy. Statist. Soc. Ser. B 39 1-38.
19. ELDÉN, L. (2007). Matrix Methods in Data Mining and Pattern Recognition. SIAM, Philadelphia.
20. FRIEDMAN, J., HASTIE, T. and TIBSHIRANI, R. (2007). Pathwise coordinate optimization. Ann. Appl. Statist. 1 302-332.
21. GROENEN, P. J. F., NALBANTOV, G. and BIOCH, J. C. (2006). Nonlinear support vector machines through iterative majorization and I-splines. Studies in Classification, Data Analysis and Knowledge Organization (H. J. Lenz and R. Decker, eds.) 149-161. Springer, Berlin.
22. HASTIE, T., TIBSHIRANI, R. and FRIEDMAN, J. (2001). The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York.
23. HEISER, W. J. (1987). Correspondence analysis with least absolute residuals. Comput. Statist. Data Anal. 5 337-356.
24. HEISER, W. J. (1995). Convergent computing by iterative majorization: theory and applications in multidimensional data analysis. In Recent Advances in Descriptive Multivariate Analysis (W. J. Krzanowski, ed.). Clarendon Press, Oxford.
25. HUBER, P. J. (1981). Robust Statistics. Wiley, New York.
26. HUNTER, D. R. (2004). MM algorithms for generalized Bradley-Terry models. Ann. Statist. 32 384-406.
27. HUNTER, D. R. and LANGE, K. (2000). Quantile regression via an MM algorithm. J. Comput. Graph. Statist. 9 60-77.
28. HUNTER, D. R. and LANGE, K. (2002). Computing estimates in the proportional odds model. Ann. Inst. Statist. Math. 54 155-168.
29. HUNTER, D. R. and LANGE, K. (2004). A tutorial on MM algorithms. Amer. Statist. 58 30-37.
30. HUNTER, D. R. and LI, R. (2005). Variable selection using MM algorithms. Ann. Statist. 33 1617-1642.
31. JAMSHIDIAN, M. and JENNRICH, R. I. (1997). Quasi-Newton acceleration of the EM algorithm. J. Roy. Statist. Soc. Ser. B 59 569-587.
32. KENT, J. T., TYLER, D. E. and VARDI, Y. (1994). A curious likelihood identity for the multivariate t -distribution. Comm. Statist. Simulation Comput. 23 441-453.
33. KIERS, H. A. L. (2002). Setting up alternating least squares and iterative majorization algorithms for solving various matrix optimization problems. Comput. Statist. Data Anal. 41 157-170.
34. KIERS, H. A. L. and TEN BERGE, J. M. F. (1992). Minimization of a class of matrix trace functions by means of refined majorization. Psychometrika 57 371-382.
35. LANGE, K. (1994). An adaptive barrier method for convex programming. Methods Appl. Anal. 1 392-402.
36. LANGE, K. (1995a). A gradient algorithm locally equivalent to the EM algorithm. J. Roy. Statist. Soc. Ser. B 57 425-437.
37. LANGE, K. (1995b). A quasi-Newton acceleration of the EM algorithm. Statist. Sinica 5 1-18.
38. LANGE, K. (2004). Optimization. Springer, New York.
39. LANGE, K. and FESSLER, J. A. (1994). Globally convergent algorithms for maximum a posteriori transmission tomography. IEEE Trans. Image Process. 4 1430-1438.
40. LANGE, K., HUNTER, D. R. and YANG, I. (2000). Optimization transfer using surrogate objective functions (with discussion). J. Comput. Graph. Statist. 9 1-20.
41. LANGE, K., LITTLE, R. J. A. and TAYLOR, J. M. G. (1989). Robust statistical modeling using the t distribution. J. Amer. Statist. Assoc. 84 881-896.
42. LANGE, K. andWU, T. T. (2008). An MM algorithm for multicategory vertex discriminant analysis. J. Comput. Graph. Statist. 17 1-18.
43. LEE, D. D. and SEUNG, H. S. (1999). Learning the parts of objects by non-negative matrix factorization. Nature 401 788-791.
44. LEE, D. D. and SEUNG, H. S. (2001). Algorithms for non-negative matrix factorization. Adv. Neural Inform. Process. Syst. 13 556-562.
45. LIAO, W. H., HUANG, S. C., LANGE, K. and BERGSNEIDER, M. (2002). Use of MM algorithm for regularization of parametric images in dynamic PET. In Brain Imaging Using PET (M. Senda, Y. Kimura, P. Herscovitch and Y. Kimura, eds.). Academic Press, New York.
46. LITTLE, R. J. A. and RUBIN, D. B. (2002). Statistical Analysis with Missing Data, 2nd ed. Wiley, New York.
47. MARSHALL, A.W. and OLKIN, I. (1979). Inequalities: Theory of Majorization and Its Applications. Academic Press, San Diego.
48. MCLACHLAN, G. J. and KRISHNAN, T. (1997). The EM Algorithm and Extensions. Wiley, New York.
49. MEILIJSON, I. (1989). A fast improvement to the EMalgorithm on its own terms. J. Roy. Statist. Soc. B 51 127-138.
50. MENG, X. L. and RUBIN, D. B. (1991). Using EM to obtain asymptotic variance-covariance matrices: The SEM algorithm. J. Amer. Statist. Assoc. 86 899-909.
51. MENG, X. L. and VAN DYK, D. (1997). The EM algorithm-an old folk-song sung to a fast new tune. J. Roy. Statist. Soc. Ser. B 59 511-567.
52. ORTEGA, J. M. (1990). Numerical Analysis: A Second Course. SIAM, Philadelphia.
53. ORTEGA, J. M. and RHEINBOLDT, W. C. (1970). Iterative Solutions of Nonlinear Equations in Several Variables. Academic Press, New York.
54. PAUCA, V. P., PIPER, J. and PLEMMONS, R. J. (2006). Nonnegative matrix factorization for spectral data analysis. Linear Algebra Appl. 416 29-47.
55. RAO, C. R. (1973). Linear Statistical Inference and Its Applications, 2nd ed. Wiley, New York.
56. SABATTI, C. and LANGE, K. (2002). Genomewide motif identification using a dictionary model. Proceedings IEEE 90 1803-1810.
57. SCHOLKOPF, B. and SMOLA, A. J. (2002). Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, Cambridge.
58. STEELE, J. M. (2004). The Cauchy-Schwarz Master Class: An Introduction to the Art of Mathematical Inequalities. Cambridge Univ. Press and Math. Assoc. Amer., Washington, DC.
59. TAKANE, Y., YOUNG, F. W. and DE LEEUW, J. (1977). Nonmetric individual differences multidimensional scaling: An alternating least squares method with optimal scaling features. Psychometrika 42 7-67.
60. VAIDA, F. (2005). Parameter convergence for EM and MM algorithms. Statist. Sinica 15 831-840.
61. VAPNIK, V. (1995). The Nature of Statistical Learning Theory. Springer, New York.
62. VARADHAN, R. and ROLAND, C. (2008). Simple and globally convergent methods for accelerating the convergence of any EM algorithm. Scand. J. Statist. 35 335-353.
63. WU, C. F. J. (1983). On the convergence properties of the EM algorithm. Ann. Statist. 11 95-103.