A Tutorial on MM Algorithms

American Statistician

Volumn 58, Issue 1, 2004, Pages 30-37

  Hunter, David R ^a   Lange, Kenneth ^b  

^a PENNSYLVANIA STATE UNIVERSITY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Constrained optimization; EM algorithm; Majorization; Minorization; Newton Raphson

Indexed keywords

EID: 1342332031     PISSN: 00031305     EISSN: 15372731     Source Type: Journal    
DOI: 10.1198/0003130042836     Document Type: Article

References (39)

1. Becker, M. P., Yang, I., and Lange, K. (1997),“EM AlgorithmsWithoutMissing Data,” Statistical Methods in Medical Research, 6, 38-54.
2. Bijleveld, C. C. J. H., and de Leeuw, J. (1991), “Fitting Longitudinal Reduced-Rank Regression Models by Alternating Least Squares,” Psychometrika, 56, 433-447.
3. Bohning, D., and Lindsay, B. G. (1988), “Monotonicity of Quadratic Approximation Algorithms,” Annals of the Institute of Statistical Mathematics, 40, 641-663.
4. Censor, Y., and Zenios, S. A. (1992), “Proximal Minimization With D-Functions,” Journal of Optimization Theory and Applications, 73, 451-464.
5. de Leeuw, J. (1994), “Block Relaxation Algorithms in Statistics, in Information Systems and Data Analysis, eds. H. H. Bock, W. Lenski, and M. M. Richter, Berlin: Springer-Verlag, pp. 308-325.
6. de Leeuw, J., and Heiser, W. J. (1977), “Convergence of Correction Matrix Algorithms for Multidimensional Scaling,” in Geometric Representations of Relational Data, eds. J. C. Lingoes, E. Roskam, and I. Borg, Ann Arbor, MI: Mathesis Press, pp. 735-752.
7. Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977), “MaximumLikelihood from Incomplete Data viathe EM Algorithm,” Journal of the Royal Statistical Society, Ser. B, 39, 1-38.
8. De Pierro, A. R. (1995), “A Modified Expectation Maximization Algorithmfor Penalized Likelihood Estimation in Emission Tomography,” IEEE Transac-tionson Medical Imaging, 14, 132-137.
9. Groenen, P. J. F. (1993), The Majorization Approachto Multidimensional Scaling: Some Problems and Extensions, Leiden, the Netherlands: DSWO Press.
10. Heiser, W. J. (1987),“Correspondence Analysis with Least Absolute Residuals,” Computational Statistical Data Analysis, 5, 337-356.
11. Heiser, W. J. (1995), “Convergent Computing by Iterative Majorization: Theory and Applications in Multidimensional Data Analysis,” in Recent Advances in Descriptive Multivariate Analysis, ed. W. J. Krzanowski, Oxford: Clarendon Press, pp. 157-189.
12. Hosmer, D. W., and Lemeshow, S. (1989), Applied Logistic Regression, New York: Wiley.
13. Huber, P. J. (1981), Robust Statistics, New York: Wiley.
14. Hunter, D. R. (2004), “MM Algorithms for Generalized Bradley-Terry Models,” The Annals of Statistics, 32, 386-408.
15. Hunter, D. R., and Lange, K. (2000a), Rejoinderto discussion of “Optimization Transfer Using Surrogate Objective Functions,” Journal of Computational and Graphical Statistics, 9, 52-59.
16. Hunter, D. R., and Lange, K. (2000b), “Quantile Regression via an MM Algorithm,” Journal of Com-putational and Graphical Statistics, 9, 60-77.
17. Hunter, D. R., and Lange, K. (2002), “Computing Estimates in the Proportional Odds Model,” Annals of the Institute of Statistical Mathematics, 54, 155-168.
18. Hunter, D. R., and Li, R. (2002), “A Connection Between Variable Selection and EM-type Algorithms,”technicalreport0201, Dept. of Statistics, Pennsylvania State University.
19. Jamshidian, M., and Jennrich, R. I. (1997), “Quasi-Newton Acceleration of the EM Algorithm,” Journal of the Royal Statistical Society, Ser. B, 59,569-587.
20. Kiers, H. A. L., and Ten Berge, J. M. F. (1992), “Minimization of a Class of Matrix Trace Functions by Means of Refined Majorization,” Psychometrika, 57, 371-382.
21. Koenker, R., and Bassett, G. (1978), “Regression Quantiles,” Econometrica, 46, 33-50.
22. Lange, K. (1994), “An Adaptive Barrier Method for Convex Programming,” Methods Applications Analysis, 1, 392-402.
23. Lange, K. (1995a), “A Gradient Algorithm Locally Equivalent to the EM Algorithm,” Journal of the Royal Statistical Society, Ser. B, 57, 425-437.
24. Lange, K. (1995b), “A Quasi-Newton Acceleration of the EM Algorithm,” Statistica Sinica, 5, 1-18.
25. Lange, K. (1999), Numerical Analysis for Statisticians, New York: SpringerVerlag.
26. Lange, K., and Fessler, J. A. (1995), “Globally ConvergentAlgorithmsforMax-imum A Posteriori Transmission Tomography,” IEEE Transactionson Image Processing, 4, 1430-1438.
27. Lange, K., Hunter, D. R., and Yang, I. (2000), “Optimization Transfer using Surrogate Objective Functions” (with discussion), Journal of Computational and Graphical Statistics, 9, 1-20.
28. Lange, K., and Sinsheimer, J. S. (1993), “Normal/Independent Distributions and Their Applications in Robust Regression,” Journal of Computational and Graphical Statistics, 2, 175-198.
29. Luenberger, D. G. (1984), Linear and Nonlinear Programming (2nd ed.), Reading, MA: Addison-Wesley.
30. Maher, M. J. (1982), “Modelling Association Football Scores,” Statistica Neer-landica, 36, 109-118.
31. Marshall, A. W., and Olkin, I. (1979), Inequalities: Theory of Majorization and its Applications, San Diego: Academic.
32. McLachlan, G. J., and Krishnan, T. (1997), The EM Algorithm and Extensions, New York: Wiley.
33. Meng, X.-L., and Rubin, D. B. (1991), “Using EM to Obtain Asymptotic Variance-Covariance Matrices: The SEM Algorithm,” Journal of the American Statistical Association, 86, 899-909.
34. Meng, X.-L., and Rubin, D. B. (1993), “Maximum Likelihood Estimation via the ECM Algorithm: A General Framework, Biometrika, 800, 267-278.
35. Oakes, D. (1999), “Direct Calculation of the Information Matrix via the EM Algorithm,” Journal of the Royal Statistical Society, Ser. B, 61, Part 2, 479-482.
36. Ortega, J. M., and Rheinboldt, W. C. (1970), Iterative Solutions of Nonlinear Equations in Several Variables, New York: Academic, pp. 253-255.
37. Sabatti, C., and Lange, K. (2002), “Genome wide Motif Identification Using a Dictionary Model,” Proceedings of the IEEE, 90, 1803-1810.
38. Schlossmacher, E. J. (1973), “An Iterative Technique for Absolute Deviations Curve Fitting,” Journal of the American Statistical Association, 68, 857-859.
39. Wu, C. F. J. (1983), “On the Convergence Properties of the EM Algorithm,” The Annals of Statistics, 11, 95-103.