A fast em algorithm for quadratic optimization subject to Convex constraints

Statistica Sinica

Volumn 17, Issue 3, 2007, Pages 945-964

  Tan, Ming ^a,c   Tian, Guo Liang ^a,c   Fang, Hong Bin ^a,c   Ng, Kai Wang ^b  

^a UNIVERSITY OF MARYLAND   (United States)

^b UNIVERSITY OF HONG KONG   (Hong Kong)

^c UNIVERSITY OF MARYLAND   (United States)

Author keywords

Bootstrap; Cholesky decomposition; Constrained optimization; Convergence rate; Data augmentation; EM algorithm; Latent variables; Working parameter

Indexed keywords

EID: 36249018992     PISSN: 10170405     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (30)

1. Breiman, L. (1995). Better subset regression using the nonnegative garrote. Technometrics 37, 373-384.
2. Coleman, T. F. and Li, Y. (1996). A reflective Newton method for minimizing a quadratic function subject to bounds on some of the variables. SIAM J. Optim. 6, 1040-1058.
3. Dempster, A. P., Laird, N. M. and Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). J. Roy. Statist. Soc. Ser. B 39, 1-38.
4. Dykstra, R. L. (1983). An algorithm for restricted least squares regression. J. Amer. Statist. Assoc. 78, 837-842.
5. Dykstra, R. L. and Robertson, T. (1982). An algorithm for isotonic regression for two or more independent variables. Amer. Statist. 10, 708-716.
6. Fraser, D. A. S. and Massam, H. (1989). A mixed primal-dual bases algorithm for regression under inequality constraints with application to concave regression. Scand. J. Statist. 16, 65-74.
7. Gill, P. E., Murray, W. and Wright, M. H. (1981). Practical Optimization. Academic Press, London.
8. Green, P. J. (1990). On the use of the EM algorithm for penalized likelihood estimation. J. Roy. Statist. Soc. Ser. B 52, 443-452.
9. Hajivassiliou, V. A. and McFadden, D. L. (1998). The method of simulated scores for the estimation of LDV models. Econometrica 66, 863-896.
10. Hildreth, C. (1954). Point estimates of ordinates of concave functions. J. Amer. Statist. Assoc. 49, 598-619.
11. Kim, D. K. and Taylor, J. M. G. (1995). The restricted EM algorithm for maximum likelihood estimation under linear restrictions on the parameters. J. Amer. Statist. Assoc. 90, 708-716.
12. Kuhn, H. W. and Tucker, A. W. (1951). Nonlinear programming. In Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability (Edited by J. Neyman). University of California Press, Berkeley.
13. Lange, K. (1999). Numerical Analysis for Statisticians. Springer, New York.
14. Liu, C. H. (2000). Estimation of discrete distributions with a class of simplex constraints. J. Amer. Statist. Assoc. 95, 109-120.
15. Liu, C. H., Rubin, D. B. and Wu, Y. N. (1998). Parameter expansion to accelerate EM: The PX-EM algorithm. Biometrika 85, 755-770.
16. Meng, X. L. (1994). On the rate of convergence of the ECM algorithm. Ann. Statist. 22, 326-339.
17. Meng, X. L. and Rubin, D. B. (1993). Maximum likelihood estimation via the ECM algorithm: A general framework. Biometrika 80, 267-278.
18. Meng, X. L. and van Dyk, D. (1997). The EM algorithm -an old folk-song sung to a fast new tune (with discussion). J. Roy. Statist. Soc. Ser. B 59, 511-567.
19. Meyer, M. C. (1999). An extension of the mixed primal-dual bases algorithm to the case of more constraints than dimensions. J. Statist. Plann. Inference 81, 13-31.
20. Nyquist, H. (1991). Restricted estimation of generalized linear models. Appl. Statist. 40, 133-141.
21. Robertson, T., Wright, F. T. and Dykstra, R. L. (1988). Order Restricted Statistical Inference. Wiley, New York.
22. Shi, N. Z., Zheng, S. R. and Guo, J. H. (2005). The restricted EM algorithm under inequality restrictions on the parameters. J. Multivariate Anal. 92(1), 53-76.
23. Silverman, B. W., Jones, M. C., Nychka, D. W. and Wilson, J. D. (1990). A smoothed EM approach to indirect estimation problems, with particular reference to stereology and emission tomography (with discussion). J. Roy. Statist. Soc. Ser. B 52, 271-324.
24. Tan, M., Tian, G. L. and Fang, H. B. (2003). Estimating restricted normal means using the EM-type algorithms and IBF sampling. In Development of Modern Statistics and Related Topics -In Celebration of Prof. Yaoting Zhang's 70th Birthday (Edited by J. Huang and H. Zhang), 53-73. World Scientific, New Jersey.
25. Tian, G. L., Ng, K. W. and Tan, M. (2005). Likelihood-based approaches for constrained parameter problems in multiple regression models. Ann. Inst. Statist. Math., in revision.
26. Tibshirani, R. J. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58, 267-288.
27. Titterington, D. M. (1985). General structure of regularization procedures in image reconstruction. Astronomy and Astrophysics 144, 381-387.
28. Vardi, Y. and Lee, D. (1993). From image deblurring to optimal investments: Maximum likelihood solutions for positive linear inverse problems (with discussion). J. Roy. Statist. Soc. Ser. B 55, 569-612.
29. Wolfe, P. (1959). The simplex method for quadratic programming. Econometrica 27, 382-398.
30. Wollan, P. C. and Dykstra, R. L. (1987). Minimizing linear inequality constrained Mahalanobis distances. Appl. Statist. 36, 234-240.