Efficient methods for estimating constrained parameters with applications to regularized (lasso) logistic regression

Computational Statistics and Data Analysis

Volumn 52, Issue 7, 2008, Pages 3528-3542

  Tian, Guo Liang ^a   Tang, Man Lai ^b   Fang, Hong Bin ^a   Tan, Ming ^a  

^a UNIVERSITY OF MARYLAND GREENEBAUM CANCER CENTER   (United States)

^b HONG KONG BAPTIST UNIVERSITY   (Hong Kong)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; BRANCH AND BOUND METHOD; CONSTRAINT THEORY; CONVERGENCE OF NUMERICAL METHODS; PARAMETER ESTIMATION;

CONSTRAINED PARAMETERS; MONOTONIC CONVERGENCE; QUADRATIC LOWER-BOUND (QLB) ALGORITHMS; REGULARIZED LOGISTIC REGRESSIONS;

REGRESSION ANALYSIS;

EID: 40249087061     PISSN: 01679473     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.csda.2007.11.007     Document Type: Article

References (35)

1. Agresti A. Categorical Data Analysis. second edition (2002), John Wiley & Sons, New York
2. Alon U., Barkai N., Notterman D.A., Gish K., Ybarra S., Mack D., and Levine A.J. Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the National Academy of Sciences of the United States of America 96 (1999) 6745-6750
3. Barlow R.E., and Brunk H.D. The isotonic regression problem and its dual. Journal of the American Statistical Association 67 (1972) 140-147
4. Böhning D. Multinomial logistic regression algorithm. Annals of the Institute of Statistical Mathematics 44 (1992) 197-200
5. Böhning D., and Lindsay B.G. Monotonicity of quadratic approximation algorithms. Annals of the Institute of Statistical Mathematics 40 (1988) 641-663
6. Collett D. Modeling Binary Data (1991), Chapman & Hall, London
7. Cox D.R., and Oakes D. Analysis of Survival Data (1984), Chapman & Hall, London
8. Craven P., and Wahba G. Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation. Numerische Mathematik 31 (1979) 377-403
9. Dempster A.P., Laird N.M., and Rubin D.B. Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society, Series B 39 (1977) 1-38
10. Dykstra R.L. An algorithm for restricted least squares regression. Journal of the American Statistical Association 78 (1983) 837-842
11. Efron B., and Tibshirani R.J. An Introduction to the Bootstrap (1993), Chapman & Hall/CRC, Boca Raton
12. Efron B., Hastie T., Johnstone I., and Tibshirani R.J. Least angle regression (with discussion). The Annals of Statistics 32 (2004) 407-499
13. Fessler J.A., Clinthorne N.H., and Rogers W.L. On complete-data spaces for PET reconstruction algorithms. IEEE Transactions on Nuclear Science 40 4 (1993) 1055-1061
14. Frank I.E., and Friedman J.H. A statistical view of some chemometrics regression tools (with discussion). Technometrics 35 2 (1993) 109-148
15. Geyer C.J. Constrained maximum likelihood exemplified by isotonic convex logistic regression. Journal of the American Statistical Association 86 (1991) 717-724
16. Gilks W.R. Full conditional distributions. In: Gilks W.R., Richardson S., and Spiegelhalter D.J. (Eds). Markov Chain Monte Carlo in Practice (1996), Chapman & Hall, London 75-88
17. Green P.J. On the use of the EM algorithm for penalized likelihood estimation. Journal of the Royal Statistical Society, Series B 52 (1990) 443-452
18. Hastie T.J., and Tibshirani R.J. Generalized Additive Models (1990), Chapman & Hall/CRC, Boca Raton
19. Hook E.B., and Fabia J.J. Frequency of down syndrome in livebirths by single-year maternal age interval: Results of a Massachusetts study. Teratology 17 (1978) 223-228
20. Khuri A.I. A constrained least squares problem. Communications in Statistics: Simulation and Computation 5 (1976) 82-84
21. Kim Y., Kwon S., and Song S.H. Multiclass spare logistic regression for classification of multiple cancer types using gene expression data. Computational Statistics and Data Analysis 51 (2006) 1643-1655
22. Liu C.H. Estimation of discrete distributions with a class of simplex constraints. Journal of the American Statistical Association 95 (2000) 109-120
23. Mehta C.R., Patel N.R., and Senchaudhuri P. Efficient Monte Carlo methods for conditional logistic regression. Journal of the American Statistical Association 95 (2000) 99-108
24. Meng X.L. On the rate of convergence of the ECM algorithm. The Annals of Statistics 22 (1994) 326-339
25. Meng X.L., and Rubin D.R. Using EM to obtain asymptotic variance-covariance matrices: The SEM algorithm. Journal of the American Statistical Association 86 (1991) 899-909
26. Meng X.L., and Rubin D.B. Maximum likelihood estimation via the ECM algorithm: A general framework. Biometrika 80 (1993) 267-278
27. Meng X.L., and van Dyk D. The EM algorithm - an old folk-song sung to a fast new tune (with discussion). Journal of the Royal Statistical Society, Series B 59 (1997) 511-567
28. Meyer M.C. An extension of the mixed primal-dual bases algorithm to the case of more constraints than dimensions. Journal of Statistical Planning and Inference 81 (1999) 13-31
29. Robertson T., Wright F.T., and Dykstra R.L. Order Restricted Statistical Inference (1988), John Wiley & Sons, New York
30. Schmoyer R.L. Sigmoidally constrained maximum likelihood estimation in quantal bioassay. Journal of the American Statistical Association 79 (1984) 448-453
31. Silvapulle M.J., and Sen P.K. Constrained Statistical Inference: Inequality, Order, and Shape Restrictions (2005), John Wiley & Sons, New York
32. Tan M., Tian G.L., and Fang H.B. Estimating restricted normal means using the EM-type algorithms and IBF sampling. In: Huang J., and Zhang H. (Eds). Development of Modern Statistics and Related Topics - In Celebration of Prof. Yaoting Zhang's 70th Birthday (2003), World Scientific, New Jersey 53-73
33. Tan M., Tian G.L., Fang H.B., and Ng K.W. A fast EM algorithm for quadratic optimization subject to convex constraints. Statistica Sinica 17 3 (2007) 945-964
34. Tibshirani R.J. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B 58 (1996) 267-288
35. Zhu C.Y., Byrd R.H., Lu P.H., and Nocedal J. Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization. ACM Transactions on Mathematical Software 23 4 (1997) 550-560