The Loss Rank Criterion for Variable Selection in Linear Regression Analysis

Scandinavian Journal of Statistics

Volumn 38, Issue 3, 2011, Pages 466-479

  Tran, Minh Ngoc ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Lasso; Loss rank principle; Model selection; Shrinkage parameter; Variable selection

Indexed keywords

EID: 80051906949     PISSN: 03036898     EISSN: 14679469     Source Type: Journal    
DOI: 10.1111/j.1467-9469.2011.00732.x     Document Type: Article

References (28)

1. Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In Proceedings of the 2nd International Symposium on Information Theory( B.N. Petrov & F. Caski ), 267-281. Akademiai Kaidó, Budapest, Hungary.
2. Bartlett, P., Boucheron, S. & Lugosi, G. (2002). Model selection and error estimation. Mach. Learn. 48, 85-113.
3. Burnham, K. P. & Anderson, D. (2002). Model selection and multimodel inference: a practical information-theoretic approach. Springer, New York.
4. Candes, E. & Tao, T. (2007). The dantzig selector: statistical estimation when p is much larger than n (with discussion). Ann. Statist. 35, 2313-2351.
5. Chambaz, A. (2006). Testing the order of a model. Ann. Statist. 34, 1166-1203.
6. Craven, P. & Wahba, G. (1979). Smoothing noisy data with spline functions: estimating the correct degree of smoothing by the methods of generalized cross-validation. Numer. Math. 31, 377-403.
7. Dudley, R. M. & Philipp, W. (1983). Invariance principles for sums of banach space valued random elements and empirical processes. Z. Wahrsch. Verw. Gebiete 62, 509-552.
8. Efron, B., Hastie, T., Johnstone, I. & Tibshirani, R. (2004). Least angle regression. Ann. Statist. 32, 407-499.
9. Fan, J. & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96, 1348-1360.
10. Friedman, J. H. (2008). Fast sparse regression and classification. Available on (accessed February 12, 2011).
11. Hutter, M. (2007). The loss rank principle for model selection. In Proceedings of the 20th Annual Conf. on Learning Theory (COLT'07), volume 4539 of LNAI (eds N. Bshouty & C. Gentile ), 589-603. Springer, Berlin and San Diego.
12. Hutter, M. & Tran, M. N. (2010). Model selection with the loss rank principle. Comput. Statist. Data Anal. 54, 1288-1306.
13. Koltchinskii, V. (2001). Rademacher penalties and structural risk minimization. IEEE Trans. Inform. Theory 47, 1902-1914.
14. Leng, C., Lin, Y. & Wahba, G. (2006). A note on the lasso and related procedures in model selection. Statist. Sinica 16, 1273-1284.
15. Meinshausen, N. & Buhlmann, P. (2006). Consistent neighbourhood selection for high-dimensional graphs the lasso. Ann. Statist. 34, 1436-1462.
16. Miller, A. (1990). Subset selection in regression. Chapman & Hall/CRC, New York.
17. Poetscher, B. M. & Leeb, H. (2009). On the distribution of penalized maximum likelihood estimators: The lasso, scad, and thresholding. Journal of Multivariate Analysis 100, 2065-2082.
18. Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6, 461-464.
19. Shao, J. (1997). An asymptotic theory for linear model selection. Statist. Sinica 7, 221-264.
20. Stamey, T., Kabalin, J., McNeal, J., Johnstone, I., Freiha, F., Redwine, E. & Yang, N. (1989). Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate ii. radical prostatectomy treated patients. J. Urol. 16, 1076-1083.
21. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58, 267-288.
22. Tran, M. N. (2009). Penalized maximum likelihood principle for choosing ridge parameter. Comm. Statist. Simulation Comput. 38, 1610-1624.
23. Tran, M. N. & Hutter, H. (2010). Model selection by loss rank for classification and unsupervised learning. Preprint: .
24. Wang, H., Li, R. & Tsai, C. L. (2007). Tuning parameter selectors for the smoothly clipped absolute deviation method. Biometrika 94, 553-568.
25. Yang, Y. (2005). Can the strengths of AIC and BIC be shared? A conflict between model indentification and regression estimation. Biometrika 92, 937-950.
26. Zhao, P. & Yu, B. (2006). On model selection consistency of lasso. J. Mach. Learn. Res. 7, 2541-2563.
27. Zou, H. (2006). The adaptive lasso and its oracle properties. J. Amer. Statist. Assoc. 101, 1418-1429.
28. Zou, H., Hastie, T. & Tibshirani, R. (2007). On the degrees of freedom of the lasso. Ann. Statist. 35, 2173-2192.