Penalized Bregman divergence for large-dimensional regression and classification

Biometrika

Volumn 97, Issue 3, 2010, Pages 551-566

  Zhang, Chunming ^a   Jiang, Yuan ^a   Chai, Yi ^a  

^a University of Wisconsin Madison   (United States)

Author keywords

Consistency; Divergence minimization; Exponential family; Loss function; Optimal Bayes rule; Oracle property; Quasilikelihood

Indexed keywords

EID: 77955858059     PISSN: 00063444     EISSN: 14643510     Source Type: Journal    
DOI: 10.1093/biomet/asq033     Document Type: Article

References (31)

1. ALTUN, Y. & SMOLA, A. (2006). Unifying divergence minimization and statistical inference via convex duality. In Learning Theory: 19th Ann. Conf. Learn. Theory, Ed. G. Lugosi and H. U. Simon, pp. 139-153 Berlin: Springer.
2. BARTLETT, M. S. (1953). Approximate confidence intervals. Biometrika 40, 12-19.
3. BICKEL, P. & LI, B. (2006). Regularization in statistics (with discussion). Test 15, 271-344.
4. BREGMAN, L. M. (1967). A relaxation method of finding a common point of convex sets and its application to the solution of problems in convex programming. USSR Comp. Math. Math. Phys. 7, 620-631
5. CHOW, Y. S. & TEICHER, H. (1988). Probability Theory, 2nd ed. New York: Springer.
6. EFRON, B. (1986). How biased is the apparent error rate of a prediction rule? J. Am. Statist. Assoc. 81, 461-470
7. EFRON, B., HASTIE, T., JOHNSTONE, I., & TIBSHIRANI, R. (2004). Least angle regression. Ann. Statist. 32, 407-499
8. FAN, J. & LI, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Statist. Assoc. 96, 1348-1360
9. FAN, J. & PENG, H. (2004). Nonconcave penalized likelihood with a diverging number of parameters. Ann. Statist. 32, 928-961
10. GOLUB, G. H. & VAN LOAN, C. F. (1996). Matrix Computations, 3rd ed. Baltimore, MD: Johns Hopkins University Press.
11. GRÜNWALD, P. D. & DAWID, A. P. (2004). Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory. Ann. Statist. 32, 1367-1433
12. GÜVENIR, H. A.,ACAR, B.,DEMIRöZ, G. & ÇEKIN, A. (1997). A supervised machine learning algorithm for arrhythmia analysis. Comp. Cardiol. 24, 433-436
13. HASTIE, T., TIBSHIRANI, R. & FRIEDMAN, J. (2001). The Elements of Statistical Learning. New York: Springer.
14. HUANG, J.,MA, S. G. & ZHANG, C. H. (2008). Adaptive lasso for sparse high-dimensional regression models. Statist. Sinica 18, 1603-1618
15. KIVINEN, J.&WARMUTH,M. K. (1999). Boosting as entropy projection. In Proc. 12th Ann. Conf. Comp. Learn. Theory, pp. 134-144 New York: ACM Press.
16. KNIGHT, K. & FU, W. J. (2000). Asymptotics for lasso-type estimators. Ann. Statist. 28, 1356-1378
17. LAFFERTY, J. D., DELLA PIESTRA, S., & DELLA PIESTRA, V. (1997). Statistical learning algorithms based on Bregman distances. In Proc. 5th Can. Workshop Info. Theory.
18. LAFFERTY, J. (1999). Additive models, boosting, and inference for generalized divergences. In Proc. 12th Ann. Conf. Comp. Learn. Theory, pp. 125-133 New York: ACM Press.
19. MCCULLAGH, P. (1983). Quasi-likelihood functions. Ann. Statist. 11, 59-67.
20. MEINSHAUSEN, N. & BUHLMANN, P. (2006). High dimensional graphs and variable selection with the lasso. Ann. Statist. 34, 1436-1462
21. ROSSET, S. & ZHU, J. (2007). Piecewise linear regularized solution paths. Ann. Statist. 35, 1012-1030
22. SHEN, X., TSENG, G. C., ZHANG, X. & WONG, W. H. (2003). On Ψ-learning. J. Am. Statist. Assoc. 98, 724-734
23. STRIMMER,K. (2003).Modeling gene expression measurement error: a quasi-likelihood approach.BMCBioinformatics 4, 10.
24. TIBSHIRANI, R. (1996). Regression shrinkage and selection via the lasso. J. R. Statist. Soc. B 58, 267-288
25. VAPNIK, V. (1996). The Nature of Statistical Learning Theory. New York: Springer.
26. VAN DER VAART, A. W. & WELLNER, J. A. (1996). Weak Convergence and Empirical Processes: With Applications to Statistics. New York: Springer.
27. WEDDERBURN, R.W.M. (1974). Quasi-likelihood functions, generalized linear models, and the Gauss-Newton method. Biometrika 61, 439-447
28. ZHANG, C. M., JIANG, Y. & SHANG, Z. (2009). New aspects of Bregman divergence in regression and classification with parametric and nonparametric estimation. Can. J. Statist. 37, 119-139
29. ZHANG, C. M. & ZHANG, Z. J. (2010). Regularized estimation of hemodynamic response function for fMRI data. Statist. Interface 3, 15-32.
30. ZHAO, P. & YU, B. (2006). On model selection consistency of lasso. J. Mach. Learn. Res. 7, 2541-2567
31. ZOU, H. (2006). The adaptive lasso and its oracle properties. J. Am. Statist. Assoc. 101, 1418-1429