New aspects of Bregman divergence in regression and classification with parametric and nonparametric estimation

Canadian Journal of Statistics

Volumn 37, Issue 1, 2009, Pages 119-139

  Zhang, Chunming ^a   Jiang, Yuan ^a   Shang, Zuofeng ^a  

^a University of Wisconsin Madison   (United States)

Author keywords

Asymptotic normality; Bayes optimal rule; Consistency; Local polynomial regression; Loss function; Prediction error

Indexed keywords

EID: 74049108277     PISSN: 03195724     EISSN: None     Source Type: Journal    
DOI: 10.1002/cjs.10005     Document Type: Article

References (25)

1. Y. Altun & A. Smola (2006). Unifying divergence minimization and statistical inference via convex duality. In Proc. COLT2006, Springer, Berlin/Heidelberg, pp. 139-153.
2. K. S. Azoury & M. K. Warmuth (2001). Relative loss bounds for on-line density estimation with the exponential family of distributions. Machine Learning, 43, 211-246.
3. A. Banerjee, X. Guo & H. Wang (2005). On the optimality of conditional expectation as a Bregman predictor. IEEE Transactions on Information Theory, 51, 2664-2669.
4. L.M. Brègman (1967). A relaxation method of finding a common point of convex set sand its application to the solution of problems in convex programming. U.S.S.R. Computational Mathematics and Mathematical Physics, 7, 620-631.
5. L. Breiman (1998). Arching classifiers (with discussion). Annals of Statistics, 26, 801-824.
6. P. Bühlmann & B. Yu (2003). Boosting with the L2 loss: regression and classification. Journal of American Statistical Association, 98, 324-339.
7. B. Efron (1986). How biased is the apparent error rate of a prediction rule? Journal of American Statistical Association, 81, 461-470.
8. B. Efron (2004). The estimation of prediction error: covariance penalties and cross-validation (with discussion). Journal of American Statistical Association, 99, 619-642.
9. J. Fan & I. Gijbels (1996). Local Polynomial Modeling and Its Applications, Chapman and Hall, London.
10. Y. Freund & R. E. Schapire (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55, 119-139.
11. J. Friedman (1997). On bias, variance, 0/1-loss, and the curse-of-dimensionality. Journal of Data Mining and Knowledge Discovery, 1, 55-77.
12. J. Friedman, T. Hastie & R. Tibshirani (2000). Additive logistic regression: a statistical view of boosting (with discussion). Annals of Statistics, 28, 337-407.
13. P. D. Grünwald & A. P. Dawid (2004). Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory. Annals of Statistics, 32, 1367-1433.
14. T. J. Hastie & R. J. Tibshirani, (1993). Varying-coefficient models (with discussion), Journal of the Royal Statistical Society, Series B, 55, 757-796.
15. T. Hastie, R. Tibshirani & J. Friedman (2001). The Elements of Statistical Learning, Springer, New York.
16. J. Kivinen & M. K. Warmuth (1999). Boosting as entropy projection. Proceedings of the Twelfth Annual Conference on Computational Learning Theory, Santa Cruz, CA. ACM Press, New York, NY, pp. 134-144.
17. J. D. Lafferty, S. Della Piestra & V. Della Piestra (1997). Statistical learning algorithms based on Bregman distances. In Proceedings of the Canadian Workshop on Information Theory.
18. J. Lafferty (1999). Additive models, boosting, and inference for generalized divergences. Proceedings of the TwelfthAnnualConferenceonComputationalLearningTheory,SantaCruz,CA.ACMPress,NewYork, NY, pp. 125-133.
19. X. Nguyen, M. J. Wainwright & M. I. Jordan (2008). On surrogate loss functions and f-divergences. Annals of Statistics, in press.
20. R. E. Schapire (2002). The boosting approach to machine learning: an overview. In MSRI Workshop on Nonlinear Estimation and Classification.
21. X. Shen, G. C. Tseng, X. Zhang & W. H. Wong (2003). On ψ-learning. Journal of American Statistical Association, 98, 724-734.
22. R. Tibshirani (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 58, 267-288.
23. R. Tibshirani & T. Hastie (1987). Local likelihood estimation. Journal of American Statistical Association, 82,559-567.
24. R. W. M. Wedderburn (1974). Quasi-likelihood functions, generalized linear models, and the Gauss-Newton method. Biometrika, 61, 439-447.
25. C. M. Zhang, Y. Jiang & Z. Shang (2007). New aspects of Bregman divergence in regression and classification with parametric and nonparametric estimation. Technical Report #1127, Department of Statistics, University of Wisconsin, Madison, WI.