Variance and bias for general loss functions

Machine Learning

Volumn 51, Issue 2, 2003, Pages 115-135

  James, Gareth M ^a  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Bias; Loss function; Prediction error; Variance

Indexed keywords

ERROR CORRECTION; ERROR DETECTION; NUMERICAL METHODS; STATISTICAL METHODS;

ERROR PREDICTION; SQUARED ERROR LOSS; VARIANCE;

LEARNING SYSTEMS;

EID: 0037403462     PISSN: 08856125     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1022899518027     Document Type: Article

References (21)

1. Breiman, L. (1996a). Bagging predictors. Machine Learning, 26:2, 123-140.
2. Breiman, L. (1996b). Bias, variance, and arcing classifiers. Technical Report 460, Statistics Department, University of California Berkeley (460).
3. Breiman, L., Friedman, J., Olshen, R., & Stone, C. (1984). Classification and Regresion Trees. Wadsworth (Since 1993 this book has been published by Chapman & Hall, New York.).
4. Cover, T., & Hart, P. (1967). Nearest neighbor pattern classification. IEEE Transactions on Information Theory, IT-13, 21-27.
5. Dietterich, T., & Bakiri, G. (1995). Solving multiclass learning problems via error-correcting output codes. Journal of Artificial Intelligence Research, 2, 263-286.
6. Dietterich, T. G., & Kong, E. B. (1995). Error-correcting output coding corrects bias and variance. In Proceedings of the 12th International Conference on Machine Learning (pp. 313-321). Morgan Kaufmann.
7. Domingos, P. (2000). A unified bias-variance decomposition and its applications. In Proceedings of the 17th International Conference on Machine Learning.
8. Efron, B. (1978). Regression and anova with zero-one data. Journal of the American Statistical Association. 73, 113-121.
9. Efron, B., & Tibshirani, R. (1993). An Introduction to the Bootstrap. London: Chapman and Hall.
10. Fisher, R. A. (1936). The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7, 179-188.
11. Fix, E., & Hodges, J. (1951). Discriminatory Analysis, Nonparametric Discrimination: Consistency Properties. Technical Report, Randolph Field, Texas: USAF School of Aviation Medicine.
12. Freund, Y., & Schapire, R. (1996). Experiments with a new boosting algorithm. In Machine Learning: Proceedings of the Thirteenth International Conference.
13. Friedman, J., & Hastie, T., & Tibshirani, R. (2000). Additive logistic regression: A statistical view of boosting. Annals of Statistics, 28, 337-407.
14. Friedman, J. H. (1996). On bias, variance, 0/1-loss, and the curse of dimensionality. Technical Report, Department of Statistics, Stanford University.
15. Geman, S., Bienenstock, E., & Doursat, R. (1992). Neural networks and the bias/variance dilemma. Neural Computation, 4, 1-58.
16. Heskes, T. (1998). Bias/variance decompositions for likelihood-based estimators. Neural Computation. 10, 1425-1433.
17. Kohavi, R., & Wolpert, D. (1996). Bias plus variance decomposition for zero-one loss functions. In Machine Learning: Proceedings of the Thirteenth International Conference.
18. Schapire, R. E., Freund, Y., Bartlett, P., & Lee, W. S. (1998). Boosting the margin: A new explanation for the effectiveness of voting methods. The Annals of Statistics, 26, 1651-1686.
19. Stone, C. (1977). Consistent nonparametric regression (with discussion). Annals of Statistics, 5, 595-645.
20. Tibshirani, R. (1996). Bias, variance & prediction error for classification rules. Technical Report, Department of Statistics University of Toronto.
21. Wolpert, D. (1997). On bias plus variance. Neural Computation, 9, 1211-1243.