On bias, variance, 0/1-loss, and the curse-of-dimensionality

Data Mining and Knowledge Discovery

Volumn 1, Issue 1, 1997, Pages 55-77

  Friedman, Jerome H ^a,b  

^a SLAC NATIONAL ACCELERATOR LABORATORY   (United States)

^b STANFORD UNIVERSITY   (United States)

Author keywords

Bagging; Bias; Classification; Curse of dimensionality; Naive Bayes; Nearest neighbors; Variance

Indexed keywords

EID: 21744462998     PISSN: 13845810     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1009778005914     Document Type: Article

References (24)

1. Bellman, R.E. 1961. Adaptive Control Processes. Princeton University Press.
2. Breiman, L. 1995. Bagging predictors. Dept. of Statistics, University of California, Berkeley, Technical Report.
3. Breiman, L. 1996. Bias, variance, and arcing classifiers. Dept. of Statistics, University of California, Technical Report (revised).
4. Breiman, L., Friedman, J.H., Olshen, R.A., and Stone, C.J. 1984. Classification and Regression Trees. Wadsworth.
5. Chow, W.S. and Chen, Y.C. 1992. A new fast algorithm for effective training of neural classifiers. Pattern Recognition, 25:423-429.
6. Dietterich, T.G. and Kong, E.B. 1995. Machine learning bias, statistical bias, and statistical variance of decision tree algorithms. Dept. of Computer Science, Oregon State University Technical Report.
7. Efron, B. and Tibshirani, R. 1995. Cross-validation and the bootstrap: Estimating the error rate of a prediction rule. Dept. of Statistics, Stanford University Technical Report.
8. Fix, E. and Hodges, J.L. 1951. Discriminatory analysis - nonparametric discrimination: Consistency properties. Randolf Field Texas: U.S. Airforce School of Aviation Medicine Technical Report No. 4.
9. Friedman, J.H. 1985. Classification and multiple response regression through projection pursuit. Dept. of Statistics, Stanford University Technical Report LCS012.
10. Geman, S., Bienenstock, E., and Doursat, R. 1992. Neural networks and the bias/variance dilemma. Neural Comp., 4:1-18.
11. Good, I.J. 1965. The Estimation of Probabilities: An Essay on Modern Bayesian Methods. M.I.T. Press.
12. Hand, D.J. 1982. Kernel discriminant analysis. Chichester: Research Studies Press.
13. Heckerman, D., Geiger, D., and Chickering, D. 1994. Learning Bayesian networks: the combination of knowledge and statistical data. In Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence, pp. 293-301, AAAI Press and M.I.T. Press.
14. Henley, W.E. and Hand, D.J. 1996. A k-nearest neighbour classifier for assessing consumer credit risk. The Statistician, 45:77-95.
15. Holte, R.C. 1993. Very simple classification rules perform well on most commonly used data sets. Machine Learning, 11:63-90.
16. Kohavi, R. and Wolpert, D.H. 1996. Bias plus variance decomposition for zero-one loss functions. Dept. of Computer Science, Stanford University Technical Report.
17. Kohonen, T. 1990. The self-organizing map. Proceedings of the IEEE, 78:1464-1480.
18. Langley, P., Iba, W., and Thompson, K. 1992. An analysis of Bayesian classifiers. In Proceedings of the Tenth National Conference on Artificial Intelligence, pp. 223-228, AAAI Press and M.I.T. Press.
19. Lippmann, R. 1989. Pattern classification using neural networks. IEEE Communications Magazine, 11:47-64.
20. McLachlan, G.J. 1992. Discriminant Analysis and Statistical Pattern Recognition. Wiley.
21. Quinlan, J.R. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann.
22. Rosen, D.B., Burke, H.B., and Goodman, P.H. 1995. Local learning methods in high dimension: Beating the bias-variance dilemma via recalibration. Workshop Machines That Learn - Neural Networks for Computing, Snowbird Utah.
23. Tibshirani, R. 1996. Bias, variance and prediction error for classification rules. Dept. of Statistics, University of Toronto Technical Report.
24. Titterington, D.M., Murray, G.D., Murray, L.S., Spiegelhalter, D.J., Skene, A.M., Habbema, J.D.F., and Gelpke, G.J. 1981. Comparison of discrimination techniques applied to a complex data set of head injured patients. J. Roy. Statist. Soc. A, 144:145-175.