Random forests and adaptive nearest neighbors

Journal of the American Statistical Association

Volumn 101, Issue 474, 2006, Pages 578-590

  Lin, Yi ^a   Jeon, Yongho ^a  

^a University of Wisconsin Madison   (United States)

Author keywords

Adaptive estimation; Boosting; Classification trees; Randomized trees; Regression trees

Indexed keywords

EID: 33745653724     PISSN: 01621459     EISSN: None     Source Type: Journal    
DOI: 10.1198/016214505000001230     Document Type: Article

References (30)

1. Amit, Y., and Geman, D. (1997), "Shape Quantization and Recognition With Randomized Trees," Neural Computation, 9, 1545-1588.
2. Breiman, L. (1996), "Bagging Predictors." Machine Learning, 26, 123-140.
3. _ (1998), "Randomizing Outputs to Increase Prediction Accuracy," Technical Report 518, University of California Berkeley. Dept. of Statistics, available at http://www.stat.berkeley.edu.
4. _ (1999), "Random Forests," technical report, University of California Berkeley, Dept. of Statistics, available at http://www.stat.berkeley. edu.
5. _ (2000), "Some Infinity Theory for Predictor Ensembles." Technical Report 577, University of California Berkeley. Dept. of Statistics, available at. http://www.stat.berkeley.edu.
6. Bühlmann, P., and Yu, B. (2002), "Analyzing Bagging," The Annals of Statistics, 30, 927-961.
7. Cutler, A. (1999), "Fast Classification Using Perfect Random Trees." Technical Report 5/99/99, Utah State University, Dept. of Mathematics and Statistics.
8. Cutler, A., and Zhao, G. (2000), "Voting Perfect Random Trees," Technical Report 5/00/100, Utah State University, Dept. of Mathematics and Statistics.
9. Dasarathy, B. (1991), Nearest-Neighbor Pattern Classification Techniques, Los Alamitos, CA: IEEE Computer Society Press.
10. Devroye, L., Györfi, L., and Lugosi, G. (1996). A Probability Theory of Pattern Recognition, New York: Springer-Verlag.
11. Dietterich, T. G. (2000), "An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization," Machine Learning, 40, 139-157.
12. Domeniconi, C., Peng, J., and Gunopulos, D. (2002), "Locally Adaptive Metric Nearest-Neighbor Classification." IEEE Transactions on Pattern Analysis and Machine Intelligence, 24, 1281-1285.
13. Fix, E., and Hodges, J. (1951), "Discriminatory Analysis- Nonparametric Discrimination: Consistency Properties," Technical Report 21-49-004, 4. U.S. Air Force, School of Aviation Medicine.
14. Freund, Y. (1995), "Boosting a Weak Learning Algorithm by Majority," Information and Computation, 121, 256-285.
15. Freund, Y., and Schapire, R. E. (1996), "Experiments With a New Boosting Algorithm," in Machine Learning: Proceedings of the Thirteenth International Conference, San Francisco: Morgan Kauffman, pp. 148-156.
16. Friedman, J. H. (1991), "Multivariate Adaptive Regression Splines" (with discussion), The Annals of Statistics, 19, 1-141.
17. _ (1994), "Flexible Metric Nearest-Neighbor Classification." preprint, available at http://www-stat.stanford.edu/~jhf/=Reports.
18. _ (2001), "Greedy Function Approximation: A Gradient Boosting Machine," The Annals of Statistics, 29, 1189-1232.
19. Friedman, J. H., and Hall, P. (1999), "On Bagging and Nonlinear Estimation," preprint, available at http:/www-stat,stanford.edu/_jhf/ #Reports.
20. Friedman, J. H., Hastie, T., and Tibshirani, R. (2000), "Additive Logistic Regression: A Statistical View of Boosting" (with discussion), The Annals of Statistics, 28, 337-407.
21. Hastie, T., and Tibshirani, R. (1996), "Discriminant Adaptive Nearest-Neighbor Classification," IEEE Transactions on Pattern Analysis and Machine Intelligence, 18, 607-616.
22. Hastie, T., Tibshirani, R., and Friedman, J. (2001), The Elements of Statistical Learning: Data Mining, Inference, and Prediction, New York: Springer-Verlag.
23. Ho, T. K. (1998), "The Random Subspace Method for Constructing Decision Forests," IEEE Transactions on Pattern Analysis and Machine Intelligence, 20, 832-844.
24. Lim, T., Loh, W., and Shih, Y. (2000), "A Comparison of Prediction Accuracy. Complexity, and Training Time of Thirty-Three Old and New Classification Algorithms," Machine Learning, 40, 203-228.
25. Mason, L., Baxter, J., Bartlett, P., and Frean, M. R. (2000), "Boosting Algorithms as Gradient Descent in Function Space," in Neural Information Processing Systems 1999, Cambridge, MA: MIT Press, pp. 512-518.
26. Myles, J., and Hand, D. (1990), "The Multiclass Metric Problem in Nearest-Neighbor Classification," Pattern Recognition, 23, 1291 -1297.
27. Ripley, B. D. (1996), Pattern Recognition and Neural Networks, Cambridge. U.K.: Cambridge University Press.
28. Schapire, R. E. (1990), "The Strength of Weak Learnability." Machine Learning. 5, 197-227.
29. Short, R., and Fukunaga, K. (1981), "The Optimal Distance Measure for Nearest-Neighbor Classification," IEEE Transactions on Information Theory, 27, 655-665.
30. Stone, C. J. (1980), "Optimal Rates of Convergence for Nonparametric Estimators," The Annals of Statistics, 8, 1348-1360.