A Neyman-Pearson approach to statistical learning

IEEE Transactions on Information Theory

Volumn 51, Issue 11, 2005, Pages 3806-3819

  Scott, Clayton ^a   Nowak, Robert ^b  

^a Rice University   (United States)

^b University of Wisconsin Madison   (United States)

Author keywords

Generalization error bounds; Neyman Pearson (NP) classification; Statistical learning theory

Indexed keywords

CONVERGENCE OF NUMERICAL METHODS; DECISION THEORY; ERROR ANALYSIS; PROBABILITY; STATISTICAL METHODS; THEOREM PROVING; TREES (MATHEMATICS);

GENERALIZATION ERROR BOUNDS; NEYMAN-PEARSON (NP) CLASSIFICATION; STATISTICAL LEARNING THEORY;

LEARNING SYSTEMS;

EID: 27744553952     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2005.856955     Document Type: Article

References (38)

1. E. Lehmann, Testing Statistical Hypotheses. New York: Wiley, 1986.
2. H. V. Poor, An Introduction to Signal Detection and Estimation. New York: Springer-Verlag, 1988.
3. H. V. Trees, Detection, Estimation, and Modulation Theory: Part I. New York: Wiley, 2001.
4. E. L. Lehmann, "The Fisher, Neyman-Pearson theories of testing hypotheses: One theory or two?," J. Amer. Statist. Assoc., vol. 88, no. 424, pp. 1242-1249, Dec. 1993.
5. P. Sebastiani, E. Gussoni, I. S. Kohane, and M. Ramoni, "Statistical challenges in functional genomics," Statist. Sci., vol. 18, no. 1, pp. 3340, 2003.
6. S. Dudoit, J. Fridlyand, and T. P. Speed, "Comparison of discrimination methods for the classification of tumors using gene expression data," J. Amer. Statist. Assoc., vol. 97, no. 457, pp. 77-87, Mar. 2002.
7. B. Zadrozny, J. Langford, and N. Abe, "Cost sensitive learning by cost-proportionate example weighting," in Proc. 3rd Int. Conf. Data Mining, Melbourne, FL, 2003.
8. P. Domingos, "Metacost: A general method for making classifiers cost sensitive," in Proc. 5th Int. Conf. Knowledge Discovery and Data Mining, San Diego, CA, 1999, pp. 155-164.
9. C. Elkan, "The foundations of cost-sensitive learning," in Proc. 17th Int. Joint Conf. Artificial Intelligence, Seattle, WA, 2001, pp. 973-978.
10. D. Margineantu, "Class probability estimation and cost-sensitive classification decisions," in Proc. 13th European Conf. Machine Learning, Helsinki, Finland, 2002, pp. 270-281.
11. D. Casasent and X.-W. Chen, "Radial basis function neural networks for nonlinear Fisher discrimination and Neyman-Pearson classification," Neural Netw., vol. 16, pp. 529-535, 2003.
12. A. Cannon, J. Howse, D. Hush, and C. Scovel. (2002) Learning With the Neyman-Pearson and min-max Criteria. Los Alamos National Laboratory. [Online]. Available: http://www.c3.lanl.gov/kelly/ml/pubs/2002minmax/ paper.pdf
13. A. Cannon, J. Howse, D. Hush, and C. Scovel. (2003) Simple Classifiers. Los Alamos National Laboratory. [Online]. Available: http://www.c3.lanl. gov/ml/pubs/2003sclassifiers/abstract.shtml
14. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
15. V. Vapnik and C. Chervonenkis, "On the uniform convergence of relative frequencies of events to their probabilities," Theory Probab. Its Applic., vol. 16, no. 2, pp. 264-280, 1971.
16. V. Vapnik and C. Chervonenkis, Theory of Pattern Recognition (in Russian). Moscow, U.S.S.R.: Nauka, 1971.
17. V. Koltchinskii, "Rademacher penalties and structural risk minimization," IEEE Trans. Inf. Theory, vol. 47, no. 5, pp. 1902-1914, Jul. 2001.
18. P. Bartlett, S. Boucheron, and G. Lugosi, "Model selection and error estimation," Mach. Learn., vol. 48, pp. 85-113, 2002.
19. V. Vapnik, Estimation of Dependencies Based on Empirical Data. New York: Springer-Verlag, 1982.
20. G. Lugosi and K. Zeger, "Concept learning using complexity regularization," IEEE Trans. Inf. Theory, vol. 42, no. 1, pp. 48-54, Jan. 1996.
21. S. Boucheron, O. Bousquet, and G. Lugosi. (2004) Theory of Classification: A Survey of Recent Advances. [Online]. Available: http://www.econ.upf.es/lugosi/
22. A. B. Tsybakov, "Optimal aggregation of classifiers in statistical learning," Ann. Statist., vol. 32, no. 1, pp. 135-166, 2004.
23. C. Scott and R. Nowak, "On the adaptive properties of decision trees," in Advances in Neural Information Processing Systems 17, L. K. Saul, Y. Weiss, and L. Bottou, Eds. Cambridge, MA: MIT Press, 2005, pp. 1225-1232.
24. I. Steinwart and C. Scovel, "Fast rates to bayes for kernel machines," in Advances in Neural Information Processing Systems 17, L. K. Saul, Y. Weiss, and L. Bottou, Eds. Cambridge, MA: MIT Press, 2005, pp. 1337-1344.
25. G. Blanchard, G. Lugosi, and N. Vayatis, "On the rate of convergence of regularized boosting classifiers," J. Mach. Learn. Res., vol. 4, pp. 861-894, 2003.
26. A. B. Tsybakov and S. A. van de Geer. (2004) Square Root Penalty: Adaptation to the Margin in Classification and in Edge Estimation. [Online]. Available: http://www.proba.jussieu.fr/pageperso/tsybakov/ tsybakov.html
27. L. Devroye, "Any discrimination rule can have an arbitrarily bad probability of error for finite sample size," IEEE Trans. Pattern Anal. Mach. Intell., vol. PAMI-4, pp. 154-157, Mar. 1982.
28. C. Scott and R. Nowak. Minimax Optimal Classification With Dyadic Decision Trees. Rice Univ., Houston, TX. [Online]. Available: http://www. stat.rice.edu/~cscott
29. C. Scott and R. Nowak, "Dyadic classification trees via structural risk minimization," in Advances in Neural Information Processing Systems 15, S. Becker, S. Thrun, and K. Obermayer, Eds. Cambridge, MA, 2003.
30. C. Scott and R. Nowak, "Near-minimax optimal classification with dyadic classification trees," in Advances in Neural Information Processing Systems 16, S. Thrun, L. Saul, and B. Schölkopf, Eds. Cambridge, MA: MIT Press, 2004.
31. R. A. DeVore, "Nonlinear approximation," Acta Numer., vol. 7, pp. 51-150, 1998.
32. A. Cohen, W. Dahmen, I. Daubechies, and R. A. DeVore, "Tree approximation and optimal encoding," Appl. Comput. Harmonic Anal., vol. 11, no. 2, pp. 192-226, 2001.
33. D. Donoho, "Wedgelets: Nearly minimax estimation of edges," Ann. Statist., vol. 27, pp. 859-897, 1999.
34. G. Blanchard, C. Schäfer, and Y. Rozenholc, "Oracle bounds and exact algorithm for dyadic classification trees," in Learning Theory: Proc. 17th Annu. Conf. Learning Theory, COLT 2004, J. Shawe-Taylor and Y. Singer, Eds. Heidelberg, Germany: Springer-Verlag, 2004, pp. 378-392.
35. D. Donoho, "CART and best-ortho-basis selection: A connection," Ann. Statist., vol. 25, pp. 1870-1911, 1997.
36. R. Durrett, Probability: Theory and Examples. Pacific Grove, CA: Wadsworth & Brooks/Cole, 1991.
37. T. Hagerup and C. Rüb, "A guided tour of Chernoff bounds," Inf. Process. Lett., vol. 33, no. 6, pp. 305-308, 1990.
38. G. Blanchard, C. Schäfer, Y. Rozenholc, and K.-R. Müller, "Optimal Dyadie Decision Trees," preprint, 2005.