Minimax bounds for active learning

IEEE Transactions on Information Theory

Volumn 54, Issue 5, 2008, Pages 2339-2353

  Castro, Rui M ^a   Nowak, Robert D ^a  

^a University of Wisconsin Madison   (United States)

Author keywords

Active learning; Adaptive sampling; Minimax lower bounds; Statistical learning theory

Indexed keywords

FEATURE EXTRACTION; INFORMATION THEORY; SAMPLING; STATISTICAL METHODS;

ACTIVE LEARNING; MINIMAX ANALYSIS; PASSIVE LEARNING;

LEARNING ALGORITHMS;

EID: 43749112531     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2008.920189     Document Type: Article

References (31)

1. D. J. C. Mackay, "Information-based objective functions for active data selection," Neural Comput. vol. 4, pp. 698-714, 1991.
2. D. Cohn, Z. Ghahramani, and M. Jordan, "Active learning with statistical models," J. Artif. Intell. Res., pp. 129-145, 1996.
3. Y. Freund, H. S. Seung, E. Shamir, and N. Tishby, "Selective sampling using the query by committee algorithm," March. Learn., vol. 28, no. 2-3, pp. 133-168, Aug. 1997.
4. N. Cesa-Bianchi, A. Conconi, and C. Gentile, "Learning probabilistic linear-threshold classifiers via selective sampling," in Lecture Notes in Artificial Intelligence. Berlin, Germany: Springer-Verlag, 2003, vol. 2777.
5. G. Blanchard and D. Geman, "Hierarchical testing designs for pattern recognition," Am. Statist., vol. 33, no. 3, pp. 1155-1202, 2005.
6. S. Dasgupta, A. Kalai, and C. Monteleoni, "Analysis of perceptron-based active learning," in Proc. 18th Annu. Conf. Learn. Theory, Bertinoro, Italy, Jun. 27-30, 2005, pp. 249-263.
7. S. Dasgupta, "Coarse sample complexity bounds for active learning," in Advances in Neural Information Processing (NIPS , Cambridge, MA: MIT Press, 2005.
8. S. Dasgupta, "Analysis of a greedy active learning strategy," in Advances in Neural Information Processing (NIPS). Cambridge, MA: MIT Press, 2004.
9. N. Balcan, A. Beygelzimer, and J. Langford, "Agostic active learning," in Proc. 23rd Int. Conf. Mach. Learn., Pittsburgh, PA, Jun. 2006, pp. 65-72.
10. R. Castro, R. Willett, and R. Nowark, "Faster rates in regression via active learning," in Advances in Neural Information Processing Systems (NIPS). Cambridge, MA: MIT Press, 2005 [Online]. Available: http://homepages.cae.wisc.edu/~rcastro/ECE-05-3.pdf.
11. M. Kääriäinen, "On active learning in the nonrealizable case," presented at the NIPS Workshop Found. Active Learn., Whistler, BC, Canada, 2005.
12. M. Horstein, "Sequential decoding using noiseless feedback," IEEE Trans. Inf. Theory, vol. IT-9, no. 3, pp. 136-143, Jul. 1963.
13. M. V. Burnashev and K. S. Zigangirov, "An interval estimation problem for controlled observations," Problems Inf. Transm., vol. 10, pp. 223-231, 1974.
14. P. Hall and I. Molchanov, "Sequential methods for design-adaptive estimation of discontinuities in regression curves and surfaces," Ann. Statist., vol. 31, no. 3, pp. 921-941, 2003.
15. G. Golubev and B. Levit, "Sequential recovery of analytical periodic edges in the binary image models," Math. Methods Statist. vol. 12, pp. 95-115, 2003.
16. B. Bryan, J. Schneider, R. C. Nichol, C. J. Miller, C. R. Genovese, and L. Wasserman, "Active learning for identifying function threshold boundaries," in Advances in Neural Information Processing (NIPS . Cambridge, MA: MTT Press, 2005.
17. A. Tsybakov, "Optimal aggegation of classifier in statistical learning," Ann Statist., vol. 32, no. 1, pp. 135-166, 2004.
18. G. Tur, D. Hakkani-Tür, and R. E. Schapire, "Combining active and semi-supervised learning for spoken language understanding," Speech Commun. vol. 45, pp. 171-186, 2005.
19. S. K. Thompson and G. A. F. Seber, Adaptive Sampling. New York: Wiley, 1996.
20. M. Ghosh, N. Mukhopadhyay, and P. Sen. Sequential Estimation. New York: Wiley, 1997.
21. P. K. Sen, Sequential Nonparametrics. New York: Wiley, 1981.
22. E. Baum, "Neural net algorithms that learn is polynomial time for examples and queries," IEEE Trans. Neural Netw., vol. 2, pp. 5-19, Sep. 1991.
23. L. Cavalier, "Nonparametric estimation of regression level sets," Statistics, vol. 29, pp. 131-160, 1997.
24. A. B. Tsybakov, "On nonparametric estimation of density level sets," Ann. Statist. vol. 25, pp. 948-969, 1997.
25. A. B. Tsybakov, Introductionà L'estimation Non-Paramétrique, ser. Mathématiques et Applications, 41. New York: Springer-Verlag, 2004.
26. R. Castro and R. Nowak, "Upper and lower bounds for active learning," in Proc. 44th Annu. Allerton Conf. Commun. Control Comput. 2006, pp. 255-234.
27. A. Korostelev and A. Tsybakov, "Minimax theory of image reconstruction." in Lecture Notes in Statistics. Berlin, Germany: Springer-Verlag, 1993.
28. D. Donono, "Wedgelets: Nearly minimax estimation of edges," Ann. Statist., vol. 27, pp. 859-897, 1999.
29. A. P. Korostelev, "On minimnx rates of convergence in image models under sequential design," Statist. Probab. Lett., vol. 43, pp. 369-375, 1999.
30. A. Korostelev and J.-C. Kim, "Rates of convergence for the sup-norm risk in image models under sequential designs," Statist. Probab. lett. vol. 46, pp. 391-399, 2000.
31. C. de Boor, "The error in polynomial tensor-product, and Chung-Yao, interpolation," in Surface Fitting and Multiresolution, Methods, A. LeMéhauté, C. Rabut, and L. Schumaker, Eds. Nashville, TN: Vanderb in Univ. Press, 1997, pp. 35-50.