Active and passive learning of linear separators under log-concave distributions

Journal of Machine Learning Research

Volumn 30, Issue , 2013, Pages 288-316

  Balcan, Maria Florina ^a   Long, Philip M ^b  

^a GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

^b MICROSOFT   (United States)

Author keywords

Active learning; Agnostic learning; ERM; Nearly log concave distributions; PAC learning; Tsybakov low noise condition

Indexed keywords

POLYNOMIAL APPROXIMATION;

ACTIVE LEARNING; AGNOSTIC LEARNING; ERM; LOG CONCAVES; LOW NOISE; PAC LEARNING;

ALGORITHMS;

EID: 84898034026     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (56)

1. K.S. Alexander. Rates of growth and sample moduli for weighted empirical processes indexed by sets. Probability Theory and Related Fields, 1987.
2. N. Alon. A non-linear lower bound for planar epsilon-nets. FOCS, pages 341-346, 2010.
3. D. Applegate and R. Kannan. Sampling and integration of near log-concave functions. In STOC, 1991.
4. P. Assouad. Plongements lipschitziens dans. R. Bull. Soc. Math. France, 111(4): 429-448, 1983.
5. M. F. Balcan, A. Beygelzimer, and J. Langford. Agnostic active learning. In ICML, 2006.
6. M.-F. Balcan, A. Broder, and T. Zhang. Margin based active learning. In COLT, 2007.
7. M.-F. Balcan, S. Hanneke, and J. Wortman. The true sample complexity of active learning. In COLT, 2008.
8. P. L. Bartlett, O. Bousquet, and S. Mendelson. Local Rademacher complexities. Annals of Statistics, 2005.
9. A. Beygelzimer, D. Hsu, J. Langford, and T. Zhang. Agnostic active learning without constraints. In NIPS, 2010.
10. A. Blumer, A. Ehrenfeucht, D. Haussler, and M. K. Warmuth. Learnability and the Vapnik-Chervonenkis dimension. JACM, 36(4): 929-965, 1989.
11. S. Boucheron, O. Bousquet, and G. Lugosi. Theory of classification: a survey of recent advances. ESAIM: Probability and Statistics, 2005.
12. N. H. Bshouty, Y. Li, and P. M. Long. Using the doubling dimension to analyze the generalization of learning algorithms. JCSS, 2009.
13. C. Caramanis and S. Mannor. An inequality for nearly log-concave distributions with applications to learning. IEEE Transactions on Information Theory, 2007.
14. R. Castro and R. Nowak. Minimax bounds for active learning. In COLT, 2007.
15. N. Cesa-Bianchi, C. Gentile, and L. Zaniboni. Learning noisy linear classifiers via adaptive and selective sampling. Machine Learning, 2010.
16. K. L. Clarkson and K. Varadarajan. Improved approximation algorithms for geometric set cover. Discrete Comput. Geom., 37(1): 43-58, 2007.
17. D. Cohn, L. Atlas, and R. Ladner. Improving generalization with active learning. In ICML, 1994.
18. S. Dasgupta. Coarse sample complexity bounds for active learning. In NIPS, Volume 18, 2005.
19. S. Dasgupta. Active learning. Encyclopedia of Machine Learning, 2011.
20. S. Dasgupta, A. Kalai, and C. Monteleoni. Analysis of perceptron-based active learning. In COLT, 2005.
21. S. Dasgupta, D.J. Hsu, and C. Monteleoni. A general agnostic active learning algorithm. Advances in Neural Information Processing Systems, 20, 2007.
22. O. Dekel, C. Gentile, and K. Sridharan. Selective sampling and active learning from single and multiple teachers. Journal of Machine Learning Research, 2012.
23. A. Ehrenfeucht, D. Haussler, M. Kearns, and L. G. Valiant. A general lower bound on the number of examples needed for learning. Information and Computation, 1989.
24. Y. Freund, H.S. Seung, E. Shamir, and N. Tishby. Selective sampling using the query by committee algorithm. Machine Learning, 28(2-3): 133-168, 1997.
25. E. J. Friedman. Active learning for smooth problems. In COLT, 2009.
26. R. J. Gardner. The Brunn-Minkowski inequality. Bull. Amer. Math. Soc., 2002.
27. E. Giné and V. Koltchinskii. Concentration inequalities and asymptotic results for ratio type empirical processes. The Annals of Probability, 34(3): 1143-1216, 2006.
28. A. Gonen, S. Sabato, and S. Shalev-Shwartz. Efficient pool-based active learning of halfspaces. In ICML, 2013.
29. S. Hanneke. A bound on the label complexity of agnostic active learning. In ICML, 2007.
30. S. Hanneke. Rates of convergence in active learning. The Annals of Statistics, 39(1): 333-361, 2011.
31. S. Hanneke and L. Yang. Surrogate losses in passive and active learning, 2012. http://arXiv.org/abs/1207.3772.
32. D. Haussler, N. Littlestone, and M. K. Warmuth. Predicting {0, 1}-functions on randomly drawn points. Information and Computation, 115(2): 129-161, 1994.
33. David Haussler and Emo Welzl. Epsilon nets and simplex range queries. Disc. Comp. Geometry, 2: 127-151, 1987.
34. A. Kalai, A. Klivans, Y. Mansour, and R. Servedio. Agnostically learning halfspaces. In Proceedings of the 46th Annual Symposium on the Foundations of Computer Science (FOCS), 2005.
35. M. Kearns and U. Vazirani. An Introduction to Computational Learning Theory. MIT Press, 1994.
36. A. R. Klivans, P. M. Long, and R. A. Servedio. Learning halfspaces with malicious noise. JMLR, 2009a.
37. A. R. Klivans, P. M. Long, and A. Tang. Baum's algorithm learns intersections of halfspaces with respect to log-concave distributions. In RANDOM, 2009b.
38. V. Koltchinskii. Rademacher complexities and bounding the excess risk in active learning. Journal of Machine Learning Research, 11: 2457-2485, 2010.
39. J. Komlós, J. Pach, and G. Woeginger. Almost tight bounds on epsilon-nets. Discrete and Computational Geometry, 7: 163-173, 1992.
40. S. R. Kulkarni, S. K. Mitter, and J. N. Tsitsiklis. Active learning using arbitrary binary valued queries. Machine Learning, 1993.
41. P. M. Long. On the sample complexity of PAC learning halfspaces against the uniform distribution. IEEE Transactions on Neural Networks, 6(6): 1556-1559, 1995.
42. P. M. Long. An upper bound on the sample complexity of PAC learning halfspaces with respect to the uniform distribution. Information Processing Letters, 2003.
43. L. Lovasz and S. Vempala. The geometry of logconcave functions and sampling algorithms. Random Structures and Algorithms, 2007.
44. E. Mammen and A.B. Tsybakov. Smooth discrimination analysis. The Annals of Statistics, 27: 1808-1829, 1999.
45. P. Massart and E. Nedelec. Risk bounds for statistical learning. The Annals of Statistics, 2006.
46. S. Mendelson. Estimating the performance of kernel classes. Journal of Machine Learning Research, 4: 759-771, 2003.
47. R. Nowak. The Geometry of Generalized Binary Search. IEEE Transactions on Information Theory, 2011.
48. J. Pach and P.K. Agarwal. Combinatorial Geometry. John Wiley and Sons, 1995.
49. M. Raginsky and A. Rakhlin. Lower bounds for passive and active learning. In NIPS, 2011.
50. L.G. Valiant. A theory of the learnable. Communications of the ACM, 27(11): 1134-1142, 1984.
51. S. van de Geer. Empirical processes in M-estimation. Cambridge Series in Statistical and Probabilistic Methods, 2000.
52. A. van der Vaart and J. A. Wellner. Weak Convergence and Empirical Processes With Applications to Statistics. Springer, 1996.
53. V. Vapnik and A. Chervonenkis. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications, 16(2): 264-280, 1971.
54. V. N. Vapnik. Estimation of Dependencies based on Empirical Data. Springer Verlag, 1982.
55. V. N. Vapnik. Statistical Learning Theory. John Wiley and Sons, 1998.
56. S. Vempala. A random-sampling-based algorithm for learning intersections of halfspaces. JACM, 57(6), 2010.