The power of localization for efficiently learning linear separators with noise

Proceedings of the Annual ACM Symposium on Theory of Computing

Volumn , Issue , 2014, Pages 449-458

  Awasthi, Pranjal ^a   Balcan, Maria Florina ^b   Long, Philip M ^c  

^a PRINCETON UNIVERSITY   (United States)

^b GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

^c MICROSOFT   (United States)

Author keywords

Adversarial label noise; Malicious noise; Noise tolerant learning; Passive and active learning

Indexed keywords

ARTIFICIAL INTELLIGENCE; POLYNOMIALS; PROBABILITY DISTRIBUTIONS;

ACTIVE LEARNING; ACTIVE-LEARNING ALGORITHM; ADVERSARIAL LABEL NOISE; COMPUTATIONALLY EFFICIENT; LOCALIZATION TECHNIQUE; MALICIOUS NOISE; NOISE-TOLERANT LEARNING; POLYNOMIAL-TIME ALGORITHMS;

LEARNING ALGORITHMS;

EID: 84904299163     PISSN: 07378017     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2591796.2591839     Document Type: Conference Paper

References (44)

1. S. Arora, L. Babai, J. Stern, and Z. Sweedyk. The hardness of approximate optima in lattices, codes, and systems of linear equations. In Proceedings of the 1993 IEEE 34th Annual Foundations of Computer Science, 1993.
2. P. Awasthi, M.-F. Balcan, and P. M. Long. The power of localization for efficiently learning linear separators with noise, 2014. Arxiv, 1307.8371v7.
3. M.-F. Balcan, A. Beygelzimer, and J. Langford. Agnostic active learning. In ICML, 2006.
4. M.-F. Balcan, A. Broder, and T. Zhang. Margin based active learning. In COLT, 2007.
5. M.-F. Balcan and V. Feldman. Statistical active learning algorithms. NIPS, 2013.
6. M.-F. Balcan and S. Hanneke. Robust interactive learning. In COLT, 2012.
7. M.-F. Balcan, S. Hanneke, and J. Wortman. The true sample complexity of active learning. In COLT, 2008.
8. M.-F. Balcan and P. M. Long. Active and passive learning of linear separators under log-concave distributions. In Conference on Learning Theory, 2013.
9. P. L. Bartlett, O. Bousquet, and S. Mendelson. Local Rademacher complexities. Annals of Statistics, 33(4):1497-1537, 2005.
10. E. B. Baum. The perceptron algorithm is fast for nonmalicious distributions. Neural Computation, 2:248-260, 1990.
11. A. Beygelzimer, D. Hsu, J. Langford, and T. Zhang. Agnostic active learning without constraints. In NIPS, 2010.
12. D. Bienstock and A. Michalka. Polynomial solvability of variants of the trust-region subproblem, 2013. Optimization Online.
13. A. Blum, A. Frieze, R. Kannan, and S. Vempala. A polynomial time algorithm for learning noisy linear threshold functions. Algorithmica, 22(1/2):35-52, 1997.
14. A. Blum, M. L. Furst, M. J. Kearns, and R. J. Lipton. Cryptographic primitives based on hard learning problems. In Proceedings of the 13th Annual International Cryptology Conference on Advances in Cryptology, 1994.
15. S. Boucheron, O. Bousquet, and G. Lugosi. Theory of classification: a survey of recent advances. ESAIM: Probability and Statistics, 9:9:323-375, 2005.
16. N. H. Bshouty, Y. Li, and P. M. Long. Using the doubling dimension to analyze the generalization of learning algorithms. JCSS, 2009.
17. R. Castro and R. Nowak. Minimax bounds for active learning. In COLT, 2007.
18. D. Cohn, L. Atlas, and R. Ladner. Improving generalization with active learning. Machine Learning, 15(2), 1994.
19. N. Cristianini and J. Shawe-Taylor. An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, 2000.
20. S. Dasgupta. Coarse sample complexity bounds for active learning. In NIPS, volume 18, 2005.
21. S. Dasgupta. Active learning. Encyclopedia of Machine Learning, 2011.
22. S. Dasgupta, D. Hsu, and C. Monteleoni. A general agnostic active learning algorithm. NIPS, 20, 2007.
23. O. Dekel, C. Gentile, and K. Sridharan. Selective sampling and active learning from single and multiple teachers. JMLR, 2012.
24. Y. Freund, H. Seung, E. Shamir, and N. Tishby. Selective sampling using the query by committee algorithm. Machine Learning, 28(2-3):133-168, 1997.
25. M. R. Garey and D. S. Johnson. Computers and Intractability; A Guide to the Theory of NP-Completeness. 1990.
26. A. Gupta, M. Hardt, A. Roth, and J. Ullman. Privately releasing conjunctions and the statistical query barrier. In Proceedings of the 43rd annual ACM symposium on Theory of computing, 2011.
27. V. Guruswami and P. Raghavendra. Hardness of learning halfspaces with noise. In Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science, 2006.
28. S. Hanneke. A bound on the label complexity of agnostic active learning. In ICML, 2007.
29. S. Hanneke. Rates of convergence in active learning. The Annals of Statistics, 39(1):333-361, 2011.
30. D. S. Johnson and F. Preparata. The densest hemisphere problem. Theoretical Computer Science, 6(1):93-107, 1978.
31. A. T. Kalai, A. R. Klivans, Y. Mansour, and R. A. Servedio. Agnostically learning halfspaces. In Proceedings of the 46th Annual IEEE Symposium on Foundations of Computer Science, 2005.
32. M. Kearns and M. Li. Learning in the presence of malicious errors. In Proceedings of the twentieth annual ACM symposium on Theory of computing, 1988.
33. M. Kearns and U. Vazirani. An introduction to computational learning theory. MIT Press, Cambridge, MA, 1994.
34. M. J. Kearns, R. E. Schapire, and L. M. Sellie. Toward efficient agnostic learning. Mach. Learn., 17(2-3), Nov. 1994.
35. A. R. Klivans, P. M. Long, and R. A. Servedio. Learning halfspaces with malicious noise. Journal of Machine Learning Research, 10, 2009.
36. V. Koltchinskii. Rademacher complexities and bounding the excess risk in active learning. Journal of Machine Learning Research, 11:2457-2485, 2010.
37. C. Monteleoni. Efficient algorithms for general active learning. In Proceedings of the 19th annual conference on Learning Theory, 2006.
38. M. Raginsky and A. Rakhlin. Lower bounds for passive and active learning. In NIPS, 2011.
39. O. Regev. On lattices, learning with errors, random linear codes, and cryptography. In Proceedings of the thirty-seventh annual ACM symposium on Theory of computing, 2005.
40. J. Sturm and S. Zhang. On cones of nonnegative quadratic functions. Mathematics of Operations Research, 28:246-267, 2003.
41. L. G. Valiant. Learning disjunction of conjunctions. In Proceedings of the 9th International Joint Conference on Artificial intelligence, 1985.
42. V. Vapnik. Statistical Learning Theory. Wiley-Interscience, 1998.
43. L. Wang. Smoothness, Disagreement Coefficient, and the Label Complexity of Agnostic Active Learning. JMLR, 2011.
44. T. Zhang. Information theoretical upper and lower bounds for statistical estimation. IEEE Transactions on Information Theory, 52(4):1307-1321, 2006.