Agnostically learning halfspaces

SIAM Journal on Computing

Volumn 37, Issue 6, 2007, Pages 1777-1805

  Kalai, Adam Tauman ^a   Klivans, Adam R ^b   Mansour, Yishay ^c   Servedio, Rocco A ^d  

^a Georgia Institute of Technology   (United States)

^b University of Texas at Austin   (United States)

^c TEL AVIV UNIVERSITY   (Israel)

^d Columbia University ^*  (United States)

Author keywords

Agnostic learning; Fourier; Halfspaces

Indexed keywords

AGNOSTIC LEARNING; COMPUTATIONAL LEARNING THEORIES; COMPUTATIONALLY EFFICIENT; ERROR RATES; FOURIER; FOURIER ALGORITHMS; HALFSPACES; OPEN PROBLEMS; POLYNOMIAL REGRESSIONS; RUNNING TIMES; UNIFORM DISTRIBUTIONS;

ALGORITHMS; BOOLEAN FUNCTIONS; EDUCATION; LEARNING SYSTEMS;

LEARNING ALGORITHMS;

EID: 55249114173     PISSN: 00975397     EISSN: None     Source Type: Journal    
DOI: 10.1137/060649057     Document Type: Conference Paper

References (45)

1. E. BAUM, The Perceptron algorithm is fast for nonmalicious distributions, Neural Comput., 2 (1990), pp. 248-260.
2. E. B. BAUM AND Y-D. LYUU, The transition to perfect generalization in perceptrons, Neural Comput., 3 (1991), pp. 3S6-401.
3. A. BLUM, Machine learning: A tour through some favorite results, directions, and open problems, FOCS 2003 tutorial slides; available online from http://www- 2.cs.cmu.edu/̃avrim/Talks/FOCS03/tutorial.ppt (2003).
4. A. BLUM, A. FRIEZE, R. KANNAN, AND S. VEMPALA, A polynomial time algorithm for learning noisy linear threshold functions, Algorithmica, 22 (1997), pp. 35-52.
5. A. BLUM, M. L. FURST, J. JACKSON, M. J. KEARNS, Y. MANSOUR, AND S. RUDICH, Weakly learning dnf and characterizing statistical query learning using Fourier analysis, in Proceedings of the 26th Annual ACM Symposium on Theory of Computing, 1994, pp. 253-262.
6. A. BLUM, A. KALAI, AND H. WASSERMAN, Noise-tolerant learning, the parity problem, and the statistical query model, J. ACM, 50 (2003), pp. 506-519.
7. S. BONAN AND D. CLARK, Estimates of the Hermite and the Freud polynomials, J. Approx. Theory, 63 (1990), pp. 210-224.
8. N. BSHOUTY AND C. TAMON, On the Fourier spectrum of monotone functions, J. ACM, 43 (1996), pp. 747-770.
9. K. L. CLARKSON, Subgradient and sampling algorithms for l1 regression, in Proceedings of the Sixteenth Annual ACM-SIAM Symposium on Discrete Algorithms, 2005, pp. 257-266.
10. S. DECATUR, Statistical queries and faulty PAC oracles, in Proceedings of the Sixth Annual ACM Conference on Computational Learning Theory, 1993, pp. 262-268.
11. T. ERICSON AND V. ZINOVIEV, Codes on Euclidean Spheres, North-Holland Math. Library 63, North-Holland, Amsterdam, 2001.
12. V. FELDMAN, Optimal hardness results for maximizing agreements with monomials, in Proceedings of the 21st Annual IEEE Conference on Computational Complexity, 2006, pp. 226-236.
13. V. FELDMAN, P. GOPALAN, S. KHOT, AND A. PONNUSWAMI, New results for learning noisy parities and half spaces, in Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science, 2006, pp. 563-574.
14. Y. FREUND AND R. SCHAPIRE, A decision-theoretic generalization of on-line learning and an application to boosting, J. Comput. System Sci., 55 (1997), pp. 119-139.
15. Y. FREUND AND R. SCHAPIRE, Large margin classification using the Perceptron algorithm, in Proceedings of the Eleventh Annual ACM Conference on Computational Learning Theory, 1998, pp. 209-217.
16. Y. FREUND AND R. SCHAPIRE, A short introduction to boosting, J. Japanese Soc. Artificial Intelligence, 14 (1999), pp. 771-780.
17. S. GOLDMAN, M. KEARNS, AND R. SCHAPIRE, On the sample complexity of weakly learning, Inform. Comput., 117 (1995), pp. 276-287.
18. V. GURUSWAMI AND P. RAGHAVENDRA, Hardness of learning halfspaces with noise, in Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science, 2006, pp. 543-552.
19. A. HAJNAL, W. MAASS, P. PUDLAK, M. SZEGEDY, AND G. TURAN, Threshold circuits of bounded depth, J. Comput. System Sci., 46 (1993), pp. 129-154.
20. J. JACKSON, The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice, Ph.D. thesis, Carnegie Mellon University, Pittsburgh, PA, 1995.
21. J. JACKSON, An efficient membership-query algorithm for learning DNF with respect to the uniform, distribution, J. Comput. System Sci., 55 (1997), pp. 414-440.
22. 0, in Proceedings of the 34th Annual ACM Symposium on Theory of Computing, 2002, pp. 776-784.
23. M. KEARNS, Efficient noise-tolerant learning from statistical queries, J. ACM, 45 (1998), pp. 983-1006.
24. M. KEARNS AND M. LI, Learning in the presence of malicious errors, SIAM J. Comput., 22 (1993), pp. 807-837.
25. M. KEARNS, R. SCHAPIRE, AND L. SELLIE, Toward efficient agnostic learning, Mach. Learn., 17 (1994), pp. 115-141.
26. A. KLIVANS, R. O'DONNELL, AND R. SERVEDIO, Learning intersections and thresholds of half-spaces, J. Comput. System Sci., 68 (2004), pp. 808-840.
27. A. KLIVANS AND R. SERVEDIO, Boosting and hard-core sets, in Proceedings of the 40th Annual IEEE Symposium on Foundations of Computer Science, 1999, pp. 624-633.
28. A. R. KLIVANS AND A. A. SHERSTOV, Cryptographic hardness for learning intersections of half-spaces, in Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science, 2006, pp. 553-562.
29. W. LEE, P. BARTLETT, AND R. WILLIAMSON, Efficient agnostic learning of neural networks with bounded fan-in, IEEE Trans. Inform. Theory, 42 (1996), pp. 2118-2132.
30. W. S. LEE, P. L. BARTLETT, AND R. C. WILLIAMSON, On efficient agnostic learning of linear combinations of basis functions, in Proceedings of the Eighth Annual ACM Conference on Computational Learning Theory, 1995, pp. 369-376.
31. N. LINIAL, Y. MANSOUR, AND N. NISAN, Constant depth circuits, Fourier transform and learnability, J. Assoc. Comput. Mach., 40 (1993), pp. 607-620.
32. P. LONG, On the sample complexity of PAC learning halfspaces against the uniform distribution, IEEE Trans. Neural Networks, 6 (1995), pp. 1556-1559.
33. P. LONG, An upper bound on the sample complexity of pac learning halfspaces with respect to the uniform distribution, Inform. Process. Lett., 87 (2003), pp. 229-234.
34. L. LOVÁSZ AND S. VEMPALA, Log concave functions: Geometry and efficient sampling algorithms, in Proceedings of the 44th Annual IEEE Symposium on Foundations of Computer Science, 2003, pp. 650-659.
35. Y. MANSOUR AND M. PARNAS, Learning conjunctions with noise under product distributions, Inform. Process. Lett., 68 (1998), pp. 189-196.
36. N. NISAN AND M. SZEGEDY, On the degree of Boolean functions as real polynomials, in Proceedings of the 24th Annual ACM Symposium on Theory of Computing, 1992, pp. 462-467.
37. R. O'DONNELL AND R. SERVEDIO, New degree bounds for polynomial threshold functions, in Proceedings of the 35th Annual ACM Symposium on Theory of Computing, 2003, pp. 325-334.
38. R. PATURI, On the degree of polynomials that approximate symmetric Boolean functions, in Proceedings of the 24th Annual ACM Symposium on Theory of Computing, 1992, pp. 468-474.
39. O. REGEV, On lattices, learning with errors, random linear codes, and cryptography, in Proceedings of the 37th Annual ACM Symposium on Theory of Computing, 2005, pp. 84-93.
40. R. SERVEDIO, On PAC learning using Winnow, Perceptron, and a Perceptron-like algorithm, in Proceedings of the Twelfth Annual ACM Conference on Computational Learning Theory, 1999, pp. 296-307.
41. J. SHAWE-TAYLOR AND N. CRISTIANINI, An Introduction to Support Vector Machines, Cambridge University Press, Cambridge, UK, 2000.
42. G. SZEGÖ, Orthogonal Polynomials, Amer. Math. Soc. Colloq. Publ. 23, AMS, Providence, RI, 1989.
43. L. VALIANT, A theory of the learnable, Comm. ACM, 27 (1984), pp. 1134-1142.
44. L. VALIANT, Learning disjunctions of conjunctions, in Proceedings of the Ninth International Joint Conference on Artificial Intelligence, Morgan Kaufmann, San Francisco, 1985, pp. 560-566.
45. V. VAPNIK, Statistical Learning Theory, Wiley-Interscience, New York, 1998.