Unconditional lower bounds for learning intersections of halfspaces

Machine Learning

Volumn 69, Issue 2-3, 2007, Pages 97-114

  Klivans, Adam R ^a   Sherstov, Alexander A ^a  

^a University of Texas at Austin   (United States)

Author keywords

Halfspace learning; Harmonic sieve; Intersections of halfspaces; Lower bounds for learning; PAC learning; Polynomial threshold functions; Query learning; SQ learning; Statistical queries

Indexed keywords

ALGORITHMS; HARMONIC ANALYSIS; LEARNING SYSTEMS; POLYNOMIALS; QUERY PROCESSING; STATISTICAL METHODS;

HALFSPACE LEARNING; HARMONIC SIEVE; INTERSECTIONS OF HALFSPACES; LOWER BOUNDS FOR LEARNING; POLYNOMIAL THRESHOLD FUNCTION (PTF); QUERY LEARNING; SQ LEARNING; STATISTICAL QUERIES; UNIFORM DISTRIBUTION;

COMPUTATION THEORY;

EID: 35348904454     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-007-5010-1     Document Type: Conference Paper

References (26)

1. Alekhnovich, M., Braverman, M., Feldman, V., Klivans, A., & Pitassi, T. (2004). Learnability and automatizability. In Proceedings of the 45th annual symposium on foundations of computer science (FOCS).
2. Aspnes, J., Beigel, R., Furst, M. L., & Rudich, S. (1994). The expressive power of voting polynomials. Combinatorica, 14(2), 135-148.
3. Beigel, R., Reingold, N., & Spielman, D. A. (1995). PP is closed under intersection. Journal of Computer and System Sciences, 50(2), 191-202.
4. Blum, A., Frieze, A., Kannan, R., & Vempala, S. (1997). A polynomial time algorithm for learning noisy linear threshold functions. Algorithmica, 22(1/2), 35-52.
5. Blum, A., Furst, M., Jackson, J., Kearns, M., Mansour, Y., & Rudich, S. (1994). Weakly learning DNF and characterizing statistical query learning using Fourier analysis. In Proceedings of the 26th annual ACM symposium on theory of computing (STOC) (pp. 253-262). New York: ACM.
6. Blum, A., Kalai, A., & Wasserman, H. (2003). Noise-tolerant learning, the parity problem, and the statistical query model. Journal of the ACM, 50(4), 506-519.
7. Bruck, J. (1990). Harmonic analysis of polynomial threshold functions. SIAM Journal on Discrete Mathematics, 3(2), 168-177.
8. Bshouty, N. H., & Tamon, C. (1996). On the Fourier spectrum of monotone functions. Journal of the ACM, 43(4), 747-770.
9. Feldman, V., Gopalan, P., Khot, S., & Ponnuswami, A. K. (2006). New results for learning noisy parities and halfspaces. In Proceedings of the 47th annual symposium on foundations of computer science (FOCS) (pp. 563-574).
10. Jackson, J. C. (1995). The harmonic sieve: a novel application of Fourier analysis to machine learning theory and practice. PhD thesis, Carnegie Mellon University.
11. Kearns, M. (1993). Efficient noise-tolerant learning from statistical queries. In Proceedings of the 25th annual ACM symposium on theory of computing (STOC) (pp. 392-401). New York: ACM.
12. Klivans, A., O'Donnell, R., & Servedio, R. (2004). Learning intersections and thresholds of halfspaces. Journal of Computer and System Sciences, 68(4), 808-840.
13. Klivans, A., & Servedio, R. (2004). Learning intersections of halfspaces with a margin. In Proceedings of the 17th annual conference on learning theory (pp. 348-362).
14. Klivans, A. R., & Sherstov, A. A. (2006). Cryptographic hardness for learning intersections of halfspaces. In Proceedings of the 47th annual symposium on foundations of computer science (FOCS) (pp. 553-562), October 2006.
15. Krause, M., & Pudlák, P. (1997). On the computational power of depth-2 circuits with threshold and modulo gates. Theoretical Computer Science, 174(1-2), 137-156.
16. Kushilevitz, E., & Mansour, Y. (1993). Learning decision trees using the Fourier spectrum. SIAM Journal on Computing, 22(6), 1331-1348.
17. Kwek, S., & Pitt, L. (1998). PAC learning intersections of halfspaces with membership queries. Algorithmica, 22(1/2), 53-75.
18. Macwilliams, F. J., & Sloane, N. J. A. (1977). The theory of error correcting codes. Amsterdam: North-Holland.
19. Minsky, M. L., & Papert, S. A. (1988). Perceptrons: expanded edition. Cambridge: MIT.
20. O'Donnell, R. (2003). Computational applications of noise sensitivity. PhD thesis, Massachusetts Institute of Technology.
21. O'Donnell, R., & Servedio, R. A. (2003). New degree bounds for polynomial threshold functions. In Proceedings of the 25th annual ACM symposium on theory of computing (STOC) (pp. 325-334). New York: ACM.
22. O'Neil, P. (1971). Hyperplane cuts of an n-cube. Discrete Mathematics, 1(2), 193-195.
23. Saks, M. E. (1993). Slicing the hypercube. Surveys in Combinatorics, 1993, 211-255.
24. Valiant, L. G. (1984). A theory of the learnable. Communications of the ACM, 27(11), 1134-1142.
25. Vempala, S. (1997). A random sampling based algorithm for learning the intersection of halfspaces. In Proceedings of the 38th annual symposium on foundations of computer science (FOCS) (pp. 508-513).
26. Yang, K. (2005). New lower bounds for statistical query learning. Journal of Computer and System Sciences, 70(4), 485-509.