Agnostic learning of monomials by halfspaces is hard

SIAM Journal on Computing

Volumn 41, Issue 6, 2012, Pages 1558-1590

  Feldman, Vitaly ^a   Guruswami, Venkatesan ^b   Raghavendra, Prasad ^c   Wu, Yi ^a  

^a IBM ALMADEN RESEARCH CENTER   (United States)

^b Carnegie Mellon University ^*  (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Agnostic learning; Dictatorship tests; Hardness of learning; PCPs

Indexed keywords

ABSOLUTE VALUES; AGNOSTIC LEARNING; ARBITRARY CONSTANTS; CONCEPT CLASS; COVER PROBLEM; DECISION LISTS; DICTATORSHIP TESTS; HALF SPACES; HALF-SPACE; HARDNESS OF LEARNING; HARDNESS RESULT; HYPERCUBE; INAPPROXIMABILITY; INVARIANCE PRINCIPLE; LEARNING THEORY; LIST DECODING; LONG LINE; NEGATIVE EXAMPLES; NP-HARD; PCPS; PROPER LEARNING; UNIQUE GAMES; UNIQUE GAMES CONJECTURE;

COMPUTATIONAL COMPLEXITY; GEOMETRY;

HARDNESS;

EID: 84871531284     PISSN: 00975397     EISSN: None     Source Type: Journal    
DOI: 10.1137/120865094     Document Type: Conference Paper

References (52)

1. E. AMALDI AND V. KANN, On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems, Theoret. Comput. Sci., 109(1998), pp. 237-260. (Pubitemid 128450440)
2. D. ANGLUIN AND P. LAIRD, Learning from noisy examples, Machine Learning, 2(1988), pp. 343-370.
3. B. APPLEBAUM, B. BARAK, AND D. XIAO, On basing lower-bounds for learning on worstcase assumptions, in Proceedings of the 49th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2008, pp. 211-220.
4. S. ARORA, L. BABAI, J. STERN, AND Z. SWEEDYK, The hardness of approximate optima in lattices, codes, and systems of linear equations, J. Comput. System. Sci., 54(1997), pp. 317-331. (Pubitemid 127433376)
5. P. AUER AND M. K. WARMUTH, Tracking the best disjunction, Machine Learning, 32(1998), pp. 127-150. (Pubitemid 40626078)
6. S. BEN-DAVID, N. EIRON, AND P. M. LONG, On the difficulty of approximately maximizing agreements, J. Comput. System Sci., 66(2003), pp. 496-514.
7. A. BLUM, A. FRIEZE, R. KANNAN, AND S. VEMPALA, A polynomial-time algorithm for learning noisy linear threshold functions, Algorithmica, 22(1998), pp. 35-52. (Pubitemid 128684248)
8. A. BLUMER, A. EHRENFEUCHT, D. HAUSSLER, AND M. K. WARMUTH, Occam's razor, Inform. Process. Lett., 24(1987), pp. 377-380.
9. N. BSHOUTY AND L. BURROUGHS, Bounds for the minimum disagreement problem with applications to learning theory, in Proceedings of the 22nd Conference on Computational Learning Theory (COLT), 2002, pp. 271-286.
10. N. BSHOUTY AND L. BURROUGHS, Maximizing agreements and coagnostic learning, Theoret. Comput. Sci., 350(2006), pp. 24-39. (Pubitemid 41817578)
11. T. BYLANDER, Learning linear threshold functions in the presence of classification noise, in Proceedings of the 7th Annual ACM Conference on Computational Learning Theory (COLT), 1994, pp. 340-347.
12. S. CHATTERJEE, A simple invariance theorem, preprint, 2005; arxiv:math/0508213v1.
13. E. COHEN, Learning noisy perceptrons by a perceptron in polynomial time, in Proceedings of the 38th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 1997, pp. 514-523.
14. I. DIAKONIKOLAS, P. GOPALAN, R. JAISWAL, R. A. SERVEDIO, AND E. VIOLA, Bounded independence fools halfspaces, in Proceedings of the 50th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2009, pp. 171-180.
15. I. DIAKONIKOLAS, R. O'DONNELL, R. SERVEDIO, AND Y. WU, Hardness results for agnostically learning low-degree polynomial threshold functions, in Proceedings of the 22nd Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2011, pp. 1590-1606.
16. V. FELDMAN, Optimal hardness results for maximizing agreements with monomials, in Proceedings of the 21st Annual IEEE Conference on Computational Complexity (CCC), 2006, pp. 226-236.
17. V. FELDMAN, A complete characterization of statistical query learning with applications to evolvability, in Proceedings of the 50th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2009, pp. 375-384.
18. V. FELDMAN, P. GOPALAN, S. KHOT, AND A. K. PONNUSWAMI, On agnostic learning of parities, monomials, and halfspaces, SIAM J. Comput., 39(2009), pp. 606-645.
19. Y. FREUND, Boosting a weak learning algorithm by majority, Inform. and Comput., 121(1995), pp. 256-285.
20. S. GALANT, Perceptron based learning algorithms, IEEE Trans. Neural Networks, 1(2) (1990), pp. 179-192.
21. M. GAREY AND D. S. JOHNSON, Computers and Intractability, W. H. Freeman, San Francisco, CA, 1979.
22. C. GENTILE AND M. K. WARMUTH, Linear hinge loss and average margin, in Proceedings of the 1998 Conference on Advances in Neural Information Processing Systems (NIPS), 1998, pp. 225-231.
23. P. GOPALAN, S. KHOT, AND R. SAKET, Hardness of reconstructing multivariate polynomials over finite fields, SIAM J. Comput., 39(2010), pp. 2598-2621.
24. V. GURUSWAMI AND P. RAGHAVENDRA, Hardness of learning halfspaces with noise, SIAM J. Comput., 39(2009), pp. 742-765.
25. V. GURUSWAMI, P. RAGHAVENDRA, R. SAKET, AND Y. WU, Bypassing UGC from some optimal geometric inapproximability results, in Proceedings of the 23rd Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2012, pp. 699-717.
26. D. HAUSSLER, Decision theoretic generalizations of the PAC model for neural net and other learning applications, Inform. and Comput., 100(1992), pp. 78-150.
27. K. HOFFGEN, K. VAN HORN, AND H. U. SIMON, Robust trainability of single neurons, J. Comput. System Sci., 50(1995), pp. 114-125.
28. D. S. JOHNSON AND F. P. PREPARATA, The densest hemisphere problem, Theoret. Comput. Sci., 6(1978), pp. 93-107.
29. A. T. KALAI, A. R. KLIVANS, Y. MANSOUR, AND R. A. SERVEDIO, Agnostically learning halfspaces, SIAM J. Comput., 37(2008), pp. 1777-1805.
30. M. KEARNS, Efficient noise-tolerant learning from statistical queries, J. ACM, 45(1998), pp. 983-1006.
31. M. KEARNS, R. SCHAPIRE, AND L. SELLIE, Toward efficient agnostic learning, Machine Learning, 17(1994), pp. 115-141.
32. M. KEARNS AND M. LI, Learning in the presence of malicious errors, SIAM J. Comput., 22(1993), pp. 807-837. (Pubitemid 23677152)
33. M. J. KEARNS AND R. E. SCHAPIRE, Efficient distribution-free learning of probabilistic concepts, J. Comput. System Sci., 48(1994), pp. 464-497. (Pubitemid 124013300)
34. S. KHOT, On the power of unique 2-prover 1-round games, in in Proceedings of the 34th ACM Symposium on Theory of Computing (STOC), 2002, pp. 767-775. (Pubitemid 35009440)
35. S. KHOT, New Techniques for Probabilistically Checkable Proofs and Inapproximability Results (Thesis), Technical Report TR-673-03, Princeton University, Princeton, NJ, 2003.
36. S. KHOT, G. KINDLER, E. MOSSEL, AND R. O'DONNELL, Optimal inapproximability results for MAX-CUT and other 2-variable CSPs?, SIAM J. Comput., 37(2007), pp. 319-357. (Pubitemid 351321999)
37. S. KHOT AND O. REGEV, Vertex cover might be hard to approximate to within 2-epsilon, J. Comput. System Sci., 74(2008), pp. 335-349.
38. S. KHOT AND R. SAKET, Hardness of minimizing and learning DNF expressions, in Proceedings of the 49th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2008, pp. 231-240.
39. A. KLIVANS AND A. SHERSTOV, Unconditional lower bounds for learning intersections of halfspaces, Machine Learning, 69(2007), pp. 97-114. (Pubitemid 47574311)
40. A. KLIVANS AND A. SHERSTOV, Cryptographic hardness for learning intersections of halfspaces, J. Comput. System Sci., 75(2009), pp. 2-12.
41. W. S. LEE, P. L. BARTLETT, AND R. C. WILLIAMSON, On efficient agnostic learning of linear combinations of basis functions, in Proceedings of the 8th Annual Conference on Computational Learning Theory (COLT), 1995, pp. 369-376.
42. N. LITTLESTONE, Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm, Machine Learning, 2(1987), pp. 285-318.
43. K. MATULEF, R. O'DONNELL, R. RUBINFELD, AND R. A. SERVEDIO, Testing halfspaces, in Proceedings of the 20th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), 2009, pp. 256-264.
44. E. MOSSEL, Gaussian bounds for noise correlation of functions, in Proceedings of the 49th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2008, pp. 156-165.
45. E. MOSSEL, R. O'DONNELL, AND K. OLESZKIEWICZ, Noise stability of functions with low influences: Invariance and optimality, in Proceedings of the 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS), 2005, pp. 21-30. (Pubitemid 44375716)
46. R. O'DONNELL AND R. A. SERVEDIO, The Chow parameters problem, in Proceedings of the 40th Annual ACM Symposium on Theory of Computing (STOC), 2008, pp. 517-526.
47. R. RIVEST, Learning decision lists, Machine Learning, 2(1987), pp. 229-246.
48. F. ROSENBLATT, The perceptron: A probabilistic model for information storage and organization in the brain, Psychological Rev., 65(1958), pp. 386-407.
49. R. SCHAPIRE, The strength of weak learnability, Machine Learning, 5(1990), pp. 197-227.
50. R. A. SERVEDIO, Every linear threshold function has a low-weight approximator, Comput. Complex., 16(2007), pp. 180-209.
51. L. VALIANT, A theory of the learnable, Comm. ACM, 27(1984), pp. 1134-1142.
52. V. VAPNIK, Statistical Learning Theory, John Wiley, New York, 1998.