Agnostic learning of monomials by halfspaces is hard

Proceedings - Annual IEEE Symposium on Foundations of Computer Science, FOCS

Volumn , Issue , 2009, Pages 385-394

  Feldman, Vitaly ^a   Guruswami, Venkatesan ^b   Raghavendra, Prasad ^c   Wu, Yi ^b  

^a IBM ALMADEN RESEARCH CENTER   (United States)

^b Carnegie Mellon University   (United States)

^c MICROSOFT RESEARCH   (United States)

Author keywords

Agnostic learning; Dictatorship tests; Hardness of learning; PCPs

Indexed keywords

AGNOSTIC LEARNING; ARBITRARY CONSTANTS; DECISION LISTS; DICTATORSHIP TESTS; HALF SPACES; HALF-SPACE; HARDNESS OF LEARNING; HARDNESS RESULT; HYPERCUBE; INVARIANCE PRINCIPLE; LEARNING THEORY; LIST DECODING; NP-HARD; PROPER LEARNING; SPARSE APPROXIMATIONS; UNIQUE GAMES;

COMPUTATIONAL COMPLEXITY; COMPUTER SCIENCE; FIBER OPTIC CHEMICAL SENSORS;

HARDNESS;

EID: 77952373092     PISSN: 02725428     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/FOCS.2009.26     Document Type: Conference Paper

References (42)

1. E. Amaldi and V. Kann, "On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems," Theoretical Computer Science, vol. 109, pp. 237-260, 1998.
2. D. Angluin and P. Laird, "Learning from noisy examples," Machine Learning, vol. 2, pp. 343-370, 1988.
3. S. Arora, L. Babai, J. Stern, and Z. Sweedyk, "The hardness of approximate optima in lattices, codes, and systems of linear equations," J. Comput. Syst. Sci., vol. 54, no. 2, pp. 317-331, 1997.
4. P. Auer and M. K. Warmuth, "Tracking the best disjunction," Machine Learning, vol. 32, no. 2, pp. 127-150, 1998. (Pubitemid 40626078)
5. S. Ben-David, N. Eiron, and P. M. Long, "On the difficulty of approximately maximizing agreements." J. Comput. Syst. Sci., vol. 66, no. 3, pp. 496-514, 2003.
6. A. Blum, A. Frieze, R. Kannan, and S. Vempala, "A polynomial-time algorithm for learning noisy linear threshold functions," Algorithmica, vol. 22, no. 1-2, pp. 35-52, 1998.
7. A. Blumer, A. Ehrenfeucht, D. Haussler, and M. K. Warmuth, "Occam's razor," Inf. Process. Lett., vol. 24, no. 6, pp. 377-380, 1987.
8. N. Bshouty and L. Burroughs, "Bounds for the minimum disagreement problem with applications to learning theory," in Proceedings of COLT, 2002, pp. 271-286.
9. -, "Maximizing agreements and coagnostic learning," Theoretical Computer Science, vol. 350, no. 1, pp. 24-39, 2006.
10. T. Bylander, "Learning linear threshold functions in the presence of classification noise," in Proceedings of COLT, 1994, pp. 340-347.
11. S. Chatterjee, "A simple invariance theorem," arxiv:math/0508213v1., 2005.
12. E. Cohen, "Learning noisy perceptrons by a perceptron in polynomial time," in IEEE FOCS, 1997, pp. 514-523.
13. I. Diakonikolas, P. Gopalan, R. Jaiswal, R. Servedio, and E. Viola, "Bounded independence fools halfspaces," ECCC, vol. TR09-016, 2009.
14. V. Feldman, "Optimal hardness results for maximizing agreements with monomials," in IEEE CCC, 2006, pp. 226-236.
15. V. Feldman, P. Gopalan, S. Khot, and A. K. Ponnuswami, "New results for learning noisy parities and halfspaces," in IEEE FOCS, 2006, pp. 563-574.
16. S. Galant, "Perceptron based learning algorithms," IEEE Trans. on Neural Networks, vol. 1(2), 1990.
17. M. Garey and D. S. Johnson, Computers and Intractability, 1979.
18. C. Gentile and M. K. Warmuth, "Linear hinge loss and average margin," in Proceedings of NIPS, 1998, pp. 225-231.
19. V. Guruswami and P. Raghavendra, "Hardness of learning halfspaces with noise," in IEEE FOCS, 2006, pp. 543-552.
20. D. Haussler, "Decision theoretic generalizations of the PAC model for neural net and other learning applications," Information and Computation, vol. 100, no. 1, pp. 78-150, 1992.
21. K. Hoffgen, K. van Horn, and H. U. Simon, "Robust train-ability of single neurons," J. Comput. Syst. Sci, vol. 50, no. 1, pp. 114-125, 1995.
22. D. S. Johnson and F. P. Preparata, "The densest hemisphere problem." Theoretical Computer Science, vol. 6, pp. 93-107, 1978.
23. A. Kalai, A. Klivans, Y. Mansour, and R. Servedio, "Agnostically learning halfspaces," SIAM Journal on Computing, vol. 37, no. 6, pp. 1777-1805, 2008.
24. M. Kearns, "Efficient noise-tolerant learning from statistical queries," Journal of the ACM, vol. 45, no. 6, pp. 983-1006, 1998.
25. M. Kearns, R. Schapire, and L. Sellie, "Toward efficient agnostic learning," Machine Learning, vol. 17, pp. 115-141, 1994.
26. M. J. Kearns and M. Li, "Learning in the presence of malicious errors," SIAM J Comput., vol. 22, no. 4, pp. 807-837, 1993.
27. M. J. Kearns and R. E. Schapire, "Efficient distribution-free learning of probabilistic concepts," J. Comput. Syst. Sci., vol. 48, no. 3, pp. 464-497, 1994.
28. S. Khot, "New techniques for probabilistically checkable proofs and inapproximability results (thesis)," Princeton University Technical Reports, vol. TR-673-03, 2003.
29. -, "On the power of unique 2-Prover 1-Round games," in ACM STOC, May 19-21 2002, pp. 767-775. (Pubitemid 35009440)
30. S. Khot, G. Kindler, E. Mossel, and R. O'Donnell, "Optimal inapproximability results for MAX-CUT and other 2-variable CSPs?" SIAM J. Comput, vol. 37, no. 1, pp. 319-357, 2007.
31. S. Khot and R. Saket, "Hardness of minimizing and learning DNF expressions," in IEEE FOCS, 2008, pp. 231-240.
32. W. S. Lee, P. L. Bartlett, and R. C. Williamson, "On efficient agnostic learning of linear combinations of basis functions," in Proceedings of COLT, 1995, pp. 369-376.
33. N. Littlestone, "Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm," Machine Learning, vol. 2, pp. 285-318, 1987.
34. K. Matulef, R. O'Donnell, R. Rubinfeld, and R. A. Servedio, "Testing halfspaces," in SODA, 2009, pp. 256-264.
35. E. Mossel, "Gaussian bounds for noise correlation of functions," IEEE FOCS, 2008.
36. E. Mossel, R. O'Donnell, and K. Oleszkiewicz, "Noise stability of functions with low influences: Invariance and optimality," in IEEE FOCS, 2005.
37. R. O'Donnell and R. A. Servedio, "The chow parameters problem," in ACM STOC, 2008, pp. 517-526.
38. R. Rivest, "Learning decision lists," Machine Learning, vol. 2, no. 3, pp. 229-246, 1987.
39. F. Rosenblatt, "The perceptron: a probabilistic model for information storage and organization in the brain," Psychological Review, vol. 65, pp. 386-407, 1958.
40. R. A. Servedio, "Every linear threshold function has a low-weight approximator," Comput. Complex., vol. 16, no. 2, pp. 180-209, 2007.
41. L. Valiant, "A theory of the learnable," Communications of the ACM, vol. 27, no. 11, pp. 1134-1142, 1984.
42. V. Vapnik, Statistical Learning Theory, 1998.