Learning DNF in time 2Õ(n(1/3))

Journal of Computer and System Sciences

Volumn 68, Issue 2, 2004, Pages 303-318

  Klivans, Adam R ^a   Servedio, Rocco A ^b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b HARVARD UNIVERSITY   (United States)

Author keywords

Computational learning theory; Disjunctive normal form; Polynomial threshold function

Indexed keywords

ALGORITHMS; BOOLEAN FUNCTIONS; DATA STRUCTURES; POLYNOMIALS; SET THEORY; THEOREM PROVING;

COMPUTATIONAL LEARNING THEORY; DISJUNCTIVE NORMAL FORM; POLYNOMIAL THRESHOLD FUNCTION;

COMPUTATIONAL COMPLEXITY;

EID: 1842510602     PISSN: 00220000     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jcss.2003.07.007     Document Type: Article

References (46)

1. D. Angluin, Queries and concept learning, Mach. Learning 2 (1988) 319-342.
2. M. Anthony, Classification by polynomial surfaces, Discrete Appl. Math. 61 (1995) 91-103.
3. J. Aspnes, R. Beigel, M. Furst, S. Rudich, The expressive power of voting polynomials, Combinatorica 14(2) (1994) 1-14.
4. Earlier version in: Proceedings of the 23rd ACM Symposium on Theory of Computation, New Orleans, LA, 1991, pp. 402-409.
5. R. Beigel, The polynomial method in circuit complexity, in: Proceedings of the eighth Conference on Structure in Complexity Theory, San Diego, CA, 1993, pp. 82-95.
6. R. Beigel, personal communication, 2000.
7. R. Beigel, N. Reingold, D. Spielman, The perception strikes back, in: Proceedings of the Sixth Conference on Structure in Complexity Theory, Chicago, IL, 1991, pp. 286-291.
8. C. Berg, S. Ulfberg, A lower bound for perceptrons and an oracle separation of the PP hierarchy, J. Comput. System Sci. 56 (3) (1998) 263-271.
9. A. Blum, Rank-r decision trees are a subclass of r-decision lists, Inform. Process. Lett. 42 (4) (1992) 183-185.
10. A. Blum, A. Frieze, R. Kannan, S. Vempala, A polynomial time algorithm for learning noisy linear threshold functions, in: Proceedings of the 37th Symposium on Foundations of Computer Science, Burlington, VT, 1996, pp. 330-338.
11. A. Blum, M. Furst, J. Jackson, M. Kearns, Y. Mansour, S. Rudich, Weakly learning DNF and characterizing statistical query learning using Fourier analysis, in: Proceedings of the 26th Annual Symposium on Theory of Computing, Montreal, Quebec, Canada, 1994, pp. 253-262.
12. A. Blum, S. Rudich, Fast learning of k-term DNF formulas with queries, J. Comput. System Sci. 51 (3) (1995) 367-373.
13. A. Blumer, A. Ehrenfeucht, D. Haussler, M.K. Warmuth, Learnability and the Vapnik-Chervonenkis dimension, J. Assoc. Comput. Mach. 36 (4) (1989) 929-965.
14. N. Bshouty, A subexponential exact learning algorithm for DNF using equivalence queries, Inform. Process. Lett. 59 (1996) 37-39.
15. T. Bylander, Worst-case analysis of the perceptron and exponentiated update algorithms, Artif. Intell. 106 (1998) 335-352.
16. E.W. Cheney, Introduction to Approximation Theory, McGraw-Hill, New York, 1966.
17. E. Cohen, Learning noisy perceptrons by a perceptron in polynomial time, in: Proceedings of the 38th Symposium on Foundation of Computer Science, Miami Beach, FL, 1997, pp. 514-523.
18. A. Ehrenfeucht, D. Haussler, Learning decision trees from random examples, Inform. and Comput. 82 (3) (1989) 231-246.
19. A. Ehrenfeucht, D. Haussler, M. Kearns, L. Valiant, A general lower bound on the number of examples needed for learning, Inform. and Comput. 82 (3) (1989) 247-251.
20. Y. Freund, Boosting a weak learning algorithm by majority, Inform. and Comput. 121 (2) (1995) 256-285.
21. Y. Freund, R. Schapire, Large margin classification using the perceptron algorithm, in: Proceedings of the 11th Annual Conference on Computer Learning Theory, Madison, WI, 1998, pp. 209-217.
22. B. Fu, Separating PH from PP by relativization, Acta Math. Sinica 8 (3) (1992) 329-336.
23. M. Furst, J. Saxe, M. Sipser, Parity, circuits and the polynomial-time hierarchy, Math. Systems Theory 17 (1) (1984) 13-27.
24. M. Goldmann, J. Håstad, A. Razborov, Majority gates vs. general weighted threshold gates, Comput. Complexity 2 (1992) 277-300.
25. T. Hancock, Y. Mansour, Learning monotone k-μ DNF formulas on product distributions, in: Proceedings of the Fourth Annual Workshop on Computer Learning Theory, Santa Cruz, CA, 1991, pp. 179-183.
26. J. Håstad, Computational limitations for small depth circuits, Ph.D. Thesis, MIT Press, Cambridge, 1986.
27. D. Helmbold, R. Sloan, M. Warmuth, Learning nested differences of intersection-closed concept classes, Mach. Learning 5 (1990) 165-196.
28. J. Jackson, An efficient membership-query algorithm for learning DNF with respect to the uniform distribution, J. Comput. System Sci. 55 (1997) 414-440.
29. M. Kearns, Efficient noise-tolerant learning from statistical queries, J. Assoc. Comput. Mach. 45 (6) (1998) 983-1006.
30. R. Khardon, On using the Fourier transform to learn disjoint DNF, Inform Process Lett. 49 (1994) 219-222.
31. J. Kivinen, M. Warmuth, P. Auer, The perception algorithm vs. winnow: linear vs. logarithmic mistake bounds when few input variables are relevant, in: Proceedings of the eight Conference on Computational Learning Theory, Santa Cruz, CA, 1995, pp. 289-296.
32. L. Kucera, A. Marchetti-Spaccamela, M. Protassi, On learning monotone DNF formulae under uniform distributions, Inform and Comput. 110 (1994) 84-95.
33. E. Kushilevitz, D. Roth, On learning visual concepts and DNF formulae, in: Proceedings of the Sixth Annual ACM Conference on Computational Learning Theory, Santa Cruz, CA, 1993, pp. 317-326.
34. N. Linial, N. Nisan, Approximate inclusion-exclusion, Combinatorica 10 (4) (1990) 349-365.
35. N. Littlestone, Learning quickly when irrelevant attributes abound: a new linear-threshold algorithm, Mach. Learning 2 (1988) 285-318.
36. S. Lokam, personal communication, 2001.
37. W. Maass, G. Turan, How fast can a threshold gate learn?, in: S.J. Hanson, G. Drastal, R. Rivest (Eds.), Computational Learning Theory and Natural Learning Systems: Constraints and Prospects, Vol. I, MIT Press, Cambridge, 1994, pp. 381-414.
38. M. Minsky, S. Papert, Perceptrons, MIT Press, Cambridge, 1968 (expanded edition 1988).
39. Y. Sakai, A. Maruoka, Learning monotone log-term DNF formulas under the uniform distribution, Theory Comput. Systems 33 (2000) 17-33.
40. M. Schmitt, Identification criteria and lower bounds for Perceptron-like learning rules, Neural Comput. 10 (1998) 235-250.
41. R. Servedio, On PAC learning using Winnow, Perceptron, and a Perceptron-like algorithm, in: Proceedings of the 12th Annual Conference on Computer Learning Theory, 1999, pp. 296-307.
42. J. Tarui, T. Tsukiji, Learning DNF by approximating inclusion-exclusion formulae, in: Proceedings of the IEEE Conference on Computational Complexity, Atlanta, GA, 1999, pp. 215-220.
43. L.G. Valiant, A theory of the learnable, Comm. ACM 27 (11) (1984) 1134-1142.
44. K. Verbeurgt, Learning DNF under the uniform distribution in quasi-polynomial time, in: Proceedings of the Third Annual Workshop on Computational Learning Theory, Rochester, NY, 1990, pp. 314-326.
45. K. Verbeurgt, Learning sub-classes of monotone DNF on the uniform distribution, in: Proceedings of the Ninth Conference on Algorithmic Learning Theory, Otzenhausen, Germany, 1998, pp. 385-399.
46. N. Vereshchagin, On the power of PP, in: Proceedings of the Seventh Conference on Structure in Complexity Theory, Boston, MA, 1992, pp. 138-143.