A complete characterization of statistical query learning with applications to evolvability

Journal of Computer and System Sciences

Volumn 78, Issue 5, 2012, Pages 1444-1459

  Feldman, Vitaly ^a  

^a IBM ALMADEN RESEARCH CENTER   (United States)

Author keywords

Agnostic learning; Complexity of learning; Evolvability; Statistical query

Indexed keywords

EFFICIENCY; EVOLUTIONARY ALGORITHMS; LEARNING SYSTEMS;

AGNOSTIC LEARNING; COMPLEXITY OF LEARNING; EVOLUTION ALGORITHMS; EVOLVABILITY; LOSS PERFORMANCE; PAC LEARNING MODEL; STATISTICAL PROPERTIES; STATISTICAL QUERIES;

LEARNING ALGORITHMS;

EID: 84861597048     PISSN: 00220000     EISSN: 10902724     Source Type: Journal    
DOI: 10.1016/j.jcss.2011.12.024     Document Type: Conference Paper

References (42)

1. J. Aslam, and S. Decatur General bounds on statistical query learning and PAC learning with noise via hypothesis boosting Inform. and Comput. 141 2 1998 85 118
2. J. Aslam, and S. Decatur Specification and simulation of statistical query algorithms for efficiency and noise tolerance J. Comput. System Sci. 56 1998 191 208
3. J. Balcázar, J. Castro, D. Guijarro, J. Köbler, and W. Lindner A general dimension for query learning J. Comput. System Sci. 73 6 2007 924 940
4. N. Bansal, A. Blum, S. Chawla, Correlation clustering, in: Proceedings of FOCS, 2002, pp. 238-247.
5. B. Barak, M. Hardt, S. Kale, The uniform hardcore lemma via approximate Bregman projections, in: Proceedings of SODA, 2009, pp. 1193-1200.
6. A. Blum, C. Burch, J. Langford, On learning monotone boolean functions, in: Proceedings of FOCS, 1998, pp. 408-415.
7. A. Blum, C. Dwork, F. McSherry, K. Nissim, Practical privacy: The SuLQ framework, in: Proceedings of PODS, 2005, pp. 128-138.
8. A. Blum, A. Frieze, R. Kannan, and S. Vempala A polynomial time algorithm for learning noisy linear threshold functions Algorithmica 22 1/2 1997 35 52
9. A. Blum, M. Furst, J. Jackson, M. Kearns, Y. Mansour, S. Rudich, Weakly learning DNF and characterizing statistical query learning using Fourier analysis, in: Proceedings of STOC, 1994, pp. 253-262.
10. A. Blum, A. Kalai, and H. Wasserman Noise-tolerant learning, the parity problem, and the statistical query model J. ACM 50 4 2003 506 519
11. N. Bshouty, and V. Feldman On using extended statistical queries to avoid membership queries J. Mach. Learn. Res. 2 2002 359 395
12. N. Bshouty, and C. Tamon On the Fourier spectrum of monotone functions J. ACM 43 4 1996 747 770
13. T. Bylander, Learning linear threshold functions in the presence of classification noise, in: Proceedings of COLT, 1994, pp. 340-347.
14. C. Chu, S. Kim, Y. Lin, Y. Yu, G. Bradski, A. Ng, K. Olukotun, Map-reduce for machine learning on multicore, in: Proceedings of NIPS, 2006, pp. 281-288.
15. J. Dunagan, S. Vempala, A simple polynomial-time rescaling algorithm for solving linear programs, in: Proceedings of STOC, 2004, pp. 315-320.
16. V. Feldman, Evolvability from learning algorithms, in: Proceedings of STOC, 2008, pp. 619-628.
17. V. Feldman, A complete characterization of statistical query learning with applications to evolvability, in Proceedings of FOCS, 2009, pp. 375-384.
18. V. Feldman, Robustness of evolvability, in: Proceedings of COLT, 2009, pp. 277-292.
19. V. Feldman, Distribution-specific agnostic boosting, in: Proceedings of Innovations in Computer Science, 2010, pp. 241-250.
20. V. Feldman, Distribution-independent evolvability of linear threshold functions, in: Proceedings of COLT, 2011.
21. V. Feldman, P. Gopalan, S. Khot, and A. Ponuswami On agnostic learning of parities, monomials and halfspaces SIAM J. Comput. 39 2 2009 606 645
22. V. Feldman, H. Lee, R. Servedio, Lower bounds and hardness amplification for learning shallow monotone formulas, in: Proceedings of COLT, 2011.
23. D. Haussler Decision theoretic generalizations of the PAC model for neural net and other learning applications Inform. and Comput. 100 1 1992 78 150
24. R. Impagliazzo, Hard-core distributions for somewhat hard problems, in: Proceedings of FOCS, 1995, pp. 538-545.
25. A. Kalai, V. Kanade, Potential-based agnostic boosting, in: Proceedings of NIPS, 2009, pp. 880-888.
26. A. Kalai, A. Klivans, Y. Mansour, and R. Servedio Agnostically learning halfspaces SIAM J. Comput. 37 6 2008 1777 1805
27. V. Kanade, L. G. Valiant, J. Wortman Vaughan, Evolution with drifting targets, in: Proceedings of COLT, 2010, pp. 155-167.
28. S. Kasiviswanathan, H. Lee, K. Nissim, S. Raskhodnikova, A. Smith, What can we learn privately? in: Proceedings of FOCS, 2008, pp. 531-540.
29. M. Kearns Efficient noise-tolerant learning from statistical queries J. ACM 45 6 1998 983 1006
30. M. Kearns, M. Li, and L. Valiant Learning boolean formulas J. ACM 41 6 1994 1298 1328
31. M. Kearns, R. Schapire, and L. Sellie Toward efficient agnostic learning Mach. Learn. 17 2-3 1994 115 141
32. M. Kearns, and L. Valiant Cryptographic limitations on learning boolean formulae and finite automata J. ACM 41 1 1994 67 95
33. A. Klivans, and R. Servedio Boosting and hard-core set construction Mach. Learn. 51 3 2003 217 238
34. A. Klivans, and A. Sherstov Unconditional lower bounds for learning intersections of halfspaces Mach. Learn. 69 2-3 2007 97 114
35. L. Michael, Evolving decision lists, manuscript, 2007.
36. A.A. Sherstov, Halfspace matrices, in: Proceedings of Conference on Computational Complexity, 2007, pp. 83-95.
37. H. Simon, A characterization of strong learnability in the statistical query model, in: Proceedings of Symposium on Theoretical Aspects of Computer Science, 2007, pp. 393-404.
38. B. Szörényi, Characterizing statistical query learning: Simplified notions and proofs, in: ALT, 2009, pp. 186-200.
39. L.G. Valiant A theory of the learnable Comm. ACM 27 11 1984 1134 1142
40. L.G. Valiant Evolvability J. ACM 56 1 2009 3 1-3.21
41. P. Valiant Distribution free evolvability of polynomial functions over all convex loss functions Electron. Colloq. Comput. Complexity (ECCC) 11 089 2011
42. Ke Yang New lower bounds for statistical query learning J. Comput. System Sci. 70 4 2005 485 509