Learnability and the Vapnik-Chervonenkis Dimension

Journal of the ACM (JACM)

Volumn 36, Issue 4, 1989, Pages 929-965

  Blumer, Anselm ^a   Ehrenfeucht, A ^b   Haussler, David ^c   Warmuth, Manfred K ^c  

^a TUFTS UNIVERSITY   (United States)

^b UNIVERSITY OF COLORADO   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER PROGRAMMING--ALGORITHMS; PATTERN RECOGNITION;

LEARNABILITY; PATTERN RECOGNITION ALGORITHMS; VAPNIK-CHERVONENKIS DIMENSION;

SYSTEMS SCIENCE AND CYBERNETICS;

EID: 0024750852     PISSN: 00045411     EISSN: 1557735X     Source Type: Journal    
DOI: 10.1145/76359.76371     Document Type: Article

References (69)

1. AHO, A., HOPCROFT, J., AND ULLMAN, J. The Design and Analysis of Computer Algorithms. Addison-Wesley, London, 1974.
2. ANGLUIN, D. Learning regular sets from queries and counterexamples. Inf. Comput. 75 (1987), 87-106.
3. ANGLUIN, D. Queries and concept learning. Mach. Learning 2, 2 (1988), 319-342.
4. ANGLUIN, D., AND LAIRD, P. D. Learning from noisy examples. Mach. Learning 2, 2 (1988), 343-370.
5. ANGLUIN, D., AND VALIANT, L. Fast probabilistic algorithms for Hamiltonian circuits and matchings. J. Comput. Syst. Sci. 19 (1979), 155-193.
6. ASSOUAD, P. Densité et Dimension. Ann. Inst. Fourier, Grenoble 33, 3 (1983), 233-282.
7. BAUM, E., AND HAUSSLER, D. What size net gives valid generalization. Neural Comput. I, 1 (1989) 151-160.
8. BEN-DAVID, S., BENEDEK, G., AND MANSOUR, Y. A parameterization scheme for classifying models of learnability. In Proceedings of the 2nd Workshop of Computational Learning Theory (Santa Cruz, Calif., July 31-Aug. 2). Morgan Kaufman, San Mateo, Calif., 1989, to appear.
9. BENEDEK, G., AND ITAI, A. Nonuniform learnability. In Proceedings of the 15th International Conference on Automata, Languages and Programming (July). 1988, pp. 82-92.
10. BLUM, A., AND RIVEST, R. Training a 3-node neural network is NP-complete. In Proceedings of the 1st Workshop on Computational Learning Theory (Cambridge, Mass., Aug. 3-5). Morgan Kaufmann, San Mateo, Calif., 1988, pp.9-18.
11. BLUMER, A., EHRENFEUCHT, A., HAUSSLER, D., AND WARMUTH, M. K. Occam'm Razor. Inf. Process. Lett. 24 (1987), 377-380.
12. BOLLOBÁS, B. Combinatorics. Cambridge Univ. Press, Cambridge, Mass., 1986.
13. BOUCHERON, S., AND SALLANTIN, J. Some remarks about space-complexity of learning and circuit complexity of recognizing. In Proceedings of the 1st Workshop on Computational Learning Theory (Cambridge, Mass., Aug. 3-5). Morgan Kaufmann, San Mateo, Calif., 1988, pp. 125-138.
14. COVER, T. Geometrical and statistical properties of systems of linear inequalities with applications to pattern recognition. IEEE Trans. Elect. Comp. 14 (1965), 326-334.
15. DEVROYE, L. P. Automatic pattern recognition: A study of the probability of error. IEEE Trans. Pattern Analysis and Mach. Intell. 10, 4 (1988), 530-543.
16. DEVROYE, L. P., AND WAGNER, T. J. A distribution-free performance bound in error estimation. IEEE Trans. Inf. Theorem 22 (1976), 586-587.
17. DUDA, R., AND HART, P. Pattern Classification and Scene Analysis. Wiley, New York, 1973.
18. DUDLEY, R. M. A course on empirical processes. In Lecture Notes in Mathematics, vol. 1097. Springer-Verlag, New York, 1984.
19. DUDLEY, R. M. Universal Donsker classes and metric entropy. Ann. Prob. 15, 4 (1987), 1306-1326.
20. EDELSBRUNNER, H., AND PREPARATA, F. P. Minimal polygonal separation. Inf. Comput. 77 (1988), 218-232.
21. EHRENFEUCHT, A., HAUSSLER, D., KEARNS, M., AND VALIANT, L. G. A general lower bound on the number of examples needed for learning. Inf. Comput., to appear.
22. GAREY, M., AND JOHNSON, D. Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman, San Francisco, 1979.
23. GILL, J. Probabilistic Turing machines. SIAM J. Comput. 6, 4 (1977), 675-695.
24. GINÉ, E., AND ZINN, J. Lectures on the central limit theorem for empirical processes. In Lecture Notes in Mathematics, vol. 1221. Springer-Verlag, New York, 1986.
25. GOLDREICH, O., GOLDWASSER, S., AND MICALI, S. How to construct random functions. J. ACM 33, 4 (Oct. 1986), 792-807.
26. HAMPSON, S. E., AND VOLPER, D. Linear Function neurons: Structure and training. Biol. Cyber. 53 (1986), 203-217.
27. HAUSSLER, D. Learning conjunctive concepts in structural domains. Mach. Learning 4, 1 (1989)
28. HAUSSLER, D. Applying Valiant't learning framework to AI concept learning problems. In Proceedings of the 4th International Workshop on Machine Learning. Morgan Kaufmann, San Mateo, Calif., 1987, pp. 324-336.
29. HAUSSLER, D. Quantifying inductive bias: AI learning algorithms and Valiant't learning framework. Artif. Intell. 36 (1988), 177-221.
30. HAUSSLER, D. Generalizing the PAC model: Sample size bounds from metric dimension-based uniform convergence results. In Proceedings of the 30th IEEE Symposium on Foundations of Computer Science (Research Triangle Park, N.C., Oct. 30-Nov. 1). IEEE, New York, 1989, to appear.
31. HAUSSLER, D., AND WELZL, E. Epsilon-nets and simplex range queries. Disc. Comput. Geometry 2 (1987), 127-151.
32. HAUSSLER, D., KEARNS, M., LITTLESTONE, N., AND WARMUTH, M. K. Equivalence of models for polynomial learnability. Tech. Rep. UCSC-CRL-88-06. Univ. California at Santa Cruz, Santa Cruz, Calif., 1988.
33. HAUSSLER, D., LITTLESTONE, N., AND WARMUTH, M. K. Predicting {0, 1}-functions on randomly drawn points. In Proceedings of the 29th IEEE Symposium on Foundations of Computer Science (White Plains, N.Y., Oct.). IEEE, New York, 1988, pp. 100-109.
34. JOHNSON, D. S. Approximation algorithms for combinatorial problems. J. Comput. Syst. Sci. 9 (1974), 256-278.
35. KARMARKAR, N. A new polynomial-time algorithm for linear programming. Combinatorica 4 (1984), 373-395.
36. KEARNS, M., AND LI, M. Learning in the presence of malicious errors. In Proceedings of the 20th Symposium on Theory of Computing (Chicago, III., May 2-4). ACM, New York, 1988, pp. 267-280.
37. KEARNS, M., AND VALIANT, L. Cryptographic limitations on learning Boolean formulae and finite automata. In Proceedings of the 21st ACM Symposium on Theory of Computing (Seattle, Wash., May 15-17). ACM, New York, 1989, pp. 433-444.
38. KEARNS, M., LI, M., PITT, L., AND VALIANT, L. On the learnability of Boolean formulae. In Proceedings of the 19th ACM Symposium on Theory of Computation (New York, N.Y., May 25-27). ACM, New York, 1987, pp. 285-295.
39. KEARNS, M., LI, M., PITT, L., AND VALIANT, L. Recent results on Boolean concept learning. In Proceedings of the 4th International Workshop on Machine Learning. Morgan-Kaufmann, San Mateo, Calif., 1987, pp. 337-352.
40. KNUTH, D. E. The Art of Computer Programming, 2nd ed., vol. 1. Addison-Wesley, Reading, Mass., 1973.
41. LAIRD, P. D. Learning from good data and bad. Tech. Rep. YALEU/DCS/TR-551. Yale Univ., New Haven, Conn., 1987.
42. LEE, D. T., AND PREPARATA, F. P. Computational geometry-A survey. IEEE Trans. Comput. 33, 12 (1984), 1072-1101.
43. 4L LINIAL, N., MANSOUR, Y., AND RIVEST, R. Results on learnability and the Vapnik-Chervonenkis dimension. In Proceedings of the 29th IEEE Symposium on Foundations of Computer Science (White Plains, N.Y., Oct.). IEEE, New York, 1988, pp. 120-129.
44. LITTLESTONE, N. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Mach. Learning 2, 2 (1988), 285-318.
45. MASEK, W. J. Some NP-complete set cover problems. MIT Laboratory for Computer Science, unpublished manuscript.
46. MEGIDDO, N. Linear programming in linear time when the dimension is fixed. J. ACM 31, 1 (Jan. 1984), 114-127.
47. MEGIDDO, N. On the complexity of polyhedral separability. Discrete Comput. Geometry 3 (1988), 325-337.
48. MUROGA, S. Threshold Logic and Its Applications. Wiley, New York, 1971.
49. NATARAJAN, B. K. On learning Boolean functions. In Proceedings of the 19th Annual ACM Symposium on Theory of Computation (New York, N.Y., May 25-27). ACM, New York, 1987, pp. 296-304.
50. NATARAJAN, B. K. Learning functions from examples. Tech. Rep. CMU-RI-TR-87-19. Carnegie Mellon Univ., Pittsburgh, Pa., Aug. 1987.
51. NIGMATULLIN, R. G. The fastest descent method for covering problems (in Russian). In Proceedings of a Symposium on Questions of Precision and Efficiency of Computer Algorithms, Book 5. Kiev, 1969, pp. 116-126.
52. PEARL, J. On the connection between the complexity and credibility of inferred models. Int. J. Gen. Syst. 4 (1978), 255-264.
53. PEARL, J. Capacity and error estimates for Boolean classifiers with limited capacity. IEEE Trans. Pattern Analysis Mach. Intell. 1, 4 (1979), 350-355.
54. PITT, L., AND VALIANT, L. G. Computational limitations on learning from examples. J. ACM 35, 4 (Oct. 1988), 965-984.
55. PITT, L., AND WARMUTH, M. Reductions among prediction problems, on the difficulty of predicting automata. In Proceedings of the 3rd IEEE Structure in Complexity Theory Conference (Washington, D.C.). IEEE, New York, 1988, pp. 62-69.
56. POLLARD, D. Convergence of Stochastic Processes. Springer-Verlag, New York, 1984.
57. QUINLAN, R., AND RIVEST, R. Inferring decision trees using the minimum description length principle. Inf. Comput., to appear.
58. RIVEST, R. Learning decision-lists. Mach. Learning 2, 3 (1987), 229-246.
59. ROYDEN, H. L. Real Analysis, 2nd ed. MacMillan, New York, 1968.
60. SLOAN, R. Types of noise in data for concept learning. In Proceedings of the 1st Workshop on Computational Learning Theory (Cambridge, Mass., Aug. 3-5). Morgan Kaufmann, San Mateo, Calif., 1988, pp. 91-96.
61. VALIANT, L. G. A theory of the learnable. Commun. ACM 27, 11 (Nov. 1984), 1134-1142.
62. VALIANT, L. G. Learning disjunctions of conjunctions. In Proceedings of the 9th International Conference on Artificial Intelligence (Los Angeles, Calif., Aug.), vol. 1. Morgan Kaufmann, San Mateo, Calif., 1985, pp. 560-566.
63. VAPNIK, V. N. Estimation of Dependences Based on Empirical Data. Springer Verlag, New York, 1982.
64. VAPNIK, V. N., AND CHERVONENKIS, A. YA. On the uniform convergence of relative frequencies of events to their probabilities. Theoret. Probl. and Its Appl. 16, 2 (1971), 264-280.
65. VAPNIK, V. N., AND CHERVONENKIS, A. YA. Theory of Pattern Recognition (in Russian). Nauka, Moscow, 1974.
66. VITTER, J., AND LIN, J. H. Learning in parallel. In Proceedings of the 1st Workshop on Computational Learning Theory (Cambridge, Mass., Aug. 3-5). Morgan Kaufmann, San Mateo, Calif., 1988, pp. 106-124.
67. WATANABE, S. Pattern recognition as information compression. In Frontiers of Pattern Recognition, S. Watanabe, Ed. Academic Press, Orlando, Fla., 1972.
68. WENOCUR, R. S., AND DUDLEY, R. M. Some special Vapnik-Chervonenkis classes. Discrete Math. 33 (1981), 313-318.
69. WINSTON, P. Learning structural descriptions from examples. In The Psychology of Computer Vision. McGraw-Hill, New York, 1975.