Using the doubling dimension to analyze the generalization of learning algorithms

Journal of Computer and System Sciences

Volumn 75, Issue 6, 2009, Pages 323-335

  Bshouty, Nader H ^a   Li, Yi ^b   Long, Philip M ^c  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^b GENOME INSTITUTE OF SINGAPORE   (Singapore)

^c GOOGLE INC   (United States)

Author keywords

Doubling dimension; Doubling metric; Generalization; Learning theory; Local complexity; PAC learning; Statistical learning theory

Indexed keywords

GEOMETRY; PROBABILITY DISTRIBUTIONS;

DOUBLING DIMENSIONS; DOUBLING METRIC; GENERALIZATION; LEARNING THEORY; LOCAL COMPLEXITY; PAC LEARNING; STATISTICAL LEARNING THEORY;

LEARNING ALGORITHMS;

EID: 67349151635     PISSN: 00220000     EISSN: 10902724     Source Type: Journal    
DOI: 10.1016/j.jcss.2009.01.003     Document Type: Article

References (36)

1. K. Alexander, Rates of growth for weighted empirical processes, in: Proc. of Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, vol. 2, 1985, pp. 475-493
2. Anthony M., and Bartlett P.L. Neural Network Learning: Theoretical Foundations (1999), Cambridge Univ. Press
3. n. Bull. Soc. Math. France 111 4 (1983) 429-448
4. Bartlett P.L., Bousquet O., and Mendelson S. Local Rademacher complexities. Ann. Statist. 33 4 (2005) 1497-1537
5. Bartlett P.L., and Long P.M. Prediction, learning, uniform convergence, and scale-sensitive dimensions. J. Comput. System Sci. 56 2 (1998) 174-190
6. Benedek G., and Itai A. Learnability with respect to fixed distributions. Theoret. Comput. Sci. 86 2 (1991) 377-389
7. A. Beygelzimer, S. Kakade, J. Langford, Cover trees for nearest neighbor, in: ICML, 2006
8. Blumer A., Ehrenfeucht A., Haussler D., and Warmuth M.K. Learnability and the Vapnik-Chervonenkis dimension. J. ACM 36 4 (1989) 929-965
9. H.T.H. Chan, A. Gupta, B.M. Maggs, S. Zhou, On hierarchical routing in doubling metrics, in: SODA, 2005
10. Dudley R.M. Central limit theorems for empirical measures. Ann. Probab. 6 6 (1978) 899-929
11. Ehrenfeucht A., Haussler D., Kearns M., and Valiant L.G. A general lower bound on the number of examples needed for learning. Inform. and Comput. 82 3 (1989) 247-251
12. A. Gupta, R. Krauthgamer, J.R. Lee, Bounded geometries, fractals, and low-distortion embeddings, in: FOCS, 2003
13. Har-Peled S., and Mendel M. Fast construction of nets in low dimensional metrics, and their applications. SIAM J. Comput. 35 5 (2006) 1148-1184
14. Haussler D. Sphere packing numbers for subsets of the Boolean n-cube with bounded Vapnik-Chervonenkis dimension. J. Combin. Theory Ser. A 69 2 (1995) 217-232
15. Haussler D., Littlestone N., and Warmuth M.K. Predicting {0, 1}-functions on randomly drawn points. Inform. and Comput. 115 2 (1994) 129-161
16. Heinonen J. Lectures on Analysis on Metric Spaces (2001), Springer-Verlag
17. Helmbold D.P., Littlestone N., and Long P.M. Apple tasting. Inform. and Comput. 161 2 (2000) 85-139 preliminary version in FOCS'92
18. J. Kleinberg, A. Slivkins, T. Wexler, Triangulation and embedding using small sets of beacons, in: FOCS, 2004
19. Kolmogorov A.N., and Tihomirov V.M. ε{lunate}-Entropy and ε{lunate}-capacity of sets in functional spaces. Amer. Math. Soc. Transl. Ser. 2 vol. 17 (1961) 277-364
20. R. Krauthgamer, J.R. Lee, The black-box complexity of nearest neighbor search, in: ICALP, 2004
21. R. Krauthgamer, J.R. Lee, Navigating nets: Simple algorithms for proximity search, in: SODA, 2004
22. F. Kuhn, T. Moscibroda, R. Wattenhofer, On the locality of bounded growth, in: PODC, 2005
23. Lidl R., and Niederreiter H. Finite Fields (1997), Cambridge Univ. Press
24. Long P.M. On the sample complexity of PAC learning halfspaces against the uniform distribution. IEEE Trans. Neural Networks 6 6 (1995) 1556-1559
25. Long P.M. An upper bound on the sample complexity of PAC learning halfspaces with respect to the uniform distribution. Inform. Process. Lett. 87 5 (2003) 229-234
26. Mendelson S. Estimating the performance of kernel classes. J. Mach. Learn. Res. 4 (2003) 759-771
27. Motwani R., and Raghavan P. Randomized Algorithms (1995), Cambridge Univ. Press
28. Shawe-Taylor J., Anthony M., and Biggs N. Bounding the sample size with the Vapnik-Chervonenkis dimension. Discrete Appl. Math. 42 (1993) 65-73
29. A. Slivkins, Distance estimation and object location via rings of neighbors, in: PODC, 2005
30. K. Talwar, Bypassing the embedding: Approximation schemes and compact representations for low dimensional metrics, in: STOC, 2004
31. van de Geer S. Empirical Processes in M-Estimation. Camb. Ser. Stat. Probab. Methods (2000)
32. van der Vaart A., and Wellner J.A. Weak Convergence and Empirical Processes With Applications to Statistics (1996), Springer-Verlag
33. Vapnik V.N. Estimation of Dependencies Based on Empirical Data (1982), Springer-Verlag
34. Vapnik V.N. Statistical Learning Theory (1998), Wiley, New York
35. Vapnik V.N., and Chervonenkis A.Y. On the uniform convergence of relative frequencies of events to their probabilities. Theory Probab. Appl. 16 2 (1971) 264-280
36. Zhang T. Information theoretic upper and lower bounds for statistical estimation. IEEE Trans. Inform. Theory 52 4 (2006) 1307-1321