Concept learning using complexity regularization

IEEE Transactions on Information Theory

Volumn 42, Issue 1, 1996, Pages 48-54

  Lugosi, Gábor ^a   Zeger, Kenneth ^b  

^a BUDAPEST UNIVERSITY OF TECHNOLOGY AND ECONOMICS   (Hungary)

^b University of Illinois at Urbana Champaign   (United States)

Author keywords

Classification; Estimation; Learning theory; Pattern recognition

Indexed keywords

CLASSIFICATION (OF INFORMATION); DECISION THEORY; ESTIMATION; PATTERN RECOGNITION; PROBABILITY; RANDOM PROCESSES; SET THEORY;

BAYESIAN RISK; COMPLEXITY REGULARIZATION; CONCEPT LEARNING; ERROR PROBABILITY; LEARNING THEORY; VAPNIK-CHERNOVNENKIS DIMENSION;

LEARNING SYSTEMS;

EID: 0029754587     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.481777     Document Type: Article

References (25)

1. H. Akaike, "A new look at the statistical model identification," IEEE Trans. Automat. Contr., vol. AC-19, pp. 716-723, 1974.
2. A. R. Barron, "Logically smooth density estimation," Tech. Rep. TR 56, Dept. of Statistics, Stanford Univ., Stanford, CA, 1985.
3. _, "Complexity regularization with application to artificial neural networks," in G. Roussas, Ed., Nonparametric functional Estimation and Related Topics (NATO ASI ser.). Dordrecht, The Netherlands: Kluwer, 1991.
4. A. R. Barron and T. M. Cover, "Minimum complexity density estimation," IEEE Trans. Inform. Theory, vol. 37, pp. 1034-1054, 1991.
5. G. M. Benedek and A. Itai, "Nonuniform learnability," J. Comput. Syst. Sci., vol. 48. pp. 311-323, 1994.
6. A. Blumer, A. Ehrenfeucht, D. Haussler, and M. K. Warmuth, "Learnability and the Vapnik-Chervonenkis dimension," J. ACM, vol. 36, pp. 929-965, 1989.
7. K. L. Buescher, "Learning and smooth simultaneous estimation based on empirical data," Ph.D. dissertation. Univ. of Illinois, UrbanaChampaign. 1992.
8. [81 K. L. Buescher and P. R. Kumar, "Learning by canonical smooth estimation, Part II: Learning and choice of model complexity," to appear in IEEE Trans. Automat. Contr., 1995.
9. L. Devroye, "Any discrimination rule can have an arbitrarily bad probability of error for finite sample size," IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-4, pp. 154-157, 1982.
10. _, "Bounds for the uniform deviation of empirical measures," J. Multivariate Anal, vol. 12, pp. 72-79, 1982.
11. _, "Automatic pattern recognition: A study of the probability of error," IEEE Trans. Pattern Anal. Machine Intell., vol. 10, pp. 530-543, 1988.
12. L. Devroye and G. Lugosi, "Lower bounds in pattern recognition and learning," Pattern Recogn., 1995, to appear.
13. L. Devroye and T. J. Wagner, "Notiparametric discrimination and density estimation," Tech. Rep. 183, Electronics Res. Ctr., Univ. of Texas, Austin, 1976.
14. [141 A. Faragó and G. Lugosi, "Strong universal consistency of neural network classifiers," IEEE Trans. Inform. Theory, vol. 39, pp. 1146-1151, 1993.
15. D. Haussler, N. Littlestone, and M. Warmuth, "Predicting {0, 1} functions from randomly drawn points," in Proc. 29th IEEE Symp. on Foundations of Computer Science. Los Alamitos, CA: IEEE Computer Soc. Press, 1988, pp. 100-109.
16. G. Lugosi, "Improved upper bounds for probabilities of uniform deviations," Statist. Probability Lett., vol. 25. pp. 71-77, 1995.
17. J. Rissanen, "A universal prior for integers and estimation by minimum description length," Annals Statist., vol. 11, pp. 416-431, 1983.
18. _, "Universal coding, information, prediction and estimation," IEEE Trans. Inform. Theory, vol. IT-30, pp. 629-636, 1984.
19. G. Schwarz, "Estimating the dimension of a model," Annals Statist., vol. 6, pp. 461-464, 1978.
20. L. G. Valiant, "A theory of the learnable," Commun. ACM, vol. 27, pp. 1134-1142, 1984.
21. V. N. Vapnik, Estimation of Dependencies Based on Empirical Data. New York: Springer, 1982.
22. _, "Inductive principles of the search for empirical dependencies (methods based on weak convergence of probability measures," in Proc. 2nd Annual Workshop on Computational Learning Theory, 1989, pp. 3-24.
23. V. N. Vapnik and A. Ya. Chervonenkis, "On the uniform convergence of relative frequencies of events to their probabilities," Theory of Probability and its Applications, vol. 16, pp. 264-280, 1971.
24. _, Theory of Pattern Recognition. Moscow: Nauka, 1974 (in Russian);
25. German translation: Theorie der Zeichenerkennung. Berlin: Akademie Verlag, 1979.