Learning by canonical smooth estimation - Part I: Simultaneous estimation

IEEE Transactions on Automatic Control

Volumn 41, Issue 4, 1996, Pages 545-556

  Buescher, Kevin L ^a,b   Kumar, P R ^a,c  

^a IEEE

^b LOS ALAMOS NATIONAL LABORATORY   (United States)

^c University of Illinois at Urbana Champaign   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ERRORS; MATHEMATICAL MODELS; OPTIMIZATION; PROBABILITY; SET THEORY; STATISTICS;

CANONICAL SMOOTH ESTIMATION; PROBABLY APPROXIMATELY CORRECT MODEL; SIMULTANEOUS ESTIMATION; SMOOTH ESTIMATOR;

PARAMETER ESTIMATION;

EID: 0030128623     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/9.489275     Document Type: Article

References (42)

1. K. L. Buescher and P. R. Kumar, "Learning by canonical smooth estimation, Part II: Learning and choice of model complexity," IEEE Trans. Automat. Contr., vol. 41, no. 4, pp. 557-569, 1996.
2. L. Valiant, "A theory of the learnable," Communications ACM, vol. 27, pp. 1134-1142, Nov. 1984.
3. D. Angluin, "Queries and concept learning," Machine Learning, vol. 2, no. 4, pp. 319-342, 1988.
4. A. Blumer, A. Ehrenfeucht, D. Haussler, and M. K. Warmuth, "Learnability and the Vapnik-Chernonenkis dimension," J. ACM, vol. 36, no. 3, pp. 939-965, 1989.
5. M. Anthony and N. Biggs, Computational Learning Theory. Cambridge, MA: Cambridge Univ., 1992.
6. D. Haussler, M. Kearns, N. Littlestone, and M. K. Warmuth, "Equivalence of models of polynomial learnability," Inform. Computation, vol. 95, pp. 129-161, 1991.
7. D. Angluin and P. D. Laird, "Learning from noisy examples," Machine Learning, vol. 2, no. 4, pp. 343-370, 1988.
8. G. M. Benedek and A. Itai, "Nonuniform learnability," in Proc. 15th Int. Colloquium Automata, Languages, Programming, vol. 317, July 1988. New York: Springer-Verlag, pp. 81-92.
9. _, "Learnability by fixed distributions," in Computational Learning Theory: Proc. 1988 Workshop. San Mateo, CA: Morgan Kaufmann, 1988, pp. 80-90.
10. M. Linial, Y. Mansour, and R. L. Rivest, "Results on learnability and the Vapnik-Chervonenkis dimension," in 29th Annu. IEEE Symp. Foundations Computer Sci., 1988, pp. 120-129.
11. B. K. Natarajan, "Learning over classes of distributions," in Computational Learning Theory: Proc. 1988 Workshop. San Mateo, CA: Morgan Kaufmann, 1988, pp. 408-409.
12. R. Sloan, "Types of noise in data for concept learning," in Computational Learning Theory: Proc. 1988 Workshop. San Mateo, CA: Morgan Kaufmann, 1988, pp. 91-96.
13. B. K. Natarajan, "On learning sets and functions," Machine Learning, vol. 4, no. 1, pp. 67-97, 1989.
14. S. Ben-David, G. M. Benedek, and Y. Mansour, "A parameterization scheme for classifying models of learnability," in Computational Learning Theory: Proc. 2nd Workshop. San Mateo, CA: Morgan Kaufmann, 1989, pp. 285-302.
15. D. Haussler, "Generalizing the PAC model: Sample size bounds from metric dimension-based uniform convergence results," 30th Annu. IEEE Symp. Foundations Computer Sci., 1989, pp. 40-45.
16. M. J. Kearns and R. E. Schapire, "Efficient distribution-free learning of probabilistic concepts," 31st Ann. IEEE Symp. Foundations Computer Sci., 1990, pp. 382-391.
17. A. T. Ogielski, "Information, probability, and learning from examples," unpublished course notes, Bell Communications Res., Morristown, NJ, May 1990.
18. S. R. Kulkarni, "Problems of computational and information complexity in machine vision and learning," Center for Intelligent Control Systems, M.I.T., Cambridge, MA, Tech. Rep. CICS-TH-298, July 1991; also Ph.D. dissertation, Dep. Electrical Engineering and Computer Science.
19. B. K. Natarajan, "Probably approximate learning of sets and functions," SIAM J. Computing, vol. 20, no. 2, pp. 328-351, 1991.
20. _, Machine Learning: A Theoretical Approach. San Mateo, CA: Morgan Kaufmann, 1991.
21. D. Haussler, "Decision theoretic generalizations of the PAC model for neural net and other learning applications," Inform. Computation, vol. 100, pp. 78-150, 1992.
22. M. J. Kearns, R. K. Schapire, and L. M. Sellie, "Toward efficient agnostic learning," in Proc. 5th Annu. ACM Workshop Computational Learning Theory, 1992, pp. 341-352.
23. V. N. Vapnik, Estimation of Dependences Based on Empirical Data. New York: Springer-Verlag, 1982.
24. S. Ben-David, N. Cesa-Bianchi, and P. M. Long, "Characterizations of learnability for classes of {0, ⋯, n}-valued functions," in Proc. 5th Ann. ACM Workshop Computational Learning Theory, pp. 333-340, 1992.
25. V. N. Vapnik and A. Chervonenkis, "On the uniform convergence of relative frequencies of events to their probabilities," Theory Probability Appl., vol. 16, no. 2, pp. 264-280, 1971.
26. _, "Necessary and sufficient conditions for the uniform convergence of means to their expectations," Theory Probability Appl., vol. 26, no. 3, pp. 532-553, 1981.
27. R. M. Dudley, "Central limit theorems for empirical measures," Annals Probability, vol. 6, no. 6, pp. 899-929, 1978.
28. _, "A course on empirical processes," in Lecture Notes in Mathematics, vol. 1097. New York: Springer-Verlag, 1984.
29. D. Pollard, Convergence of Stochastic Processes. New York: Springer-Verlag, 1984.
30. _, Empirical Processes: Theory and Applications. Hayward, CA: Institute Mathematical Statistics, 1990.
31. M. Talagrand, "The Glivenko-Cantelli problem," Annals Probability, vol. 15, no. 3, pp. 837-870, 1987.
32. P. Gaenssler, Empirical Processes, vol. 3 of IMS Lecture Notes - Monograph Series. Hayward, CA: Institute of Mathematical Statistics, 1983.
33. E. Gine and J. Zinn, "Some limit theorems for empirical processes," Annals Probability, vol. 12, no. 4, pp. 929-989, 1984.
34. R. Duda and P. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
35. L. Devroye, "Automatic pattern recognition: A study of the probability of error," IEEE Trans. Pattern Analysis Machine Intell., vol. 10, no. 4, pp. 530-543, 1988.
36. K. Fukunaga, Statistical Pattern Recognition. 2nd ed. New York: Academic, 1990.
37. L. Pitt and L. G. Valiant, "Computational limitations on learning from examples," J. ACM, vol. 35, no. 4, pp. 965-984, 1988.
38. S. R. Kulkarni, "On metric entropy, Vapnik-Chervonenkis dimension, and learnability for a class of distributions," Center for Intelligent Control Systems, M.I.T., Cambridge, MA, Tech. Rep. CICS-P-160, October 1989.
39. S. R. Kulkarni and M. Vidyasagar, "Learning decision rules for pattern classification under a family of probability measures," preprint, Oct., 1993.
40. W. Hoeffding, "Probability inequalities for sums of bounded random variables," J. Amer. Statistical Assoc., vol. 58, pp. 13-30, 1963.
41. A. N. Kolmogorov and V. M. Tihomirov, "ε-entropy and ε-capacity of sets in functional spaces," Amer. Math. Soc. Translations, Ser. 2, vol. 17, pp. 277-364, 1961.
42. A. B. Nobel, "On uniform laws of averages," Stanford University, Dep. Statistics, Stanford, CA, Tech. Rep. 81, June 1992; also Ph.D. dissertation, Dep. Electrical Engineering.