The importance of convexity in learning with squared loss

IEEE Transactions on Information Theory

Volumn 44, Issue 5, 1998, Pages 1974-1980

  Lee, Wee Sun ^a,b   Bartlett, Peter L ^a,b   Williamson, Robert C ^a,b  

^a UNIVERSITY OF NEW SOUTH WALES   (Australia)

^b AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

Agnostic learning, artificial neural networks, computational learning theory, convex hull, sample complexity

Indexed keywords

COMPUTATIONAL COMPLEXITY; LEARNING ALGORITHMS; MATHEMATICAL MODELS; NEURAL NETWORKS; PROBABILITY DISTRIBUTIONS;

AGNOSTIC LEARNING; CONVEX HULL;

ARTIFICIAL INTELLIGENCE;

EID: 0032166052     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.705577     Document Type: Article

References (21)

1. M. Anthony and N. Biggs, "Computational learning theory," in Cambridge Tracts in Theoretical Computer Science, vol. 30. Cambridge, U.K.: Cambridge Univ. Press, 1992.
2. A. R. Barron, "Complexity regularization with applications to artificial neural networks," in Nonparametric Functional Estimation, G. Roussa, Ed. Boston, MA /Dordrecht, The Netherlands: Kluwer, 1990, pp. 561-576.
3. _, "Neural net approximation," in P roc. 7th Yale Workshop on Adaptive and Learning Systems, 1992.
4. _, "Universal approximation bounds for superposition of a sigmoidal function," IEEE Trans. Inform. Theory, vol. 39, pp. 930-945, 1993.
5. P. L. Bartlett, S. R. Kulkarni, and S. E. Posner, "Covering numbers for real-valued function classes," submitted to IEEE Trans. Inform, Theory, 1997.
6. S. Ben-David and M. Lindenbaum, "Learning distributions by their density levels-A paradigm for learning without a teacher," in Computational Learning Theory: EUROCOLT'95, 1995, pp. 53-68.
7. G. Benedek and A. Itai, "Learnability with respect to fixed distributions," Theor. Comput. Sei, vol. 86, no. 2, pp. 377-389, 1991.
8. D. Braess, Nonlinear Approximation Theory. Berlin, Germany: Springer-Verlag, 1986.
9. D. Haussler, "Decision theoretic generalizations of the PAC model for neural net and other learning applications," Inform. Comp., vol. 100, no. 1, pp. 78-150, Sept. 1992.
10. L. K. Jones, "A simple lemma on greedy approximation in Hubert space and convergence rates for projection pursuit regression and neural network training," A. Stat., vol. 20, pp. 608-613, 1992.
11. M. J. Kearns and R. E. Schapire, "Efficient distribution-free learning of probabilistic concepts," in Proc. 31st Symp. Foundations of Computer Science. Los Alamitos, CA: IEEE Computer Soc., 1990, pp. 382-391.
12. M. J. Kearns, R. E. Schapire, and L. M. Sellie, "Toward efficient agnostic learning," Mach. Learn., vol. 17, no. 2, pp. 115-142, 1994.
13. A. N. Kolmogorov and S. V. Fomin, Introductory Real Analysis. New York: Dover, 1970.
14. W. S. Lee, P. L. Bartlett, and R. C. Williamson, "On efficient agnostic learning of linear combinations of basis functions," in Proc. 8th Annu. Workshop Computer Learning Theory. New York: ACM Press, 1995, pp. 369-376.
15. _, "Efficient agnostic learning of neural networks with bounded fan-in," IEEE Trans. Inform. Theory, vol. 42, pp. 2118-2132, Nov. 1996.
16. W. Maass, "Agnostic PAC-learning of functions on analog neural networks," Neur. Comp., vol. 7, no. 5, pp. 1054-1078, 1995.
17. Y. Makovoz, "Random approximants and neural networks," J. Approx. Theory, vol. 85, pp. 98-109, 1996.
18. D. F. McCaffrey and A. R. Gallant, "Convergence rates for single hidden layer feedforward networks," Neur. Net., vol. 7, no. 1, pp. 147-158, 1994.
19. D. Pollard, Convergence of Stochastic Processes. Berlin, Germany: Springer-Verlag, 1984.
20. _, "Uniform ratio limit theorems for empirical processes," Scand. J. Stat., vol. 22, no. 3, pp. 271-278, 1995.
21. A. W. van der Vaart and J. A. Wellner, Weak Convergence and Empirical Processes. New York: Springer-Verlag, 1996.