Efficient agnostic learning of neural networks with bounded fan-in

IEEE Transactions on Information Theory

Volumn 42, Issue 6 PART 2, 1996, Pages 2118-2132

  Lee, Wee Sun ^b   Bartlett, Peter L ^a,c   Williamson, Robert C ^a,c  

^a IEEE

^b UNIVERSITY OF NEW SOUTH WALES   (Australia)

^c AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

Agnostic learning; Artificial neural networks; Bounded fan in neural networks; Computational learning theory; Iterative approximation; Polynomial time learning algorithm; Rate of convergence

Indexed keywords

EID: 0001556720     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.556601     Document Type: Article

References (30)

1. A. Aho, J. Hopcroft, and J. Ullman, The Design and Analysis of Computer Algorithms. London, UK: Addison-Wesley, 1974.
2. M. Anthony and N. Biggs, Computational Learning Theory (Cambridge Tracts in Theoretical Computer Science vol. 30). Cambridge, UK: Cambridge Univ. Press, 1992.
3. A. R. Barron, "Complexity regularization with applications to artificial neural networks," in G. Roussa, Ed., Nonparametric Functional Estimation. Boston, MA, and Dordrecht, The Netherlands: Kluwer, 1990, pp. 561-576.
4. _, "Neural net approximation," in Proc. 7th Yale Workshop on Adaptive and Learning Systems, 1992.
5. _, "Universal approximation bounds for superposition of a sigmoidal function," IEEE Trans. Inform. Theory, vol. 39, pp. 930-945, 1993.
6. _, "Approximation and estimation bounds for artificial neural networks," Mach. Learning, vol. 14, pp. 115-133, 1994.
7. A. Blum and R. Rivest, "Training a 3-node neural network is NP-complete," Neural Net., vol. 5, pp. 117-127, 1992.
8. B. E. Boser, I. M. Guyon, and V. N. Vapnik, "A training algorithm for optimal margin classifiers," in Proc. 5th Annu. Workshop on Comput. Learning Theory. New York: ACM Press, 1992, pp. 144-152.
9. C. C. Craig, "On the Tchebychef inequality of Bernstein," Ann. Math. Statist., vol. 4, pp. 94-102, 1933.
10. A. Farago and G. Lugosi, "Strong universal consistency of neural network classifiers," IEEE Trans. Inform. Theory, vol. 39, pp. 1146-1151, 1993.
11. D. Haussler, "Decision theoretic generalizations of the (PAC) model for neural net and other learning applications," Inform. Comput., vol. 100, no. 1, pp. 78-150, Sept. 1992.
12. K. Hornik, M. Stinchcombe, and H. White, "Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks," Neural Net., vol. 3, pp. 551-560, 1990.
13. M. Jerrum, "Simple translation-invariant concepts are hard to learn," Inform. Comput., vol. 113, no. 2,. pp. 300-311, Sept. 1994.
14. L. K. Jones, "A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training," Ann. Statist., vol. 20, pp. 608-613, 1992.
15. J. S. Judd, Neural Network Design and the Complexity of Learning. Cambridge, MA: MIT Press, 1990.
16. M. J. Kearns and R. E. Schapire, "Efficient distribution-free learning of probabilistic concepts," J. Comput. Syst. Sci., vol/. 48, no. 3, p. 464, 1994.
17. M. J. Kearns, R. E. Schapire, and L. M. Sellie, "Toward efficient agnostic learning," Mach. Learning, vol. 17, no. 2, p. 115, 1994.
18. P. Koiran, "Efficient learning of continuous neural networks," in Proc. 7th Annu. ACM Workshop on Comput Learning Theory. New York: ACM Press, 1994, pp. 348-355.
19. W. S. Lee, P. L. Bartlett, and R. C. Williamson, "The importance of convexity in learning with squared loss.," submitted to IEEE Trans. Inform. Theory, 1995.
20. W. S. Lee, P. L. Bartlett, and R. C. Williamson, "On efficient agnostic learning of linear combinations of basis functions," in Proc. 8th Annu. Workshop on Comput. Learning Theory. New York:ACM Press, 1995, pp. 369-376.
21. W. Maass, "Agnostic PAC-learning of functions on analog neural networks" Neural Comput., vol. 7, no. 5, pp. 1054-1078, 1995.
22. D. F. McCaffrey and A. R. Gallant, "Convergence rates for single hidden layer feedforward networks," Neural Net., vol. 7, no. 1, pp. 147-158, 1994.
23. M. Minsky and S. Papert, "Perceptrons. Cambridge, MA: MIT Press, 1969.
24. D. Pollard, Convergence of Stochastic Processes. Berlin, Germany: Springer-Verlag, 1984.
25. _, "Uniform ratio limit theorems for empirical processes," Scandinavian J. Statist., Theory and Appl., vol. 22, no. 3, p. 271, 1995.
26. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propagation," in Parallel Distributed Processing - Explorations in the Microstructure of Cognition. Cambridge, MA: MIT Press, 1986, ch. 8, pp. 318-362.
27. N. Sauer, "On the density of families of sets," J. Comb. Theory (Ser. A), vol. 13, pp. 145-147, 1972.
28. J. Shawe-Taylor, M. Anthony, and N. L. Biggs, "Bounding the sample size with the Vapnik-Chervonenkis dimension," Discr. Appl. Math., vol. 42, pp. 65-73, 1993.
29. L. G. Valiant, "A theory of the learnable," Commun. ACM, vol. 27, no. 11, pp. 1134-1142, Nov. 1984.
30. V. N. Vapnik and A. Y. Chervonenkis, "On the uniform convergence of relative frequencies of events to their probabilities," Theory of Probab. and its Appl., vol. 16, no. 2, pp. 264-280, 1971.