Training neural networks with noisy data as an ill-posed problem

Advances in Computational Mathematics

Volumn 13, Issue 4, 2000, Pages 335-354

  Burger, Martin ^a   Engl, Heinz W ^a  

^a JOHANNES KEPLER UNIVERSITY LINZ   (Austria)

Author keywords

Ill posed problems; Least squares collocation; Network training; Neural networks; Regularization

Indexed keywords

EID: 0034558491     PISSN: 10197168     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1016641629556     Document Type: Article

References (40)

1. U. Amato and D.T. Vuza, Besov regularization, thresholding and wavelets for smoothing data, Numer. Funct. Anal. Optim. 18 (1997) 461-493.
2. K.E. Atkinson and I.H. Sloan, The numerical solution of first-kind logarithmic-kernel integral equations on smooth open arcs, Math. Comp. 56 (1991) 119-139.
3. A.R. Barron, Universal approximation bounds for superpositions of a sigmoidal function, IEEE Trans. Inform. Theory 39 (1993) 930-945.
4. C.M. Bishop, Neural Networks for Pattern Recognition (Clarendon Press, Oxford, 1995).
5. C.M. Bishop, Regularization and complexity control in feed-forward networks, EC2 & Cie (1995) 141-148.
6. M. Burger and A. Neubauer, Error bounds for approximation with neural networks, submitted.
7. C.K. Chui and X. Li, Approximation by ridge functions and neural networks with one hidden layer, J. Approx. Theory 70 (1992) 131-141.
8. I. Daubechies, Ten Lectures on Wavelets (SIAM, Philadelphia, PA, 1992).
9. H.W. Engl, Regularization by least-squares collocation, in: Numerical Treatment of Inverse Problems in Differential and Integral Equations, eds. P. Deuflhard and E. Hairer (Birkhäuser, Boston, 1983) pp. 345-354.
10. H.W. Engl, M. Hanke and A. Neubauer, Regularization of Inverse Problems (Kluwer, Dordrecht, 1996).
11. H.G. Feichtinger and T. Strohmer, eds., Gabor Analysis and Applications (Birkhäuser, Boston, 1998).
12. W. Freeden and F. Schneider, Regularization wavelets and multiresolution, Inverse Problems 14 (1998) 225-243.
13. F. Girosi and G. Anzellotti, Convergence rates of approximation by translates, AI Memo 1288, AI Laboratory, MIT, Cambridge, MA (1995).
14. F. Girosi and T. Poggio, A theory of networks for approximation and learning, AI Memo 1140, AI Laboratory, MIT, Cambridge, MA (1989).
15. F. Girosi and T. Poggio, Networks and the best approximation property, Biol. Cybern. 63 (1990) 169-176.
16. F. Girosi, M. Jones and T. Poggio, Regularization theory and neural networks architectures, Neural Comput. 7 (1995) 219-269.
17. C.W. Groetsch, Generalized inverses and generalized splines, Numer. Funct. Anal. Optim. 2 (1980) 93-97.
18. W. Hackbusch, Integral Equations (Birkhäuser, Basel, 1995).
19. M. Hanke and O. Scherzer, Numerical differentiation as an example for inverse problems, Preprint 98/16, University Karlsruhe (1998).
20. M. Hegland and R.S. Anderssen, A mollification framework for improperly posed problems, Numer. Math. 78 (1998) 549-575.
21. K. Hornik, M. Stinchcombe and H. White, Multilayer feedforward networks are universal approxi-mators, Neural Networks 2 (1989) 359-366.
22. R. Kress, Linear Integral Equations, 2nd edition (Springer, New York, 1999).
23. A.K. Louis, Inverse und Schlecht Gestellte Probleme (Teubner, Stuttgart, 1989).
24. A.K. Louis and P. Maaß, Smoothed projection methods for the moment problem, Numer. Math. 59 (1991) 277-294.
25. A.K. Louis, Approximate inverse for linear and some nonlinear problems, Inverse Problems 12 (1996) 175-190.
26. Y. Makovoz, Uniform approximation by neural networks, J. Approx. Theory 95 (1998) 215-228.
27. Y. Meyer, Wavelets, Algorithms and Applications (SIAM, Philadelphia, PA, 1993).
28. D. A. Murio, The Mollification Method and the Numerical Solution of Ill-posed Problems (Wiley, New York, 1993).
29. H.N. Mhaskar and C.A. Micchelli, Degree of approximation by neural and translation networks with a single hidden layer, Adv. Appl. Math. 16 (1995) 151-183.
30. I.T. Nabney, Efficient training of RBF networks for classification, NCRG Technical Report 99/02 (1999).
31. M.Z. Nashed and G. Wahba, Convergence rates of approximate least squares solutions to linear integral and operator equations of the first kind, Math. Comp. 28 (1974) 69-80.
32. F. Natterer, Regularisierung schlecht gestellter Probleme durch Projektionsverfahren, Numer. Math. 28 (1977) 329-341.
33. P. Niyogi and F. Girosi, Generalization bounds for function approximation from scattered noisy data, Adv. Comput. Math. 10 (1999) 51-80.
34. A. Sard, Approximations based on nonscalar observations, J. Approx. Theory 8 (1973) 315-334.
35. J. Sjöberg, Q. Zhang, L. Ljung, A. Benveniste, B. Deylon, P.Y. Glorennec, H. Hjalmarsson and A. Juditsky, Nonlinear black-box modeling in system identification: A unified overview, Automatica 31 (1995) 1691-1724.
36. A.N. Tikhonov and V.Y. Arsenin, Solutions of Inverse Problems (Wiley, New York, 1977).
37. G. Wahba, Convergence rates of certain approximate solutions to fredholm integral equations of the first kind, J. Approx. Theory 7 (1973) 167-185.
38. G. Wahba, A class of approximate solutions to linear operator equations, J. Approx. Theory 9 (1973) 61-77.
39. Y. Xu and Y Zhao, Quadratures for boundary integral equations of the first kind with logarithmic kernels, J. Integral Equations Appl. 8 (1996) 239-268.
40. P. Yee and S. Haykin, Pattern classification as an ill-posed, inverse problem: a regularization approach, in: Proc. ICASSP-93 (Minneapolis, 1993).