Best approximation of Gaussian neural networks with nodes uniformly spaced

IEEE Transactions on Neural Networks

Volumn 19, Issue 2, 2008, Pages 284-298

  Mulero Martinez, Juan Ignacio ^a  

^a TECHNICAL UNIVERSITY OF CARTAGENA   (Spain)

Author keywords

Best approximation; Existence; Functions band limited in frequency; Gaussian radial basis functions (RBFs); Sampling theory; Truncation; Uniqueness

Indexed keywords

GAUSSIAN DISTRIBUTION; HILBERT SPACES; SAMPLING;

BAND-LIMITED FUNCTION; ENERGY-BASED ARGUMENT;

RADIAL BASIS FUNCTION NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; NONLINEAR SYSTEM; NORMAL DISTRIBUTION;

COMPUTER SIMULATION; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; NORMAL DISTRIBUTION; PATTERN RECOGNITION, AUTOMATED;

EID: 40549110625     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2007.905851     Document Type: Article

References (70)

1. S. M. Nikolskii, Approximation of Functions of Several Variables and Imbedding Theorems. Berlin, Germany: Springer-Verlag, 1975.
2. M. Plancherel and G. Pólya, "Functions entières et intégrales de fourier multiples," Comment. Math. Hel., vol. 10, pp. 110-163, 1937-1938.
3. C. E. Shannon, "Communication in the presence of noise," Proc. IEEE vol. 86, no. 2, pp. 447-457, Feb. 1998.
4. H. N. Mhaskar, "On the representation of band limited functions using finitely many bits," J. Complexity, vol. 18, no. 2, pp. 449-478, Jun. 2002.
5. M. Unser, "Sampling - 50 years after Shannon," Proc. IEEE, vol. 88, no. 4, pp. 569-587, Apr. 2000.
6. N. Wiener, The Fourier Integral and Certain of its Applications. New York: Dover, 1933.
7. J. Park and I. W. Sandberg, "Universal approximation using radial-basis-function networks," Neural Comput., vol. 3, no. 2, pp. 246-257, 1991.
8. H. N. Mhaskar, "Versatile Gaussian networks," in Proc. IEEE Workshop Nonlinear Image Process., 1995, pp. 70-73.
9. J. Bruna, X. Massaneda, and J. Ortega-Cerdà, "Connections between signal processing and complex analysis," Contrib. Sci., vol. 2, no. 3, pp. 345-357, 2003.
10. F. Girosi and G. Anzellotti, "Convergence rates of approximation by translates," Massachusetts Inst. Technol., Artif. Intell. Lab., Cntr. Biol. Inf. Process., Whitaker College, Cambridge, MA, Tech. Rep. A.I. Memo. C.B.I.P. Paper No. 73, 1995.
11. A. T. Dingankar, "On applications of approximation theory to identification, control and classification," Ph.D. dissertation, Dept. Electr. Comp. Eng., Inf. Syst. Eng. Area, Univ. Texas, Arlington, TX, 1995.
12. A. R. Barron, "Universal approximation bounds for superpositions of a sigmoidal function," IEEE Trans. Inf. Theory, vol. 39, no. 3, pp. 930-945, May 1993.
13. M. A. Kowalski, "Optimal complexity recovery of band- and energy-limited signals," J. Complexity, vol. 2, no. 3, pp. 239-254, Sep. 1986.
14. M. A. Kowalski and F. Stenger, "Optimal complexity recovery of band-and energy-limited signals ii," J. Complexity, vol. 5, no. 1, pp. 45-59, Mar. 1989.
15. R. M. Sanner and J. E. Slotine, "Gaussian networks for direct adaptive control," IEEE Trans. Neural Netw., vol. 3, no. 6, pp. 837-863, Nov. 1992.
16. R. Zbikowski and K. J. Hunt, Neural Adaptive Control Technology. Singapore: World Scientific, 1996, vol. 15.
17. C. A. Micchelli, "Interpolation of scattered data: Distance matrices and conditionally positive definite functions," Constructive Approx. vol. 2, pp. 11-12, 1986.
18. F. Girosi and T. Poggio, "Networks and the best approximation property," Biol. Cybern., vol. 63, pp. 169-176, 1990.
19. F. Girosi and T. Poggio, "Networks and the best approximation property," Massachusetts Inst. Technol., Artif. Intell. Lab., Cntr. Biol. Inf. Process., Whitaker College, Cambridge, MA, Tech. Rep., 1989.
20. T. L. Fine, Feedforward Neural Network Methodology, ser. Statistics for Engineering and Information Science. New York: Springer-Verlag, 1999.
21. G. Chen and G. Fang, "The discrete characterization of Paley-Wiener space with several variables," Prog. Natural Sci., vol. 13, pp. 300-303, 2003.
22. G. Fang, "Whittaker-Kotelnikov-Shannnon sampling theorem and aliasing error," J. Approx. Theory, vol. 85, pp. 115-131, 1996.
23. J. R. Higgins, "Five short stories about the cardinal series," Bull Amer. Math. Soc., vol. 12, pp. 45-89, 1985.
24. G. Hinsen, "Irregular sampling of bandlimited LP-functions," J. Approx. Theory, vol. 72, pp. 346-364, 1993.
25. B. Y. Levin, Lectures on Entire Functions. Providence, RI: American Mathematical Society, 1996.
26. K. T. Smith, D. C. Solomon, and S. L. Wagner, "Practical and mathematical aspects of the problem of reconstructing objects from radiographs," Bull. Amer. Math. Soc., vol. 83, pp. 1227-1270, 1977.
27. E. M. Stein and W. Guido, Introduction to Fourier Analysis on Euclidean Spaces, 6th ed. Princeton, NJ: Princeton Univ. Press, 1990.
28. J. Ortega-Cerdá and K. Seip, "Fourier frames," Ann. Math, vol. 155, no. 3, pp. 789-806, 2002.
29. M. Vidyasagar, Learning and Generalization With Applications to Neural Networks, ser. Communications and Control Engineering. London, U.K.: Springer-Verlag, 2003.
30. N. Sadegh, "A perceptron network for functional identification and control of nonlinear systems," IEEE Trans. Neural Netw., vol. 4, no. 6, pp. 982-988, Nov. 1993.
31. J. Stewart, "Positive de nite functions and generalizations, an historical survey," Rocky Mountain J. Math, vol. 6, pp. 409-434, 1976.
32. P. C. Kainen, V. Kurková, and A. Vogt, "Approximation by neural networks is not continuous," Neurocomputing, vol. 29, pp. 47-56, 1999.
33. R. Hecht-Nielsen, "On the algebraic structure of feedforward network weight spaces," Adv. Neural Comput., pp. 129-135, 1990.
34. H. Sussmann, "Uniqueness of the weights for minimal feedforward nets with a given input-output map," Neural Netw., vol. 5, pp. 589-594, 1992.
35. C. Chui, X. Li, and H. Mhaskar, "Neural networks for localized approximation," Math. Comput., vol. 63, pp. 607-623, 1994.
36. V. Kurkova and R. Neruda, "Uniqueness of functional representations by Gaussian basis function networks," in Proc. Int. Conf. Artif. Neural Netw., London, U.K., 1994, pp. 471-474.
37. K. Hornik, M. Stinchcombe, and H. White, "Multilayer feedforward networks are universal approximators," Neural Netw., vol. 2, pp. 359-366, 1989.
38. N. E. Cotter, "The Stone-Weierstrass theorem and its application to neural networks," IEEE Trans. Neural Netw., vol. 1, no. 4, pp. 290-295, Dec. 1990.
39. R. Hecht-Nielsen, Neurocomputing. Reading, MA: Addison-Wesley, 1990.
40. K.-I. Funahashi, "On the approximate realization of continuous mappings by neural networks," Neural Netw., vol. 2, pp. 183-192, 1989.
41. V. Kurkova, "Kolmogorov's theorem and multilayer neural networks," Neural Netw., vol. 5, pp. 501-506, 1992.
42. H. Katsuura and D. Sprecher, "Computational aspects of Kolmogorov's superposition theorem," Neural Netw., vol. 7, pp. 455-461, 1994.
43. D. Sprecher, "A universal mapping for Kolmogorov's superposition theorem," Neural Netw., vol. 6, pp. 1089-1094, 1993.
44. K. Hornik, "Approximation capabilities of multilayer feedforward networks," Neural Netw., vol. 4, pp. 251-257, 1991.
45. L. Jones, "Good weights and hyperbolic kernels for neural networks, projection pursuit, and pattern classification: Fourier strategies for extracting information from high-dimensional data," IEEE Trans. Inf. Theory, vol. 40, no. 2, pp. 439-454, Mar. 1994.
46. P. Koiran, "On the complexity of approximating mappings using feed-forward networks," Neural Netw., vol. 6, pp. 649-653, 1993.
47. M. Leshno, V. Y. Lin, A. Pinkus, and S. Schocken, "Multilayer feedforward networks with a nonpolynomial activation can approximate any function," Neural Netw., vol. 6, pp. 861-867, 1993.
48. H. Mhaskar, "Approximation properties of a multilayered feedforward artificial neural network," in Advances in Computational Mathematics Dordrecht, The Netherlands: Springer-Verlag, 1993, vol. 1, pp. 61-80.
49. G. Cybenko, "Approximation by superpositions of a sigmoidal function," Math. Control Signals Syst., vol. 2, no. 4, pp. 303-314, 1989.
50. A. F. Timan, C. J. Hyman, and N. I. Achieser, Theory of Approximation of Functions of a Real Variable. New York: Dover, 1993.
51. P. C. Kainen, V. Kurková, and A. Vogt, "Best approximation by heaviside perceptron networks," Neural Netw., vol. 13, pp. 695-697, 2000.
52. H. N. Mhaskar and D. V. Pai, Fundamentals of Approximation Theory. India: Narosa House, 2000.
53. B. Liu and J. Si, "The best approximation to C2 functions and its error bounds using regular-center Gaussian networks," IEEE Trans. Neural Netw., vol. 5, no. 5, pp. 845-847, Sep. 1994.
54. B. Liu and J. Si, "The best approximation properties and error bounds of Gaussian networks," in Proc. IEEE 32nd Conf. Decision Control, San Antonio, TX, Dec. 1993, pp. 2798-2803.
55. B. Liu and J. Si, "The best approximation to C2 functions and its error bounds using regular-center Gaussian networks," in Proc. Int. Conf. Neural Netw. Control, 1994, pp. 2400-2406.
56. J. Moody and C. Darken, "Fast learning in networks of locally-tuned processing units," Neural Comput., vol. 1, no. 2, pp. 281-294, 1989.
57. T. Poggio and F. Girosi, "Networks for approximation and learning," Proc. IEEE, vol. 78, no. 9, pp. 1481-1497, Sep. 1990.
58. T. Poggio and S. Edelman, "A network that learns to recognize 3D objects," Nature, vol. 343, pp. 263-266, 1990.
59. R. Klopfenstein and R. Sverdlove, "Approximation by uniformly spaced Gaussian functions," in Approximation Theory IV, E. Cheney, Ed. New York: Academic, 1983, pp. 575-580.
60. P. C. Kainen, V. K. Urkova, and M. Sanguineti, "Minimization of error functionals over variable-basis functions," Inst. Comput. Sci., Acad. Sci. Czech Republic, Prague, Czech Republic, Tech. Rep. 864, 2002.
61. J. R. Rice, The Approximation of Functions. Reading, MA: Addison-Wesley, 1964, vol. 1.
62. P. Koosis, "Leçons sur le théorème de beurling et malliavin," Centre de recherches mathématiques, Université de Montréal, Montreal, QC, Canada, 1996.
63. J. Kallkuhl, K. J. Hunt, R. Zbikowski, and A. J. Dzielinshi, Applications of Neural Adaptive Control Technology. Singapore: World Scientific, 1997.
64. N. A. Borghese and S. Ferrari, "Hierarchical RBF networks and local parameters estimate," Neurocomputing, pp. 259-283, 1998.
65. S. Ferrari, M. Maggioni, and N. A. Borghese, "Multiscale approximation with hierarchical radial basis functions networks," IEEE Trans. Neural Netw., vol. 15, no. 1, pp. 178-188, Jan. 2004.
66. J. R. Munkres, Elements of Algebraic Topology. Reading, MA: Perseus Books, 1993.
67. E. L. Escardó, Análisis Matemático II. Madrid, Spain: Universidad Nacional de Educación a Distancia (UNED), 1991.
68. N. Lebedev, Special Functions and their Applications. New York: Dover, 1972.
69. R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge, U.K.: Cambridge Univ. Press, 1999.
70. T. M. Apostol, Calculus. One-Variable Calculus, With an Introduction to Linear Algebra, 2nd ed. New York: Wiley, 1967.