Towards a black box algorithm for nonlinear function approximation over high-dimensional domains

SIAM Journal on Scientific Computing

Volumn 29, Issue 3, 2007, Pages 941-963

  Jamshidi, Arta A ^a   Kirby, Michael J ^a  

^a Department of Environmental and Radiological Health Sciences   (United States)

Author keywords

Algorithms for nonlinear functional approximation; Curve fitting; Manifold learning; Radial basis functions

Indexed keywords

CURVE FITTING; MATHEMATICAL MODELS; NONLINEAR ANALYSIS; RADIAL BASIS FUNCTION NETWORKS;

MANIFOLD LEARNING; NONLINEAR FUNCTIONAL APPROXIMATION;

APPROXIMATION THEORY;

EID: 44649173689     PISSN: 10648275     EISSN: None     Source Type: Journal    
DOI: 10.1137/050646457     Document Type: Article

References (48)

1. H. AKAIKE, A new look at statistical model identification, IEEE Trans. Automat. Control, 19 (1974), pp. 716-723.
2. M. ANDERLES AND M. KIRBY, Correlation feedback resource allocation RBFs, in Proceedings of the 2001 IEEE International Joint Conference on Neural Networks (IJCNN), Washington, DC, 2001, Vol. 3, pp. 1949-1953.
3. M. ANDERLE, Modeling Geometric Structure in Noisy Data, Ph.D. dissertation, Department of Mathematics, Colorado State University, Fort Collins, CO, 2001.
4. C. ANDRIEU, N. FREITAS, AND A. DOUCET, Robust full Bayesian learning for radial basis networks, Neural Comput., 13 (2001), pp. 2359-2407.
5. C. M. BISHOP, Neural Networks for Pattern Recognition, Oxford University Press, Oxford, UK, 1995.
6. P. J. BROCKWELL aND R. A. DAVIS, Time Series: Theory and Methods, 2nd ed., Springer Ser. Statist., Springer, New York, 1991.
7. D. S. BROOMHEAD AND M. KIRBY, A new approach for dimensionality reduction: Theory and algorithms, SIAM J. Appl. Math., 60 (2000), pp. 2114-2142.
8. D. S. BROOMHEAD AND M. KIRBY, The Whitney reduction network: A method for computing autoassociative graphs, Neural Comput., 13 (2001), pp. 2595-2616.
9. D. S. BROOMHEAD AND M. KIRBY, Large dimensionality reduction using secant-based projection methods: The induced dynamics in projected systems, Nonlinear Dynam., 41 (2005), pp. 47-67.
10. D. S. BROOMHEAD AND D. LAOWE, Multivariable functional interpolation and adaptive networks, Complex Systems, 2 (1988), pp. 321-355.
11. J. V. CANDY, Signal Processing: The Model-Based Approach, McGraw-Hill Series in Electrical Engineering, Communications and Signal Processing, McGraw-Hill, New York, 1986.
12. Y. LE CUN, J. S. DENKER, AND S. A. SOLLA, Optimal brain damage, in Advances in Neural Information Processing Systems, D. S. Touretzky, ed., Morgan Kaufmann, San Mateo, CA, 1990, Vol. 2, pp. 598-605.
13. S. E. FAHLMAN AND C. LEBIERE, The cascade-correlation learning architecture, in Proceedings of the Connectionist Models Summer School, D. S. Touretzky, ed., Morgan Kaufmann, San Mateo, CA, 1988, Vol. 2, pp. 524-432.
14. M. FREAN, An upstart algorithm: A method for constructing and training feedforward neural networks, Neural Comput., 2 (1990), pp. 198-209.
15. B. FRITZKE, Fast learning with incremental RBF networks, Neural Process. Lett., 1 (1994), pp. 2-5.
16. B. FRITZKE, Supervised learning with growing cell structures, in Advances in Neural Information Processing Systems, G. Tesauro J. Cowan, and J. Alspector, eds., Morgan Kaufmann, San Mateo, CA, 1994, Vol. 6, pp. 255-262.
17. F. GIROSI, M. JONES, AND T. POGGIO, Regularization theory and neural network architectures, Neural Comput., 7 (1995), pp. 219-269.
18. S. HAYKIN, Neural Networks: A Comprehensive Foundation, 2nd ed., Prentice-Hall, Englewood Cliffs, NJ, 1999.
19. C. C. HOLMES AND B. K. MAI.LICK, Bayesian radial basis functions of variable dimension, Neural Comput., 10 (1998), pp. 1217-1233.
20. A. A. JAMSHIDI AND M. J. KIRBY, Examples of compactly supported functions for radial basis approximations, in Proceedings of the 2006 International Conference on Machine Learning; Models, Technologies and Applications, H. R. Arabnia, E. Kozerenko, and S. Shaumyan, eds., Las Vegas, 2006, CSREA Press, Irvine, CA, 2006, pp. 155-160.
21. A. A. JAMSHIDI, A New Spatio-Temporal Resource Allocation Network (ST-RAN), Master's thesis, Department of Mathematics, Colorado State University, Fort Collins, CO, 2004.
22. A. A. JAMSHIDI, An Adaptive Underwater Target Classification System with a Decision Feedback Mechanism, Master's thesis, Department of Electrical and Computer Engineering, Colorado State University, Fort Collins, CO, 2002.
23. K. KADIRKAMANATHAN AND M. NIRANJAN, A function estimation approach to sequential learning with neural networks, Neural Comput., 5 (1993), pp. 954-975.
24. K. KADIRKAMANATHAN, A statistical inference based growth criterion for RBF network, in Proceedings of the IEEE Workshop on Neural Networks for Signal Processing IV, Ermioni, Greece, IEEE Press, Piscataway, NJ, 1994, pp. 12-21.
25. M. KIRBY AND C. ANDERSONS, Geometric analysis for the characterization of nonstationary time-series, in Springer Applied Mathematical Sciences Series Celebratory Volume for the Occasion of the 70th Birthday of Larry Sirovich, E. Kaplan, J. Marsden, and K. R. Sreenivasan, eds., Springer-Verlag, New York, 2003, pp. 263-292.
26. Y. LI. N. SUNDARARAJAN, AND P. SARATCHANDRAN, Analysis of minimal radial basis function network algorithm for real-time identification of nonlinear dynamic systems, IEE Proc. Control Theory Appl., 147 (2000), pp. 476-484.
27. L. LJUNG, System Identification Theory for the User, 2nd ed., Prentice-Hall PTR, Englewood Cliffs, NJ, 1999.
28. D. J. C. MACKAY, A practical Bayesian framework for backpropagation networks, Neural Comput., 4 (1989), pp. 448-472.
29. A. MACLACHLAN, An improved novelty criterion for resource allocating networks, in Proceedings of the IEE 5th International Conference on Artificial Neural Networks, London, 1997, Cambridge, UK, 1997, pp. 48-52.
30. A. D. MARRS, An application of reversible-jump MCMC to multivariate spherical Gaussian mixtures, in Advances in Neural Information Processing Systems, M. I. Kearns and S. A. Solla, eds., MIT press, Cambridge, MA, 1998, Vol. 10, pp. 577-583.
31. J. MOODY AND C. DARKEN, Fast learning in networks of locally tuned processing units, Neural Comput., 1 (1989), pp. 281-294.
32. M. J. L. ORR, Introduction to Radial Basis Function Networks (1996), Technical report, Center for Cognitive Science, University of Edinburgh, Edinburgh, Scotland, 1996.
33. M. J. L. ORR, An EM algorithm for regularized RBF networks, in International Conference on Neural Networks and Brain, Beijing, China, Publishing House of Electronics Industry, China, 1998.
34. M. J. L. ORR, Recent Advances in Radial Basis Function Networks, Technical report, Institute for Adaptive and Neural Computation, Division of Informatics, Edinburgh University, Edinburgh, Scotland, 1999.
35. J. PLATT, A resource allocating network for function interpolation, Neural Comput., 3 (1991), pp. 213-225.
36. M. J. D. POWELL, Radial basis functions for multivariable interpolation: A review, in Algorithms for Approximation, J. C. Mason and M. G. Cox, eds., Clarendon Press, Oxford, UK, 1987, pp. 143-167.
37. M. J. D. POWELL, The theory of radial basis functions in 1990, in Advances in Numerical Analysis, W. Light, ed., Wavelets, Subdivision Algorithms, and Radial Basis Functions 2, Oxford University Press, Oxford, UK, 1992, pp. 105-210.
38. D. INSUA RIOS AND P. MULLER, Feedforward neural networks for nonparametric regression, in Practical Nonparametric and Semiparametric Bayesian Statistics, P. Muller, D. K. Dey, and D. Sinha, eds., Springer-Verlag, New York, 1998, pp. 181-191.
39. D. E. RUMELHART AND J. L. MCCLELLAND, Parallel Distributed Processing, MIT Press, Cambridge, MA, 1986.
40. R. SCHABACK, Reconstruction of Multivariate Functions from Scattered Data, preprint, University of Go¨ttingen, Go¨ttingen, Germany, 1997.
41. P. SMYTH, On stochastic complexity and admissible models for neural network classifiers, in Proceedings of Neural Information Processing Systems, D. S. Touretzky, R. Lippmann, and J. Moody, eds., Morgan Kaufmann, San Mateo, CA, 1991, Vol. 3, pp. 818-824.
42. A. N. TIKHONOV AND V. Y. ARSENIN, Solutions of Ill-Posed Problems, John Wiley & Sons, New York, 1977.
43. G. WAHBA, Spline bases, regularization, and generalized cross validation for solving approximation problems with large quantities of data, in Approximation Theory III, W. Cheney, ed., Academic Press, New York, 1980, pp. 905-912.
44. P. J. WERBOS, Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences, Ph.D. dissertation, Harvard University, Cambridge, MA, 1974.
45. B. WIDROW AND M. E. HOFF, Adaptive switching circuits, in 1960 WESCON Convention Record Part IV, reprinted in Neurocomputing: Foundations of Research, J. A. Anderson and E. Rosenfeld, eds., MIT Press, Cambridge, MA, 1988, pp. 96-104.
46. L. YINGWEI, N. SUNDARARAJAN, AND P. SARATCHANDRAN, A sequential scheme for function approximation using minimal radial basis function neural networks, Neural Comput., 9 (1997), pp. 461-478.
47. L. YINGWEI, N. SUNDARARAJAN, AND P. SARATCHANDRAN, Performance evaluation of a sequential minimal radial basis function (RBF) neural network learning algorithm, IEEE Trans. Neural Networks, 9 (1998), pp. 308-318.
48. In EDWARD J. KANSA AND YIU-CHUNG HON, ED., Computers and mathematics with applications, R. Lippmann, J. Moody, D. S. Touretzky, eds., Vol. 43, Elsevier, 2002, pp. 275-619.