On global-local artificial neural networks for function approximation

IEEE Transactions on Neural Networks

Volumn 17, Issue 4, 2006, Pages 942-952

  Wedge, David ^a   Ingram, David ^b   McLean, David ^a   Mingham, Clive ^a   Bandar, Zuhair ^a  

^a MANCHESTER METROPOLITAN UNIVERSITY   (United Kingdom)

^b UNIVERSITY OF EDINBURGH   (United Kingdom)

Author keywords

Global; Hybrid; Local; Overtopping; Regularization

Indexed keywords

APPROXIMATION THEORY; ARTIFICIAL INTELLIGENCE; MATHEMATICAL MODELS; OPTIMIZATION; RADIAL BASIS FUNCTION NETWORKS; REGRESSION ANALYSIS;

FUNCTION APPROXIMATION; MODEL SELECTION; REAL WORLD DATA;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; LEARNING; STATISTICS;

ALGORITHMS; LEARNING; NEURAL NETWORKS (COMPUTER); STATISTICS;

EID: 33746915070     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2006.875972     Document Type: Article

References (57)

1. D. Donoho and I. Johnstone, "Projection-based approximation and a duality with kernel methods," Ann. Statist., vol. 17, pp. 58-106, 1989.
2. T. Poggio and F. Girosi, A theory of networks for approximation and learning Massachusetts Inst. Tech., Artificial Intelligence Lab., Cambridge, MA, Jul. 1989, AI Memo 1140.
3. G. Auda and M. Kamel, "Modular neural networks: A survey," Int. J. Neural Syst., vol. 9, pp. 129-151, 1999.
4. T. Hrycej, Modular Learning in Neural Networks: A Modularized Approach to Classification. New York: Wiley, 1992.
5. P. Poirazi, C. Neocleous, C. Pattichis, and C. Schizas, "Classification capacity of a modular neural network implementing neurally inspired architecture and training rules," IEEE Trans. Neural Netw., vol. 15, no. 3, pp. 597-612, May 2004.
6. S. Johannes, B. Wieringa, M. Matzke, and T. Munte, "Hierarchical visual stimuli: Electrophysical evidence for separating left hemispheric global and local processing mechanisms in humans," Neurosci. Lett., vol. 210, pp. 111-114, 1996.
7. R. Hubner, "Hemispheric differences in global/local processing revealed by same-different judgements," Visual Cogn., vol. 5, no. 4, pp. 457-478, 1998.
8. J. Piaget, Meine Theorie der geistigen Intelligenz. Frankfurt, Germany: Fischer Taschenbuch Verlag, 1983.
9. M. Moussa, "Combining expert neural networks using reinforcement feedback for learning primitive grasping behavior," IEEE Trans. Neural Netw., vol. 15, no. 3, pp. 629-638, May 2004.
10. T. Poggio and F. Girosi, "Networks for approximation and learning," Proc. IEEE, vol. 78, no. 9, pp. 1481-1497, Sep. 1990.
11. F. Girosi, M. Jones, and T. Poggio, "Regularization theory and neural networks architectures," Neural Comput., vol. 7, pp. 219-269, 1995.
12. J. Moody, "Fast learning in multi resolution hierarchic," in Advances in Neural Information Processing Systems I, D. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1989, pp. 29-39.
13. S. Ferrari, M. Maggioni, and N. Borghese, "Multiscale approximation with hierarchical radial basis functions networks," IEEE Trans. Neural Netw., vol. 15, no. 1, pp. 178-188, Jan. 2004.
14. J. de Rouck, Crest level assessment of coastal structures by full scale monitoring, neural network prediction and hazard analysis on permissible wave overtopping [Online]. Available: http://www.clash-eu.org
15. D. Wedge, D. Ingram, D. McLean, C. Mingham, and Z. Bandar, "Neural network architectures and wave overtopping," Proc. Inst. Civil Engineering 2005: Maritime Engineering, vol. 158, no. MA3, pp. 123-133, 2005.
16. D. Broomhead and D. Lowe, "Multivariable functional interpolation and adaptive networks," Complex Syst., vol. 2, pp. 321-355, 1988.
17. S. Lawrence, A. Tsoi, and A. Back, "Function approximation with neural networks and local methods: Bias, variance and smoothness," in Proc. Australian Conf. Neural Networks (ACNN96), P. Bartlett, A. Burkitt, and R. Williamson, Eds., 1996, pp. 16-21.
18. J. Friedman and W. Stuetzle, "Projection pursuit regression," J. Amer. Statist. Assoc., vol. 76, pp. 817-823, 1981.
19. C. Campbell, "Constructive learning techniques for designing neural network systems," in Optimization Techniques, ser. Neural Network Systemes Techniques and Applications, C. Leondes, Ed. San Diego, CA: Academic, 1998, vol. 2.
20. M. Orr, "Regularisation in the selection of radial basis function centres," Neural Comput., vol. 7, pp. 606-623, 1995.
21. N. Jiang, Z. Zhao, and L. Ren, "Design of structural modular neural networks with genetic algorithm," Adv. Software Eng., vol. 34, pp. 17-24, 2003.
22. Y. Liu and X. Yao, "Evolutionary design of artificial neural networks with different nodes, evolutionary computation," in Proc. 3rd IEEE Int. Conf. Evolutionary Computation, 1996, pp. 670-675.
23. S. Cohen and N. Intrator, "A hybrid projection-based and radial basis function architecture: Initial values and global optimisation," Pattern Anal. Applicat., vol. 5, pp. 113-120, 2002.
24. M. Orr, K. Takezawa, A. Murray, S. Ninomiya, M. Oide, and T. Leonard, "Combining regression trees and radial basis function networks," Int. J. Neural Syst., vol. 10, no. 6, pp. 453-465, 2000.
25. M. Jordan and R. Jacobs, "Hierarchical mixtures of experts and the em algorithm," Neural Comput., vol. 6, pp. 181-214, 1994.
26. G. Cybenko, "Approximation by superpositions of a sigmoidal function," Math. Contr. Signals Syst., vol. 2, pp. 303-314, 1989.
27. J. Park and I. Sandberg, "Approximation and radial-bais-function networks," Neural Comput., vol. 5, pp. 304-316, 1993.
28. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd ed. Upper Saddle River, NJ: Prentice-Hall, 1999.
29. C. Bishop, Neural Networks for Pattern Recognition. Oxford, U.K.: Clarendon, 1995.
30. J. Moody and C. Darken, "Fast learning in networks of locally-tuned processing units," Neural Comput., vol. 1, pp. 281-294, 1989.
31. G. Hinton, J. McClelland, and D. Rumelhart, "Distributed representations," in Parallel Distributed Processing, ser. Explorations in the Microstructure of Cognition. Cambridge, MA: MIT Press, 1986, vol. 1.
32. M. Orr, Introduction to radial basis function networks Inst. Adaptive Neural Computation, Univ. Edinburgh, Apr. 1996 [Online]. Available: http://www.anc.ed.ac.uk/mjo/papers/intro.ps.gz, Tech. Rep.
33. D. Navon, "Forest before trees: The precedence of global features in visual perception," Cogn. Psychol., vol. 9, pp. 353-383, 1977.
34. S. Christman, "Individual differences in stroop and local-global processing: A possible role of interhemispheric interaction," Brain Cogn., vol. 45, pp. 97-118, 2001.
35. S. Arisariyawong and S. Charoenseang, "Dynamic self-organised learning for optimizing the complexity growth of radial basis function neural networks," in Proc. IEEE Int. Conf. Industrial Technology, Dec. 11-14, 2002, pp. 655-660.
36. S. Geman, E. Bienenstock, and R. Doursat, "Neural networks and the bias/variance dilemma," Neural Comput., vol. 4, pp. 1-58, 1992.
37. Y. LeCun, L. Bottou, G. Orr, and K.-R. Muller, "Efficient backprop," in Neural Networks: Tricks of the Trade, ser. Lecture Notes in Computer Science, G. Orr and K. Muller, Eds. Berlin, Germany: Springer-Verlag, 1998, vol. 1524.
38. M. Hagan and M. Menhaj, "Training feedforward networks with the Marquardt algorithm," IEEE Trans. Neural Netw., vol. 5, no. 6, pp. 989-993, Nov. 1994.
39. T. Masters, Advanced Algorithms for Neural Networks: C++ Source Book. Frankfurt, Germany: Wiley, 1995.
40. S. Chen, C. Cowan, and P. Grant, "Orthogonal least squares learning for radial basis function networks," IEEE Trans. Neural Netw., vol. 2, no. 2, pp. 302-309, Mar. 1991.
41. J. Friedman, "Multivariate adaptive regression splines," Ann. Statist., vol. 19, pp. 1-141, 1991.
42. D. Mackay, "Bayesian interpolation," Neural Comput., vol. 4, pp. 415-447, 1992.
43. P. Besley, Overtopping of seawalls: Design and assessment manual HR Wallingford, Environment Agency, Tech. Rep. W178, 1999.
44. J. Shiach, C. Mingham, D. Ingram, and T. Bruce, "The applicability of the shallow water equations for modelling violent wave overtopping," Coastal Eng., vol. 51, pp. 1-15, 2004.
45. J. de Rouck, Second detailed interim rep. CLASH, Ghent Univ., Tech. Rep. CLA127/296, Feb. 2004.
46. M. Hassoun, Fundamentals of Artificial Neural Networks. Cambridge, MA: MIT Press, 1995.
47. H. Mhaskar, "Neural networks for localized approximation of real functions," in IEEE Workshop Neural Networks Signal Processing, Sep. 6-9, 1993.
48. C. Chui, X. Li, and H. Mhaskar, "Neural networks for localized approximation," Math. Comput., vol. 63, pp. 607-623, 1994.
49. M. Orr, J. Hallam, A. Murray, and T. Leonard, "Assessing RBF networks using delve," Int. J. Neural Syst., vol. 10, no. 5, pp. 397-415, 2000.
50. K. Hu, C. Mingham, and D. Causon, "Numerical simulation of wave overtopping of coastal structures using the non-linear shallow water equations," Coastal Eng., vol. 41, pp. 433-465, 2000.
51. D. Mackay, "Probable networks and plausible predictions - A review of practical Bayesian methods for supervised neural networks," Network: Comput. Neural Syst., vol. 6, pp. 469-505, 1995.
52. E. Kolman and M. Margaliot, "Are artificial neural networks white boxes?," IEEE Trans. Neural Netw., vol. 16, no. 4, pp. 844-851, Jul. 2005.
53. M. Orr, Matlab functions for radial basis function networks Inst. for Adaptive and Neural Computation (Univ. of Edinburgh), Jun. 1999 [Online]. Available: http://www.anc.ed.ac.uk/mjo/software/rbf2.zip, Rep. Tech.
54. F. Schwenker, H. Kestler, and G. Palm, "Three learning phases for radial-basis-function networks," Neural Netw., vol. 14, pp. 439-458, 2001.
55. G. Schwarz, "Estimating the dimension of a model," Ann. Statist., vol. 6, pp. 461-464, Mar. 1978.
56. H. Lee, "Model selection for neural network classification," J. Class., vol. 18, pp. 227-243, 2001.
57. D. Mackay, "A practical Bayesian framework for backprop networks," Neural Comput., vol. 4, pp. 448-472, 1992.