Constructive algorithms for structure learning in feedforward neural networks for regression problems

IEEE Transactions on Neural Networks

Volumn 8, Issue 3, 1997, Pages 630-645

  Kwok, Tin Yau ^b   Yeung, Dit Yan ^a,b  

^a IEEE

^b HONG KONG UNIVERSITY OF SCIENCE AND TECHNOLOGY   (Hong Kong)

Author keywords

Cascade correlation; Constructive algorithm; Dynamic node creation; Group method of data handling; Projection pursuit regression; Resource allocating network; State space search; Structure learning

Indexed keywords

COMPUTER ARCHITECTURE; DATA HANDLING; FEEDFORWARD NEURAL NETWORKS; REGRESSION ANALYSIS; STATE SPACE METHODS;

CONSTRUCTIVE ALGORITHMS; STRUCTURE LEARNING;

LEARNING ALGORITHMS;

EID: 0031146959     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.572102     Document Type: Article

References (140)

1. Y. Chauvin, "A backpropagation algorithm with optimal use of hidden units," in Advances in Neural Information Processing Systems 1, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1989, pp. 519-526.
2. S. J. Hanson and L. Y. Pratt, "Comparing biases for minimal network construction with backpropagation," in Advances in Neural Information Processing Systems 1, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1989, pp. 177-185.
3. A. S. Weigend, D. E. Rumelhart, and B. A. Huberman, "Generalization by weight-elimination with application to forecasting," in Advances in Neural Information Processing Systems 3, R. Lippmann, J. Moody, and D. Touretzky, Eds. San Mateo, CA: Morgan Kaufmann, 1991, pp. 875-882.
4. G. H. Golub, M. Heath, and G. Wahba, "Generalized cross-validation as a method for choosing a good ridge parameter," Technometrics, vol. 21, no. 2, pp. 215-223, 1979.
5. W. L. Buntine and A. S. Weigend, "Bayesian backpropagation," Complex Syst., vol. 5, pp. 603-643, 1991.
6. D. J. C. Mackay, "Bayesian interpolation," Neural Computa., vol. 4, no. 3, pp. 415-447, May 1992.
7. R. M. Neal, "Bayesian learning for neural networks," Ph.D. dissertation, Dep. Computer Sci., Univ. Toronto, Canada, 1995.
8. H. H. Thodberg, "A review of Bayesian neural networks with an application to near infrared spectroscopy," IEEE Trans. Neural Networks, vol. 7, pp. 56-72, 1996.
9. P. M. Williams, "Bayesian regularization and pruning using a Laplace prior," Neural Computa., vol. 7, pp. 117-143, 1995.
10. D. J. C. MacKay, "A practical Bayesian framework for backpropagation networks," Neural Computa., vol. 4, no. 3, pp. 448-472, May 1992.
11. J. S. Judd, Neural Network Design and the Complexity of Learning. Cambridge, MA: MIT Press, 1990.
12. A. L. Blum and R. L. Rivest, "Training a 3-node neural network is NP-complete," Neural Networks, vol. 5, pp. 117-127, 1992.
13. J. H. Lin and J. S. Vitter, "Complexity results on learning by neural nets," Machine Learning, vol. 6, pp. 211-230, 1991.
14. J. Sima, "Loading deep networks is hard," Neural Computa., vol. 6, pp. 842-850, 1994.
15. E. B. Baum, "Book review on 'Neural-network design and the complexity of learning,'" IEEE Trans. Neural Networks, vol. 2, pp. 181-182, 1991.
16. L. G. Valiant, "A theory of the learnable," Commun. ACM, vol. 27, no. 11, pp. 1134-1142, 1984.
17. E. B. Baum, "A proposal for more powerful learning algorithms," Neural Computa., vol. 1, no. 2, pp. 201-207, 1989.
18. R. Reed, "Pruning algorithms - A survey," IEEE Trans. Neural Networks, vol. 4, pp. 740-747, Sept. 1993.
19. E. D. Karnin, "A simple procedure for pruning backpropagation trained neural networks," IEEE Trans. Neural Networks, vol. 1, pp. 239-242, June 1990.
20. M. C. Mozer and P. Smolensky, "Skeletonization: A technique for trimming the fat from a network via relevance assessment," in Advances in Neural Information Processing Systems 1, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1989, pp. 107-115.
21. B. Hassibi and D. G. Stork, "Second-order derivatives for network pruning: Optimal brain surgeon," in Advances in Neural Information Processing Systems 5. San Mateo, CA: Morgan Kaufmann, 1993, pp. 164-171.
22. Y. Le Cun, J. S. Denker, and S. A. Solla, "Optimal brain damage," in Advances in Neural Information Processing Systems 2, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, 1990, pp. 598-605.
23. J. H. Friedman, "An overview of predictive learning and function approximation," in From Statistics to Neural Networks: Theory and Pattern Recognition Applications, J. H. Friedman and H. Wechsler, Eds., ASI Proc., Subseries F. New York: Springer-Verlag, 1994.
24. E. Alpaydin, "GAL: Networks that grow when they learn and shrink when they forget," Int. Computer Sci. Inst., Berkeley, CA, Tech. Rep. 91-032, May 1991.
25. G. Deffuant, "Neural units recruitment algorithm for generation of decision trees," in Proc. 1990 IEEE Int. Joint Conf. Neural Networks, San Diego, CA, vol. 1, June 1990, pp. 637-642.
26. M. Frean, "The upstart algorithm: A method for constructing and training feedforward neural networks," Neural Computa., vol. 2, pp. 198-209, 1990.
27. M. Marchand, M. Golea, and P. Ruján, "A convergence theorem for sequential learning in two-layer perceptrons," Europhys. Lett., vol. 11, no. 6, pp. 487-492, 1990.
28. T. Ash and G. Cottrell, "Topology modifying neural network algorithms," in Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. Cambridge, MA: MIT Press, 1995, pp. 990-993.
29. T. Ash and G. W. Cottrell, "A review of learning algorithms that modify network topologies," Computer Sci. Eng., Univ. California, San Diego, Tech. Rep. CS94-348, 1994.
30. E. Fiesler, "Comparative bibliography of ontogenic neural networks," in Proc. Int. Conf. Artificial Neural Networks, Sorrento, Italy, vol. 1, May 1994, pp. 793-796.
31. D. E. Nelson and S. K. Rogers, "A taxonomy of neural network optimality," in Proc. IEEE Nat. Aerospace Electron. Conf., Dayton, OH, vol. 3, May 1992, pp. 894-899.
32. G. Gong, "Cross-validation, the jackknife, and the bootstrap: Excess error estimation in forward logistic regression," J. Amer. Statist. Assoc., vol. 81, no. 393, pp. 108-113, May 1986.
33. A. R. Barron, "Approximation and estimation bounds for artificial neural networks," Machine Learning, vol. 14, pp. 115-133, 1994.
34. S. Geman, E. Bienenstock, and R. Doursat, "Neural networks and the bias/variance dilemma," Neural Computa., vol. 4, pp. 1-58, 1992.
35. R. E. Parker and M. Tummala, "Identification of Volterra systems with a polynomial neural network," in Proc. 1992 IEEE Int. Conf. Acoust., Speech, Signal Processing, San Francisco, CA, vol. 4, Mar. 1992, pp. 561-564.
36. J. Moody, "Prediction risk and architecture selection for neural networks," in From Statistics to Neural Networks: Theory and Pattern Recognition Applications, V. Cherkassky, J. H. Friedman, and H. Wechsler, Eds., vol. 136 of NATO ASI Series F. New York: Springer-Verlag, 1994, pp. 147-165.
37. M. Stone, "Cross-validatory choice and assessment of statistical predictions (with discussion)," J. Roy. Statist. Soc. Series B, vol. 36, pp. 111-147, 1974.
38. B. Efron and R. J. Tibshirani, An Introduction to the Bootstrap, vol. 57 of Monographs on Statistics and Applied Probability. New York: Chapman and Hall, 1993.
39. A. S. Weigend and B. LeBaron, "Evaluating neural network predictors by bootstrapping," in Proc. Int. Conf. Neural Inform. Processing, Seoul, Korea, vol. 2, Oct. 1994, pp. 1207-1212.
40. H. Akaike, "A new look at the statistical model identification," IEEE Trans. Automat. Contr., vol. AC-19, pp. 716-723, Dec. 1974.
41. G. Schwartz, "Estimating the dimension of a model," Ann. Statist., vol. 6, pp. 461-464, 1978.
42. H. Akaike, "Statistical predictor identification," Ann. Instit. Statist. Math., vol. 22, pp. 203-217, 1970.
43. P. Craven and G. Wahba, "Smoothing noisy data with spline functions: Estimating the correct degree of smoothing by the method of generalized cross-validation," Numer. Math., vol. 31, pp. 377-403, 1979.
44. A. Barron, "Predicted squared error: A criterion for automatic model selection," in Self-Organizing Methods in Modeling, S. Farlow, Ed. New York: Marcel Dekker, 1984.
45. J. Rissanen, "Modeling by shortest data description," Automatica, vol. 14, pp. 465-471, 1975.
46. J. E. Moody, "Note on generalization, regularization, and architecture selection in nonlinear learning systems," in Proc. 1991 IEEE Wkshp. Neural Networks for Signal Processing, B. H. Juang, S. Y. Kung, and C. A. Kamm, Eds. Princeton, NJ, Sept. 1991, pp. 1-10.
47. B. D. Ripley, "Choosing network complexity," in Probabilistic Reasoning and Bayesian Belief Networks, A. Gammerman, Ed., 1995, pp. 97-108.
48. B. D. Ripley, "Statistical ideas for selecting network architectures," in Neural Networks: Artificial Intelligence and Industrial Applications, B. Kappen and S. Gielen, Eds. New York: Springer-Verlag, 1995, pp. 183-190.
49. M. F. Tenorio and W. T. Lee, "Self-organizing network for optimum supervised learning," IEEE Trans. Neural Networks, vol. 1, pp. 100-110, Mar. 1990.
50. E. Hartman and J. D. Keeler, "Predicting the future: Advantages of semilocal units," Neural Computa., vol. 3, pp. 566-578, 1991.
51. J. Moody and C. Darken, "Learning with localized receptive fields," in Proc. 1988 Connectionist Models Summer School, 1988, pp. 133-143.
52. S. Renals, "Radial basis function network for speech pattern classification," Electron. Lett., vol. 25, no. 7, pp. 437-439, 1988.
53. G. Cybenko, "Approximation by superpositions of a sigmoidal function," Math. Contr., Signals, Syst., vol. 2, pp. 303-314, 1989.
54. K. I. Funahashi, "On the approximate realization of continous mappings by neural networks," Neural Networks, vol. 2, pp. 183-192, 1989.
55. K. Hornik, "Approximation capabilities of multilayer feedforward networks," Neural Networks, vol. 4, pp. 251-257, 1991.
56. _, "Some new results on neuralnetwork approximation," Neural Networks, vol. 6, pp. 1069-1072, 1993.
57. E. Hartman, J. Keeler, and J. Kowalski, "Layered neural networks with Gaussian hidden units as universal approximations," Neural Computa., vol. 2, pp. 210-215, 1990.
58. J. Park and I. Sandberg, "Universal approximation using radial-basis-function networks," Neural Computa., vol. 3, pp. 246-257, 1991.
59. _, "Approximation and radial-basis-function networks," Neural Computa., vol. 5, pp. 305-316, 1993.
60. A. N. Kolmogorov and S. V. Fomin, Introductory Real Analysis. New York: Dover, 1975.
61. G. Lugosi and K. Zeger, "Nonparametric estimation via empirical risk minimization," IEEE Trans. Inform. Theory, vol. 41, pp. 677-687, May 1995.
62. A. R. Barron, "Universal approximation bounds for superpositions of a sigmoidal function," IEEE Trans. Inform. Theory, vol. 39, pp. 930-945, May 1993.
63. L. K. Jones, "A simple lemma on greedy approximation in Hilbert space and convergence rates for projection pursuit regression and neural network training," Ann. Statist., vol. 20, no. 1, pp. 608-613, 1992.
64. F. L. Chung and T. Lee, "Network-grwoth approach to design of feedforward neural networks," in IEE Proceedings. Control Theory and Applications, Sept. 1995.
65. P. Courrieu, "A convergent generator of neural networks," Neural Networks, vol. 6, no. 6, pp. 835-844, 1993.
66. J. Luo, "A bias architecture with rank-expanding algorithm for neural networks supervised learning problem," in Proc. World Congr. Neural Networks, San Diego, CA, vol. 3, June 1994, pp. 742-747.
67. G. Weiss, "Neural networks and evolutionary computation - Part 1: Hybrid approaches in artificial intelligence," in Proc. 1st IEEE Conf. Evolutionary Computa., Orlando, FL, vol. 1, June 1994, pp. 268-272.
68. X. Yao, "A review of evolutionary artificial neural networks," Int. J. Intell. Syst., vol. 8, pp. 539-567, 1993.
69. G. F. Miller, P. M. Todd, and S. U. Hegde, "Designing neural networks using genetic algorithms," in Proc. 3rd Int. Conf. Genetic Algorithms, 1989, pp. 379-384.
70. C. C. Teng and B. W. Wah, "An automated design system for finding the minimal configuration of a feedforward neural network," in Proc. IEEE Int. Conf. Neural Networks, Orlando, FL, vol. 3, June 1994, pp. 1295-1300.
71. R. Battiti, "First- and second-order methods for learning: Between steepest descent and Newton's method," Neural Computa., vol. 4, pp. 141-166, 1992.
72. T. T. Jervis and W. J. Fitzgerald, "Optimization schemes for neural networks," Cambridge Univ. Eng. Dep., Cambridge, U.K., Rep. CUED/F-INFENG/TR 144, 1993.
73. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C, 2nd ed. New York: Cambridge Univ. Press, 1992.
74. L. Xu, S. Klasa, and A. Yuille, "Recent advances on techniques of static feedforward networks with supervised learning," Int. J. Neural Syst., vol. 3, no. 3, pp. 253-290, 1992.
75. G. S. G. Beveridge and R. S. Schechter, Optimization: Theory and Practice New York: McGraw-Hill, 1970.
76. SAS Institute Inc., SAS/STAT User's Guide, release 6.03 edition, 1988.
77. M. R. Azimi-Sadjadi, S. Sheedvash, and F. O. Trujillo, "Recursive dynamic node creation in multilayer neural networks," IEEE Trans. Neural Networks, vol. 4, pp. 242-256, Mar. 1993.
78. P. S. Lewis and J. N. Hwang, "Recursive least-squares learning algorithms for neural networks," in Proc. SPIE Advanced Signal-Processing Algorithms, Architectures, and Implementations, vol. 1348, 1990, pp. 28-39.
79. S. E. Fahlman, "Faster learning variations on backpropagation: An empirical study," in Proc. 1988 Connectionist Models Summer School, D. S. Touretzky, G. E. Hinton, and T. J. Sejnowski, Eds. San Mateo, CA: Morgan Kaufmann, 1988, pp. 38-51.
80. R. Setiono and L. C. K. Hui, "Use of a quasi-Newton method in a feedforward neural network construction algorithm," IEEE Trans. Neural Networks, vol. 6, pp. 273-277, 1995.
81. J. H. Friedman and W. Stuetzle, "Projection pursuit regression," J. Amer. Statist. Assoc., vol. 76, no. 376, pp. 817-823, 1981.
82. S. Sjøgaard, "A conceptual approach to generalization in dynamic neural networks," Ph.D. dissertation, Computer Sci. Dep., Aarhus University, Aarhus, Denmark, 1991.
83. S. E. Fahlman and C. Lebiere, "The cascade-correlation learning architecture," in Advances in Neural Information Processing Systems 2, D. S. Touretzky, Ed. San Mateo, CA: Morgan Kaufmann, pp. 524-532, 1990.
84. T. Ash, "Dynamic node creation in backpropagation networks," Connection Sci., vol. 1, no. 4, pp. 365-375, 1989.
85. E. B. Bartlett, "Dynamic node architecture learning: An information theoretic approach," Neural Networks, vol. 7, no. 1, pp. 129-140, 1994.
86. Y. Hirose, K. Yamashita, and S. Hijiya, "Backpropagation algorithm which varies the number of hidden units," Neural Networks, vol. 4, pp. 61-66, 1991.
87. K. Khorasani and W. Weng, "Structure adaptation in feedforward neural networks," in Proc. IEEE Int. Conf. Neural Networks, Orlando, FL, vol. 3, June 1994, pp. 1403-1408.
88. B. T. Zhang, "An incremental learning algorithm that optimizes network size and sample size in one trial," in Proc. IEEE Int. Conf. Neural Networks, Orlando, FL, vol. 1, June 1994, pp. 215-220.
89. T. E. Flick, L. K. Jones, R. G. Priest, and C. Herman, "Pattern classification using projection pursuit," Pattern Recognition, vol. 23, no. 12, pp. 1367-1376, 1990.
90. J. H. Friedman, E. Grosse, and W. Stuetzle, "Multidimensional additive spline approximation," SIAM J. Sci. Statist. Computing, vol. 4, no. 2, pp. 291-301, June 1983.
91. J. N. Hwang, S. R. Lay, M. Maechler, D. Martin, and J. Schimert, "Regression modeling in backpropagation and projection pursuit learning," IEEE Trans. Neural Networks, vol. 5, pp. 342-353, May 1994.
92. S. Lay, J. Hwang, and S. You, "Extensions to projection pursuit learning networks with parametric smoothers," in Proc. IEEE Int. Conf. Neural Networks, Orlando, FL, vol. 3, June 1994, pp. 1325-1330.
93. C. B. Roosen and T. J. Hastie, "Automatic smoothing spline projection pursuit," J. Computa. Graphical Statist., vol. 3, pp. 235-248, 1994.
94. A. Saha, C. L. Wu, and D. S. Tang, "Approximation, dimension reduction, and nonconvex optimization using linear superpositions of Gaussians," IEEE Trans. Computers, vol. 42, pp. 1222-1233, Oct. 1993.
95. Y. Shin and J. Ghosh, "Ridge polynomial networks," IEEE Trans. Neural Networks, vol. 6, May 1995.
96. W. Verkooijen and H. Daniels, "Connectionist projection pursuit regression," Computa. Economics, vol. 7, pp. 155-161, 1994.
97. J. L. Yuan and T. L. Fine, "Forecasting demand for electric power," in Advances in Neural Information Processing Systems 5, S. J. Hanson, J. D. Cowan, and C. L. Giles, Eds. San Mateo, CA: Morgan Kaufmann, 1993, pp. 739-746.
98. Y. Zhao and C. G. Atkeson, "Some approximation properties of projection pursuit learning networks," in Advances in Neural Information Processing Systems 4, J. E. Moody, S. J. Hanson, and R. P. Lippman, Eds. San Mateo, CA: Morgan Kaufmann, 1992, pp. 936-943.
99. Statistical Sciences, Inc., Seattle, WA, S-Plus User's Manual, 3rd ed., Sept. 1991.
100. J. Moody and N. Yarvin, "Networks with learned unit response functions," in Advances in Neural Information Processing Systems 4, J. E. Moody, S. J. Hanson, and R. P. Lippmann, Eds. San Mateo, CA: Morgan Kaufmann, 1992, pp. 1048-1055.
101. J. H. Friedman, "Exploratory projection pursuit," J. Amer. Statist. Assoc., vol. 82, no. 397, pp. 249-266, Mar. 1987.
102. Y. Shin and J. Ghosh, "The pi-sigma network: An efficient higher-order neural network for pattern classification and function approximation," in Proc. Int. Joint Conf. Neural Networks, Seattle, WA, vol. 1, July 1991, pp. 13-18.
103. N. Intrator, "Combining exploratory projection pursuit and projection pursuit regression with application to neural networks," Neural Computa., vol. 5, pp. 443-455, May 1993.
104. T. Y. Kwok and D. Y. Yeung, "Bayesian regularization in constructive neural networks," in Proc. Int. Conf. Artificial Neural Networks, Bochum, Germany, July 1996, pp. 557-562.
105. J. Ghosh and Y. Shin, "Efficient higher-order neural networks for classification and function approximation," Int. J. Neural Syst., vol. 3, no. 4, pp. 323-350, 1992.
106. L. K. Jones, "On a conjecture of Huber concerning the convergence of projection pursuit regression," Ann. Statist., vol. 15, no. 2, pp. 880-882, 1987.
107. T. Y. Kwok and D. Y. Yeung, "Use of bias term in projection pursuit learning improves approximation and convergence properties," IEEE Trans. Neural Networks, vol. 7, pp. 1168-1183, 1996.
108. W. Fang and R. C. Lacher, "Network complexity and learning efficiency of constructive learning algorithms," in Proc. IEEE Int. Conf. Neural Networks, Orlando, FL, vol. 1, June 1994, pp. 366-369.
109. E. Littmann and H. Ritter, "Cascade LLM networks," in Proc. Int. Conf. Artificial Neural Networks, Brighton, U.K., vol. 1, Sept. 1992, pp. 253-257.
110. _, "Cascade network architectures," in Proc. Int. Joint Conf. Neural Networks, Baltimore, MD, vol. 2, June 1992, pp. 398-404.
111. D. S. Phatak and I. Koren, "Connectivity and performance tradeoffs in the cascade correlation learning architecture," IEEE Trans. Neural Networks, vol. 5, pp. 930-935, Nov. 1994.
112. N. Simon, H. Corporaal, and E. Kerckhoffs, "Variations on the cascade-correlation learning architecture for fast convergence in robot control," in Proc. 5th Int. Conf. Neural Networks Applicat., Nimes, France, Nov. 1992, pp. 454-464.
113. S. Sjøgaard, "Generalization in cascade-correlation networks," in Proc. IEEE - SP Wkshp. Neural Networks for Signal Processing II, Helsingoer, Denmark, Sept. 1992, pp. 59-68.
114. I. G. Smotroff, D. H. Friedman, and D. Connolly, "Self organizing modular neural networks," in Proc. Int. Joint Conf. Neural Networks, Seattle, WA, July 1991.
115. P. L. Springer and S. Gulati, "Parallelizing the cascade-correlation algorithm using time warp," Neural Networks, vol. 8, no. 4, pp. 571-577, 1995.
116. D. Y. Yeung, "A neural network approach to constructive induction," in Proc. 8th Int. Wkshp. Machine Learning, Evanston, IL, 1991.
117. L. K. Hansen and M. W. Pedersen, "Controlled growth of cascade correlation nets," in Proc. Int. Conf. Artificial Neural Networks, Sorrento, Italy, vol. 1, May 1994, pp. 797-800.
118. O. Fujita, "Optimization of the hidden unit function in feedforward neural networks," Neural Networks, vol. 5, pp. 755-764, 1992.
119. V. Kurková and B. Beliczynski, "Incremental approximation by one-hidden-layer neural networks," in Proc. Int. Conf. Artificial Neural Networks, Paris, France, vol. 1, Oct. 1995, pp. 5505-5510.
120. T. Y. Kwok and D. Y. Yeung, "Objective functions for training new hidden units in constructive neural networks," submitted to IEEE Trans. Neural Networks.
121. G. P. Drago and S. Ridella, "Convergence properties of cascade correlation in function approximation," Neural Comput. Applicat., vol. 2, pp. 142-147, 1994.
122. G. Deco and J. Ebmeyer, "Coarse coding resource-allocating network," Neural Computa., vol. 5, pp. 105-114, 1993.
123. B. Fritzke, "Growing cell structures - A self-organizing network for unsupervised and supervised learning," Neural Networks, vol. 7, no. 9, pp. 1441-1460, 1994.
124. S. Lee and R. M. Kil, "A Gaussian potential function network with hierarchically self-organizing learning," Neural Networks, vol. 4, pp. 207-224, 1991.
125. J. Platt, "A resource-allocating network for function interpolation," Neural Computa., vol. 3, pp. 213-225, 1991.
126. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
127. V. Kadirkamanathan and M. Niranjan, "A function estimation approach to sequential learning with neural networks," Neural Computa., vol. 5, no. 6, pp. 954-975, 1993.
128. S. Roberts and L. Tarassenko, "A probabilistic resource allocating network for novelty detection," Neural Computa., vol. 6, no. 2, pp. 270-284, Mar. 1994.
129. S. J. Farlow, Ed., Self-Organizing Methods in Modeling: GMDH Type Algorithms, vol. 54 of Statistics: Textbooks and Monographs. New York: Marcel Dekker, 1984.
130. T. M. Nabhan and A. Y. Zomaya, "Toward generating neural network structures for function approximation," Neural Networks, vol. 7, no. 1, pp. 89-90, 1994.
131. C. Darken, M. Donahue, L. Gurvits, and E. Sontag, "Rate of approximation results motivated by robust neural network learning," Siemens Corporate Res., Inc., Princeton, NJ, Tech. Rep., Apr. 1994.
132. K. Hornik, M. Stinchcombe, H. White, and P. Auer, "Degree of approximation results for feedforward networks approximating unknown mappings and their derivations," Neural Computa., vol. 6, pp. 1262-1275, 1994.
133. L. Xu, A. Krzyzak, and A. Yuille, "On radial basis function nets and kernel regression: Statistical consistency, convergence rates, and receptive field size," Neural Networks, vol. 7, no. 4, pp. 609-628, 1994.
134. A. R. Barron and R. L. Barron, "Statistical learning networks: A unifying view," in Proc. 20th Symp. Interface Computing Sci. Statist., E. Wegman, Ed. Washington, D.C.: Amer. Statist. Assoc., 1988, pp. 192-203.
135. B. Cheng and D. M. Titterington, "Neural networks: A review from a statistical perspective (with discussion)," Statist. Sci., vol. 9, no. 1, pp. 2-54, 1994.
136. V. Cherkassky, "Neural networks and nonparametric regression," in Proc. IEEE-SP Wkshp., Helsingoer, Denmark, Aug. 1992, pp. 511-521.
137. B. D. Ripley, "Statistical aspects of neural networks," in Networks and Chaos - Statistical and Probabilistic Aspects, O. E. Barndorff-Nielsen, J. L. Jensen, and W. S. Kendall, Eds. London: Chapman and Hall, 1993, pp. 40-123.
138. B. D. Ripley, "Neural networks and related methods for classification (with discussion)," J. Roy. Statist. Soc. Series B, vol. 56, 1994.
139. W. S. Sarle, "Neural networks and statistical models," in Proc. 19th Annu. SAS Users Group Int. Conf., Apr. 1994.
140. D. J. C. MacKay, "Probable networks and plausible predictions - A review of practical Bayesian methods for supervised neural networks," Network: Computa. Neural Syst., vol. 6, no. 3, pp. 469-505, Aug. 1995.