Evaluation of constructive neural networks with cascaded architectures

Neurocomputing

Volumn 48, Issue 1-4, 2002, Pages 573-607

  Lahnajärvi, Jani J T ^a   Lehtokangas, Mikko I ^a   Saarinen, Jukka P P ^a  

^a TAMPERE UNIVERSITY OF TECHNOLOGY   (Finland)

Author keywords

Cascade correlation; Constructive neural networks; Generalization; Model selection; Network training

Indexed keywords

ALGORITHMS; COMPUTER SIMULATION; CORRELATION METHODS;

CASCADED ARCHITECTURES;

NEURAL NETWORKS;

ALGORITHM; ARTIFICIAL NEURAL NETWORK; COMPUTER SIMULATION; CONTROLLED STUDY; CYTOARCHITECTURE; EVALUATION; MATHEMATICAL COMPUTING; MONTE CARLO METHOD; PRIORITY JOURNAL; REGRESSION ANALYSIS; REVIEW; STUDENT T TEST;

EID: 0036825537     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0925-2312(01)00630-0     Document Type: Review

References (51)

1. R. Beale, T. Jackson, Neural Computing: An Introduction, IOP Publishing Ltd, Bristol, England, 1990.
2. C.M. Bishop, Regularization and complexity control in feed-forward networks, Technical Report: NCRG/95/022, Neural Computing Research Group, Department of Computer Science and Applied Mathematics, Aston University, Birmingham, UK, 1995.
3. T.L. Burrows, M. Niranjan, The Use of Feed-Forward and Recurrent Neural Networks for System Identification, Technical Report CUED/F-INFENG/TR158, Cambridge University Engineering Department, Cambridge, England, December 1993.
4. A. Cichocki, R. Unbehauen, Neural Networks for Optimization and Signal Processing, Wiley, New York, 1993.
5. G. Cybenko, Approximation by Superpositions of a Sigmoidal Function, Mathmatics of Control, Signals and Systems, Vol. 2, 1989, pp. 303-314.
6. S.E. Fahlman, C. Lebiere, The cascade correlation learning architecture, Technical Report CMU-CS-90-100, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 1990.
7. E. Fiesler, Comparative Bibliography of Ontogenic Neural Networks, Proceedings of the International Conference on Artificial Neural Networks, ICANN'94, Sorrento, Italy, May 26-29, 1994, pp. 793-796.
8. J. Freeman, D. Skapura, Neural Networks: Algorithms, Applications and Programming Techniques, Addison-Wesley Publishing Company, Reading, MA, 1991.
9. J. Ghosh, K. Tumer, Structural adaptation and generalization in supervised feed-forward networks, J. Artif. Neural Networks 1 (4) (1994) 431-458.
10. S. Haykin, Neural Networks: A Comprehensive Foundation, MacMillan College Publishing Company, New York, 1994.
11. D.R. Hush, B. Horne, J.M. Salas, Error Surfaces for Multilayer Perceptrons, IEEE Trans. Systems, Man, Cybernet. 22 (5) (1992) 1152-1161.
12. T.-Y. Kwok, D.-Y. Yeung, Experimental analysis of input weight freezing in constructive neural networks, Proceedings of the 1993 IEEE International Conference on Neural Networks, ICNN'93, San Francisco, CA, USA, March 1993, pp. 511-516.
13. T.-Y. Kwok, D.-Y. Yeung, Constructive neural networks: some practical considerations, Proceedings of the 1994 IEEE International Conference on Neural Networks, ICNN'94, Orlando, FL, USA, June, 1994, pp. 198-203.
14. T.-Y. Kwok, D.-Y. Yeung, Constructive feedforward neural networks for regression problems: a survey, Technical Report HKUST-CS95-43, Department of Computer Science, Hong Kong University of Science and Technology, Kowloon, Hong Kong, September 1995.
15. T.-Y. Kwok, D.-Y. Yeung, Bayesian regularization in constructive neural networks, Proceedings of the 1996 International Conference on Artificial Neural Networks, ICANN'96, Bochum, Germany, July 1996, pp. 557-562.
16. T.-Y. Kwok, D.-Y. Yeung, Constructive algorithms for structure learning in feedforward neural networks for regression problems, IEEE Trans. Neural Networks 8 (3) (1997) 630-645.
17. T.-Y. Kwok, D.-Y. Yeung, Objective functions for training new hidden units in constructive neural networks, IEEE Trans. Neural Networks 8 (5) (1997) 1131-1148.
18. J. Lahnajärvi, M. Lehtokangas, J. Saarinen, Filtering ground reflection with neural networks, Proceedings of the Fifth European Congress on Intelligent Techniques and Soft Computing, EUFIT'97, Vol. 1, Aachen, Germany, September 8-11, 1997, pp. 470-474.
19. J. Lahnajärvi, M. Lehtokangas, J. Saarinen, Fast constructive methods for regression problems, Proceedings of the 18th IASTED International Conference on Modelling, Identification and Control, MIC'99, Innsbruck, Austria, February 15-18, 1999, pp. 442-445.
20. J. Lahnajärvi, M. Lehtokangas, J. Saarinen, Comparison of constructive neural networks for structure learning, Proceedings of the 18th IASTED International Conference on Modelling, Identification and Control, MIC'99, Innsbruck, Austria, February 15-18, 1999, pp. 446-449.
21. J. Lahnajärvi, M. Lehtokangas, J. Saarinen, Fixed cascade error - a novel constructive neural network for structure learning, Proceedings of the Artificial Neural Networks in Engineering Conference, ANNIE'99, St. Louis, Missouri, USA, November 7-10, 1999, pp. 25-30.
22. J. Lahnajärvi, Constructive neural networks in structure learning, Licentiate Thesis, Tampere University of Technology, Signal Processing Laboratory, Tampere, Finland, 1999.
23. J. Lahnajärvi, M. Lehtokangas, J. Saarinen, Forward selection initialization method for constructive neural networks, Proceedings of the Fourth World Multiconference on Circuits, Systems, Communications & Computers, CSCC'2000, Vouliagmeni, Greece, July 10-15, 2000, pp. 3151-3156. Also published as a book chapter in: N. Mastorakis (Ed.), Systems and Control: Theory and Applications, World Scientific and Engineering Society Press, 2000, pp. 263-268.
24. J. Lahnajärvi, M. Lehtokangas, J. Saarinen, Constructive neural networks - automated allocation of hidden units into hidden layers, Proceedings of the 2001 WSES International Conference on Neural Networks and Applications, NNA'01, Puerto De La Cruz, Spain, February 11-15, 2001, pp. 4041-4046. Also published as a book chapter in: N. Mastorakis (Ed.), Advances in Neural Networks and Applications, World Scientific and Engineering Society Press, 2001, pp. 146-151.
25. A. Lapedes, R. Farber, Nonlinear signal processing using neural networks: prediction and signal modeling, Technical Report LA-UR-87-2662, Los Alamos National Laboratory, Los Alamos, NM, 1987.
26. M. Lehtokangas, J. Saarinen, P. Huuhtanen, K. Kaski, Initializing weights of a multilayer perceptron network by using the orthogonal least squares algorithm, Neural Comput. 7 (5) (1995) 982-999.
27. M. Lehtokangas, Modeling with layered feedforward neural networks, Dr. Technical Dissertation, Tampere University of Technology, Electronics Laboratory, Publications 163, Tampere, Finland, 1995.
28. M. Lehtokangas, J. Saarinen, Weight Initialization with Reference Patterns, Neurocomputing 20 (1998) 265-278.
29. M. Lehtokangas, Fast initialization for cascade-correlation learning, IEEE Trans. Neural Networks 10 (2) (1999) 410-414.
30. E. Littmann, H. Ritter, Cascade LLM networks, Proceedings of the 1992 International Conference on Artificial Neural Networks, ICANN-92, Vol. 1, Brighton, United Kingdom, September 4-7, 1992, pp. 253-257.
31. E. Littmann, H. Ritter, Generalization abilities of cascade network architectures, in: S.J. Hanson, J.D. Cowan, C.L. Giles (Eds.), Advances in Neural Information Processing Systems 5, Morgan Kaufmann Publishers, San Mateo, CA, 1993, pp. 188-195.
32. A. Maren, C. Harston, R. Pap, Handbook of Neural Computing Applications, Academic Press, San Diego, 1990.
33. J.L. McClelland, D.E. Rumelhart, Parallel Distributed Processing, MIT Bradford Press, Cambridge, MA, 1986.
34. Oregon Graduate Institute of Science and Technology, Data Distribution WWW site (http://www.ece.ogi.edu/ ~ericwan/data.html).
35. L. Prechelt, PROBEN1-A set of neural network benchmark problems and benchmarking rules, Technical Report 21/94, Fakultät für Informatik, Universität Karlsruhe, D-76128 Karlsruhe, Germany, September 1994.
36. L. Prechelt, Investigation of the CasCor family of learning algorithms, Neural Networks 10 (5) (1997) 885-896.
37. R. Reed, Pruning algorithms - a survey, IEEE Trans. Neural Networks 4 (5) (1993) 740-747.
38. M. Riedmiller, RPROP - description and implementation details, Technical Report, January 1994, Institut für Logik, Komplexität und Deduktionssysteme, University of Karlsruhe, Germany, 1994.
39. M. Riedmiller, H. Braun, A direct adaptive method for faster backpropagation learning: the RPROP algorithm, Proceedings of the IEEE International Conference on Neural Networks, San Francisco, CA, March 28-April 1, 1993, pp. 586-591.
40. R. Rojas, Optimal weight initialization for neural networks, Proceedings of the International Conference on Artificial Neural Networks, ICANN'94, Vol. 1, Sorrento, Italy, May 26-29, 1994, pp. 577-580.
41. D. Rumelhart, G. Hinton, R. Williams, Learning internal representations by error propagation, in: D.E. Rumelhart, J.L. McClelland (Eds.), Parallel Distributed Processing, The MIT Press, Cambridge, MA, Vol. 1, 1986, pp. 318-330.
42. The Santa Fe Time Series Competition Data WWW site (http://www.stem.nyu.edu/ ~aweigend/Time-Series/SantaFe.html).
43. F.J. Smieja, Neural network constructive algorithms: trading generalization for learning efficiency? Circuits, Systems Signal Processing 12 (2) (1993) 331-374.
44. Strange Attractors, Chaos and Time-Series Analysis WWW site, (http://sprott.physics.wisc.edu/phys505/lect06.htm).
45. G. Thimm, E. Fiesler, High order and multilayer perceptron initialization, IEEE Trans. Neural Networks 8 (2) (1997) 349-359.
46. N.K. Treadgold, T.D. Gedeon, Extending CasPer: a regression survey, Proceedings of the 1997 International Conference on Neural Information Processing and Intelligent Information Systems, Vol. 1, 1997, pp. 310-313.
47. The UCI Machine Learning Repository WWW Site, (http://www.ics.uci.edu/ ~mlearn/ MLRepository.html).
48. P. Werbos, Beyond regression: new tools for prediction and analysis in the behavioral sciences, Ph.D. Dissertation, Harvard University, Cambridge, 1974.
49. L.F.A. Wessels, E. Barnard, Avoiding false local minima by proper initialization of connections, IEEE Trans. Neural Networks 3 (6) (1992) 899-905.
50. B. Widrow, S.D. Stearns, Adaptive Signal Processing, Prentice-Hall, Englewood Cliffs, NJ, 1985.
51. W.H. Wolberg, O.L. Mangasarian, Multisurface Method of Pattern Separation for Medical Diagnosis Applied to Breast Cytology, Proceedings of the National Academy of Sciences, USA, Vol. 87, December 1990, pp. 9193-9196.