A systematic comparison of flat and standard cascade-correlation using a student-teacher network approximation task

Connection Science

Volumn 19, Issue 3, 2007, Pages 223-244

  Dandurand, F ^a   Berthiaume, V ^a   Shultz, T R ^a  

^a MCGILL UNIVERSITY   (Canada)

Author keywords

Cascade correlation; Connectivity; Function approximation; Network depth; Neural network architectures

Indexed keywords

CASCADE-CORRELATION; COMPUTATIONAL COSTS; COMPUTATIONAL POWER; CONNECTION WEIGHTS; FUNCTION APPROXIMATION; INPUT-OUTPUT MAPPING; NETWORK APPROXIMATION; NETWORK DEPTHS;

NEURAL NETWORKS; STUDENTS;

COMPLEX NETWORKS;

EID: 34548348927     PISSN: 09540091     EISSN: 13600494     Source Type: Journal    
DOI: 10.1080/09540090701528951     Document Type: Article

References (39)

1. A. Adams and S. Waugh, "Function evaluation and die cascade-correlation architecture", in The IEEE International Conference on Neural Networks, 1995, pp. 942-946.
2. S. Amari, N. Murata, K.R. Muller, M. Finke and H.H. Yang, "Asymptotic statistical theory of overtraining and cross-validation", IEEE Trans. Neural Net., 8, pp. 985-996, 1997.
3. P.T. Baffes and J.M. Zelle, "Growing layers of perceptrons: introducing the extentron algorithm", in Proceedings of the International Joint Conference on Neural Networks 2, 1992, pp. 392-397.
4. S. Baluja and S.E.Fahlman, "Reducing network depth in the cascade-correlation". Technical Report CMU-CS-94-209, Carnegie Mellon University (1994).
5. D. Buckingham and T.R. Shultz, 'The developmental course of distance, time, and velocity concepts: a generative connectionist model", J. Cognit. Dev., 1, pp. 305-345, 2000.
6. D.C Donderi, "An information theory analysis of visual complexity and dissimilarity", Perception, 35, pp. 823-835, 2006.
7. D.C. Donderi and S. McFadden, "Compressed file length predicts search time anderrors on visual displays", Displays, 26, pp. 71-78, 2005.
8. S. Draghici, "Improving the speed of some constructive algorithms by using a locking detection mechanism", Neural Net. World, 6, pp. 563-575, 1996.
9. S. Draghici, "The constraint based decomposition (CBD) training architecture", Neural Net., 14, pp. 527-550, 2001.
10. S.E. Fahlman, "Faster-learning variations on back-propagation: an empirical study", in Proceedings of the 1988 Connectionist Models Summer School, T.J. Sejnowski, G.E. Hinton and D.S. Touretzky, Eds, San Mateo, CA: Morgan Kaufmann, 1988.
11. S.E. Fahlman and C. Lebiere, "The cascade-correlation learning architecture", in Advances in Neural Information Pmcessing Systems 2, D.S. Touretzky, Ed., Los Altos, CA: Morgan Kaufmann, 1990, pp. 524-532.
12. M. Frean, "The upstart algorithm: a method for constructing and training feedforward neural networks", Neural Comput., 2, pp. 198-209, 1990.
13. L.K. Hansen and M.W. Pedersen, "Controlled growth of cascade-correlation nets", in Proceedings of ICANN, 1994, pp. 797-800.
14. J. Hwang, S. You, S. Lay and I. Jou, "The cascade-correlation learning: a projection pursuit learning perspective", IEEE Trans. Neural Net., 7, pp. 278-289, 1996.
15. H. Klagges and M. Soegtrop, "Limited fan-in random, wired cascade-correlation", Neuroprose Archive, Munich, IBM Research Division, Physics Group (1992).
16. J.J.T. Lahnajarvi, M.I. Lehtokangas and J.P.P. Saarinen, "Evaluation of constructive neural networks with cascaded architectures", Neurocomputing, 48, pp. 573-607, 2002.
17. S. Lawrence, C.L. Giles and A.C. Tsoi, "Lessons in neural network training: overfitting may be harder than expected", in Proceedings of the Fourteenth National Conference on Artificial Intelligence, AAA1-97, 1997a, pp. 540-545.
18. S. Lawrence, A.D. Back, A.C. Tsoi and C.L. Giles, "On the distribution of performance from multiple neural network trials", IEEE Trans. Neural Net., 8, pp. 1507-1517, 1997b.
19. E. Littmann and H. Ritter, "Generalization abilities of cascade network architectures", in Advances in Neural Information Processing System. 5, S.J. Hanson, J. Cowan and C.L. Giles, Eds, Los Altos, CA: Morgan Kaufmann, 1993, pp. 188-195.
20. M. Mezard and J. Nadal, "Learning feedforward networks: the tiling algorithm", J. Phys. A: Math. Gen., 22, pp. 2191-2203, 1989.
21. R. Parekh, J. Yang and V. Honavar, "Constructive neural network learning algorithms for pattern classification", IEEE Trans. Neural Net., 11, pp. 436-451, 2000.
22. H. Park, N. Murata and S. Amari, "Improving generalization performance of natural gradient learning using optimized regularization by NIC", Neural Comput, 16, pp. 355-382, 2004.
23. D.S. Phatak and I. Koren, "Connectivity and performance tradeoffs in the cascade correlation learning architecture", IEEE Trans. Neural Net., 5, pp. 930-935, 1994.
24. L. Prechelt, "PROBENl - a set of neural network benchmark, problems and benchmarking rules". Technical Report, Karlsruhe, Germany (1994).
25. L. Prechelt, "Investigation of the CasCor family of learning algorithms", Neural Net., 10, pp. 885-896, 1997.
26. M.Y. Rafiq, G. Bugmann and D.J. Easterbrook, "Neural network, design for engineering applications", Comput. Struct., 79, pp. 1541-1552, 2001.
27. S.G. Romaniuk and L.O. Hall, "Divide and conquer neural networks", Neural Net., 6, pp. 1105-1116, 1993.
28. D. Saad (Ed.), On-line Learning in Neural Networks, Cambridge: Cambridge University Press, 1999.
29. T.R. Shultz, Computational. Developmental Psychology, Cambridge, MA: MIT Press, 2003.
30. T.R. Shultz, "Constructive learning in the modeling of psychological development", in Processes of Change in Brain and Cognitive. Development: Attention and Performance XXI, Y. Munakata and M. H. Johnson, Eds, Oxford:
31. Oxford University Press, 2006, pp. 61-86.
32. T.R. Shultz, D. Mareschal and W.C Schmidt, "Modeling cognitive development on balance scale phenomena", Machine Learn., 16, pp. 57-86, 1994.
33. T.R. Shultz, S.P. Mysore and S.R. Quartz, "Why let networks grow?", in Newoconstmctivism: Perspectives and Prospects, Vol. 2, D. Mareschal, S. Sirois, G. Westermann and M.H. Johnson, Eds, Oxford: Oxford University Press, 2007, pp. 65-98.
34. T.R. Shultz and R Rivest, "Knowledge-based cascade-correlation: using knowledge to speed learning", Connection Sci, 13, pp. 43-72, 2001.
35. S. Sjogaard, "A conceptual approach to generalization in dynamic neural networks". PhD thesis, Aarhus University (1991).
36. Y. Takane, Y. Oshima-Takane and T.R. Shultz, "Network analyses: the case of first and second person pronouns", in Proceedings of the 1995 IEEE International Conference on Systems, Man and Cybernetics, 1995, pp. 3594-3599.
37. J.P. Thivierge, F. Rivest and T.R. Shultz, "A dual-phase technique for pruning constructive networks", in Proceedings of the 2003 IEEE International. Joint Conference on Neural Networks, 2003, pp. 559-564.
38. S. Waugh and A. Adams, "Connection strategies in cascade-correlation", in Proceedings: The Fifth Australian Conference on Neural Networks, 1994, pp. 1-4.
39. E.J. Wisniewski, "Prior knowledge and functionally relevant features in concept learning", J. Exp. Psychol: Learn. Mem. Cognit., 21, pp. 449-468, 1995.