Structure optimization of neural networks with the A*-algorithm

IEEE Transactions on Neural Networks

Volumn 8, Issue 6, 1997, Pages 1434-1445

  Doering, Axel ^b   Galicki, Miroslaw ^b   Witte, Herbert ^a,b  

^a IEEE

^b FRIEDRICH SCHILLER UNIVERSITY JENA   (Germany)

Author keywords

EEG processing; Evaluation of neural networks; Feedforward neural networks; Generalization capability; Heuristic search; Structure optimization; System identification

Indexed keywords

ALGORITHMS; DATA STRUCTURES; GRAPH THEORY; HEURISTIC METHODS; IDENTIFICATION (CONTROL SYSTEMS); OPTIMIZATION;

OBJECTIVE FUNCTION; STRUCTURE OPTIMIZATION;

FEEDFORWARD NEURAL NETWORKS;

EID: 0031269563     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.641466     Document Type: Article

References (38)

1. B. Kosko, Neural Networks and Fuzzy Systems. Reading, MA: Addison-Wesley, 1992.
2. K. Hornik, M. Stinchcombe, and H. White, "Multilayer feedforward networks are universal approximators," Neural Networks, vol. 2, no. 5, pp. 359-366, 1989.
3. G. Cybenko, "Approximation by superpositions of a sigmoidal function," Math. Contr., Signals, and Syst., vol. 2, pp. 303-314, 1989.
4. S. Geman, E. Bienenstock, and R. Doursat, "Neural networks and the bias/variance dilemma," Neural Computa., vol. 4, no. 1, pp. 1-58, 1992.
5. N. Murata, S. Yoshizawa, and S. Amari, "Network information criterion - Determining the number of hidden units for an artificial neural network model," IEEE Trans. Neural Networks, vol. 5, pp. 865-872, Nov. 1994.
6. A. N. Kolmogorov, "On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition," Dokl. Akad. Nauk USSR, vol. 114, pp. 953-956, 1957.
7. M. Mézard and J.-P. Nadal, "Learning in feedforward layered networks: The tiling algorithm," J. Phys. A, vol. 22, pp. 2191-2204, 1989.
8. M. Frean, "The upstart algorithm: A method for constructing and training feedforward neural networks," Neural Computa., vol. 2, pp. 198-209, 1990.
9. Y. Hirose, K. Yamashita, and S. Hijiya, "Back-propagation algorithm which varies the number of hidden units," Neural Networks, vol. 4, no. 1, pp. 61-66, 1991.
10. S. E. Fahlman and C. Lebiere, "The cascade-correlation learning architecture," in Advances in Neural Information Processing Systems, D. S. Touretzky, Ed., vol. 2. San Mateo, CA: Morgan Kaufmann, 1990, pp. 524-532.
11. J. Sietsma and R. J. F. Dow, "Creating artificial neural networks that generalize," Neural Networks, vol. 4, no. 1, pp. 67-79, 1991.
12. H. White, "Learning in artificial neural networks: A statistical perspective," Neural Computa., vol. 1, no. 4, pp. 425-464, 1989.
13. Y. Le Cun, J. S. Denker, and S. A. Solla, "Optimal brain damage," in Advances in Neural Information Processing Systems, D. S. Touretzky, Ed., vol. 2. San Mateo, CA: Morgan Kaufmann, 1990, pp. 598-605.
14. S. A. Harp, T. Samad, and A. Guha, "Designing application-specific neural networks using the genetic algorithm," in Advances in Neural Information Processing Systems, D. S. Touretzky, Ed., vol. 2. San Mateo, CA: Morgan Kaufmann, 1989, pp. 447-454.
15. G. F. Miller, P. M. Todd, and S. U. Hedge, "Designing neural networks using genetic algorithms," in Proc. 3rd Int. Conf. Genetic Algorithms, J. D. Schaffer, Ed. San Mateo, CA: Morgan Kaufmann, 1989, pp. 379-384.
16. R. Zollner, H. J. Schmitz, F. Wünsch, and U. Krey, "Fast generating algorithm for a general three-layer perceptron," Neural Networks, vol. 5, 1992.
17. B. Andree, "Constructive learning algorithms for feed-forward neural networks", Ph.D. dissertation, Univ. Utrecht, The Netherlands, 1995.
18. S.-C. Huang and Y.-F. Huang, "Bounds on the number of hidden neurons in multilayer perceptrons," IEEE Trans. Neural Networks, vol. 2, pp. 47-55, Jan. 1991.
19. A. S. Weigend, D. E. Rumelhart, and B. A. Huberman, "Generalization by weight-elimination with application to forecasting," in Advances in Neural Information Processing Systems 3, R. P. Lippmann, J. E. Moody, and D. S. Touretzky, Eds. San Mateo, CA: Morgan Kaufmann, 1991, pp. 875-882.
20. B. D. Ripley, Pattern Recognition and Neural Networks. Cambridge, U.K.: Cambridge Univ. Press, 1996.
21. Y. S. Abu-Mostafa, "The Vapnik-Chervonenkis Dimension: Information versus complexity in learning," Neural Computa., vol. 1, no. 3, pp. 312-317, 1989.
22. V. N. Vapnik and A. Y. Chervonenkis, "On the uniform convergence of relative frequencies of events to their probabilities," Theory of Probability Applicat., vol. 16, pp. 264-280, 1971.
23. E. B. Baum and D. Haussler, "What size net gives valid generalization?," Neural Computa., vol. 1, pp. 151-160, 1989.
24. D. Haussler, "Decision theoretic generalizations of the PAC model for neural net and other learning applications," Inform. Computa., vol. 100, pp. 78-150, 1992.
25. S. B. Holden and M. Niranjan, "On the practical applicability of VC dimension bounds," Neural Computa., vol. 7, no. 6, pp. 1265-1288, 1995.
26. N. Nilsson, Principals of Artificial Intelligence. New York: Springer-Verlag, 1980.
27. J. Pearl, Heuristics - Intelligent Search Strategies for Computer Problem Solving. Reading, MA: Addison-Wesley, 1984.
28. R. Dechter and J. Pearl, "Generalized best-first search strategies and the optimality of A*," JACM, vol. 32, no. 3, pp. 505-536, 1985.
29. J. N. Hwang, S. S. You, S. R. Lay, and I. C. Jou, "The cascade-correlation learning: A projection pursuit learning perspective," IEEE Trans. Neural Networks, vol. 7, pp. 278-289, Mar. 1996.
30. G. P. Drago and S. Ridella, "On the convergence of a growing topology neural algorithm," Neurocomputing, vol. 12, pp. 171-185, 1996.
31. L. Prechelt, "Some notes on neural learning algorithm benchmarking," Neurocomputing, vol. 9, no. 3, pp. 343-347, 1995.
32. G. W. Stewart, "The efficient generation of random orthogonal matrices with an application to condition estimators," SIAM J. Numer. Anal., vol. 17, pp. 403-409, 1980.
33. D. Michie, D. J. Spiegelhalter, and C. C. Taylor, Eds., Machine Learning, Neural and Statistical Classification. New York: Ellis Horwood, 1994.
34. R. C. Eberhart, R. W. Dobbins, and W. R. S. Webber, "Casenet: A neural network tool for EEG wave form classification.," in Proc. IEEE Symp. Computer-Based Med. Syst., 1989, pp. 60-89.
35. T. Pietila, S. Vapaakoski, U. Nousiainen, A. Varri, H. Frey, V. Hakkinen, and Y. Neuvo, "Evaluation of a computerized system for recognition of epileptic activity during long-term EEG recording," Electroencephalography and Clinical Neurophysiology, vol. 90, pp. 438-443, 1994.
36. D. Lehmann and W. Skrandies, "Reference-free identification of the components of checkerboard evoked multichannel potential fields," Electroencephalography and Clinical Neurophysiology, vol. 48, pp. 609-621, 1980.
37. H. Witte, M. Eiselt, S. Petranek, G. Griessbach, V. Krajca, and M. Rother, "Use of Discrete Hilbert Transformation for automatic spike mapping: a methodological investigation," Med. Biol. Eng. Computing, vol. 29, pp. 242-248, May 1991.
38. A. Doering and H. Witte, "Feedforward neural networks for Bayes-optimal classification: Investigations on the influence of the composition of the training set on the cost function," in Proc. 13th ICPR, vol. IV, Los Alamitos, CA: IEEE Computer Society Press, 1996, pp. 219-223.