Improving generalization capabilities of dynamic neural networks

Neural Computation

Volumn 16, Issue 6, 2004, Pages 1253-1282

  Galicki, Miroslaw ^a,b   Leistritz, Lutz ^a   Zwick, Ernst Bernhard ^c   Witte, Herbert ^a  

^a FRIEDRICH SCHILLER UNIVERSITY JENA   (Germany)

^b UNIVERSITY OF ZIELONA GÓRA   (Poland)

^c UNIVERSITY OF GRAZ   (Austria)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; BIOLOGICAL MODEL; COMPUTER SIMULATION;

ALGORITHMS; COMPUTER SIMULATION; MODELS, NEUROLOGICAL; NEURAL NETWORKS (COMPUTER);

EID: 2442550796     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976604773717603     Document Type: Article

References (47)

1. Abu-Mostafa, Y. S. (1989). The Vapnik-Chervonenkis dimension. Information versus complexity in learning. Neural Comp., 1, 312-317.
2. Azoff, E. M. (1993). Reducing error in neural network time series forecasting. Neural Computation and Applications, 1, 140-247.
3. Bhat, N. V., & McAvoy, T. J. (1990). Use of neural nets for dynamic modelling and control of chemical process systems. Computers and Chemical Engineering, 14, 573-583.
4. Bhat, N. V., & McAvoy, T. J. (1992). Determining model structure for neural models by network stripping. Computers and Chemical Engineering, 16, 271-281.
5. Billings, S., & Zheng, G. L. (1995). Radial basis function network configuration using genetic algorithms. Neural Networks, 8, 877-890.
6. Bishop, Ch. (1991). Improving the generalization properties of radial basis function neural networks. Neural Comp., 3, 579-588.
7. Bishop, Ch. (1995a). Training with noise is equivalent to Tikhonov regularization. Neural Comp., 7, 108-116.
8. Bishop, Ch. (1995b). Neural networks for pattern recognition. New York: Oxford University. Press.
9. Cohen, B., Saad, D. & Marom E. (1997). Efficient training of recurrent neural network with time delays. Neural Networks, 10, 51-59.
10. Czernichow, T. (1997). A double gradient algorithm to optimize regularization. In Proc. Artificial Neural Networks-ICANN'97, (pp. 289-294). Berlin: Springer-Verlag.
11. DasGupta, B., & Sontag, E. D. (1996). Sample complexity for learning recurrent perceptron mappings. IEEE Trans. Information Theory, 42, 1479-1487.
12. Doering, A., Galicki, M., & Witte, H. (1997). Structure optimization of neural networks with the A*-algorithm. IEEE Trans. Neural Networks, 8, 1434-1445.
13. Fabri, S., & Kadirkamanathan, V. (1996). Dynamic structure of neural networks for stable adaptive control of nonlinear systems. IEEE Trans. Neural Networks, 7, 1151-1166.
14. Fahlman, S. (1991). The recurrent cascade-correlation architecture, In R. Lippman, J. Moody & D. Touretzky (Eds.), Advances in neural information processing systems, 3 (pp. 190-196). San Mateo, CA: Morgan Kaufmann.
15. Fedorenko, R. R. (1978). Approximate solutions of optimal control problems. Moscow: Nauka. (in Russian)
16. Filippov, A. E (1962). On certian questions in the theory of optimal control. SIAM J. Control, 1, 76-84.
17. Fukunaga, K. (1990). Introduction to statistical pattern recognition. San Diego, CA: Academic Press.
18. Galicki, M. (1998). The planning of robotic optimal motions in the presence of obstacles. Int. J. Robot. Res., 17, 248-259.
19. Galicki, M., Leistritz, L., & Witte, H. (1999). Learning continuous trajectories in recurrent neural networks with time-dependent weights. IEEE Trans. Neural Networks, 10, 741-756.
20. Galicki, M., Leistritz, L., & Witte, H. (2000). Improved learning of multiple continuous trajectories with initial network state. In Proc. 2000 Int. Joint Conference on Neural Networks, IJCNN'2000 (pp 15-20) Los Alamitos, CA: IEEE Computer Society.
21. Geman, S., Bienenstock, E., & Doursat, E. R. (1992). Neural networks and the bias/variance dilemma. Neural Comp., 4, 1-58.
22. Giles, C. L., Chen, D., & Sun, G. Z. (1995). Constructive learning of recurrent neural networks. Limitations of recurrent cascade correlation and a simple solution. IEEE Trans. Neural Networks, 6, 829-836.
23. Girosi, F., Jones, M., & Poggio, T. (1995). Regularization theory and neural networks architectures. Neural Comp., 7, 219-269.
24. Hammer, B. (1997). Generalization of Elman networks. In Proc. Artificial Neural Networks-ICANN'97, (pp. 409-414). Berlin: Springer-Verlag.
25. Hansen, L. K., & Rasmussen, C. E. (1994). Pruning from adaptive regularization. Neural Comput., 6, 1223-1232.
26. Hinton, G. E. (1987). Learning translation invariant recognition in massively parallel networks. In J. W. de Bakker, A. J. Nijman, & P. C. Treleaven (Eds.), Proc. PARLE Conf. Parallel Architectures and Languages Europe, (pp. 1-13). Berlin: Springer-Verlag.
27. Holden, S. B., & Niranjan, M. (1995). On the practical applicability of VC dimension bounds. Neural Comp., 7, 1265-1288.
28. Kiliom, J., & Siegelmann, H. T. (1996). The dynamic universality of sigmoidal neural networks. Information and Computation, 128, 48-56.
29. Leistritz, L., Galicki, M., & Witte, H. (1998). Training continuous trajectories by means of dynamic neural networks with time dependent weights. In Proc. Int. ICSC/IFAC Symp. Neural Networks, (pp. 591-596). Vienna, Austria.
30. Leung, C. S., Tsoi, A. C., & Chan, L. W. (2001). Two regularizers for recursive least squared algorithms in feedforward multilayered neural networks. IEEE Trans. Neural Networks, 12, 1314-1332.
31. Ma, S., & Ji, C. (1998). Fast training of recurrent networks based on the EM algorithm. IEEE Trans. Neural Networks, 9, 11-26.
32. Morozov, V. A. (1984). Methods for solving incorrectly posed problems. Berlin: Springer-Verlag.
33. Murata, N., Yoshizawa, S., & Amari, S. (1994). Network information criterion-determining the number of hidden units for an artificial neural network model. IEEE Trans. Neural Networks, 5, 865-872.
34. Pearlmutter, B. A. (1995). Gradient calculation for dynamic recurrent neural networks: A survey. IEEE Trans. Neural Networks, 6, 1212-1228.
35. Pontryagin, L. C., Boltyansky, V. C., Gamkrelidze, R. V., & Mischenko, E. F. (1961). Mathematical theory of optimal problem. Moscow: Nauka. (in Russian)
36. Rementeria, S., & Olabe X. B. (2000). Predicting sunspots with a self-configuring neural system. In Proc. of the 8th Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems (IPMU 2000). Madrid, Spain.
37. Sakawa, Y., & Shindo, Y. (1980). On global convergence of an algorithm for optimal control. IEEE Trans. Automatic Control, 25, 1149-1153.
38. Simard, P., Victorri, B., Le Cun, Y., & Denker J. (1991). Tangent Prop-A formalism for specifying selected invariances in an adaptive network. In J. E. Moody, J. S.s Hanson, & R. P. Lippmann (Eds.), Advances in neural information processing systems 4 (pp. 895-903). San Mateo, CA: Morgan Kaufmann.
39. Sridhar, D. V., Bartlett E. B., & Seagrave, R. C. (1998). Information theoretic subset selection for neural network models. Computers and Chemical Engineering, 22, 613-626.
40. Srochko, V. A. (1986). Application of maximum principle for numerical solution of optimal control problems with terminal state constraints. Kibernetika, 1, 73-77. (in Russian)
41. Strend, S., & Balden, J. (1990). A comparison of constrained optimal control algorithms. In V. Utkin & V. Jaaksoo (Eds.), Prep. 11th IFAC World Congress, (pp. 192-198). Tallinn, Estonia: Technical University.
42. Sundareshan, M., & Condarcure, T. (1998). Recurrent neural networks training by a learning automaton approach for trajectory learning and control system design. IEEE Trans. Neural Networks, 9, 354-368.
43. Taniguchi, M., & Tresp, V. (1997). Combining regularized networks. In Proc. Artificial Neural Networks-ICANN'97, (pp. 349-354). Berlin: Springer-Verlag.
44. Tikhonov A. N., & Arsenin, V. Y. (1977). Solutions of ill-posed problems. Washington, D. C.: W. H. Winston.
45. Vapnik, V. N., & Chervonenkis, A. Y. (1971). On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability Applicat., 16, 264-280.
46. Wan, E. A. (1997). Combining fossil and sunspot data: Committee predictions. In Proc. of the 1997 International Conference on Neural Networks (ICNN 97). Berlin: Spring-Verlag.
47. Wu, L., & Moody, J. (1996). A smoothing regularizer for feedforward and recurrent neural networks. Neural Comput., 8, 461-489.