A comparison of first- And second-order training algorithms for dynamic neural networks

IEEE International Conference on Neural Networks - Conference Proceedings

Volumn 2, Issue , 2004, Pages 837-842

  Bajramovic, Ferid ^a,b   Gruber, Christian ^a,b   Sick, Bernhard ^a,b  

^a UNIVERSITY OF PASSAU   (Germany)

^b Institute for Computer Architectures   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

DATA SETS; DYNAMIC NEURAL NETWORK (DYNN); MULTILAYER PERCEPTRONS (MLP); NOISE SUPPRESSION;

ACOUSTIC NOISE; BACKPROPAGATION; DATA REDUCTION; ENERGY UTILIZATION; FIR FILTERS; LEARNING ALGORITHMS; OPTIMIZATION; TIME SERIES ANALYSIS;

NEURAL NETWORKS;

EID: 10944241890     PISSN: 10987576     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2004.1380038     Document Type: Conference Paper

References (23)

1. G. Dorffner, "Neural networks for time series processing," in Neural Network World, vol. 6 (4), pp. 447 - 468, 1996.
2. C. Ulbricht, G. Dorffner, S. Canu, D. Guillemyn, G. Marijuán, J. Olarte, C. Rodríguez, and I. Martín, "Mechanisms for handling sequences with neural networks," in Intelligent Engineering Systems through Artificial Neural Networks, C. H. Dagli, L. I. Burke, and Y. C. Shin, Eds. 1992, vol. 2, pp. 273 - 278, ASME Press, New York.
3. C. Gruber and B. Sick, "Fast and efficient second-order training of the dynamic neural network paradigm," in Proc. of the IJCNN, Portland, 2003, pp. 2482 - 2487.
4. T. Koskela, M. Lehtokangas, J. Saarinen, and K. Kaski, "Time series prediction with multilayer perceptron, FIR and Elman neural networks," in Proc. of the WCNN, San Diego, 1996, pp. 491 - 496, INNS Press.
5. E. A. Wan, Finite Impulse Response Neural Networks with Applications in Time Series Prediction, Ph.D. thesis, Stanford University, Department of Electrical Engineering, 1993.
6. T. Lin, B. G. Horne, P. Tiňo, and C. L. Giles, "Learning long-term dependencies in NARX recurrent neural networks," in IEEE Tr. on Neural Networks, vol. 7 (6), pp. 1329 - 1338. 1996.
7. S. Haykin, Neural Networks -A Comprehensive Foundation, Macmillan College Publishing Company, New York, 1994.
8. H. T. Siegelmann, B. G. Horne, and C. L. Giles, "Computional capabilities of recurrent NARX neural networks," IEEE Tr. on Systems, Man and Cybernetics - Part B, vol. 27 (2), pp. 208 - 215, 1997.
9. A. J. Shepherd, Second Order Methods for Neural Networks - Fast and Reliable Training Methods for Multi-Layer Perceptrons, Springer-Verlag, London, Berlin, Heidelberg, 1997.
10. N. Benvenuto, F. Piazza, and A. Uncini, "Comparison of four learning algorithms for multilayer perceptron with FIR synapses," in Proc. of the IEEE WCCI, 1994, vol. 1, pp. 309 - 314.
11. K. Patan and T. Parisini, "Stochastic learning methods for dynamic neural networks: simulated and real-data comparisons," in Proc. of the ACC, 2002, vol. 4, pp. 2577 - 2582.
12. P. Werbos, "Backpropagation through time: What it does and how to do it," Proc. of the IEEE, vol. 78 (10), pp. 1550 - 1560, 1990.
13. A. Zell, Simulation Neuronaler Netze, Addison-Wesley Publishing, Bonn, 2000.
14. M. Riedmiller, "Advanced supervised learning in multi-layer perceptrons - from backpropagation to adaptive learning algorithms," in International Journal of Computer Standards and Interfaces, vol. 16 (3), pp. 265 - 278. 1994, (Special Issue on Artificial Neural Networks).
15. J. E. Dennis and R. B. Schnabel, Numerical Methods for Unconstrained Optimization and Nonlinear Equations, Prentice Hall, Englewood Cliffs, New Jersey, 1983.
16. W. H. Press, W. T. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C: The Art of scientific Programming, Cambridge University Press, Cambridge, 2 edition, 1992.
17. R. Battiti, "First- and second-order methods for learning: Between steepest descent and newton's method," Neural Computation, vol. 4 (2), pp. 141 - 166, 1992.
18. K. Saito and R. Nakano, "Partial BFGS-update and efficient step-length calculation for three-layer neural networks," Neural Computation, vol. 9 (1), pp. 123 - 141, 1997.
19. M. F. Moller, "A scaled conjugate algorithm for fast supervised learning," Neural Networks, vol. 6 (1), pp. 525 - 533, 1993.
20. C. M. Bishop, Neural Networks for Pattern Recognition, Clarendon Press, Oxford, 1995.
21. L. Prechelt, "PROBEN 1 - A Set of Neural Network Benchmark Problems and Benchmarking Rules," Tech. rep. 21/94, University of Karlsruhe, 1994.
22. C. Hook, J. Kempf, and G. Scharfenberg, "New pen device for biometrical 3D pressure analysis of handwritten characters, words and signatures," in Proc. of the ACM SIGMM workshop on Biometrics methods and applications. 2003, pp. 38 - 44, ACM Press.
23. C. Gruber and B. Sick, "Processing short-term and long-term information with a combination of hard- and soft-computing techniques," in Proc. of the IEEE SMC Conf., Washington D.C., 2003, pp. 126 - 133.