Processing short-term and long-term information with a combination of polynomial approximation techniques and time-delay neural networks

IEEE Transactions on Neural Networks

Volumn 20, Issue 9, 2009, Pages 1450-1462

  Fuchs, Erich ^a   Gruber, Christian ^b   Reitmaier, Tobias ^a   Sick, Bernhard ^a  

^a UNIVERSITY OF PASSAU   (Germany)

^b Elektrobit   (Germany)

Author keywords

Orthogonal polynomials; Short term and long term information; Time series; Time delay neural networks

Indexed keywords

ARTIFICIAL DATA; LONG-TERM BEHAVIOR; LONG-TERM TREND; NEW APPROACHES; ON-LINE TOOLS; ORTHOGONAL EXPANSION; ORTHOGONAL POLYNOMIAL; ORTHOGONAL POLYNOMIALS; REAL-WORLD APPLICATION; SHORT-TERM AND LONG-TERM INFORMATION; TAPPED DELAY LINE; TEMPORAL INFORMATION; TIME-DELAY NEURAL NETWORKS; USER NUMBER;

FEEDFORWARD NEURAL NETWORKS; FOURIER ANALYSIS; LEAST SQUARES APPROXIMATIONS; POLYNOMIAL APPROXIMATION; TIME DELAY; TIME SERIES;

COMPUTER NETWORKS;

ALGORITHM; APPARATUS, EQUIPMENT AND SUPPLIES; ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER; COMPUTER SIMULATION; FACTUAL DATABASE; FORECASTING; HUMAN; METHODOLOGY; NERVE CELL; NONLINEAR SYSTEM; PERIODICITY; REGRESSION ANALYSIS; STATISTICAL MODEL; TIME;

ALGORITHMS; COMPUTER SIMULATION; COMPUTERS; DATABASES, FACTUAL; EQUIPMENT AND SUPPLIES; FORECASTING; HUMANS; LEAST-SQUARES ANALYSIS; LINEAR MODELS; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); NEURONS; NONLINEAR DYNAMICS; PERIODICITY; TIME FACTORS;

EID: 70349238624     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2009.2024679     Document Type: Article

References (50)

1. G. Dorffner, "Neural networks for time series processing," Neural Netw. World, vol. 6, no. 4, pp. 447-468, 1996.
2. S. J. Ovaska and B. Sick, "Fusion of soft computing and hard computing: Applications and research opportunities," in Computational Intelligence: Principles and Practice, G. Yen and D. B. Fogel, Eds. Piscataway, NJ: IEEE, 2006, ch. 3, pp. 47-72.
3. B. Sick and S. J. Ovaska, "Fusion of soft and hard computing: Multidimensional categorization of computationally intelligent hybrid systems," Neural Comput. Appl., vol. 16, no. 2, pp. 125-137, 2007.
4. E. Fuchs and K. Donner, "Fast least-squares polynomial approximation in moving time windows," in Proc. Int. Conf. Acoust. Speech Signal Process., Munich, Germany, 1997, vol. 3, pp. 1965-1968.
5. E. Fuchs, Schnelle Quadratmittelapproximation in gleitenden Zeitfenstern mit diskreten orthogonalen Polynomen, ser. Berichte aus der Informatik. Aachen, Germany: Shaker-Verlag, 2000.
6. A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. J. Lang, "Phoneme recognition using time-delay neural networks," IEEE Trans. Acoust. Speech Signal Process., vol. 37, no. 3, pp. 328-339, Mar. 1989.
7. E. A.Wan, "Finite impulse response neural networks with applications in time series prediction," Ph.D. dissertation, Dept. Electr. Eng., Stanford Univ., Stanford, CA, 1993.
8. E. A. Wan, "Time series prediction by using a connectionist network with internal delay lines," in Time Series Prediction: Forecasting the Future and Understanding the Past, ser. SFI Studies in the Sciences of Complexity, A. S.Weigend and N. A. Gershenfeld, Eds. Reading, MA: Addison-Wesley, 1993, pp. 195-217.
9. C. Ulbricht, G. Dorffner, S. Canu, D. Guillemyn, G. Marijuán, J. Olarte, C. Rodríguez, and I. Martín, "Mechanisms for handling sequences with neural networks," in Intelligent Engineering Systems Through Artificial Neural Networks, C. H. Dagli, L. I. Burke, and Y. C. Shin, Eds. New York: ASME Press, 1992, vol. 2.
10. D.-T. Lin, J. E. Dayhoff, and P. A. Ligomenides, "Adaptive time-delay neural network for temporal correlation and prediction," in SPIE-Intell. Robots Comput. Vis. XI, 1992, vol. 1826, pp. 170-181.
11. J. A. Howell and H. Buxton, "Learning gestures with time-delay RBF networks," in Proc. 8th Int. Conf. Artif. Neural Netw., Skövde, L. Niklasson, M. Bodén, and T. Ziemke, Eds., London, U.K., 1998, vol. 1, pp. 239-244.
12. S. Haykin, Neural Networks-A Comprehensive Foundation. New York: Macmillan, 1994.
13. A. C. Tsoi, "Recurrent neural network architectures-An overview," in Adaptive Processing of Sequences and Data Structures, ser. Lecture Notes in Computer Science, C. L. Giles and M. Gori, Eds. New York: Springer-Verlag, 1998, vol. 1387.
14. S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Comput., vol. 9, no. 8, pp. 1735-1780, 1997.
15. F. Gers, N. Schraudolph, and J. Schmidhuber, "Learning precise timing with LSTM recurrent networks," J. Mach. Learn. Res., vol. 3, pp. 115-143, 2002.
16. U. Bodenhausen, "The tempo-algorithm: Learning in a neural network with variable time-delays," in Proc. Int. Joint Conf. Neural Netw., Washington, DC, 1990, vol. 1, pp. 597-600.
17. T.-N. Lin, C. L. Giles, B. G. Horne, and S.-Y. Kung, "A delay damage model selection algorithm for NARX neural networks," IEEE Trans. Signal Process., vol. 45, Special Issue on Neural Networks, no. 11, pp. 2719-2730, Nov. 1997.
18. T. Lin, B. G. Horne, P. Tiňo, and C. L. Giles, "Learning long-term dependencies in NARX recurrent neural networks," IEEE Trans. Neural Netw., vol. 7, no. 6, pp. 1329-1338, Nov. 1996.
19. T. Koskela, M. Lehtokangas, J. Saarinen, and K. Kaski, "Time series prediction with multilayer perceptron, FIR and Elman neural networks," in Proc.World Congr. Neural Netw., San Diego, CA, 1996, pp. 491-496.
20. B.-L. Zhang, R. Coggins, M. Jabri, D. Dersch, and B. Flower, "Multiresolution forecasting for futures trading using wavelet decompositions," IEEE Trans. Neural Netw., vol. 12, no. 4, pp. 765-775, Jul. 2001.
21. V. Alarcon-Aquino and J. Barria, "Multiresolution fir neural-networkbased learning algorithm applied to network traffic prediction," IEEE Trans. Syst. Man Cybern. C, Appl. Rev., vol. 36, no. 2, pp. 208-220, Mar. 2006.
22. A. Zell, Simulation Neuronaler Netze. Reading, MA: Addison-Wesley, 1994.
23. D. Bassi, "Short- and long-term online combined learning for robotic control," in Proc. IEEE Int. Joint Conf. Neural Netw., Seattle, WA, 1991, vol. 3, pp. 2019-2026.
24. D. Pham and M. Sukkar, "A supervised neural network for dynamic systems identification," in Proc. 4th IEEE Int. Conf. Universal Personal Commun., Tokyo, Japan, 1995, pp. 697-701.
25. J. T.-H. Lo, "Robust adaptive identification of dynamic systems by neural networks," in Proc. Int. Conf. Neural Netw., Houston, TX, 1997, vol. 2, pp. 1121-1126.
26. J. Lo and D. Bassu, "Adaptive multilayer perceptrons with long- and short-term memories," IEEE Trans. Neural Netw., vol. 13, no. 1, pp. 22-33, Jan. 2002.
27. D. Prokhorov, "Training recurrent neurocontrollers for real-time applications," IEEE Trans. Neural Netw., vol. 18, no. 4, pp. 1003-1015, Jul. 2007.
28. A. Nikiforov, S. Suslov, and V. Uvarov, Classical Orthogonal Polynomials of a Discrete Variable, ser. Computational Physics. New York: Springer-Verlag, 1991.
29. E. Fuchs, "On discrete polynomial least-squares approximation in moving time windows," in Applications and Computation of Orthogonal Polynomials, ser. Numerical Mathematics, W. Gautschi, G. Golub, and G. Opfer, Eds. Basel, Switzerland: Birkhäuser, 1999, pp. 93-107, no. 131.
30. W. Gautschi, "On generating orthogonal polynomials," SIAM J. Sci. Statist. Comput., vol. 3, pp. 289-317, Sep. 1982.
31. W. Gautschi, "An algorithmic implementation of the generalized Christoffel theorem," in Numerical Integration, ser. Numerical Mathematics, G. Hämmerlin, Ed. Basel, Switzerland: Birkhäuser, 1982, vol. 57, pp. 89-106.
32. D. Galant, "An implementation of Christoffel's theorem in the theory of orthogonal polynomials," Math. Comput., vol. 25, no. 113, pp. 111-113, 1971.
33. C. Gruber and B. Sick, "Fast and efficient second-order training of the dynamic neural network paradigm," in Proc. IEEE-INNS-ENNS Int. Joint Conf. Neural Netw., Portland, OR, 2003, vol. 4, pp. 2482-2487.
34. F. Bajramovic, C. Gruber, and B. Sick, "A comparison of first- and second-order training algorithms for dynamic neural networks," in Proc. Int. Joint Conf. Neural Netw., Budapest, Hungary, 2004, vol. 2, pp. 837-842.
35. M. Riedmiller and H. Braun, "A direct method for faster backpropagation learning: The RPROP algorithm," in Proc. IEEE Int. Conf. Neural Netw., San Francisco, CA, 1993, vol. 1, pp. 586-591.
36. M. Riedmiller, "RPROP-Description and Implementation Details, Universität Karlsruhe," Institut für Logik, Komplexität und Deduktionssysteme, Karlsruhe, Germany, Tech. Rep., 1994.
37. M. F. Moller, "A scaled conjugate algorithm for fast supervised learning," Neural Netw., vol. 6, no. 1, pp. 525-533, 1993.
38. A. J. Shepherd, Second Order Methods for Neural Networks - Fast and Reliable Training Methods for Multi-Layer Perceptrons. New York: Springer-Verlag, 1997.
39. H. Liu and H. Motoda, Feature Selection for Knowledge Discovery and Data Mining. Boston, MA: Kluwer, 1998.
40. H. Liu and H. Motoda, Eds., Feature Extraction, Construction, and Selection: A Data Mining Perspective. Boston, MA: Kluwer, 1998.
41. C. Gruber, M. Coduro, and B. Sick, "Signature verification with dynamic RBF networks and time series motifs," in Proc. 10th Int. Workshop Frontiers Handwriting Recognit., La Baule, France, 2006, pp. 455-460.
42. B. Chiu, E. Keogh, and S. Lonardi, "Probabilistic discovery of time series motifs," in Proc. 9th Int. Conf. Knowl. Disc. Data Mining, Washington, DC, 2003, pp. 493-498.
43. L. J. Tashman, "Out-of-sample tests of forecasting accuracy: An analysis and review," Int. J. Forecasting, vol. 16, pp. 437-450, 2000.
44. R. J. Hyndman and A. B. Koehler, "Another look at measures of forecast accuracy," Int. J. Forecasting, vol. 22, pp. 679-688, 2006.
45. C. Gruber and B. Sick, "Processing short-term and long-term information with a combination of hard- and soft-computing techniques," in Proc. IEEE Int. Conf. Syst. Man Cybern., Washington, DC, 2003, vol. 1, pp. 126-133.
46. E. Fuchs and B. Sick, "Using temporal information in input features of neural networks," in Proc. 9th Int. Conf. Artif. Neural Netw., Edinburgh, U.K., 1999, vol. 1, pp. 467-472.
47. B. Sick, "Fusion of soft and hard computing techniques in indirect, online tool wear monitoring," IEEE Trans. Syst. Man Cybern. C, Appl. Rev., vol. 32, no. 2, pp. 80-91, May 2002.
48. S. V. Kamarthi, "On-line tool wear estimation in turning through sensor data fusion and neural networks," Ph.D. dissertation, Dept. Ind. Manuf. Eng., Pennsylvania State Univ., State College, PA, 1994.
49. S. Elhay, G. H. Golub, and J. Kautsky, "Updating and downdating of orthogonal polynomials with data fitting applications," SIAM J. Matrix Anal. Appl., vol. 12, no. 2, pp. 327-353, 1991.
50. B. A. Pearlmutter, "Dynamic recurrent neural networks," School Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. CMU-CS-90-196, 1990.