Recurrent Neural Networks and Robust Time Series Prediction

IEEE Transactions on Neural Networks

Volumn 5, Issue 2, 1994, Pages 240-254

  Connor, Jerome T ^a   Martin, R Douglas ^b   Atlas, L E ^b  

^a TELCORDIA TECHNOLOGIES   (United States)

^b UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; COMPUTER SIMULATION; DIFFERENCE EQUATIONS; LEARNING SYSTEMS; LEAST SQUARES APPROXIMATIONS; MATHEMATICAL MODELS; PARAMETER ESTIMATION; PREDICTIVE CONTROL SYSTEMS; ROBUSTNESS (CONTROL SYSTEMS); SENSITIVITY ANALYSIS; SIGNAL FILTERING AND PREDICTION; TIME SERIES ANALYSIS;

AUTOREGRESSIVE MOVING AVERAGE; DATA SETS; FEEDFORWARD NETWORKS; FILTERED DATA; PUGET POWER ELECTRIC DEMAND TIME SERIES; RECURRENT NEURAL NETWORKS; ROBUST TIME SERIES PREDICTION; SYNTHETIC DATA; TARGET FUNCTION;

NEURAL NETWORKS;

EID: 0028401357     PISSN: 10459227     EISSN: 19410093     Source Type: Journal    
DOI: 10.1109/72.279188     Document Type: Article

References (42)

1. B. Anderson, and J. Moore, Optimal Filtering. Englewood Cliffs, NJ: Prentice-Hall, 1979.
2. R. Battiti, “First- and second-order methods for learning: Between steepest descent and Newton's method,” Neural Computation, vol. 4, pp. 141–166, 1992.
3. G. E. Box and G. M. Jenkins, Time series analysis: forecasting and control, Holden-Day, 1976.
4. G. E. Box and G. C. Tiao, “Intervention analysis with applications to economic and environmental problems,” J. Amer. Stat. Assoc., vol. 70, 1975.
5. L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and Regression Trees. Belmont, CA: Wadsworth International Group, 1984.
6. D. R. Brillinger, “The identification of polynomial systems by means of higher order spectra,” J. Sound Vib., vol. 12, pp. 301–313, 1970.
7. J. T. Connor, “Time series and neural network modeling,” doctoral dissertation, Univ. of Washington, 1993.
8. J. T. Connor, L. E. Atlas, and R. D. Martin, “Recurrent and NARMA modeling,” Advances in Neural Information Processing Systems 4, J. E. Moody, S. J. Hanson, and R. P. Lippmann, Eds. San Matteo, CA: Morgan Kauffman, pp. 301–308, 1992.
9. A. P. Dempster, N. M. Laird, and D. B. Rubin, “Maximum likelihood from incomplete data via the EM algorithm,” Journal of the Royal Statistical Society, B, vol. 39, pp. 1–38, 1977.
10. J. L. Elman, “Finding structure in time,” Cognitive Science, vol. 14, pp. 179–21, 1990.
11. J. H. Friedman, “Multivariate adaptive regression splines,” The Annals of Statistics, vol. 19, pp. 1–141, 1991.
12. J. H. Friedman and W. Stuetzle, “Projection Pursuit regression,” J. Amer. Statist. Assoc., pp. 817–823, 1981.
13. S. Grossberg, “Adaptive pattern classification and universal recoding: 1. Parallel development and coding of neural feature detectors,” Biol. Cybern., vol. 23, pp. 121–134, 1976.
14. A. C. Harvey, The econometric analysis of time series. Cambridge, MA: MIT Press, 1990.
15. G. E. Hinton, and T. J. Sejnowski, “Learning and relearning in Boltzman machines,” D. E. Rumelhart, J. L. McClelland, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 1: Foundations, Cambridge, MA: MIT Press, Chapter 7, 1986.
16. J. J. Hopfield “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. USA, vol. 79, pp. 2554–2558, 1982.
17. A. Jazwinski, Stochastic processes and filtering theory. New York: Academic Press, 1970.
18. M. 1. Jordan, “Attractor Dynamics and parallelism in a connectionist sequential machine,” in Proceedings of the Eighth Annual Conference of the Cognitive Science Society, Hillsdale, NJ: Erlbaum, 1987, pp. 531–546.
19. B. Kleiner, R. D. Martin, and D. J. Thomson, “Robust estimation of power spectra,” J. Roy. Stat. Soc. Ser. B vol 41, pp. 313–351, 1979.
20. B. Kosko, Neural Networks and Fuzzy Systems: A Dynamical Systems Approach to Machine Intelligence. Englewood Cliffs, NJ: Prentice Hall, 1992.
21. A. Lapedes, and R. Farber, “Nonlinear signal processing using neural networks: Prediction and Modeling,” Technical Report, LA-UR87-2662, Los Alamos National Laboratory, Los Alamos, New Mexico, 1987.
22. D. J. C. MacKay, “Bayesian non-linear modeling for the energy prediction completion,” Draft 1.3., 1993.
23. R. D. Martin, and R. Fraiman, “Approximate conditional mean non-Gaussian filters,” Tech. Report, Dept. of Statistics, Univ. of Washington, 1993.
24. R. D. Martin, “Robust estimation of autoregressive models (with discussion),” in Directions in Time Series, D. R. Brillinger and G. C. Tiao, Eds. Institute of Mathematical Statistics, Hayward, Calif., 1980, pp. 228–262.
25. R. D. Martin, and D. J. Thompson, “Robust-resistant spectrum estimation,” Proc. IEEE, vol. 70, pp. 1097–1115, 1982.
26. R. D. Martin and V. J. Yohai, “Highly robust estimation of autoregression integrated time series models,” Tech. Report., 1985.
27. R. D. Martin, “Robust estimation for time series autoregressions,” in Robustness in Statistics, R. L. Launer and G. N. Wilkinson, Eds. New York: Academic Press, pp. 147–176, 1979.
28. C. J. Masreliez, “Approximate non-Gaussian filtering with linear state and observation relations,” IEEE Transactions on Automatic Control. pp. 107–110, 1975.
29. J. S. Meditch, Stochastic Optimal Linear Estimation and Control. New York: McGraw-Hill, 1969.
30. M. C. Mozer, “Neural net architectures for temporal sequence processing,” in Predicting the Future and Understanding the Past, A. Weigend and N. Gershenfeld, Eds. Redwood City, CA: Addison-Wesley.
31. S. J. Nowlan, “Soft competitive adaptation: Neural network learning algorithm based on fitting statistical mixtures,” Tech. Report, CMU-CS-91-126, CMU, Pittsburgh.
32. D. C. Park, M. A. El-Sharkawi, and R. J. Marks, “An adaptively trained neural network,” IEEE Trans. on Neural Networks. vol. 2, no. 3, 1991.
33. F. J. Pineda, “Generalization of backpropagation to recurrent and higher order networks,” in Proc. IEEE Conf. Neural Inform. Proc. Syst. 1987.
34. A. J. Robinson and F. Fallside, “The utility driven dynamic error propagation network,” Tech Report CUED/F-1NFENG/TR.L Cambridge: Cambridge University, Department of Engineering.
35. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, vol. 1, D. E. Rumelhart, and J. L. McCelland, Eds. Cambridge, MA: MIT Press, pp. 318–362, 1986.
36. J. Schmidhuber, “A fixed size storage O(n3) time complexity learning algorithm for fully recurrent continually running networks,” Neural Computation, vol. 4, pp. 243–248, 1992.
37. T. Subba Rao, “On the theory of bilinear models,”.1. Roy. Statist. Soc. Ser. B, vol. 43, pp. 244–255, 1981.
38. H. Tong, Threshold Models in Non-linear Time Series Analysis. New York: Springer-Verlag, 1983.
39. A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. J. Lang, “Phoneme recognition using time delay neural networks,” IEEE Trans. on Acous. Speech and Sig. Proc., vol. 37, pp. 324–339, 1989.
40. P. J. Werbos, “Backpropagation through time: What it does and how to do it,” Proc. of the IEEE, pp. 1550–1560, vol. 78, no. 10, Oct. 1990.
41. R. Williams and D. Zipser, “A teaming algorithm for continually running fully recurrent neural networks,” Neural Computation, vol. 1, pp. 270–280, 1989.
42. Jen-Lun Yuan and T. L. Fine, “Forecasting demand for electric power,” Advances in Neural Information Processing Systems vol. 5. S. J. Hanson, J. D. Cowan, and C. L. Giles, Eds. San Matteo, CA: Morgan Kauffman, pp. 739–746, 1993.