Time series analysis and prediction using gated experts with application to energy demand forecasts

Applied Artificial Intelligence

Volumn 10, Issue 6, 1996, Pages 583-624

  Weigend, Andreas S ^a  

^a NEW YORK UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ELECTRIC LOAD FORECASTING; ENERGY UTILIZATION; MARKOV PROCESSES; PROBABILITY; TIME SERIES ANALYSIS;

NONLINEAR COMPETING EXPERTS; NONLINEAR GRATING MODELS;

EXPERT SYSTEMS;

EID: 0030291724     PISSN: 08839514     EISSN: 10876545     Source Type: Journal    
DOI: 10.1080/088395196118443     Document Type: Article

References (67)

1. Baldi, P., and Y. Chauvin. 1994. Smooth online learning algorithms for hidden Markov models. Neural Computation 6: 307-318.
2. Bengio, Y., and P. Frasconi. 1995. An input output HMM architecture. In Advances in neural information processing systems 7 (NIPS 94) eds, G. Tesauro, D. S. Touretzky, and T. K. Leen, 427-434. Cambridge, Mass.: MIT Press.
3. Bishop, C. M. 1994. Mixture density networks. Technical report. Aston University.
4. Bollerslev, T. 1986. Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics 21: 307-328.
5. Bollerslev, T., R. Y. Chou, N. Jayaraman, and K. F. Kroner. 1990. ARCH modeling in finance: A review of the theory and empirical evidence. Journal of Econometrics 52(1): 5-60.
6. Bridle, J. S. 1989. Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. In Neuro-computing: Algorithms, architectures and applications, eds, F. Fogelman-Soulie and J. Hérault, 227-236. New York: Springer-Verlag.
7. Broomhead, D. S., and D. Lowe. 1988. Multivariable functional interpolation and adaptive networks. Complex Systems 2: 321-355.
8. Buntine, W. L., and A. S. Weigend. 1991. Bayesian back-propagation. Complex Systems 5: 603-643.
9. Cacciatore, T. W., and S. J. Nowlan. 1994. Mixtures of controllers for jump linear and nonlinear plants. In Advances in neural information processing systems 6 (NIPS*93), eds, J. D. Cowen, G. Tesauro, and J. Alspector, 719-726. San Francisco, Calif.: Morgan Kaufmann.
10. Casdagli, M. 1989. Nonlinear prediction of chaotic time series. Physica D 35: 335-356.
11. Cottrell, M., B. Girard, Y. Girard, M. Mangeas, and C. Muller. 1995. Neural modeling for time series: A statistical stepwise method for weight elimination. IEEE Transactions on Neural Networks 6: 1355-1364.
12. Dempster, A. P., N. M. Laird, and D. B. Rubin. 1977. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B 39: 1-38.
13. Diebold, F. X., and G. D. Rudebusch. 1996. Measuring business cycles: A modern perspective. Review of Economics and Statistics.
14. Doutriaux, A., and D. Zipser. 1990. Unsupervised discovery of speech segments using recurrent networks. In Proceedings of the 1990 connectionist models summer school, eds, D. S. Touretzky, J. L. Elman, T. J. Sejnowski, and G. E. Hinton, 303-309, San Francisco, Calif.: Morgan Kaufmann.
15. Elman, J. L. 1990. Finding structure intime. Cognitive Science 14: 179-211.
16. Engle, R. F. 1982. Autoregressive conditional heteroskedasticity with estimates of the variance of United Kingdom inflation. Econometrica 50: 987-1007.
17. Finnoff, W., F. Hergert, and H. G. Zimmermann. 1993. Improving generalization performance by nonconvergent model selection methods. Neural Networks 6: 771-783.
18. Fraser, A. M., and A. Dimitriadis. 1994. Forecasting probability densities by using hidden Markov models. In Time series prediction: Forecasting the future and understanding the past, eds, A. S. Weigend and N. A. Gershenfeld, 265-282, Reading, Mass.: Addison-Wesley.
19. Friedman, J. H. 1991. Multivariate adaptive regression splines. Annals of Statistics 19: 1-142.
20. Granger, C. W. J. 1994. Forecasting in economics. In Time series prediction: Forecasting the future and understanding the past, eds, A. S. Weigend and N. A. Gershenfeld, 529-538, Reading, Mass.: Addison-Wesley.
21. Granger, C. W. J., and T. Terasvirta. 1993. Modelling nonlinear economic relationships. Oxford, UK: Oxford University Press.
22. Hamilton, J. D. 1989. A new approach to the economic analysisof nonstationary time series and the business cycle. Econometrica 57: 357-384.
23. Hamilton, J. D. 1990. Analysis of time series subject to changes in regime. Journal of Econometrics 45: 39-70.
24. Hamilton, J. D. 1994. Time series analysis. Princeton, N.J.: Princeton University Press.
25. Ivanova, T. O., V. V. Mottle, and I. B. Muchnik. 1994. Estimation of the parameters of hidden Markov models of noise-like signals with abruptly changing probabilistic properties. Automation and Remote Control 55: 1299-1315, 1428-1445.
26. Jacobs, R. A. 1995. Methods for combining experts’ probability assessments. Neural Computation 7: 867-888.
27. Jacobs, R. A., M. I. Jordan, S. J. Nowlan, and G. E. Hinton. 1991. Adaptive mixtures of local experts. Neural Computation 3: 79-87.
28. Johnson, N. L., and S. Kotz. 1972. Distributions in statistics. Continuous multivariate distributions. New York: Wiley.
29. Jordan, M. I., and R. A. Jacobs. 1994. Hierarchical mixtures of expertsand the EM algorithm. Neural Computation 6: 181-214.
30. Jordan, M. I., and L. Xu. 1995. Convergence results for the EM approach to mixtures of experts architectures. Neural Networks 8: 1409-1431.
31. Lapedes, A., and R. Farber. 1987. Nonlinear signal processing using neural networks. Technical Report LA-UR-87-2662, Los Alamos National Laboratory, Los Alamos, N. M.
32. Levin, E. 1991. Modeling time varying systems using hidden control neural architecture. In Advances in neural information processing systems 3 (NIPS*90), eds, R. P. Lippmann, J. E. Moody, and D. S. Touretzky, 147-154. San Francisco, Calif.: Morgan Kaufmann.
33. Lewis, P. A. W., B. K. Ray, and J. G. Stevens. 1994. Modeling time series using multivariate adaptive regression splines (MARS). In Time series prediction: Forecasting the future and understanding the past, eds, A. S. Weigend and N. A. Gershenfeld, 296-318. Reading, Mass.: Addison-Wesley.
34. Mangeas, M., C. Muller, and A. S. Weigend. 1995. Forecasting electricity demand using a mixture of nonlinear experts. In World congress on neural networks (WCNN’ 95), 1148-53.
35. McCullagh, P., and J. A. Nelder. 1989. Generalized linear models. London: Chapman and Hall.
36. Nadas, A., and R. L. Mercer. 1996. Hidden Markov models and some connections with artificial neural networks. In Mathematical Perspectives on Neural Networks, eds, P. Smolensky, M. C. Mozer, and D. E. Rumelhart, Hillsdale, N. J.: Lawrence Erlbaum Associates.
37. Nix, D. A., and A. S. Weigend. 1995. Learning local error bars for nonlinear regression. In Advances in neural information processing systems 7 (NIPS*94), eds, G. Tesauro, D. S. Touretzky, and T. K. Leen, 488-496. Cambridge, Mass.: MIT Press.
38. Nowlan, S. J. 1991. Soft competitive adaptation: Neural network learning algorithms based on fitting statistical mixtures. Technical Report CMU-CS-91-126, School of Computer Science, Carnegie Mellon University, Pittsburgh, Pa.
39. Nowlan, S. J., and G. E. Hinton. 1992. Simplifying neural networks by soft weight-sharing. Neural Computation 4: 473-493.
40. Pawelzik, K., J. Kohlmorgen, and K.-R. Muller. 1996. Annealed competition of experts for a segmentation and classification of switching dynamics. Neural Computation 8: 340-356.
41. Pearson, K. 1894. Contributions to the mathematical theory of evolution. Philosophical Transitions of the Royal Society 185: 71-110. (Seealso 185A: 195.)
42. Perrone, M. P. 1994. General averaging results for complex optimization. In Proceedings of the 1993 connectionist models summer school, eds, M. C. Mozer, P. Smolensky, D. S. Touretzky, J. L. Elman, and A. S. Weigend, 364-371. Hillsdale, N. J.: Lawrence Erlbaum Associates.
43. Poggio, T., and F. Girosi. 1990. Networks for approximation and learning. Proceedings of the IEEE 78(9): 1481-1497.
44. Poritz, A. B. 1988. Hidden Markov models: A guided tour. In IEEE international conference on acoustics, speech and signal processing, 7-13.
45. Predoviciu, M. M., and M. I. Jordan. 1995. Learning the parameters of HMMs with auxilliary input. Technical report, Massachusetts Institute of Technology, Cambridge, Mass.
46. Press, W. H., B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling. 1992. Numerical recipes in C: The art of scientific computing. Cambridge: Cambridge University Press.
47. Quandt, R. E. 1958. The estimation of parameters of linear regression system obeying two separate regimes. Journal of the American Statistical Association 55: 873-880.
48. Rabiner, L. R. 1989. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77 (2): 257-286.
49. Redner, R. A., and H. F. Walker. 1984. Mixture densities, maximum likelihood and the EM algorithm.SIAM Review 26: 195-237.
50. Rose, K., E. Gurewitz, and G. C. Fox. 1990. Statistical mechanics and phase transitions in clustering. Physical Review Letters 65: 945-948.
51. Rumelhart, D. E., R. Durbin, R. Golden, and Y. Chauvin. 1995. Backpropagation: The basic theory. In Back-propagation: Theory, architecture, and applications, eds, Y. Chauvin, and D. E. Rumelhart, 1-34. Hillsdale, N. J.: Lawrence Erlbaum Associates.
52. Rumelhart, D. E., J. L. McClelland, and P. R. Group. 1986. Parallel distributed processing: Exploration in the micro structure of cognition. Volume I: Cognition. Cambridge, Mass.: MIT Press.
53. Seber, G. A. F., and C. J. Wild. 1989. Nonlinear regression. New York: Wiley.
54. Shi, S., and A. S. Weigend. 1997. Hidden Markov experts. Technical report. Stern School of Business, New York University, New York, N.Y.
55. Tanner, M. A. 1993. Tools for statistical inference, 2nd ed. New York: Springer-Verlag.
56. Titterington, D., A. F. M. Smith, and U. E. Makov. 1985. Statistical analysis of finite mixture distributions. New York: John Wiley.
57. Tong, H., and K. S. Lim. 1980. Threshold autoregression, limit cycles and cyclical data. Journal of the Royal Statistical Society B 42: 245-292.
58. Waterhouse, S. R., D. J. C. MacKay, and A. J. Robinson. 1996. Bayesian models for mixtures of experts. In Advances in neural information processing systems 8 (N1PS*95). Cambridge, Mass.: MIT Press.
59. Waterhouse, S. R., and A. J. Robinson. 1995. Non-linear prediction of acoustic vectors using hierarchical mixture of experts. In Advances in neural information processing systems 7 (NIPS*94), eds, G. Tesauro, D. S. Touretzky, and T. K. Leen, 835-842. Cambridge, Mass.: MIT Press.
60. Weigend, A. S., and N. A. Gershenfeld. 1994. Time series prediction: Forecasting the future and understanding the past. eds, A. S. Weigend and N. A. Gershenfeld, Reading, Mass.: Addison-Wesley.
61. Weigend, A. S., B. A. Huberman, and D. E. Rumelhart. 1990. Predicting the future: A connectionist approach. International Journal of Neural Systems 1: 193-209.
62. Weigend, A. S., and B. LeBaron. 1994. Evaluating neural network predictors by bootstrapping. In Proceedings of international conference on neural information processing (ICONIP'94), 1207-1212.
63. Weigend, A. S., M. Mangeas, and A. N. Srivastava. 1995. Nonlinear gated experts for time series: Discovering regimes and avoiding overfitting. International Journal of Neural Systems 6: 373-399.
64. Weigend, A. S., and D. A. Nix. 1994. Predictions with confidence intervals (local error bars). In Proceedings of the international conference on neural information processing (ICONIP'94), 1207-1212.
65. Weigend, A. S., and A. N. Srivastava. 1995. Predicting conditional probability distributions: A connectionist approach. International Journal of Neural Systems 6: 109-118.
66. Xu, L. 1994. Signal segmentation by finite mixture model and EM algorithm. In Proceedings of the 1994 international symposium on artificial neural networks (ISANN’94), 453-458.
67. Yule, G. 1927. On a method of investigating periodicity in disturbed series with special reference to Wolfer's sunspot numbers. Philosophical Transactions of the Royal Society of London A 226: 267-298.