Mixtures-of-experts of autoregressive time series: Asymptotic normality and model specification

IEEE Transactions on Neural Networks

Volumn 16, Issue 1, 2005, Pages 39-56

  Carvalho, Alexandre X ^a,b   Tanner, Martin A ^c  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^b Inst of Applied Economic Research   (Brazil)

^c NORTHWESTERN UNIVERSITY   (United States)

Author keywords

Asymptotic properties; Maximum likelihood estimation; Mixture of experts (ME); Nonlinear time series

Indexed keywords

APPROXIMATION THEORY; ASYMPTOTIC STABILITY; COMPUTER SIMULATION; FUNCTIONS; MATHEMATICAL MODELS; MAXIMUM LIKELIHOOD ESTIMATION; MONTE CARLO METHODS; NONLINEAR EQUATIONS; REGRESSION ANALYSIS; SYSTEM STABILITY; TIME SERIES ANALYSIS;

ASYMPTOTIC NORMALITY; ASYMPTOTIC PROPERTIES; AUTOREGRESSIVE TIME SERIES; MIXTURE-OF-EXPERT (ME); MODEL SELECTION CRITERIA; NONLINEAR TIME SERIES;

EXPERT SYSTEMS;

ALGORITHM; ARTICLE; CLUSTER ANALYSIS; COMPUTER ANALYSIS; EVALUATION; EXPERT SYSTEM; MATHEMATICAL COMPUTING; NONLINEAR SYSTEM; STATISTICAL MODEL; TIME;

ALGORITHMS; CLUSTER ANALYSIS; COMPUTING METHODOLOGIES; EXPERT SYSTEMS; MODELS, STATISTICAL; NONLINEAR DYNAMICS; NUMERICAL ANALYSIS, COMPUTER-ASSISTED; TIME FACTORS;

EID: 13844296867     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2004.839356     Document Type: Article

References (47)

1. H. Akaike, "Likelihood of a model and information criteria," in Proc. 2nd Int. Symp. Information Theory, 1973, pp. 267-281.
2. H. Akaike, "Information theory and the extension of the maximum likelihood principle," J. Econometrics, vol. 16, pp. 3-14, 1981.
3. T. Amemiya, Advanced Econometrics. Cambridge, MA: Havard Univ. Press, 1985.
4. J. Berkowitz, "Testing density forecasts, with applications to risk management," Grad. School Management, Univ. California, Irvine, Tech. Rep., 2000.
5. H. Bierens, Topics in Advanced Econometrics: Estimation, Testing and Specification of Cross-Section and Time Series Models. Cambridge, U.K.: Cambridge Univ. Press, 1996.
6. P. Brockwell and R. Davis, Introduction to Time Series and Forecasting. New York: Springer-Verlag, 1996.
7. A. Carvalho, "Mixtures-of-experts of generalized linear time series," Ph.D., Dept. Statist., Northwestern Univ., Evanston, IL, 2002.
8. A. Carvalho and G. Skoulakis, "Ergodicity and existence of moments for mixtures of linear autoregressions," Dept. Statist., Northwestern Univ., Evanston, IL, 2003.
9. A. Carvalho and M. Tanner, "Hypothesis testing in mixtures-of-experts of generalized linear time series," in Proc. IEEE Conf. Computational Intelligence in Financial Engineering, Hong Kong, 2003, pp. 285-292.
10. H. Chipman, E. George, and R. McCulloch, "Bayesian treed models," Mach. Learn., vol. 48, pp. 299-320, 2002.
11. J. Davidson, Stochastic Limit Theory - An Introduction to Econometricians. London, U.K.: Oxford Univ. Press, 1994.
12. F. Diebold, T. Gunther, and A. Tay, "Evaluating density forecasts with applications to financial risk management," Int. Economic Rev., vol. 39, pp. 863-882, 1998.
13. R. Durret, Probability: Theory and Examples. New York: Duxbury Press, 1996.
14. Z. Ghahramani and G. Hinton, "The em algorithm for mixtures of factor analyzers," Dept. Comput. Sci., Univ. Toronto, Toronto, ON, Canada, Tech. Rep., 1996.
15. S. Gutta, J. Huang, P. Jonathon, and H. Wechsler, "Mixture of experts for classification of gender, ethnic origin, and pose of human faces,"IEEE Trans. Neural Netw., vol. 11, no. 4, pp. 948-960, Jul. 2000.
16. J. Hamilton, Time Series Analysis. Princeton, NJ: Princeton Univ. Press, 1994.
17. A. Harvey, Forecasting, Structural Time Series Models and the Kalman Filter. Cambridge, U.K.: Cambridge Univ. Press, 1994.
18. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference and Prediction. New York: Springer-Verlag, 2001.
19. G. Huerta, W. Jiang, and M. Tanner, "Mixtures of time series models," J. Computat. Graph. Statist., vol. 10, no. 1, pp. 82-89, 2001.
20. R. Jacobs, M. Jordan, S. Nowlan, and G. Hinton, "Adaptative mixtures of local experts," Neural Computat., vol. 3, pp. 79-87, 1991.
21. N. Jeffries and R. Pfeiffer, "A mixture model for the probability distribution of rain rate," Environmetrics, vol. 12, pp. 1-10, 2001.
22. W. Jiang and M. Tanner, "Hierarchical mixtures-of-experts for exponential family regression models: Approximation and maximum likelihood estimation," Ann. Statist., vol. 27, no. 3, pp. 987-1011, 1999.
23. W. Jiang and M. Tanner, "On the approximation rate of hierarchical mixtures-of-experts for generalized linear models," Neural Computat. vol. 11, pp. 1183-1198, 1999.
24. W. Jiang and M. Tanner, "On the identifiability of mixtures-of-experts," Neural Netw., vol. 12, pp. 1253-1258, 1999.
25. W. Jiang and M. Tanner, "On the asymptotic normality of hierarchical mixtures-of-experts for generalized linear models," IEEE Trans. Inf. Theory, vol. 46, no. 3, pp. 1005-1013, May 2000.
26. M. Jordan and R. Jacobs, "Hierarchical mixtures-of-experts and the em algorithm," Neural Computat., vol. 6, pp. 181-214, 1994.
27. L. Kullback and R. Leibler, "On information and sufficiency," Ann. Math. Statist., vol. 22, pp. 79-86, 1951.
28. R. Kurnik, J. Oliver, S. Waterhouse, T. Dunn, Y. Jayalaksmi, M. Lesho, M. Lopatina, J. Tamada, and C. Wei, "Application of mixture of experts algorithm for signal processing in a noninvasive glucose monitoring system," Sens. Actuators B, Chem., vol. 60, pp. 19-26, 1999.
29. H. Lin, C. McCulloch, B. Turnbull, E. Slate, and L. Clark, "A latent class mixed model for analysing biomarker trajectories with irregularly scheduled observations," Statist. Medicine, vol. 19, pp. 1303-1318, 2000.
30. M. McCullagh and J. Nelder, Generalized Linear Models (Monographs on Statistics and Applied Probability, 37). London, U.K.: Chapman & Hall, 1998.
31. S. Meyn and R. Tweedie, Markov Chains and Stochastic Stability. New York: Springer-Verlag, 1993.
32. W. Newey, "Uniform convergence in probability and stochastic equicontinuity," Econometrica, vol. 59, pp. 1161-1167, 1991.
33. W. Newey and K. West, "A simple positive-semi-definite, heterokedasticity and autocorrelation consistent covariance matrix," Econometrica, vol. 55, pp. 703-708, 1987.
34. B. Quinn, G. McLachlan, and H. Hjort, "A note on the Aitkin-Rubin approach to hypothesis testing in mixture models," J. Roy. Statist. Soc. B, vol. 39, pp. 311-314, 1987.
35. M. Rosenblatt, "Remarks on a multivariate transformation," Ann. Math. Statist., vol. 23, pp. 470-472, 1952.
36. G. Schwarz, "Estimating the dimension of a model," Ann. Statist. vol. 6, pp. 461-464, 1978.
37. M. Tipping and C. Bishop, "Mixtures of probabilistic principal component analyzers," Neural Computat., vol. 11, pp. 443-482, 1999.
38. H. Tong, Threshold Models in Non-Linear Time Series Analysis. New York: Springer-Verlag, 1983.
39. A. Weigend, M. Mangeas, and A. Srivastava, "Nonlinear gated experts for time series: Discovering regimes and avoid overfitting," Int. J. Neural Syst., vol. 6, pp. 373-399, 1995.
40. H. White, Estimation Inference and Specification Analysis, Econometric Society Monographs. Cambridge, U.K.: Cambridge Univ. Press, 1996.
41. H. White, Asymptotic Theory for Econometricians. New York: Academic, 2001.
42. C. Wong and W. Li, "On a logistic mixture autoregressive model," Biometrika, vol. 88, no. 3, pp. 833-846, 2001.
43. W. Wong, "Theory of partial likelihood," Ann. Statist., vol. 14, no. 1, pp. 88-123, 1986.
44. S. Wood, W. Jiang, and M. Tanner, "Bayesian mixture of splines for spatially adaptive nonparametric regression," Biometrika, vol. 89, no. 3, pp. 513-528, 2001.
45. J. Wooldridge, Econometric Analysis of Cross Section and Panel Data Cambridge, MA: MIT Press, 2002.
46. A. Zeevi and R. Meir, "Density estimation through convex combinations of densities: Approximation and estimation bounds," Neural Netw., vol. 10, no. 1, pp. 99-109, 1996.
47. M. Zeevi, R. Meir, and R. Adler, (1999) Nonlinear models for time series using mixtures of autorregressive models. [Online]. Available: Tech. Rep. Available http://www.isl.stanford.edu/azeevi/.