On the asymptotic normality of hierarchical mixtures-of-experts for generalized linear models

IEEE Transactions on Information Theory

Volumn 46, Issue 3, 2000, Pages 1005-1013

  Jiang, Wenxin ^a   Tanner, Martin A ^a  

^a NORTHWESTERN UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ASYMPTOTIC STABILITY; FUNCTIONS; MATHEMATICAL MODELS; MAXIMUM LIKELIHOOD ESTIMATION; POISSON DISTRIBUTION; SET THEORY;

ASYMPTOTIC NORMAL DISTRIBUTION; FISHER INFORMATION; GATING FUNCTIONS; GENERALIZED LINEAR MODELS; HIERARCHICAL MIXTURES OF EXPERTS;

INFORMATION THEORY;

EID: 0034188427     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.841177     Document Type: Article

References (26)

1. S. Amari and N. Murata, "Statistical theory of learning curves under entropie loss criterion," Neural Comput., vol. 5, pp. 140-153, 1993.
2. C.M. Bishop, Neural Networks for Pattern Recognition. New York: Oxford Univ. Press, 1995.
3. T.W. Cacciatore and S. J. Nowlan, "Mixtures of controllers for jump linear and nonlinear plants," in Advances in Neural Informations Processing Systems 6, G. Tesauro, D. S. Touretzky, and T. K. Leen, Eds. San Mateo, CA: Morgan Kaufmann, 1994.
4. Z. Ghahramani and G. E. Hinton, "The EM Algorithm for Mixtures of Factor Analyzers," Dep. Comput. Sei., Univ. Toronto, Toronto, Ont., Canada, Tech. Rep. CRG-TR-96-1, 1996.
5. S. Haykin, Neural Networks. New York: Macmillan College Pub., 1994.
6. J.T. G. Hwang and A. A. Ding, "Prediction intervals for artificial neural networks," J. Amer. Statist. Assoc., vol. 92, pp. 748-757, 1997.
7. T.S. Jaakkola and M. I. Jordan, "Improving the mean field approximation via the use of mixture distributions," in Learning in Graphical Models, M. I. Jordan, Ed. Norwell, MA: Kluwer , 1998.
8. R.A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, "Adaptive mixtures of local experts," Neural Comp., vol. 3, pp. 79-87, 1991.
9. W. Jiang and M. A. Tanner, "On the approximation rate of hierarchical mixtures-of-experts for generalized linear models," Neural Comp., vol. 11, pp. 1183-1198, 1999.
10. "Hierarchical mixtures-of-experts for exponential family regression models: Approximation and maximum likelihood estimation," Ann. Statist., vol. 27, pp. 987-1011, 1999.
11. "On the Asymptotic Normality of Hierarchical Mixtures-of-Experts for Generalized Linear Models," Dep. Statist., Northwestern Univ., Evanston, IL, Tech. Rep., 1999.
12. M.I. Jordan and R. A. Jacobs, "Hierarchical mixtures of experts and the EM algorithm," Neural Comp., vol. 6, pp. 181-214, 1994.
13. M.I. Jordan and L. Xu, "Convergence results for the EM approach to mixtures-of-experts architectures," Neural Networks, vol. 8, pp. 1409-1431, 1995.
14. E.L. Lehmann, Theory of Point Estimation. Monterey, CA: Wads worth, 1991.
15. P. McCullagh and J. A. Neider, Generalized Linear Models. London, U.K.: Chapman and Hall, 1989.
16. M. Meilä and M. I. Jordan, "Learning Fine Motion by Markov Mixtures of Experts," Artificial Intelligence Lab., MIT, Cambridge, MA, A.I. Memo 1567, 1995.
17. F. Peng, R. A. Jacobs, and M. A. Tanner, "Bayesian inference in mixtures-of-experts and hierarchical mixtures-of-experts models with an application to speech recognition," J. Amer. Statist. Assoc., vol. 91, pp. 953-960, 1996.
18. B.C. Peters and H. F Walker, "An iterative procedure for obtaining maximum-likelihood estimates of the parameters for a mixture of normal distributions," S1AM J. Appl. Math., vol. 35, pp. 362-378, 1978.
19. R.A. Redner and H. F. Walker, "Mixture densities, maximum likelihood and the EM algorithm," SIAM Rev., vol. 26, pp. 195-202, 1984.
20. P.K. Sen and J. M. Singer, Large Sample Methods in Statistics: An Introduction with Applications. New York: Chapman and Hall, 1993.
21. R.J. Serfling, Approximation Theorems of Mathematical Statistics. New York: Wiley, 1980.
22. M.E. Tipping and C. M. Bishop, "Mixtures of Probabilistic Principal Component Analyzers," Dep. Comput. Sei. Appl. Math., Aston Univ., Birmingham, U.K., Tech. Rep. NCRG-97-003, 1997.
23. H. White, "Learning in artificial neural networks: A statistical perspective," Neural Computation, vol. 1, pp. 425-4164, 1989a.
24. "Some asymptotic results for learning in single hidden-layer feed forward network models," J. Amer. Statist. Assoc., vol. 84, pp. 1003-1013, 1989b.
25. Estimation, Inference and Specification Analysis, Cambridge, U.K.: Cambridge Univ. Press, 1994.
26. A. Zeevi, R. Meir, and V. Maiorov, "Error bounds for functional approximation and estimation using mixtures of experts," IEEE Trans. Inform. Theory, vol. 44, pp. 1010-1025, May 1998.