Hierarchical mixtures-of-experts for exponential family regression models: Approximation and maximum likelihood estimation

Annals of Statistics

Volumn 27, Issue 3, 1999, Pages 987-1011

  Jiang, Wenxin ^a   Tanner, Martin A ^a  

^a NORTHWESTERN UNIVERSITY   (United States)

Author keywords

Approximation rate; Exponential family; Generalized linear models; Hellinger distance; Hierarchical mixtures of experts; Kullback leibler divergence; Maximum likelihood estimation; Mean square error

Indexed keywords

EID: 0033248628     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/aos/1018031265     Document Type: Article

References (25)

1. BICKEL, P. J. and DOKSUM, K. A. (1977). Mathematical Statistics. Prentice-Hall, Englewood Cliffs, NJ.
2. BISHOP, C.M. (1995). Neural Networks for Pattern Recognition. Oxford Univ. Press.
3. CACCIATORE, T. W. and NOWLAN, S. J. (1994). Mixtures of controllers for jump linear and non-linear plants. In Advances in Neural Informations Processing Systems 6 (G. Tesauro, D. S. Touretzky and T. K. Leen, eds.). Morgan Kaufmann, San Mateo, CA.
4. 1 View. Wiley, New York.
5. FRITSCH, J., FINKE, M. and WAIBEL, A. (1997). Adaptively growing hierarchical mixtures of experts. In Advances in Neural Informations Processing Systems 9 (M. C. Mozer, M. I. Jordan and T. Petsche, eds.). MIT Press.
6. GHAHRAMANI, Z. and HINTON, G. E. (1996). The EM algorithm for mixtures of factor analyzers. Technical Report CRG-TR-96-1, Dept. Computer Science, Univ. Toronto.
7. HAUSSLER, D. and OPPER, M. (1995). General bounds on the mutual information between a parameter and n conditionally independent observations. In Proceedings of the Eighth Annual Computational Learning Theory Conference (COLT), 1995, Santa Cruz, CA. ACM Press, New York.
8. HAYKIN, S. (1994). Neural Networks. Macmillan, New York.
9. JAAKKOLA, T. S. and JORDAN, M. I. (1998). Improving the mean field approximation via the use of mixture distributions. In Learning in Graphical Models (M. I. Jordan, ed.). Kluwer, Dordrecht.
10. JACOBS, R. A., JORDAN, M. I., NOWLAN, S. J. and HINTON, G. E. (1991). Adaptive mixtures of local experts. Neural Comp. 3 79-87.
11. JENNRICH, R. I. (1969). Asymptotic properties of nonlinear least squares estimators. Ann. Math. Statist. 40 633-643.
12. JIANG, W. and TANNER, M. A. (1998). Hierarchical mixtures-of-experts for exponential family regression models: approximation and maximum likelihood estimation. Technical report, Dept. Statistics, Northwestern Univ., Evanston, IL.
13. JIANG, W. and TANNER, M. A. (1999). On the approximation rate of hierarchical mixtures-of-experts for generalized linear models. Neural Comp. To appear.
14. JORDAN, M. I. and JACOBS, R. A. (1994). Hierarchical mixtures of experts and the EM algorithm. Neural Comp. 6 181-214.
15. JORDAN, M. I. and XU, L. (1995). Convergence results for the EM approach to mixtures-of-experts architectures. Neural Networks 8 1409-1431.
16. LEHMANN, E. L. (1991). Theory of Point Estimation. Wadsworth, Monterey, CA.
17. MCCULLAGH, P. and NELDER, J. A. (1989). Generalized Linear Models. Chapman and Hall, London.
18. MEILǍ, M. and JORDAN, M. I. (1995). Learning fine motion by Markov mixtures of experts. A.I. Memo 1567, Artificial Intelligence Lab., Massachusetts Institute Technology.
19. MHASKAR, H. N. (1996). Neural networks for optimal approximation of smooth and analytic functions. Neural Comp. 8 164-177.
20. PENG, F., JACOBS, R. A. and TANNER, M. A. (1996). Bayesian inference in mixtures-of-experts and hierarchical mixtures-of-experts models with an application to speech recognition. J. Amer. Statist. Assoc. 91 953-960.
21. TIPPING, M. E. and BISHOP, C. M. (1997). Mixtures of probabilistic principal component analysers. Technical Report NCRG-97-003, Dept. Computer Science and Applied Mathematics, Aston Univ., Birmingham, UK.
22. TITTERINGTON, D. M., SMITH, A. F. M. and MAKOV, U. E. (1985). Statistical Analysis of Finite Mixture Distributions. Wiley, New York.
23. WHITE, H. (1994). Estimation, Inference and Specification Analysis. Cambridge Univ. Press.
24. ZEEVI, A. and MEIR, R. (1997). Density estimation through convex combinations: approximation and estimation bounds. Neural Networks 10 99-106.
25. ZEEVI, A., MEIR, R. and MAIOROV, V. (1998). Error bounds for functional approximation and estimation using mixtures of experts. IEEE Trans. Information Theory 44 1010-1025.