Extending the akaike information criterion to mixture regression models

Journal of the American Statistical Association

Volumn 102, Issue 477, 2007, Pages 244-254

  Naik, Prasad A ^a   Shi, Peide ^b   Tsai, Chih Ling ^a,c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b Nuclear Safety Solutions Limited   (Canada)

^c Peking University   (China)

Author keywords

Akaike information criterion; Cluster analysis; EM algorithm; Mixture models; Model selection; Variable selection

Indexed keywords

EID: 33947277966     PISSN: 01621459     EISSN: None     Source Type: Journal    
DOI: 10.1198/016214506000000861     Document Type: Article

References (58)

1. Akaike, H. (1973), "Information Theory and an Extension of the Maximum Likelihood Principle," in The 2nd International Symposium on Information Theory, eds. B. N. Petrov and F. Csaki, Budapest: Akademia Kiado, pp. 267-281.
2. _ (1985), "Prediction and Entropy," in A Celebration of Statistics, eds. A. C. Atkinson and S. E. Fienberg, New York: Springer-Verlag, pp. 1-24.
3. Altham, F. M. E. (1984), "Improving the Precision of Estimation by Fitting a Model," Journal of the Royal Statistical Society, Ser. B, 46, 118-119.
4. Biernacki, C., Celeux, G., and Govaert, G. (2000), "Assessing a Mixture Model for Clustering With the Integrated Completed Likelihood," IEEE Transactions on Pattern Analysis and Machine Intelligence, 22, 719-725.
5. Brockwell, P. J., and Davis, R. A. (1991), Time Series: Theory and Methods (2nd ed.), New York: Springer-Verlag.
6. Brooks, S. P., Friel, N., and King, R. (2003), "Classical Model Selection via Simulated Annealing," Journal of the Royal Statistical Society, Ser. B, 65, 503-520.
7. Bryant, P. G., and Williamson, J. A. (1978), "Asymptotic Behaviour of Classification Maximum Likelihood Estimates," Biometrika, 65, 273-281.
8. Burnham, K. P. (2002), Discussion of "Bayesian Measures of Model Complexity and Fit," by D. J. Spieglehalter, N. G. Best, B. P. Carlin, and A. van der Linde, Journal of the Royal Statistical Society, Ser. B, 64, 583-639.
9. Burnham, K. P., and Anderson, D. R. (2002), Model Selection and Inference: A Practical Information-Theoretic Approach (2nd ed.), New York: Springer-Verlag.
10. _ (2004), "Multimodel Inference: Understanding AIC and BIC in Model Selection," Sociological Methods & Research, 33, 261-304.
11. Cai, Z., Naik, P. A., and Tsai, C. L. (2000), "Denoised Least Squares Estimators: An Application to Estimating Advertising Effectiveness," Statistica Sinica, 10, 1231-1241.
12. Chen, H., Chen, J., and Kalbfleisch, J. D. (2004), "Testing for a Finite Mixture Model With Two Components," Journal of the Royal Statistical Society, Ser. B, 66, 95-115.
13. Chen, K., and Jin, Z. (2006), "Partial Linear Regression Models for Clustered Data," Journal of the American Statistical Association, 101, 195-204.
14. Cleveland, W. S., and Devlin, S. J. (1988), "Locally Weighted Regression: An Approach to Regression Analysis by Local Fitting," Journal of the American Statistical Association, 83, 596-610.
15. Cochran, W. G. (1934), "The Distribution of Quadratic Forms in a Normal System, With Applications to the Analysis of Covariance," Proceedings of the Cambridge Philosophical Society, 30, 178-191.
16. Cravens, D. W., Woodruff, R. B., and Stamper, J. C. (1972), "An Analytical Approach for Evaluating Sales Territory Performance," Journal of Marketing, 36, 31-37.
17. Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977), "Maximum Likelihood From Incomplete Data via the EM Algorithm" (with discussion), Journal of the Royal Statistical Society, Ser. B, 39, 1-38.
18. DeSarbo, W. S., and Corn, W. L. (1988), "A Maximum Likelihood Methodology for Clusterwise Linear Regression," Journal of Classification, 5, 249-282.
19. Dielman, T. E. (2001), Applied Regression Analysis for Business and Economics (3rd ed.), New York: Duxbury.
20. Gelman, A., Carlin, J. B., Stern, H. S., and Rubin, D. B. (2004), Bayesian Data Analysis (2nd ed.), New York: Chapman & Hall/CRC.
21. Green, P. J. (1995), "Reversible-Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination," Biometrika, 82, 711-732.
22. Hartigan, J. A. (1975), Clustering Algorithms, New York: Wiley.
23. Hastie, T., Tibshirani, R., and Friedman, J. (2001), The Elements of Statistical Learning: Data Mining, Inference, and Prediction, New York: Springer-Verlag.
24. Hathaway, R. J. (1985), "A Constraint Formulation of Maximum-Likelihood Estimation for Normal Mixture Distributions," The Annals of Statistics, 13, 795-800.
25. Heiland, I. S., and Almøy, T. (1994), "Comparison of Prediction Methods When Only a Few Components Are Relevant," Journal of the American Statistical Association, 89, 583-591.
26. Hjort, N., and Claskens, G. (2003), "Frequentist Model Average Estimators" (with discussion), Journal of the American Statistical Association, 98, 879-899.
27. Hurvich, C. M., and Tsai, C. L. (1989), "Regression and Time Series Model Selection in Small Samples," Biometrika, 76, 297-307.
28. _ (1994), "Model Selection for Extended Quasi-Likelihood Models in Small Samples," Biometrics, 51, 1077-1084.
29. Jorgensen, B. (1987), "Exponential Dispersion Models" (with discussion), Journal of the Royal Statistical Society, Ser. B, 49, 127-162.
30. Le, N. D., Martin, R. D., and Raftery, A. E. (1996), "Modeling Flat Stretches, Bursts, and Outliers in Time Series Using Mixture Transition Distribution Models," Journal of the American Statistical Association, 91, 1504-1514.
31. Leroux, B. (1992), "Consistent Estimation of a Mixing Distribution." The Annals of Statistics, 20, 1350-1360.
32. Loader, C. (1999), Local Regression and Likelihood, New York: Springer-Verlag.
33. MacQueen, J. (1967), "Some Methods for Classification and Analysis of Multivariate Observations," in Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, 1, Berkeley. CA: University of California Press, pp. 281-297.
34. McCullagh, P., and Neider, J. A. (1989), Generalized Linear Models (2nd ed.), London: Chapman & Hall.
35. McLachlan, J. G. (1987). "On Bootstrapping the Likelihood Ratio Test Statistics for the Number of Components in a Normal Mixture," Applied Statistics. 36, 318-324.
36. McLachlan. J. G., and Peel, D. (2000). Finite Mixture Models, New York: Wiley.
37. McQuarrie, A. D. R., and Tsai, C. L. (1998). Regression and Time Series Model Selection, Singapore: World Scientific Publishing.
38. Naik, P. A., and Tsai, C. L. (2000). "Partial Least Squares Estimator for Single-Index Models," Journal of the Royal Statistical Society, Ser. B, 62, 763-771.
39. _ (2001), "Single-Index Model Selections," Biometrika, 88, 821-832.
40. Nelder, J. A., and Pregibon, D. (1987), "An Extended Quasi-Likelihood Function," Biometrika, 74, 221-232.
41. Raftery, A. E., and Dean, N. (2006), "Variable Selection for Model-Based Clustering," Journal of the American Statistical Association, 101, 168-179.
42. Redner, R. A., and Walker, H. F. (1984), "Mixture Densities. Maximum Likelihood and the EM Algorithm," SIAM Review, 26, 195-239.
43. Richardson, S. (2002), Discussion of "Bayesian Measures of Model Complexity and Fit," by D. J. Spieglehalter, N. G. Best, B. P. Carlin, and A. van der Linde, Journal of the Royal Statistical Society, Ser. B, 64, 583-639.
44. Satterthwaite, F. E. (1946). "An Approximate Distribution of Estimates of Variance Components." Biometrics Bulletin, 2, 110-114.
45. Schwarz, G. (1978), "Estimating the Dimension of a Model," The Annals of Statistics, 6, 461-464.
46. Shao, J. (1997), "An Asymptotic Theory for Linear Model Selection" (with discussion), Statistica Sinica, 7, 221-264.
47. Shi, P., and Tsai, C. L. (1999). "Semiparametric Regression Model Selections." Journal of Statistical Planning and Inference, 77, 119-139.
48. _ (2002). "Regression Model Selection: A Residual Likelihood Approach," Journal of the Royal Statistical Society. Ser. B, 64, 237-252.
49. Shibata, G. (1981), "An Optimal Selection of Regression Variables," Biometrika, 68, 45-54.
50. _ (1984), "Approximate Efficiency of a Selection Procedure for the Number of Regression Variables," Biometrika, 71, 43-49.
51. Spieglehalter, D. J., Best, N. G., Carlin, B. P., and van der Linde, A. (2002). "Bayesian Measures of Model Complexity and Fit" (with discussion), Journal of the Royal Statistical Society. Ser. B, 64, 583-639.
52. Tadesse, M. G., Sha, N., and Vannucci, M. (2005). "Bayesian Variable Selection in Clustering High-Dimensional Data." Journal of the American Statistical Association, 100, 602-617.
53. Tibshirani, R., Walther, G., and Hastie, T. (2001), "Estimating the Number of Clusters in a Data Set via the Gap Statistic." Journal of the Royal Statistical Society, Ser. B, 63, 411-423.
54. Titterington, D. M., Smith, A. F. M., and Makov, U. E. (1985), Statistical Analysis of Finite Mixture Distributions. New York: Wiley.
55. Wedel, M., and Kamakura, W. (2000). Market Segmentation: Conceptual and Methodological Foundations (2nd ed.), Boston: Kluwer Academic.
56. Wong, C. S., and Li, W. K. (2000), "On a Mixture Autoregressive Model." Journal of the Royal Statistical Society, Ser. B, 62, 95-115.
57. Wu, C. F. J. (1983), "On the Convergence Properties of the EM Algorithm." The Annals of Statistics, 11, 95-103.
58. Zhu, H. T., and Zhang, H. (2004), "Hypothesis Testing in Mixture Regression Models," Journal of the Royal Statistical Society, Ser. B, 66, 3-16.