A note on density model size testing

IEEE Transactions on Information Theory

Volumn 50, Issue 3, 2004, Pages 576-581

  Biau, Gérard ^a   Devroye, Luc ^b  

^a SORBONNE UNIVERSITÉ   (France)

^b MCGILL UNIVERSITY   (Canada)

Author keywords

Hypothesis testing; Mixture densities; Nonparametric estimation; Penalization; Vapnik Chervonenkis dimension

Indexed keywords

COMPUTATIONAL COMPLEXITY; MATHEMATICAL MODELS; PARAMETER ESTIMATION; PROBABILITY DENSITY FUNCTION; RANDOM PROCESSES; THEOREM PROVING;

HYPOTHESIS TESTING; MIXTURE DENSITIES; NONPARAMETRIC ESTIMATION; PENALIZATION; VAPNIK-CHERVONENKIS DIMENSION;

INFORMATION THEORY;

EID: 1842429140     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2004.825250     Document Type: Letter

References (38)

1. L. Devroye and G. Lugosi, Combinatorial Methods in Density Estimation. New York: Springer-Verlag, 2001.
2. N. Dunford and J. T. Schwartz, Linear Operators Part I. New York: Wiley, 1963.
3. L. F. James, C. E. Priebe, and D. J. Marchette, "Consistent estimation of mixture complexity," Ann. Statist., vol. 29, pp. 1281-1296, 2001.
4. G. J. McLachlan, "On bootstrapping the likelihood ratio test statistic for the number of components in a normal mixture," J. Appl. Statist., vol. 36, pp. 318-324, 1987.
5. D. Dacunha-Castelle and E. Gassiat, "Testing in locally conic models, and application to mixture models," ESAIM Probab. Statist., vol. 1, pp. 285-317, 1997.
6. ____, "Testing the order of a model using locally conic parametrization: Population mixtures and stationary arma processes," Ann. Statist., vol. 27, pp. 1178-1209, 1999.
7. G. Biau and L. Devroye, "Density estimation by the penalized combinatorial method," McGill University, Montreal, QC, Canada, Tech. Rep., 2002.
8. L. Devroye, L. Györfi, and G. Lugosi, "A note on robust hypothesis testing," IEEE Trans. Inform. Theory, vol. 48, pp. 2111-2114, July 2002.
9. V. N. Vapnik and A. Y. Chervonenkis, "On the uniform convergence of relative frequencies of events to their probabilities," Theory Probab. Applic., vol. 16, pp. 264-280, 1971.
10. C. McDiarmid, "On the method of bounded differences," in Surveys in Combinatorics 1989. Cambridge, U.K.: Cambridge Univ. Press, 1989.
11. M. Talagrand, "Sharper bounds for Gaussian and empirical processes," Ann. Probab., vol. 22, pp. 28-76, 1994.
12. P. Billingsley, Probability and Measure, 3rd ed. New York: Wiley, 1995.
13. B. S. Everitt and D. J. Hand, Finite Mixture Distributions. London, U.K.: Chapman and Hall, 1981.
14. D. M. Titterington, A. F. M. Smith, and U. E. Makov, Statistical Analysis of Finite Mixture Distributions. Chichester, U.K.: Wiley, 1985.
15. G. J. McLachlan and K. E. Basford, Mixture Models: Inference and Applications to Clustering. New York: Marcel Dekker, 1988.
16. G. J. McLachlan and D. Peel, Finite Mixture Models. New York: Wiley, 2000.
17. R. A. Redner and H. F. Walker, "Mixture densities, maximum likelihood and the EM algorithm," SIAM Rev., vol. 26, pp. 195-239, 1984.
18. J. Diebolt and C. P. Robert, "Estimation of finite mixture distributions through bayesian sampling," J. Roy. Stat. Soc. Ser. B Stat. Methodol., vol. 56, pp. 363-375, 1994.
19. S. Richardson and P. J. Green, "On Bayesian analysis of mixtures with an unknown number of components," J. Roy. Stat. Soc. Ser. B Stat. Methodol., vol. 59, pp. 731-792, 1997.
20. K. Roeder and L. Wasserman, "Practical Bayesian density estimation using mixtures of normals," J. Amer. Statist. Assoc., vol. 92, pp. 894-902, 1997.
21. G. Celeux, M. Hurn, and C. P. Robert, "Computational and inferential difficulties with mixture posterior distributions," J. Amer. Statist. Assoc., vol. 95, pp. 957-970, 2000.
22. M. Hurn, A. Justel, and C. P. Robert, "Estimating mixtures of regressions," J. Comput. Graph. Statist., vol. 12, pp. 1-25, 2003.
23. C. L. Bishop, "Mixture density networks," Dept. Comp. Sci. Applied Math., Aston Univ., Birmingham, U.K., Neural Computing Research Group Rep. NCRG/94/004, 1994.
24. M. I. Jordan and R. A. Jacobs, "Hierarchical mixtures of experts and the EM algorithm," Neural Comput., vol. 6, pp. 181-214, 1994.
25. A. Zeevi and R. Meir, "Density estimation through convex combinations of densities; approximation and estimation bounds," Neural Networks, vol. 10, pp. 90-109, 1997.
26. M. A. T. Figueiredo and A. K. Jain, "Unsupervised learning of finite mixture models," IEEE Trans. Pattern Anal. Machine Intell., vol. 24, pp. 381-396, Mar. 2002.
27. J. Hartigan, "A failure of likelihood asymptotics for normal mixtures," in Proc. Berkeley Conf. Honor of Jerzy Neyman and Jack Kiefer, vol. II, 1985, pp. 807-810.
28. K. Fukumizu, "Likelihood ratio of unidentifiable models and multilayer neural networks," Ann. Statist., vol. 31, pp. 833-851, 2003.
29. D. Dacunha-Castelle and E. Gassiat, "The estimation of the order of a mixture model," Bernoulli, vol. 3, pp. 279-299, 1997.
30. C. E. Priebe, "Adaptive mixtures," J. Amer. Statist. Assoc., vol. 89, pp. 796-806, 1994.
31. G. W. Rogers, D. J. Marchette, and C. E. Priebe, "A procedure for model complexity selection in semiparametric mixture model density estimation," Naval Surface Warfare Center, Dahlgren Division, Dahlgren, VA, Tech. Rep., 2002.
32. A. Barron, L. Birgé and P. Massart, "Risk bounds for model selection via penalization," Probab. Theory Related Fields, vol. 113, pp. 301-413, 1999.
33. G. Castellan, "Sélection d'histogrammes à l'aide d'un critère de type Akaike," C. R. Acad. Sci. Paris Sér. I Math., vol. 330, pp. 729-732, 2000.
34. P. Massart, "Some applications of concentration inequalities to statistics," Ann. Fac. Sci. Toulouse Math., vol. 9, pp. 245-303, 2000.
35. J. Rissanen, "On Bayesian analysis of mixtures with an unknown number of components," Automatica, vol. 14, pp. 465-471, 1978.
36. ____, "Stochastic complexity (with discussion)," J. Roy. Statist. Soc. Ser. B Statist. Methodol., vol. 49, pp. 223-239, 1987.
37. J. Rissanen, T. Speed, and B. Yu, "Density estimation by stochastic complexity," IEEE Trans. Inform. Theory, vol. 38, pp. 315-323, Mar. 1992.
38. L. Devroye, A Course in Density Estimation. Boston, MA: Birkhäuser, 1987.