Density estimation by the penalized combinatorial method

Journal of Multivariate Analysis

Volumn 94, Issue 1, 2005, Pages 196-208

  Biau, Gérard ^a   Devroye, Luc ^b  

^a SORBONNE UNIVERSITÉ   (France)

^b MCGILL UNIVERSITY   (Canada)

Author keywords

Mixture densities; Multivariate density estimation; Penalization; Vapnik Chervonenkis dimension

Indexed keywords

EID: 14644399896     PISSN: 0047259X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jmva.2004.04.011     Document Type: Article

References (49)

1. A. Barron, Complexity regularization with application to artificial neural networks, in: G. Roussas (Ed.), Nonparametric Functional Estimation and Related Topics, NATO ASI Series, Kluwer Academic Publishers, Dordrecht, 1991, pp. 501-576.
2. Barron A. Birgé L. Massart P. Risk bounds for model selection via penalization Probab. Theory Related Fields 113 1999 301-413
3. Barron A.R. Sheu C.H. Approximation of density functions by sequences of exponential families Ann. Statist. 19 1991 1347-1369
4. Biau G. Devroye L. A note on density model size testing IEEE Trans. Inform. Theory 50 2004 576-581
5. Billingsley P. Probability and Measure 3rd Edition 1995 Wiley New York
6. C.L. Bishop, Mixture density networks, Neural Computing Research Group Report NCRG/94/004, Department of Computer Science and Applied Mathematics, Aston University, Birmingham, 1994.
7. Bunea F. Wegkamp M. A note on penalized minimum distance estimation in nonparametric regression Canad. J. Statist. 31 2003 267-274
8. Castellan G. Sélection d'histogrammes à l'aide d'un critère de type Akaike C. R. Acad. Sci. Paris 330 2000 729-732
9. Celeux G. Hurn M. Robert C.P. Computational and inferential difficulties with mixture posterior distributions J. Amer. Statist. Assoc. 95 2000 957-970
10. Dacunha-Castelle D. Gassiat E. Testing in locally conic models, and application to mixture models ESAIM: Probab. Statist. 1 1997 285-317
11. Dacunha-Castelle D. Gassiat E. The estimation of the order of a mixture model Bernoulli 3 1997 279-299
12. Dacunha-Castelle D. Gassiat E. Testing the order of a model using locally conic parametrization: Population mixtures and stationary ARMA processes Ann. Statist. 27 1999 1178-1209
13. Devroye L. A Course in Density Estimation 1987 Birkhäuser Boston
14. Devroye L. Universal smoothing factor selection in density estimation: theory and practice (with discussion) Test 6 1997 223-320
15. Devroye L. Györfi L. Lugosi G. A Probabilistic Theory of Pattern Recognition 1996 Springer New York
16. Devroye L. Lugosi G. Combinatorial Methods in Density Estimation 2001 Springer New York
17. Diebolt J. Robert C.P. Estimation of finite mixture distributions through Bayesian sampling J. Roy. Statist. Soc. Ser. B 56 1994 363-375
18. Dudley R.M. Central limit theorems for empirical measures Ann. Probab. 6 1978 899-929
19. Dunford N. Schwartz J.T. Linear Operators Part I 1963 Wiley New York
20. Everitt B.S. Hand D.J. Finite Mixture Distributions 1981 Chapman & Hall London
21. Figueiredo M.A.T. Jain A.K. Unsupervised learning of finite mixture models IEEE Trans. Pattern Anal. Mach. Intell. 24 2002 381-396
22. Freund Y. Schapire R.E. A short introduction to boosting J. Japan Soc. Artificial Intell. 14 1999 771-780
23. Fukumizu K. Likelihood ratio of unidentifiable models and multilayer neural networks Ann. Statist. 31 2003 833-851
24. Györfi L. Kohler M. Krzyzak A. Walk H. A Distribution-Free Theory of Nonparametric Regression 2002 Springer New York
25. J. Hartigan, A failure of likelihood asymptotics for normal mixtures, Proceedings of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, Vol. II, 1985, pp. 807-810.
26. Hurn M. Justel A. Robert C.P. Estimating mixtures of regressions J. Comput. Graphical Statist. 12 2003 1-25
27. James L.F. Priebe C.E. Marchette D.J. Consistent estimation of mixture complexity Ann. Statist. 29 2001 1281-1296
28. Jordan M.I. Jacobs R.A. Hierarchical mixtures of experts and the EM algorithm Neural Comput. 6 1994 181-214
29. Kohler M. Nonparametric regression function estimation using interaction least squares splines and complexity regularization Metrika 47 1998 147-163
30. Krzyzak A. Linder T. Radial basis function networks and complexity regularization in function learning IEEE Trans. Neural Networks 9 1998 247-256
31. Lugosi G. Vayatis N. On the Bayes-risk consistency of regularized boosting methods (with discussion) Ann. Statist. 32 2004 30-55
32. Massart P. Some applications of concentration inequalities to statistics Ann. Fac. Sci. Toulouse 9 2000 245-303
33. McDiarmid C. On the method of bounded differences in: Siemons J. (Ed.), Surveys in Combinatorics, London Math. Society Lecture Note Series 141 1989 148-188 Cambridge University Press Cambridge
34. McLachlan G.J. Basford K.E. Mixture Models: Inference and Applications to Clustering 1988 Marcel Dekker New York
35. McLachlan G.J. Peel D. Finite Mixture Models 2000 Wiley New York
36. Priebe C.E. Adaptive mixtures J. Amer. Statist. Assoc. 89 1994 796-806
37. Redner R.A. Walker H.F. Mixture densities, maximum likelihood and the EM algorithm SIAM Rev. 26 1984 195-239
38. Richardson S. Green P.J. On Bayesian analysis of mixtures with an unknown number of components J. Roy. Statist. Soc. Ser. B 59 1997 731-792
39. Roeder K. Wasserman L. Practical Bayesian density estimation using mixtures of normals J. Amer. Statist. Assoc. 92 1997 894-902
40. G.W. Rogers, D.J. Marchette, C.E. Priebe, A procedure for model complexity selection in semiparametric mixture model density estimation, Technical Report, Naval Surface Warfare Center, Dahlgren Division, Virginia, 2002.
41. Schölkopf B. Support Vector Learning 1997 Oldenbourg Verlag Munich
42. Stone C.J. Large sample inference for log-spline models Ann. Statist. 18 1990 717-741
43. Titterington D.M. Smith A.F.M. Makov U.E. Statistical Analysis of Finite Mixture Distributions 1985 Wiley Chichester
44. Vapnik V.N. Estimation of Dependencies Based on Empirical Data 1982 Springer New York
45. Vapnik V.N. The Nature of Statistical Learning Theory 2nd Edition 2000 Springer New York
46. Vapnik V.N. Chervonenkis A.Ya. On the uniform convergence of relative frequencies of events to their probabilities Theory Probab. Appl. 16 1971 264-280
47. V.N. Vapnik, A.Ya. Chervonenkis, Theory of Pattern Recognition, Nauka, Moscow, 1974 (in Russian); Theorie der Zeichenerkennung, Akademie Verlag, Berlin, 1979 (German translation)
48. Yatracos Y.G. Rates of convergence of minimum distance estimators and Kolmogorov's entropy Ann. Statist. 13 1985 768-774
49. Zeevi A. Meir R. Density estimation through convex combinations of densities; approximation and estimation bounds Neural Networks 10 1997 90-109