Statistical inference for the ε{lunate}-entropy and the quadratic Rényi entropy

Journal of Multivariate Analysis

Volumn 101, Issue 9, 2010, Pages 1981-1994

  Leonenko, Nikolaj ^a   Seleznjev, Oleg ^b  

^a CARDIFF UNIVERSITY   (United Kingdom)

^b UMEÅ UNIVERSITY   (Sweden)

Author keywords

Entropy estimation; Quadratic R nyi entropy; U statistics

Indexed keywords

EID: 77955058246     PISSN: 0047259X     EISSN: 10957243     Source Type: Journal    
DOI: 10.1016/j.jmva.2010.05.009     Document Type: Article

References (43)

1. Baryshnikov Yu., Penrose M., Yukich J.E. Gaussian limits for generalized spacings. Ann. Appl. Probab. 2009, 19:158-185.
2. Beirlant J., Dudewicz E.J., Gyorfi L., van der Meulen E.C. Non-parametric entropy estimation: an overview. Internat. J. Math. Statist. Sci. 1997, 6:17-39.
3. Bennett Ch.H., Brassard G., Crépeau C., Maurer U.M. Generalized privacy amplification. IEEE Trans. Inform. Theory 1995, 41:1915-1923.
4. Bickel P.J., Ritov Y. Estimating integrated squared density derivatives: sharp best order of convergence estimates. Sankhya Ser. A 1988, 50:381-393.
5. Dereich S. Small ball probabilities around random centers of Gaussian measures and application to quantization. J. Theoret. Probab. 2003, 16:427-449.
6. Evans D., Jones A.J., Schmidt W.M. Asymptotic moments of nearest-neighbor density estimates. Proc. R. Soc. Lond. Ser. A, Math. Phys. Eng. Sci. 2002, 458:2839-2849.
7. Giné E., Nickl R. A simple adaptive estimator of the integrated square of a density. Bernoulli 2008, 14:47-61.
8. Heyde C.C., Leonenko N.N. Student processes. Adv. Appl. Probab. 2005, 37:342-365.
9. Jammalamadaka S.R., Janson S. Limit theorems for a triangular scheme of U-statistics with applications to inter-point distances. Ann. Probab. 1986, 14:1347-1358.
10. Jiménez R., Yukich J.E. Asymptotics for statistical distances based on Voronoi tessellations. J. Theoret. Probab. 2002, 15:503-541.
11. Johnson O., Vignat Ch. Some results concerning maximum Rényi entropy distributions. Ann. Inst. H. Poincaré Probab. Statist. 2007, 43:339-351.
12. Kapur J.N. Maximum-Entropy Models in Science and Engineering 1989, Wiley, New York.
13. Kapur J.N., Kesavan H.K. Entropy Optimization Principles with Applications 1992, Academic Press, New York.
14. Kolmogorov A.N. Theory of transmission of information. Amer. Math. Soc. Translation Ser. 2 1963, 33:291-321.
15. Kolmogorov A.N. On certain asymptotic characteristics of completely bounded metric spaces. Dokl. Akad. Nauk SSSR 1956, 108(3):385-388. (in Russian).
16. Koroljuk V.S., Borovskich Yu.V. Theory of U-Statistics 1994, Kluwer, London.
17. Kozachenko L.F., Leonenko N.N. On statistical estimation of entropy of random vector. Problems Inform. Transmiss. 1987, 23:95-101.
18. Kuelbs J., Li W.V. Metric entropy and the small ball problem for Gaussian measures. J. Funct. Anal. 1993, 116:133-157.
19. Learned-Miller E., Fisher J. ICA using spacing estimation of entropy. J. Mach. Learn. Res. 2003, 4:1271-1295.
20. Lee A.J. U-Statistics: Theory and Practice 1990, Marcel Dekker, New York.
21. Leonenko N., Pronzato L., Savani V. A class of Rényi information estimators for multidimensional densities. Ann. Statist. 2008, 36:2153-2182.
22. Liu Z.J., Rao C.R. Asymptotic distribution of statistics based on quadratic entropy and bootstrapping. J. Stat. Plan. Inference 1995, 43:1-18.
23. Luczak T., Szpankowski W. A suboptimal lossy data compression based in approximate pattern matching. IEEE Trans. Inform. Theory 1997, 41:1439-1451.
24. Posner E.C., Rodemich E.E., Rumsey H. Epsilon entropy of Gaussian processes. Ann. Math. Stat. 1969, 40:1272-1296.
25. Prohorov Yu.V., Rozanov Yu.A. Probability Theory 1969, Springer, New York.
26. Ranneby B., Jammalamadaka S.R., Teterukovskiy A. The maximum spacing estimation for multivariate observations. J. Stat. Plan. Inference 2005, 129:427-446.
27. Redmond C., Yukich J.E. Asymptotics for Euclidean functionals with power-weighted edges. Stochastic Process. Appl. 1996, 61:289-304.
28. Rényi A. On measures of entropy and information. 4th Berkeley Symp. Math. Statist. Prob. 1961, vol. I:547-561. Univ. Calif. Press, Berkeley.
29. Rényi A. Probability Theory 1970, North-Holland, London.
30. Seleznjev O., Thalheim B. Average case analysis in database problems. Methodol. Comput. Appl. Probab. 2003, 5:395-418.
31. Seleznjev O., Thalheim B. Random databases with approximate record matching. Methodol. Comput. Appl. Probab. 2008, (or in print 12 (2010) 63-89). 10.1007/s11009-008-9092-4.
32. Serfling R.J. Approximation Theorems of Mathematical Statistics 2002, Wiley, New York. second ed.
33. Shannon C.E. A mathematical theory of communication. Bell Syst. Tech. J. 1948, 27:379-423. 623-656.
34. Silverman B., Brown T. Short distances, flat triangles and Poisson limits. J. Appl. Probab. 1978, 15:815-825.
35. Szpankowski W. Average Case Analysis of Algorithms on Sequences 2001, Wiley, New York.
36. Thalheim B. Entity-Relationship Modeling. Foundations of Database Technology 2000, Springer-Verlag, Berlin.
37. Tsallis C. Possible generalization of Boltzmann-Gibbs statistics. J. Stat. Phys. 1988, 52:479-487.
38. Tsybakov A.B., Van der Meulen E.C. Root-n consistent estimators of entropy for densities with unbounded support. Scand. J. Statist. 1996, 23:75-83.
39. Vatutin V.A., Michailov V.G. Statistical estimation of entropy of discrete random variables with large numbers of results. Uspeh. Math. Nauk 1995, 50:121-134. (in Russian).
40. Zografos K. On maximum entropy characterization of Pearson's type II and VII multivariate distributions. J. Multivariate Anal. 1999, 71:67-75.
41. Zografos K. On Mardia's and Song's measures of kurtosis in elliptical distributions. J. Multivariate Anal. 2008, 99:858-879.
42. Zografos K., Nadarajah S. Expressions for Rényi and Shannon entropies for multivariate distributions. Statist. Probab. Lett. 2005, 71:71-84.
43. Zyczkowski K. Rényi extrapolation of Shannon entropy. Open Sys. Inf. Dyn. 2003, 10:297-310.