REGO: Rank-based estimation of Rényi information using Euclidean graph optimization

Journal of Machine Learning Research

Volumn 9, Issue , 2010, Pages 605-612

  Póczos, Barnabás ^a   Kirshner, Sergey ^b   Szepesvári, Csaba ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

^b PURDUE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EMPIRICAL ESTIMATE; EUCLIDEAN GRAPH; MULTIVARIATE DISTRIBUTIONS; NEGATIVE ENTROPIES; NON-PARAMETRIC ESTIMATIONS; SHANNON INFORMATION; SIMULATED EXPERIMENTS;

ARTIFICIAL INTELLIGENCE; CRYSTAL SYMMETRY; OPTIMIZATION;

ESTIMATION;

EID: 84862281592     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (25)

1. Beirlant, J., Dudewicz, E. J., Györfi, L., and Meulen, E. C. (1997). Nonparametric entropy estimation: An overview. Int. J. Math. Stat. Sci., 6:17-39.
2. Dickerson, M. and Eppstein, D. (1996). Algorithms for proximity problems in higher dimensions. Comput. Geometry: Theory and Applications, 5(5):277-291.
3. Dvoretzky, A., Kiefer, J., andWolfowitz, J. (1956). Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator. Annals of Mathematical Statistics, 27.
4. Hero, A. O., Ma, B., Michel, O. J. J., and Gorman, J. (2002). Applications of entropic spanning graphs. IEEE Signal Processing Magazine, 19(5):85-95.
5. Hero, A. O. and Michel, O. J. J. (1998). Robust entropy estimation strategies based on edge weighted random graphs. In Proc. of Meeting of Intl. Soc. for Optical Engin., pages 250-261.
6. Hero, A. O. and Michel, O. J. J. (1999a). Asymptotic theory of greedy approximations to minimal K-point random graphs. IEEE Transactions on Information Theory, 45(6):1921-1938.
7. Hero, A. O. and Michel, O. J. J. (1999b). Estimation of Rényi information divergence via pruned minimal spanning trees. In Proc. IEEE Workshop Higher Order Statistics, Caesaria, Israel.
8. Huber, P. J. (1981). Robust Statistics. Wiley.
9. Huber, P. J. (2009). On the non-optimality of optimal procedures. In Optimality: The Third Erich L. Lehmann Symposium, volume 57, pages 31-46. Institute of Mathematical Statistics, Beachwood, OH, USA.
10. Jiménez, R. and Yukich, J. E. (2005). Statistical distances based on Euclidean graphs. In Recent Advances in Probability and its Applications, pages 223-239. Springer.
11. Kozachenko, L. F. and Leonenko, N. N. (1987). A statistical estimate for the entropy of a random vector. Problems Infor. Transmiss, 23:9-16.
12. Kraskov, A., Stögbauer, H., and Grassberger, P. (2004). Estimating mutual information. Phys. Rev. E, 69:066138.
13. Kybic, J. (2006). Incremental updating of nearest neighbor-based high-dimensional entropy estimation. In Proc. Acoustics, Speech and Signal Processing.
14. Leonenko, N. N., Pronzato, L., and Savani, V. (2008). A class of Rényi information estimators for multidimensional densities. Annals of Statistics, 36:2153-2182.
15. Massart, P. (1990). The tight constant in the Dvoretzky- Kiefer-Wolfowitz inequality. The Annals of Probability, 18(3):1269-1283.
16. Pál, D., Szepesvári, C., and Póczos, B. (2010). Estimation of Rényi entropy and mutual information based on generalized nearest-neighbor graphs. http://arxiv.org/abs/1003.1954.
17. Prim, R. C. (1957). Shortest connection networks and some generalizations. Bell Syst. Tech. Journal, 36:1389-1401.
18. Rényi, A. (1961). On measure of entropy and information. In Proc. of the Fourth Berkeley Symposium on Mathematics, Statistics, and Probability, pages 547-561.
19. Scott, D. W. (1979). On optimal and data-based histograms. Biometrika, 66:605-610.
20. Spearman, C. (1904). The proof and measurement of association between two things. American Journal of Psychology, 15:72-101.
21. Steele, J. M. (1988). Growth rates of Euclidean minimal spanning trees with power weighted edges. Annals of Probability, 16:1767-1787.
22. Steele, J. M. (1996). Probability Theory and Combinatorial Optimization. SIAM.
23. Tukey, J. W. (1970). Exploratory Data Analysis. Mimeograph. Addison-Wesley.
24. Vajda, I. (1989). Theory of Statistical Inference and Information. Kluwer, Boston.
25. Yukich, Y. E. (1998). Probability Theory Of Classical Euclidean Optimization Problems, volume 1675 of Lecture Notes in Mathematics. Springer.