Mutual information approximation via maximum likelihood estimation of density ratio

IEEE International Symposium on Information Theory - Proceedings

Volumn , Issue , 2009, Pages 463-467

  Suzuki, Taiji ^a   Sugiyama, Masashi ^b   Tanaka, Toshiyuki ^c  

^a UNIVERSITY OF TOKYO   (Japan)

^b TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^c KYOTO UNIVERSITY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

A-DENSITY; CONVERGENCE PROPERTIES; DENSITY ESTIMATION; DENSITY RATIO; EXISTING METHOD; GLOBAL OPTIMAL SOLUTIONS; LIKELIHOOD ESTIMATION; MODEL SELECTION CRITERIA; MUTUAL INFORMATIONS; NUMERICAL EXPERIMENTS; USEFUL PROPERTIES;

ESTIMATION; FEATURE EXTRACTION; INFORMATION THEORY; NUMERICAL METHODS;

MAXIMUM LIKELIHOOD ESTIMATION;

EID: 70449490877     PISSN: 21578102     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ISIT.2009.5205712     Document Type: Conference Paper

References (11)

1. G. A. Darbellay and I. Vajda. Estim ation of the information by an adaptive part itioning of the observat ion space. IEEE Transactions on Information Theory, 45(4):1315-1321,1999.
2. S. Ghosal and A. W. van der Vaart. Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. Annals of Statistics, 29:1233-1263, 2001.
3. M. M. Van Hulle, Edgeworth approximat ion of multivariate differential entropy. Neural Computation, 17(9) :1903-1910,2005.
4. A. Kraskov, H. Stögbau er, and P. Grassberger. Estimating mutual information. Physical Review E, 69:066138, 2004.
5. X. Nguyen, M. J. Wainwright, and M. I. Jordan. Nonparametrie estimation of the likelihood ratio and divergence functionals. IEEE International Symposium on Information Theory, pages 2016-2020, Nice, France, 2007.
6. F. Pérez-Cruz. Kullback-Leibler divergence estimation of continuous distributions. IEEE International Symposium on Information Theory, pages 1666-1670, Toronto, Canada, 2008.
7. G. Rockafellar. Convex Analysis. Princeton University Press, 1970.
8. J. Silva and S. Narayanan. Universal consistency of data-driven partitions for divergence estimation. IEEE International Symposium on Information Theory, pages 2021-2025, Nice, France, 2007.
9. A W. van der Vaart and J. A. Wellner. Weak Convergence and Empirical Processes. With Applications to Statistics. Sprin ger, New York, 1996.
10. Q. Wang, S. R. Kulkarmi, and S. Verdú, Divergence estimation of contiunous distributions based on data-d ependent partitions. IEEE Transactions on Information Theory, 51(9):3064-3074, 2005.
11. H. White. Maximum likelihood estimation of misspecified models. Econometrica, 50(1) :1-25, 1982.