Relative entropy and score function: New information-estimation relationships through arbitrary additive perturbation

IEEE International Symposium on Information Theory - Proceedings

Volumn , Issue , 2009, Pages 814-818

  Guo, Dongning ^a  

^a Northwestern University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARBITRARY DISTRIBUTION; DE BRUIJN; GAUSSIAN NOISE MODEL; KULLBACK-LEIBLER DISTANCE; MEAN SQUARED ERROR; MEAN-SQUARED DIFFERENCES; NEW RESULTS; NON-GAUSSIAN MODELS; PROBABILITY MEASURES; RELATIVE ENTROPY; SCORE FUNCTION;

ADDITIVE NOISE; DISTRIBUTION FUNCTIONS; ESTIMATION; GAUSSIAN NOISE (ELECTRONIC); INFORMATION THEORY;

ENTROPY;

EID: 70449469739     PISSN: 21578102     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ISIT.2009.5205652     Document Type: Conference Paper

References (14)

1. A. J. Stam, "Some inequalities satisfied by the quantities of information of Fisher and Shannon," Information and Control, vol. 2, pp. 101-112, 1959.
2. D. Guo, S. Shamai, and S. Verdú, "Mutual information and minimum mean-square error in Gaussian channels," IEEE Trans. Inform. Theory,vol. 51, pp. 1261-1282, Apr. 2005.
3. D. Guo, S. Shamai, and S. Verdú, "Proof of entropy power inequalities via MMSE," in Proc. IEEE Int. Symp. Inform. Theory, pp. 1011-1015, Seattle, WA, USA, July 2006.
4. S. Verdú and D. Guo, "A simple proof of the entropy power inequality," IEEE Trans. Inform. Theory, pp. 2165-2166, May 2006.
5. M. Zakai, "On mutual information, likelihood ratios, and estimation error for the additive Gaussian channel," IEEE Trans. Inform. Theory, vol. 51, pp. 3017-3024, Sept. 2005.
6. D. Guo, S. Shamai, and S. Verdú, "Additive non-Gaussian noise channels: Mutual information and conditional mean estimation," in Proc. IEEE Int. Symp. Inform. Theory, Adelaide, Australia, Sept. 2005.
7. D. Palomar and S. Verdú, "Representation of mutual information via input estimates," IEEE Trans. Inform. Theory, vol. 53, pp. 453-470, Feb. 2007.
8. C. Measson, A. Montanari, and R. Urbanke, "Maxwell Construction: The Hidden Bridge Between Iterativew and Maximum a Posteriori Decoding," IEEE Trans. Inform. Theory, vol. 54, pp. 5277-5307, 2008.
9. S. Kullback, Information Theory and Statistics. New York: Dover, 1968.
10. S. Verdú, "Mismatched estimation and relative entropy," in Proc. IEEE Int. Symp. Inform. Theory, Seoul, Korea, 2009.
11. K. R. Narayanan and A. R. Srinivasa, "On the thermodynamic temperature of a general distribution." http://arxiv.org/abs/0711.1460v2, 2007.
12. I. Csiszr, "Axiomatic characterizations of information measures," Entropy, vol. 10, pp. 261-273, 2008.
13. C. Tsallis, "Possible generalization of Boltzmann-Gibbs statistics," Journal of Statistical Physics, vol. 52, pp. 479-487, 1988.
14. R. Esposito, "On a relation between detection and estimation indecision theory," Information and Control, vol. 12, pp. 116-120, 1968.