Mismatched estimation and relative entropy

IEEE Transactions on Information Theory

Volumn 56, Issue 8, 2010, Pages 3712-3720

  Verdú, Sergio ^a  

^a Princeton University   (United States)

Author keywords

Divergence; free probability; minimum mean square error (MMSE) estimation; mutual information; relative entropy; Shannon theory; statistics

Indexed keywords

DIVERGENCE; DIVERGENCE FREE; MINIMUM MEAN-SQUARE-ERROR ESTIMATIONS; MUTUAL INFORMATIONS; RELATIVE ENTROPY; SHANNON THEORY;

ENTROPY; ERROR STATISTICS; RANDOM VARIABLES; SIGNAL TO NOISE RATIO;

ESTIMATION;

EID: 77954590943     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2010.2050800     Document Type: Article

References (32)

1. D. Guo, S. Shamai, and S. Verdú, "Mutual information and minimum mean-square error in Gaussian channels," IEEE Trans. Inf. Theory, vol.51, no.4, pp. 1261-1283, Apr. 2005.
2. D. P. Palomar and S. Verdú, "Gradient of mutual information in linear vector Gaussian channels," IEEE Trans. Inf. Theory, vol.52, no.1, pp. 141-154, Jan. 2006.
3. S. Verdú and D. Guo, "A simple proof of the entropy power inequality," IEEE Trans. Inf. Theory, vol.52, no.5, pp. 2165-2166, May 2006.
4. A. M. Tulino and S. Verdú, "Monotonic decrease of the non-Gaussianness of the sum of independent random variables: A simple proof," IEEE Trans. Inf. Theory, vol.52, no.9, pp. 4295-4297, Sep. 2006.
5. M. Zakai, "On mutual information, likelihood ratios, and estimation error for the additive Gaussian channel," IEEE Trans. Inf. Theory, vol.51, no.9, pp. 3017-3024, Sep. 2005.
6. J. Binia, "Divergence and minimum mean-square error in continuoustime additive white Gaussian noise channels," IEEE Trans. Inf. Theory, vol.52, no.3, pp. 1160-1163, Mar. 2006.
7. A. Lozano, A. M. Tulino, and S. Verdú, "Optimum power allocation for parallel Gaussian channels with arbitrary input distributions," IEEE Trans. Inf. Theory, vol.52, no.7, pp. 3033-3051, Jul. 2006.
8. D. Palomar and S. Verdú, "Representation of mutual information via input estimates," IEEE Trans. Inf. Theory, vol.53, no.2, pp. 453-470, Feb. 2007.
9. E. Mayer-Wolf and M. Zakai, "Some relations between mutual information and estimation error inWiener space," Ann. Appl. Probab., vol.17, no.3, pp. 1102-1116, Jun. 2007.
10. D. Guo, S. Shamai, and S. Verdú, "Mutual information and conditional mean estimation in Poisson channels," IEEE Trans. Inf. Theory, vol.54, no.5, pp. 1837-1849, May 2008.
11. D. Guo, S. Shamai, and S. Verdú, "Estimation of non-Gaussian random variables in Gaussian noise: Properties of the MMSE," in Proc. IEEE Int. Symp. Inf. Theory, Toronto, ON, Canada, Jul. 6-11, 2008, pp. 1083-1087.
12. H. Yoshida, "Remarks on a semicircular perturbation of the free Fisher information," Infinite Dimensional Anal., Quantum Probab. Related Topics, vol.11, no.1, pp. 97-108, 2008.
13. P. Dupuis and R. E. Ellis, AWeak Convergence Approach to the Theory of Large Deviations. New York: Wiley-Interscience, 1997.
14. I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.
15. S. Kullback and R. A. Leibler, "On information and sufficiency," Ann. Math. Stat., vol.22, pp. 79-86, 1951.
16. M. Pinsker, S. Prelov, and S. Verdú, "Sensitivity of channel capacity," IEEE Trans. Inf. Theory, vol.41, no.6, pp. 1877-1888, Nov. 1995.
17. D. Guo, "Relative entropy and score function: New information estimation relationships through arbitrary additive perturbation," in Proc. IEEE Int. Symp. Inf. Theory, Seoul, Korea, Jun. 28-Jul. 3 2009, pp. 814-818.
18. Y.Wu, D. Guo, and S. Verdu, "Low-SNR behavior of mutual information with Gaussian noise," 2010, preprint.
19. V. V. Prelov and S. Verdú, "Second-order asymptotics of mutual information," IEEE Trans. Inf. Theory, vol.50, no.8, pp. 1567-1580, Aug. 2004.
20. G. Blower, Random Matrices: High Dimensional Phenomena, ser. London Mathematical Society Lecture Notes. Cambridge, U.K.: Cambridge Univ. Press, 2009.
21. A. M. Tulino and S. Verdú, "Random matrix theory and wireless communication," Found. Trends Commun. Inf. Theory, vol.1, no.1, pp. 1-182, 2004.
22. P. Biane, "Free probability for probabilists," MSRI Rep. 1998-040, 1998 [Online]. Available: arXiv:math/9809193v1
23. D. Voiculescu, "The analogues of entropy and of Fisher's information measure in free probability theory I," Commun. Math. Phys., vol.155, pp. 71-92, 1993.
24. D. Voiculescu, "The analogues of entropy and of Fisher's information measure in free probability theory II," Invent. Math., vol.118, pp. 411-440, 1994.
25. D. Voiculescu, "The analogues of entropy and of Fisher's information measure in free probability theory VI: Liberation and mutual free information," Adv. Math., vol.146, pp. 101-166, 1999.
26. S. V. Szarek and D. Voiculescu, "Volumes of restricted Minkowski sums and free analogue of the entropy power inequality," Commun. Math. Phys., vol.178, pp. 563-570, 1996.
27. D. Shlyakhtenko and H. Schultz, "Shannon's monotonicity problem for free and classical entropy," in Proc. Nat. Acad. Sci., Sep. 19, 2007, vol.104, pp. 15254-15258. (Pubitemid 47502905)
28. P. Biane and R. Speicher, "Free diffusions, free entropy and free Fisher information," Ann. Inst. H. Poincaré Probab. Stat., vol.37, pp. 581-606, 2001.
29. F. Hiai, M. Mizuo, and D. Petz, "Free relative entropy for measures and a corresponding perturbation theory," J. Math. Soc. Jpn., vol.54, pp. 679-718, 2002.
30. B. Blackadar, Operator Algebras: Theory of C*-Algebras and Von Neumann Algebras. London, U.K.: Taylor & Francis, 2006.
31. S. Verdú, "teaching it," in Proc. IEEE Int. Symp. Inf. Theory, Nice, France, Jun. 28, 2007, DOI: 10.1109/ISIT.2007.4557074.
32. T. Weissman, "The relationship between causal and non-causal mismatched estimation in continuous-time AWGN channels," IEEE Trans. Inf. Theory, 2010, to be published.