On mutual information, likelihood ratios, and estimation error for the additive Gaussian channel

IEEE Transactions on Information Theory

Volumn 51, Issue 9, 2005, Pages 3017-3024

  Zakai, Moshe ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Gaussian channel; Logarithmic Sobolev in equality; Malliavin calculus; Minimal mean square estimation error; Mutual information; Nonlinear filtering; Relative entropy

Indexed keywords

INTEGRAL EQUATIONS; MATHEMATICAL MODELS; MAXIMUM LIKELIHOOD ESTIMATION; NONLINEAR FILTERING; SIGNAL TO NOISE RATIO; THEOREM PROVING;

ADDITIVE GAUSSIAN CHANNEL; LOGARITHMIC SOBOLEV INEQUALITY; MALLIAVIN CALCULUS; MINIMAL MEAN-SQUARE ESTIMATION ERROR; RELATIVE ENTROPY;

COMMUNICATION CHANNELS (INFORMATION THEORY);

EID: 26444493543     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2005.853297     Document Type: Article

References (21)

1. E. A. Carlen, "Superadditivity of Fisher's information and logarithmic Sobolev inequalities," J. Funct. Anal., vol. 101, pp. 194-211, 1991.
2. I. Csiszár and F. Matus, "Information projections revisited," IEEE Trans. Inf. Theory, vol. 49, no. 6, pp. 1474-1490, Jun. 2003.
3. A. Dembo, M. Cover, and J. A. Thomas, "Information theoretic inequalities," IEEE Trans. Inf. Theory, vol. 37, no. 6, pp. 1501-1518, Nov. 1991.
4. R. M. Dudley, Real Analysis and Probability. Belmont, CA: Wadsworth and Brooks/Cole, 1989.
5. T. E. Duncan, "On the calculation of mutual information," in SIAM J. Appl. Math., vol. 19, Jul. 1970, pp. 215-220.
6. R. Esposito, "On a relation between detection and estimation in decision theory," Inform. Contr., vol. 12, pp. 116-120, 1968.
7. D. Guo, S. Shamai (Shitz), and S. Verdú, "Mutual information and minimum mean-square error in estimation, filtering and smoothing," IEEE Trans. Inf. Theory, vol. 51, no. 4, pp. 1261-1281, Apr. 2005.
8. C. P. Hatsell and L. W. Nolte, "Some geometric properties of the likelihood ratio," IEEE Trans. Inf. Theory, vol. IT-17, no. 5, pp. 616-618, Sep. 1971.
9. T. T. Kadota, M. Zakai, and J. Ziv, "Mutual information of the white Gaussian channel with and without feedback," IEEE Trans, Inf. Theory, vol. IT-17, no. 4, pp. 368-371, Jul. 1971.
10. T. Kailath, "A note on least squares estimates from likelihood ratios," Inform. Contr., vol. 13, pp. 534-540, 1968.
11. P. Malliavin, Stochastic Analysis. Berlin/New York: Springer-Verlag, 1997.
12. E. Mayer-Wolf and M. Zakai, "On a formula relating the Shannon information to the Fisher information for the filtering problem," in Lecture Notes in Control and Information Sciences. New York: Springer-Verlag, 1983, vol. 61, pp. 164-171.
13. D. Nualart, The Malliavin Calculus and Related Topics. Berlin, Germany: Springer-Verlag, 1995.
14. D. Nualart and M. Zakai, "Multiple Wiener-Ito integrals possessing a continuous extension," Probab. Theory Related Fields, vol. 85, pp. 131-145, 1990.
15. D. Nualart and M. Zakai, "Positive and strongly positive Wiener functionals," in Barcelona Seminar on Stochastic Analysis (St. Feliu de Guixols, 1991). Basel, Switzerland: Birkhauser, 1993, pp. 132-146. Progr. Probab., vol. 32.
16. M. S. Pinsker, Information and Information Stability of Random Variables and Processes. San Francisco, CA: Holden-Day, 1964.
17. A. S. Üstünel,"An introduction to analysis on Wiener space,"in Lecture Notes in Mathematics. Berlin, Germany: Springer-Verlag, 1996, vol. 1610.
18. A. S. Üstünel and M. Zakai, Transformation of Measure on Wiener Space. New York/Berlin: Springer-Verlag, 1999.
19. A. S. Üstünel and M. Zakai, "Embedding the abstract Wiener space in a probability space," J. Func. Anal., vol. 171, pp. 124-138, 2000.
20. C. Villani, Topics in Optimal Transportation. Providence, RI: AMS, 2003, vol. 58, American Mathematical Society, Graduate Studies in Math.
21. E. Wong and M. Zakai, "A characterization of kernels associated with the multiple integral representation of some functionals of the wiener process," Syst. Control Lett., vol. 2, pp. 94-98, 1982/1983.