The relationship between causal and noncausal mismatched estimation in continuous-time AWGN channels

IEEE Transactions on Information Theory

Volumn 56, Issue 9, 2010, Pages 4256-4273

  Weissman, Tsachy ^a,b  

^a Stanford University   (United States)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

AWGN channels; Brownian motion; continuous time; I MMSE formula; minimax estimation; minimum mean square error estimation; mismatched estimation; mutual information; nonlinear filtering; relative entropy; Shannon theory

Indexed keywords

AWGN CHANNEL; BROWNIAN MOTION; CONTINUOUS TIME; I-MMSE FORMULAS; MINIMAX ESTIMATIONS; MINIMUM MEAN-SQUARE-ERROR ESTIMATIONS; MISMATCHED ESTIMATIONS; MUTUAL INFORMATIONS; RELATIVE ENTROPY; SHANNON THEORY;

BROWNIAN MOVEMENT; CONTINUOUS TIME SYSTEMS; ENTROPY; NOISE GENERATORS; NONLINEAR FILTERING; SIGNAL TO NOISE RATIO; SQUARE WAVE GENERATORS; WHITE NOISE;

ESTIMATION;

EID: 77954570374     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2010.2054430     Document Type: Article

References (40)

1. S. Artstein, K. Ball, F. Barthe, and A. Naor, "Solution of Shannon's problem on the monotonicity of entropy," J. Amer. Math. Soc., vol.17, pp. 975-982, 2004.
2. A. R. Barron, "Entropy and the central limit theorem," Ann. Probab., vol.14, pp. 336-342, 1986.
3. J. Binia, "Divergence and minimum mean-square error in continuoustime additive white Gaussian noise channels," IEEE Trans. Inf. Theory, vol.52, no.3, pp. 1160-1163, Mar. 2006.
4. R. H. Cameron and W. T. Martin, "Transformation ofWiener integrals under translations," Ann. Math., vol.45, pp. 386-396, 1944.
5. A. Cohen, N. Merhav, and T.Weissman, "Scanning and sequential decision making for multidimensional data, part II: noisy data," IEEE Trans. Inf. Theory, vol.IT-54, no.12, pp. 5609-5631, Dec. 2008.
6. I. Csiszár and J. Körner, Information Theory: Coding theorems for discrete memoryless systems. New York: Academic, 1981.
7. A. Dembo and O. Zeitouni, Large Deviations Techniques and Applications, 2nd ed. New York: Springer, 1998.
8. T. E. Duncan, "Evaluation of likelihood functions," Inf. Contr., vol.13, pp. 62-74, 1968.
9. T. E. Duncan, "On the calculation of mutual information," SIAM J. Appl. Math., vol.19, pp. 215-220, Jul. 1970.
10. T. E. Duncan, "Mutual information for stochastic signals and Lévy processes," IEEE Trans. Inf. Theory, vol.IT-56, no.1, pp. 18-24, Jan. 2010.
11. T. E. Duncan, "Mutual information for stochastic signals and fractional Brownian motion," IEEE Trans. Inf. Theory, vol.IT-54, no.10, pp. 4432-4438, Oct. 2010.
12. R. G. Gallager, "Source coding with side information and universal coding," presented at the M.I.T. LIDS-P-937, 1979.
13. L. Gammaitoni, P. Hanggi, P. Jung, and F. Marchesoni, "Stochastic resonance," Rev. Mod. Phys., vol.70, pp. 223-287, 1998. (Pubitemid 128367336)
14. I. V. Girsanov, "On transforming a certain class of stochastic processes by absolutely continuous substitution of measures," Theory Probab. Appl., vol.5, pp. 285-301, 1960.
15. D. Guo, "Relative entropy and score function: New information-estimation relationships through arbitrary additive perturbation," presented at the IEEE Int. Symp. Information Theory, Seoul, Korea, Jul. 3, 2009.
16. D. Guo, S. Shamai, and S. Verdú, "Mutual information and minimum mean-square error in Gaussian channels," IEEE Trans. Inf. Theory, vol.IT-51, no.4, pp. 1261-1283, Apr. 2005.
17. D. Guo, S. Shamai, and S. Verdú, "Mutual information and conditional mean estimation in Poisson channels," IEEE Trans. Inf. Theory, vol.54, no.5, pp. 1837-1849, May 2008.
18. T. T. Kadota, M. Zakai, and J. Ziv, "Capacity of a continuous memoryless channel with feedback," IEEE Trans. Inf. Theory, vol.IT-17, pp. 372-378, 1971.
19. T. T. Kadota, M. Zakai, and J. Ziv, "Mutual information of the white Gaussian channel with and without feedback," Trans. Inf. Theory, vol.IT-17, pp. 368-371, 1971.
20. T. Kailath, "The structure of Radon-Nykodim derivatives with respect toWiener and related measures," Ann. Math. Statist., vol.42, no.3, pp. 1054-1067, 1971.
21. I. Karatzas and A. E. Shreve, Brownian Motion and Stochastic Calculus, 2nd ed. New York: Springer-Verlag, 1991.
22. Y. H. Kim, H. H. Permuter, and T. Weissman, "Directed information and causal estimation in continuous time," presented at the Int. Symp. Inf. Theory, Seoul, Korea, Jul. 29th, 2009.
23. A. Kolmogorov, "On the Shannon theory of information transmission in the case of continuous signals," IRE Trans. Inf. Theory, vol.2, no.4, pp. 102-108, Dec. 1956.
24. E. L. Lehmann and G. Casella, Theory of Point Estimation, 2nd ed. New York: Springer, 1998.
25. R. S. Liptser and A. N. Shiryaev, Statistics of Random Processes II: Applications, 2nd ed. New York: Springer, 2001.
26. A. Lozano, A. M. Tulino, and S. Verdú, "Optimum power allocation for parallel Gaussian channels with arbitrary input distributions," IEEE Trans. Inf. Theory, vol.52, no.7, pp. 3033-3051, Jul. 2006.
27. S. Orey, "Conditions for the absolute continuity of two diffusions," Trans. Amer. Math. Soc., vol.193, pp. 413-426, 1974.
28. M. Madiman and A. R. Barron, "Generalized entropy power inequalities and monotonicity properties of information," IEEE Trans. Inf. Theory, vol.53, no.7, pp. 2317-2329, Jul. 2007.
29. E. Mayer-Wolf and M. Zakai, "Some relations between mutual information and estimation error inWiener space," Ann. Appl. Probab., vol.17, no.3, pp. 1102-1116, Jun. 2007.
30. N. Merhav and M. Feder, "A strong version of the redundancy-capacity theorem of universal coding," IEEE Trans. Inf. Theory, vol.41, no.3, pp. 714-722, May 1995.
31. D. P. Palomar and S. Verdú, "Gradient of mutual information in linear vector Gaussian channels," IEEE Trans. Inf. Theory, vol.52, no.1, pp. 141-154, Jan. 2006.
32. D. P. Palomar and S. Verdú, "Representation of mutual information via input estimates," IEEE Trans. Inf. Theory, vol.53, no.2, pp. 453-470, Feb. 2007.
33. M. S. Pinsker, "Information and information stability of random variables and processes," presented at the Moskva: Izv. Akad. Nauk, Russian, 1960.
34. B. Y. Ryabko, "Encoding a source with unknown but ordered probabilities," Probl. Inf. Transm., pp. 134-139, Oct. 1979.
35. A. Stam, "Some inequalities satisfies by the quantities of information of Fisher and Shannon," Inf. Control, vol.2, pp. 101-112, Jun. 1959.
36. A. M. Tulino and S. Verdú, "Monotonic decrease of the non-Gaussianness of the sum of independent random variables: A simple proof," IEEE Trans. Inf. Theory, vol.52, no.9, pp. 4295-4297, Sep. 2006.
37. S. Verdú, "Mismatched estimation and relative entropy," presented at the Int. Symp. Information Theory, Seoul, Korea, Jul. 29, 2009.
38. S. Verdú and D. Guo, "A simple proof of the entropy power inequality," IEEE Trans. Inf. Theory, vol.52, no.5, pp. 2165-2166, May 2006.
39. A. D.Wyner, "A definition of conditional mutual information for arbitrary ensembles," Inf. Control, vol.38, pp. 51-59, 1978.
40. M. Zakai, "On mutual information, likelihood ratios, and estimation error for the additive Gaussian channel," IEEE Trans. Inf. Theory, vol.51, no.9, pp. 3017-3024, Sep. 2005.