MMSE dimension

IEEE Transactions on Information Theory

Volumn 57, Issue 8, 2011, Pages 4857-4879

  Wu, Yihong ^a   Verdú, Sergio ^a  

^a Princeton University   (United States)

Author keywords

Additive noise; Bayesian statistics; Gaussian noise; high SNR asymptotics; minimum mean square error (MMSE); mutual information; non Gaussian noise; Rnyi information dimension

Indexed keywords

ASYMPTOTICS; BAYESIAN STATISTICS; INFORMATION DIMENSIONS; MINIMUM MEAN-SQUARE ERROR; MUTUAL INFORMATIONS; NON-GAUSSIAN NOISE;

GAUSSIAN DISTRIBUTION; RANDOM VARIABLES;

GAUSSIAN NOISE (ELECTRONIC);

EID: 79960980740     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2011.2158905     Document Type: Article

References (45)

1. Y. Wu and S. Verdú, "MMSE dimension," in Proc. 2010 IEEE Int. Symp. Inf. Theory, Austin, TX, Jun. 2010, pp. 1463-1467.
2. A. Rěnyi, "On the dimension and entropy of probability distributions," Acta Math. Hungarica, vol. 10, no. 1-2, Mar. 1959.
3. Y. Wu and S. Verdú, "Rěnyi information dimension: Fundamental limits of almost lossless analog compression," IEEE Trans. Inf. Theory, vol. 56, no. 8, pp. 3721-3748, Aug. 2010.
4. R. M. Gray, Entropy and Information Theory. New York: Springer- Verlag, 1990.
5. T. Kawabata and A. Dembo, "The rate-distortion dimension of sets and measures," IEEE Trans. Inf. Theory, vol. 40, no. 5, pp. 1564-1572, Sep. 1994.
6. K. Falconer, Fractal Geometry: Mathematical Foundations and Applications, 2nd ed. New York: Wiley, 2003.
7. D. Guo, S. Shamai, and S. Verdú, "Mutual information and minimum mean-square error in Gaussian channels," IEEE Trans. Inf. Theory, vol. 51, no. 4, pp. 1261-1283, Apr. 2005.
8. A. Guionnet and D. Shlyakhtenko, "On classical analogues of free entropy dimension," J. Funct. Anal., vol. 251, no. 2, pp. 738-771, Oct. 2007.
9. J. K. Ghosh, Higher Order Asymptotics. Hayward, CA: Inst. Math. Stat., 1994.
10. J. K. Ghosh, B. K. Sinha, and S. N. Joshi, "Expansions for posterior probability and integrated Bayes risk," in Statistical Decision Theory and Related Topics III. New York: Academic, 1982, pp. 403-456.
11. M. V. Burnašev, "Investigation of second order properties of statistical estimators in a scheme of independent observations," Math. USSR Izvestiya, vol. 18, no. 3, pp. 439-467, 1982.
12. D. Guo, Y. Wu, S. Shamai, and S. Verdú, "Estimation in Gaussian Noise: Properties of the minimum mean-square error," IEEE Trans. Inf. Theory, vol. 57, no. 4, pp. 2371-2385, Apr. 2011.
13. M. S. Pinsker, "Optimal filtering of square-integrable signals in Gaussian noise," Problemy Peredachi Informatsii, vol. 16, no. 2, pp. 52-68, 1980.
14. M. Nussbaum, "Minimax risk: Pinsker bound," in Encyclopedia of Statistical Sciences. New York: Wiley, 1999, pp. 451-460.
15. B. Bobrovsky and M. Zakai, "Asymptotic a priori estimates for the error in the nonlinear filtering problem," IEEE Trans. Inf. Theory, vol. IT-28, no. 2, pp. 371-376, 1982. (Pubitemid 12499751)
16. Y. Wu and S. Verdú, "Functional properties of MMSE," in Proc. 2010 IEEE Int. Symp. Inf. Theory, Austin, TX, Jun. 2010, pp. 1453-1457.
17. P. Huber, Robust Statistics. New York: Wiley-Interscience, 1981.
18. L. D. Brown, "Admissible estimators, recurrent diffusions, and insoluble boundary value problems," Annals Math. Stat., vol. 42, no. 3, pp. 855-903, 1971.
19. V. V. Prelov and E. C. van der Meulen, "Asymptotics of Fisher information under weak perturbation," Probl. Inf. Transm., vol. 31, no. 1, pp. 14-22, 1995.
20. M. S. Pinsker, V. V. Prelov, and S. Verdú, "Sensitivity of channel capacity," IEEE Trans. Inf. Theory, vol. 41, no. 6, pp. 1877-1888, Nov. 1995.
21. V. V. Prelov and S. Verdú, "Second-order asymptotics of mutual information," IEEE Trans. Inf. Theory, vol. 50, no. 8, pp. 1567-1580, Aug. 2004.
22. A. M. Tulino and S. Verdú, "Random matrix theory and wireless communications," Foundations and Trends in Communications and Inf. Theory, vol. 1, no. 1, June 2004.
23. E. Çinlar, Probability and Stochastics. New York: Springer, 2011.
24. K. Falconer, Techniques in Fractal Geometry. Chichester, U.K.: Wiley, 1997.
25. Y. Wu and S. Verdú, "Functional properties of MMSE and mutual information," IEEE Trans. Inf. Theory, 2010, submitted for publication.
26. A. Stam, "Some inequalities satisfied by the quantities of information of Fisher and Shannon," Inf. Control, vol. 2, no. 2, pp. 101-112, 1959.
27. H. L. Van Trees, Detection, Estimation, and Modulation Theory, Part I. New York: Wiley, 1968.
28. L. D. Brown and L. Gajek, "Information inequalities for the Bayes risk," Ann. Stat., vol. 18, no. 4, pp. 1578-1594, 1990.
29. E. Lehmann and G. Casella, Theory of Point Estimation. New York: Springer, 1998.
30. D. L. Donoho, A. Maleki, and A. Montanari, "Message-passing algorithms for compressed sensing," Proc. Natl. Acad. Sci., vol. 106, no. 45, pp. 18 914-18 919, Nov. 2009.
31. D. Guo, D. Baron, and S. Shamai, "A single-letter characterization of optimal noisy compressed sensing," in Proc. 47th Annu. Allerton Conf. Commun., Control, Comput., Monticello, IL, Oct. 2009.
32. C. Weidmann and M. Vetterli, "Rate distortion behavior of sparse sources," IEEE Trans. Inf. Theory, Aug. 2010, submitted for publication.
33. A. Lozano, A. M. Tulino, and S. Verdú, "Optimum power allocation for parallel Gaussian channels with arbitrary input distributions," IEEE Trans. Inf. Theory, vol. 52, no. 7, pp. 3033-3051, Jul. 2006. (Pubitemid 44192966)
34. W. Rudin, Principles of Mathematical Analysis, 3rd ed. New York: McGraw-Hill, 1976.
35. K. Knopp and F. Bagemihl, Theory of Functions, Parts I and II. Mineola, NY: Dover, 1996.
36. R. Zamir, "A proof of the Fisher information inequality via a data processing argument," IEEE Trans. Inf. Theory, vol. 44, no. 3, pp. 1246-1250, Aug. 1998. (Pubitemid 128743284)
37. Y. Wu and S. Verdú, "Optimal phase transitions in compressed sensing," IEEE Trans. Inf. Theory, Jun. 2011, submitted for publication.
38. S. Verdú, "Mismatched estimation and relative entropy," IEEE Trans. Inf. Theory, vol. 56, no. 8, pp. 3712-3720, Jul. 2010.
39. T. Weissman, "The relationship between causal and noncausal mismatched estimation in continuous-time AWGN channels," IEEE Trans. Inf. Theory, vol. 56, no. 9, pp. 4256-4273, Aug. 2010.
40. P. Mattila, Geometry of Sets and Measures in Euclidean Spaces: Fractals and Rectifiability. Cambridge, U.K.: Cambridge Univ. Press, 1999.
41. L. C. Evans and R. F. Gariepy, Measure Theory and Fine Properties of Functions. Boca Raton, FL: CRC, 1992.
42. J. Doob, "Relative limit theorems in analysis," Journal d'Analyse Mathěmatique, vol. 8, no. 1, pp. 289-306, 1960.
43. E. Stein and R. Shakarchi, Real Analysis: Measure Theory, Integration, and Hilbert Spaces. Princeton, NJ: Princeton Univ. Press, 2005.
44. G. Folland, Real Analysis: Modern Techniques and Their Applications, 2nd ed. New York: Wiley-Interscience, 1999.
45. V. Prokaj, "A characterization of singular measures," Real Analysis Exchange, vol. 29, pp. 805-812, 2004. (Pubitemid 39337460)