The empirical distribution of good codes

IEEE Transactions on Information Theory

Volumn 43, Issue 3, 1997, Pages 836-846

  Shamai, Shlomo ^a,b   Verdú, Sergio ^a,c  

^a IEEE   (Israel)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^c Princeton University   (United States)

Author keywords

Approximation of output statistics; Channel capacity; Discrete memoryless channels; Divergence; Error correcting codes; Gaussian channels; Shannon theory

Indexed keywords

APPROXIMATION THEORY; CHANNEL CAPACITY; CODING ERRORS; COMMUNICATION CHANNELS (INFORMATION THEORY); ERROR CORRECTION; STATISTICS;

DISCRETE MEMORYLESS CHANNELS;

CODES (SYMBOLS);

EID: 0031142175     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.568695     Document Type: Article

References (16)

1. I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.
2. C. E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J., pp. 379-423, 623-656, July-Oct. 1948.
3. T. S. Han and S. Verdú, "Approximation theory of output statistics," IEEE Trans. Inform. Theory, vol. 39, pp. 752-772, May 1993.
4. T. S. Han and S. Verdú, "Spectrum invariancy under output approximation for discrete memoryless channels with full rank," Probl. Pered. Inform., vol. 2, pp. 9-27, 1993.
5. S. Verdú and T. S. Han, "A general formula for channel capacity," IEEE Trans. Inform. Theory, vol. 40, pp. 1147-1157, July 1994.
6. S. Kullback, J. C. Keegel, and J. H. Kullback, Topics in Statistical Information Theory (Lecture Notes in Statistics), vol. 42. Berlin, Germany: Springer, 1987.
7. I. Csiszár, "Sanov property, generalized i-projection and a conditional limit theorem," Ann. Probab., pp. 768-793, Aug. 1984.
8. R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.
9. R. E. Blahut, Principles and Practice of Information Theory. Reading, MA: Addison-Wesley, 1987.
10. Y. Steinberg and S. Verdú, "Simulation of random processes and rate-distortion theory," IEEE Trans. Inform. Theory, vol. 42, pp. 63-86, Jan. 1996.
11. E. M. Gabidulin, "Limits for the decoding error probability when linear codes are used in memoryless channels," Probl. Pered. Inform., vol. 2, pp. 55-62, 1967.
12. S. Shamai (Shitz) and S. Verdú, "Capacity of channels with side information," European Trans. Telecomm., vol. 6, pp. 587-600, Sept.-Oct. 1995.
13. J. Wolfowitz, Coding Theorems of Information Theory, 3rd ed. New York: Springer, 1978.
14. R. M. Gray, D. L. Neuhoff, and P. C. Shields, "A generalization of Ornstein's d-bar distance with applications to information theory," Ann. Probab., pp. 315-328, 1975.
15. M. Talagrand, "Transportation cost for Gaussian and other product measures," preprint, 1995.
16. K. L. Chung, A Course in Probability Theory. New York: Academic, 1974.