Fifty Years of Shannon Theory

IEEE Transactions on Information Theory

Volumn 44, Issue 6, 1998, Pages 2057-2078

  Verdü, Sergio ^a  

^a Princeton University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CHANNEL CAPACITY; DATA COMPRESSION; ENCODING (SYMBOLS); HISTORY; TELECOMMUNICATION;

SHANNON THEORY; SOURCE CODING;

INFORMATION THEORY;

EID: 0032183117     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.720531     Document Type: Article

References (443)

1. C. E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J.. vol. 27, pp. 379-423, 623-656, July-Oct. 1948.
2. H. Dudley, "The vocoder," Bell Labs. Rec., vol. 18, pp. 122-126, Dec. 1939.
3. H. Nyquist, "Certain factors affecting telegraph speed," Bell Syst. Tech. J., vol. 3, pp. 324-352, Apr. 1924.
4. K. Küpfmüller, "Über Einschwingvorgange in Wellenfiltern," Elek. Nachrichtentech., vol. 1, pp. 141-152, Nov. 1924.
5. H. Nyquist, "Certain topics in telegraph transmission theory," AIEE Trans., vol. 47, pp. 617-644, Apr. 1928.
6. V. A. Kotel'nikov, "On the transmission capacity of "ether" and wire in electrocommunications," Izd. Red. Upr. Svyazi RKKA (Moscow, USSR) (material for the first all-union conference on questions of communications), vol. 44, 1933.
7. E. T. Whittaker, "On the functions which are represented by the expansion of interpolating theory," in Proc. Roy. Soc. Edinburgh, vol. 35, pp. 181-194, 1915.
8. J. M. Whittaker, "The Fourier theory of the cardinal functions," Proc. Math. Soc. Edinburgh, vol. 1, pp. 169-176, 1929.
9. D. Gabor, "Theory of communication," J. Inst. Elec. Eng., vol. 93, pp. 42957, Sept. 1946.
10. R. V. L. Hartley, "Transmission of information," Bell Syst. Tech. J., vol. 7, pp. 535-563, July 1928.
11. N. Wiener, Extrapolation, Interpolation and Smoothing of Stationary Time Series. New York: Wiley, 1949.
12. S. O. Rice, "Mathematical analysis of random noise," Bell Syst. Tech. J., vol. 23-24, pp. 282-332 and 46-156, July 1944 and Jan. 1945.
13. F. Pratt, Secret and Urgent. Blue Ribbon Books, 1939.
14. C. E. Shannon, "A mathematical theory of cryptography," Tech. Rep. MM 45-110-02, Bell Labs. Tech. Memo., Sept. 1, 1945.
15. "Communication theory of secrecy systems," Bell Syst. Tech. J., vol. 28, pp. 656-715, Oct. 1949.
16. R. Price, "A conversation with Claude Shannon," IEEE Commun. Mag., vol. 22, pp. 123-126, May 1984.
17. J. R. Pierce, "The early days of information theory," IEEE Trans. Inform. Theory, vol. IT-19, pp. 3-8, Jan. 1973.
18. F. W. Hagemeyer, "Die Entstehung von Informationskonzepten in der Nachrichtentechnik: eine Fallstudie zur Theoriebildung in der Technik in Industrie- und Kriegsforschung," Ph.D. dissertation, Free University of Berlin, Berlin, Germany, 1979.
19. C. E. Shannon, "Letter to Vannevar Bush Feb. 16, 1939," in Claude Elwood Shannon Collected Papers, N. J. A. Sloane and A. D. Wyner, Eds. Piscataway, NJ: IEEE Press, pp. 45556, 1993.
20. "A symbolic analysis of relay and switching circuits," M.S. thesis, MIT, Cambridge, MA, 1938.
21. "An algebra for theoretical genetics," Ph.D. dissertation, MIT, Cambridge, MA, Apr. 15, 1940.
22. A. G. Clavier, "Evaluation of transmission efficiency according to Hartley's expression of information content," Elec. Commun.: ITT Tech. J., vol. 25, pp. 41420, June 1948.
23. C. W. Earp, "Relationship between rate of transmission of information, frequency bandwidth, and signal-to-noise ratio," Elec. Commun.: ITT Tech. J., vol. 25, pp. 178-195, June 1948.
24. S. Goldman, "Some fundamental considerations concerning noise re-duction and range in radar and communication," Proc. Inst. Elec. Eng., vol. 36, pp. 584-594, 1948,
25. J. Laplume, "Sur le nombre de signaux discernables en présence de bruit erratique dans un système de transmission àbande passante limitée," Comp. Rend. Acad. Sci. Paris, pp. 1348-, 1948.
26. W. G. Tuller, "Theoretical limitations on the rate of transmission of information," Ph.D. dissertation, MIT, Cambridge, MA, June 1948, published in Proc. IRE, pp. 46878, May 1949
27. N. Wiener, Cybernetics, Chaper III: Time Series, Information and Com-munication. New York: Wiley, 1948.
28. R. A. Fisher, "Probability, likelihood and quantity of information in the logic of uncertain inference," Proc. Roy. Soc. London, A, vol. 146, pp. 1-8, 1934.
29. L. Boltzmann, "Beziehung zwischen dem Zweiten Hauptsatze der Mech-anischen Waermertheorie und der Wahrscheilichkeitsrechnung Respek-tive den Saetzen über das Waermegleichgwicht," Wien. Ber., vol. 76, pp. 37335, 1877.
30. N. Chomsky, "Three models for the description of language," IEEE Trans. Inform. Theory, vol. IT-2, pp. 113-124, Sept. 1956.
31. A. I. Khinchin, "The entropy concept in probability theory," Usp. Mat. Nauk., vol. 8, pp. 3-20, 1953,
32. English translation in Mathematical Foundations of Information Theory. New York: Dover, 1957.
33. B. McMillan, "The basic theorems of information theory," Ann. Math. Statist., vol. 24, pp. 196-219, June 1953.
34. A. N. Kolmogorov, "A new metric invariant of transitive dynamical systems and automorphism in Lebesgue spaces," Dokl. Akad. Nauk. SSSR, vol. 119, pp. 861-864, 1958.
35. D. S. Ornstein, "Bernoulli shifts with the same entropy are isomorphic," Adv. Math., vol. 4, pp. 337-352, 1970.
36. R. M. Gray and L. D. Davisson, Eds., Ergodic and Information Theory. Stroudsbourg, PA: Dowden, Hutchinson & Ross, 1977.
37. I. Csiszâr, "Information theory and ergodic theory," Probl. Contr. Inform. Theory, vol. 16, pp. 3-27, 1987.
38. P. Shields, "The interactions between ergodic theory and information theory," this issue, pp. 2079-2093.
39. L. Breiman, "The individual ergodic theorems of information theory," Ann. Math. Statist., vol. 28, pp. 809-811, 1957.
40. P. Algoet and T. M. Cover, "A sandwich proof of the Shannon-McMillan-Breiman theorem," A««. Probab., vol. 16, pp. 899-909, 1988.
41. S. C. Moy, "Generalizations of the Shannon-McMillan theorem," Pa-cific J. Math., vol. 11, pp. 705-714, 1961.
42. K. Marton, "Information and information stability of ergodic sources," Probl. Inform. Transm., vol. 8, pp. 179-183, 1972.
43. J. C. Kieffer, "A simple proof of the Moy-Perez generalization of the Shannon-McMillan theorem," Pacific J. Math., vol. 351, pp. 203-206, 1974.
44. D. S. Ornstein and B. Weiss, "The Shannon-McMillan-Breiman theo-rem for amenable groups," Israel J. Math., vol. 39, pp. 53-60, 1983.
45. A. R. Barron, "The strong ergodic theorem for densities: General-ized Shannon-McMillan-Breiman theorem," Ann. Probab., vol. 13, pp. 1292-1303, 1985.
46. S. Orey, "On the Shannon-Perez-Moy theorem," Contemp. Math., vol. 41, pp. 319-327, 1985.
47. T. S. Han and S. Verdti, "Approximation theory of output statistics," IEEE Trans. Inform. Theory, vol. 39, pp. 752-772, May 1993.
48. S. Verdti and T. S. Han, "The role of the asymptotic equipartition property in noiseless source coding," IEEE Trans. Inform. Theory, vol. 43, pp. 847-857, May 1997.
49. D. Huffman, "A method for the construction of minimum redundancy codes," Proc. IRE, vol. 40, pp. 1098-1101, Sept. 1952.
50. I. S. Reed, "1982 Claude Shannon lecture: Application of transforms to coding and related topics," IEEE Inform. Theory Newslett., pp. 4-7, Dec. 1982.
51. R. Hunter and A. Robinson, "International digital facsimile coding standards," Proc. Inst. Elec. Radio Eng., vol. 68, pp. 854-867, July 1980.
52. K. Challapali, X. Lebegue, J. S. Lim, W. Paik, R. Girons, E. Petajan, V. Sathe, P. Snopko, and J. Zdepski, "The grand alliance system for US HDTV," Proc. IEEE , vol. 83, pp. 158-174, Feb. 1995.
53. R. G. Gallager, "Variations on a theme by Huffman," IEEE Trans. Inform. Theory, vol. IT-24, pp. 668-674, Nov. 1978.
54. R. M. Capocelli and A. DeSantis, "Variations on a theme by Gallager," in Image and Text Compression, J. A. Storer, Ed. Boston, NA: Kluwer, pp. 181-213, 1992.
55. L. Kraft, "A device for quantizing, grouping and coding amplitude modulated pulses," M.S. Thesis, Dept. Elec. Eng., MIT, Cambridge, MA, 1949.
56. I. S. Sokolnikoff and R. M. Redheffer, Mathematics of Physics and Modern Engineering, 2d ed. New York: McGraw-Hill, 1966.
57. B. McMillan, "Two inequalities implied by unique decipherability," IRE Trans. Inform. Theory, vol. IT-2, pp. 115-116, Dec. 1956.
58. J. Karush, "A simple proof of an inequality of McMillan," IEEE Trans. Inform. Theory, vol. IT-7, pp. 118, Apr. 1961.
59. J. Abrahams, "Code and parse trees for lossless source encoding," in Proc. Compression and Complexity of Sequences 1997, B. Carpentieri, A. De Santis, U. Vaccaro, and J. Storer, Eds. Los Alamitos, CA: IEEE Comp. Soc., 1998, pp. 145-171.
60. R. E. Krichevsky, Universal Compression and Retrieval. Dordrecht, The Netherlands: Kluwer, 1994.
61. J. Rissanen, "Generalized Kraft inequality and arithmetic coding," IBM J. Res. Devel, vol. 20, pp. 198-203, 1976.
62. R. Pasco, "Source coding algorithms for fast data compression," Ph.D. dissertation, Stanford Univ., Stanford, CA, 1976.
63. J. Rissanen and G. G. Langdon, "Arithmetic coding," IBM J. Res. Develop., vol. 23, pp. 149-162, Mar. 1979.
64. I. H. Witten, R. M. Neal, and J. G. Cleary, "Arithmetic coding for data compression," Commun. Assoc. Comp. Mach., vol. 30, pp. 520-540, June 1987
65. N. Abramson, Information Theory and Coding. New York: McGraw-Hill, 1963.
66. F. Jelinek, Probabilistic Information Theory. New York: McGraw-Hill, 1968.
67. P. Elias, personal communication, Apr. 22, 1998.
68. E. Gilbert and E. Moore, "Variable-length binary encodings," Bell Syst. Tech. J., vol. 38, pp. 933-967, 1959.
69. T. M. Cover, "Enumerative source coding," IEEE Trans. Inform. Theory, vol. IT-19, pp. 73-77, 1973.
70. R. Aravind, G. Cash, D. Duttweiler, H. Hang, B. Haskell, and A. Puri, "Image and video coding Standards," ATT Tech. J., vol. 72, pp. 67-89, Jan.-Feb. 1993.
71. B. P. Tunstall, "Synthesis of noiseless compression codes," Ph.D. dissertation, Georgia Inst. Technol., Atlanta, GA, Sept. 1967.
72. G. L. Khodak, "The estimation of redundancy for coding the messages generated by a Bernoulli source," Probl. Inform. Transm., vol. 8, pp. 28-32, 1972.
73. F. Jelinek and K. Schneider, "On variable-length to block coding," IEEE Trans. Inform. Theory, vol. IT-18, pp. 765-774, Nov. 1972.
74. T. J. Tjalkens and F. M. J. Willems, "Variable to fixed-length codes for Markov sources," IEEE Trans. Inform. Theory, vol. IT-33, pp. 246-257, Mar. 1987.
75. S. Savari and R. G. Gallager, "Generalized Tunstall codes for sources with memory," IEEE Trans. Inform. Theory, vol. 43, pp. 658-668, Mar. 1997.
76. J. Ziv, "Variable-to-fixed length codes are better than fixed-to-variable length codes for Markov sources," IEEE Trans. Inform. Theory, vol. 36, pp. 861-863, July 1990.
77. N. Merhav and D. L. Neuhoff, "Variable-to-fixed length codes provide better large deviations performance than fixed-to-variable length codes," IEEE Trans. Inform. Theory, vol. 38, pp. 135-140, Jan. 1992.
78. R. E. Krichevsky and V. K. Trofimov, "The performance of universal encoding," IEEE Trans. Inform. Theory, vol. IT-27, pp. 199-206, Mar. 1981.
79. S. W. Golomb, "Run length encodings," IEEE Trans. Inform. Theory, vol. IT-12, pp. 399-401, July 1966.
80. C. E. Shannon, "Efficient coding of a binary source with one very infrequent symbol," Bell Labs. Memo. Jan. 29, 1954, in Claude Elwood Shannon Collected Papers, N. J. A. Sloane and A. D. Wyner, Eds. Piscataway, NJ: IEEE Press, 1993, pp. 455156.
81. A. N. Kolmogorov, "Three approaches to the quantitative definition of information," Probl. Inform. Transm., vol. 1, pp. 1-7, 1965.
82. E. Gilbert, "Codes based on inaccurate source probabilities," IEEE Trans. Inform. Theory, vol. IT-17, pp. 304-314, May 1971.
83. F. Fabris, A. Sgarro, and R. Pauletti, "Tunstall adaptive coding and miscoding," IEEE Trans. Inform. Theory, vol. 42, pp. 2167-2180, Nov. 1996.
84. B. M. Fitingof, "Optimal encoding with unknown and variable message statistics," Probl. Inform. Transm., vol. 2, pp. 3-11, 1966.
85. L. Davisson, "Universal noiseless coding," IEEE Trans. Inform. Theory, vol. IT-19, pp. 783-795, Nov. 1973.
86. T. J. Tjalkens and F. M. J. Willems, "A universal variable-to-fixed length source code based on Lawrence's algorithm," IEEE Trans. Inform. Theory, vol. 38, pp. 247-253, Mar. 1992.
87. N. Faller, "An adaptive system for data compression," in 7th Asilomar Conf. on Circuits, Systems, and Computing, 1973, pp. 593-597.
88. D. E. Knuth, "Dynamic Huffman coding," J. Algorithms, vol. 1985, pp. 163-180, 1985.
89. J. S. Vitter, "Dynamic Huffman coding," ACM Trans. Math. Software, vol. 15, pp. 158-167, June 1989.
90. B. Ryabko, "A fast on-line adaptive code," IEEE Trans. Inform. Theory, vol. 38, pp. 1400-1404, July 1992.
91. "Data compression by means of a book stack," Probl. Inform. Transm., vol. 16, pp. 265-269, 1980.
92. J. Bentley, D. Sleator, R. Tarjan, and V. Wei, "Locally adaptive data compression scheme," Commun. Assoc. Comp. Mach., vol. 29, pp. 320-330, 1986.
93. P. Elias, "Interval and recency rank source coding: Two on-line adaptive variable-length schemes," IEEE Trans. Inform. Theory, vol. IT-33, pp. 3-10, Jan. 1987
94. M. Burrows and D. J. Wheeler, "A block-sorting lossless data compres-sion algorithm," Tech. Rep. 124, Digital Systems Res. Ctr., Palo Alto, CA, May 10, 1994.
95. J. Rissanen, "Universal coding, information, prediction, and estimation," IEEE Trans. Inform. Theory, vol. IT-30, pp. 629-636, July 1984.
96. A. Barron, J. Rissanen, and B. Yu, "The minimum description length principle in coding and modeling," this issue, pp. 2743-2760.
97. A. Lempel and J. Ziv, "On the complexity of an individual sequence," IEEE Trans. Inform. Theory, vol. IT-22, pp. 75-81, Jan. 1976.
98. J. Ziv and A. Lempel, "A universal algorithm for sequential data compression," IEEE Trans. Inform. Theory, vol. IT-24, pp. 337-343, May 1977.
99. "Compression of individual sequences via variable-rate coding," IEEE Trans. Inform. Theory, vol. IT-24, pp. 530-536, Sept. 1978.
100. T. A. Welch, "A technique for high performance data compression," Computer, vol. 17, pp. 8-19, June 1984.
101. J. Ziv, "Coding theorems for individual sequences," IEEE Trans. Inform. Theory, vol. IT-24, pp. 40512, July 1978.
102. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
103. A. D. Wyner and J. Ziv, "The sliding-window Lempel-Ziv algorithm is asymptotically optimal," Proc. IEEE, vol. 82, pp. 872-877, June 1994.
104. F. M. J. Willems, "Universal data compression and repetition times," IEEE Trans. Inform. Theory, vol. 35, pp. 54-58, Jan. 1989.
105. A. D. Wyner and J. Ziv, "Some asymptotic properties of the entropy of a stationary ergodic data source with applications to data compression," IEEE Trans. Inform. Theory, vol. 35, pp. 1250-1258, Nov. 1989.
106. D. S. Ornstein and B. Weiss, "Entropy and data compression schemes," IEEE Trans. Inform. Theory, vol. 39, pp. 78-83, Jan. 1993.
107. W. Szpankowski, "Asymptotic properties of data compression and suffix trees" IEEE Trans. Inform. Theory, vol. 39, pp. 1647-1659, Sept. 1993.
108. P. Jacquet and W. Szpankowski, "Asymptotic behavior of the Lem-pel-Ziv parsing scheme and digital search trees," Theor. Comp. Sci., vol. 144, pp. 161-197, June 1995.
109. A. D. Wyner, J. Ziv, and A. J. Wyner, "On the role of pattern matching in information theory," this issue, pp. 2045-2056.
110. J. Rissanen, "A universal data compression system," IEEE Trans. Inform. Theory, vol. IT-29, pp. 656-663, Sept. 1983.
111. F. M. J. Willems, Y. M. Shtarkov, and T. J. Tjalkens, "The context-tree weighting method: Basic properties," IEEE Trans. Inform. Theory, vol. IT-28, pp. 653-664, May 1995.
112. N. Merhav and M. Feder, "Universal prediction," this issue, pp. 2124-2147.
113. V.l. Levenshtein, "On the redundancy and delay of decodable coding of natural numbers," Probl. Cybern., vol. 20, pp. 173-179, 1968.
114. P. Elias, "Universal codeword sets and representation of the integers," IEEE Trans. Inform. Theory, vol. IT-21, pp. 194-203, Mar. 1975.
115. Q. F. Stout, "Improved prefix encodings of the natural numbers," IEEE Trans. Inform. Theory, vol. IT-26, pp. 607-609, Sept. 1980.
116. R. Ahlswede, T. S. Han, and K. Kobayashi, "Universal coding of integers and unbounded search trees," IEEE Trans. Inform. Theory, vol. 43, pp. 669-682, Mar. 1997.
117. C. E. Shannon, "Prediction and entropy of printed English," Bell Syst. Tech. J., vol. 30, pp. 47-51, Jan. 1951.
118. T. M. Cover and R. C. King, "A convergent gambling estimate of the entropy of English," IEEE Trans. Inform. Theory, vol. IT-24, pp. 413-421, July 1978.
119. L. Levitin and Z. Reingold, "Entropy of natural languages-Theory and practice," Chaos, Solitons and Fractals, vol. 4, pp. 709-743, May 1994.
120. P. Grassberger, "Estimating the information content of symbol se-quences and efficient codes," IEEE Trans. Inform. Theory, vol. 35, pp. 669-675, May 1989.
121. P. Hall and S. Morton, "On the estimation of entropy," A««. Inst. Statist. Math., vol. 45, pp. 69-88, Mar. 1993.
122. I. Kontoyiannis, P. Algoet, Y. Suhov, and A. J. Wyner, "Nonparametric entropy estimation for stationary processes and random fields, with applications to English text," IEEE Trans. Inform. Theory, vol. 44, pp. 1319-1327, May 1998.
123. A. N. Kolmogorov, "Logical basis for information theory and probability theory," IEEE Trans. Inform. Theory, vol. IT-14, pp. 662-664, 1968.
124. G. J. Chaitin, "On the length of programs for computing binary sequences," J. Assoc. Comput. Mach., vol. 13, pp. 547-569, 1966.
125. R. J. Solomonoff, "A formal theory of inductive inference," Inform. Contr., vol. 7, pp. 1-22, 224-254, 1964.
126. T. M. Cover, P. Gacs, and R. M. Gray, "Kolmogorov's contributions to information theory and algorithmic complexity," Ann. Probab., vol. 17, pp. 840-865, July 1989.
127. D. Slepian and J. K. Wolf, "Noiseless coding of correlated information sources," IEEE Trans. Inform. Theory, vol. IT-19, pp. 471180, 1973.
128. T. M. Cover, "A proof of the data compression theorem of Slepian and Wolf for ergodic sources," IEEE Trans. Inform. Theory, vol. IT-22, pp. 226-228, Mar. 1975.
129. A. D. Wyner, "Recent results in the Shannon theory," IEEE Trans. Inform. Theory, vol. IT-20, pp. 2-9, Jan. 1974.
130. S. Shamai (Shitz) and S. Verdti, "Capacity of channels with side information," Euro. Trans. Telecommun., vol. 6, no. 5, pp. 587-600, Sept.-Oct. 1995.
131. J. Körner and A. Orlitsky, "Zero-error information theory," this issue, pp. 2207-2229.
132. H. S. Witsenhausen, "The zero-error side information problem and chromatic numbers," IEEE Trans. Inform. Theory, vol. IT-22, pp. 592-593, Sept. 1976.
133. R. Ahlswede, "Coloring hypergraphs: A new approach to multiuser source coding," J. Comb. Inform. Syst. Sci., vol. 4, pp. 76-115, 1979; Part II,
134. vol. 5, pp. 220-268, 1980
135. A. C. Yao, "Some complexity questions related to distributive comput-ing," in P roc. llth Assoc. Comp. Mach. Symp. Theory of Computing, 1979, pp. 209-213.
136. A. Orlitsky, "Worst-case interactive communication. I: Two messages are almost optimal," IEEE Trans. Inform. Theory, vol. 36, pp. 1534-1547, Sept. 1990.
137. "Average case interactive communication," IEEE Trans. Inform. Theory, vol. 38, pp. 1534-1547, Sept. 1992.
138. M. Naor, A. Orlitsky, and P. Shor, "Three results on interactive communication," IEEE Trans. Inform. Theory, vol. 39, pp. 1608-1615, Sept. 1993.
139. R. Ahlswede, N. Cai, and Z. Zhang, "On interactive communication," IEEE Trans. Inform. Theory, vol. 43, pp. 22-37, Jan. 1997.
140. A. R. Calderbank, "The art of signaling: Fifty years of coding theory," this issue, pp. 2561-2595.
141. A. Liversidge, "Profile of Claude Shannon," Omni magazine, Aug. 1987, in Claude Elwood Shannon Collected Papers, N. J. A. Sloane and A. D. Wyner, Eds. Piscataway, NJ: IEEE Press, 1993, pp. xix-xxxiii.
142. C. E. Shannon, "The zero error capacity of a noisy channel," IRE Trans. Inform. Theory, vol. IT-2, pp. 112-124, Sept. 1956.
143. J. G. Kreer, "A question of terminology," IEEE Trans. Inform. Theory, vol. IT-3, pp. 208, Sept. 1957.
144. I. M. Gelfand, A. N. Kolmogorov, and A. M. Yaglom, "On the general definition of mutual information," Repts. Acad. Sci. USSR, vol. 3, pp. 745-748, 1956.
145. I. M. Gelfand and A. M. Yaglom, "On the computation of the mutual information between a pair of random functions," Adv. Math. Sci., vol. 12, pp. 3-52, 1957
146. I. M. Gelfand, A. N. Kolmogorov, and A. M. Yaglom, "Mutual infor-mation and entropy for continuous distributions," in Proc. 3rd All Union Math. Congr., 1958, vol. 3, pp. 521-531
147. M. S. Pinsker, Information and Information Stability of Random Vari-ables and Processes. San Francisco, CA: Holden-Day, 1964, originally published in Russian in 1960.
148. S. Muroga, "On the capacity of a discrete channel," J. Phys. Soc. Japan, vol. 8, pp. 48494, 1953.
149. C. E. Shannon, "Some geometrical results in channel capacity," Verband Deut. Elektrotechnik. Fachber., vol. 19, pp. 13-15, 1956.
150. S. Arimoto, "An algorithm for computing the capacity of arbitrary discrete memoryless channels," IEEE Trans. Inform. Theory, vol. IT-18, pp. 14-20, Jan. 1972.
151. R. E. Blahut, "Computation of channel capacity and rate-distortion functions," IEEE Trans. Inform. Theory, vol. IT-18, pp. 460173, July 1972.
152. C. E. Shannon, "Communication in the presence of noise," Proc. IRE, vol. 37, pp. 10-21, Jan. 1949.
153. A. J. Jerri, "The Shannon sampling theorem-Its various extensions and applications: A tutorial review," Proc. IEEE, vol. 65, pp. 1565-1596, Nov. 1977.
154. R. J. Marks, Introduction to Shannon Sampling and Interpolation Theory. New York: Springer, 1991.
155. M. J. E. Golay, "Note on the theoretical efficiency of information reception with PPM," Proc. IRE, vol. 37, p. 1031, Sept. 1949.
156. C. E. Shannon, "General treatment of the problem of coding," IRE Trans. Inform. Theory, vol. PGIT-1, pp. 102-104, Feb. 1953.
157. D. Slepian, "Information theory in the fifties" (Invited Paper), IEEE Trans. Inform. Theory, vol. IT-19, pp. 145-147, Mar. 1973.
158. D. Slepian, "On bandwidth," Proc. IEEE, vol. 64, pp. 292-300, Mar. 1976.
159. A. D. Wyner, "The capacity of the band-limited Gaussian channel," Bell Syst. Tech. J., vol. 45, pp. 359-371, Mar. 1966.
160. H. J. Landau, D. Slepian, and H. O. Pollack, "Prolate spheroidal wave functions, Fourier analysis and uncertainty-III: The dimension of the space of essentially time- and band-limited signals," Bell Syst. Tech. J., vol. 41, pp. 1295-1336, July 1962.
161. S. Shamai (Shitz) and S. Verdti, "The empirical distribution of good codes," IEEE Trans. Inform. Theory, vol. 43, pp. 836-846, May 1997.
162. J. L. Holsinger, "Digital communication over fixed time-continuous channels with memory with special application to telephone channels," Tech. Rep. 430, MIT Res. Lab. Electron., Cambridge, MA, 1964.
163. R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.
164. B. S. Tsybakov, "Capacity of a discrete-time Gaussian channel with a filter," Probl. Inform. Transm., vol. 6, pp. 253-256, July-Sept. 1970.
165. U. Grenander and G. Szegö, Toeplitz Forms and Their Applications. New York: Chelsea, 1958.
166. R. M. Gray, "On the asymptotic eigenvalue distribution of toeplitz matrices," IEEE Trans. Inform. Theory, vol. IT-18, pp. 725-730, Nov. 1972.
167. W. Hirt and J. L. Massey, "Capacity of the discrete-time Gaussian channel with intersymbol interference," IEEE Trans. Inform. Theory, vol. 34, pp. 380-388, May 1988.
168. J. G. Smith, "The information capacity of amplitude- and variance-constrained scalar Gaussian channels," Inform. Contr., vol. 18, pp. 203-219, 1971.
169. S. Shamai (Shitz) and I. Bar-David, "The capacity of average and peak-power-limited quadrature Gaussian channels," IEEE Trans. Inform. Theory, vol. 41, pp. 1060-1071, July 1995.
170. R. E. Blahut, Principles and Practice of Information Theory. Reading, MA: Addison-Wesley, 1987.
171. G. D. Forney and G. Ungerboeck, "Modulation and coding for linear Gaussian channels," this issue, pp. 2384-2415.
172. N. M. Blachman, "Communication as a game," in IRE Wescon Rec., 1957, vol. 2, pp. 61-66.
173. J. M. Borden, D. M. Mason, and R. J. McEliece, "Some information theoretic saddlepoints," SIAM J. Contr. Optimiz., vol. 23, pp. 129-143, Jan. 1985.
174. S. Shamai (Shitz) and S. Verdû, "Worst-case power constrained noise for binary-input channels," IEEE Trans. Inform. Theory, vol. 38, pp. 1494-1511, Sept. 1992.
175. S. Ihara, "On the capacity of channels with additive non-Gaussian noise," Inform. Contr., vol. 37, pp. 34-39, 1978.
176. A. Stam, "Some inequalities satisfied by the quantities of information of Fisher and Shannon," Inform. Contr., vol. 2, pp. 101-112, 1959.
177. V. Prelov, "Asymptotic behavior of a continuous channel with small additive noise," Probl. Inform. Transm., vol. 4, no. 2, pp. 31-37, 1968.
178. "Asymptotic behavior of the capacity of a continuous channel with large noise," Probl. Inform. Transm., vol. 6 no. 2, pp. 122-135, 1970.
179. "Communication channel capacity with almost Gaussian noise," Theory Probab. Its Applications, vol. 33, no. 2, pp. 40522, 1989.
180. M. Pinsker, V. Prelov, and S. Verdti, "Sensitivity of channel capacity," IEEE Trans. Inform. Theory, vol. 41, pp. 1877-1888, Nov. 1995.
181. E. Biglieri, J. Proakis, and S. Shamai, "Fading channels: Information-theoretic and communications aspects," this issue, pp. 2619-2692.
182. R. M. Fano, "Class notes for course 6.574: Transmission of informa-tion," MIT, Cambridge, MA, 1952.
183. S. Vembu, S. Verdü, and Y. Steinberg, "The source-channel separation theorem revisited," IEEE Trans. Inform. Theory, vol. 41, pp. 44-54, Jan. 1995.
184. J. Wolfowitz, "The coding of messages subject to chance errors," Illinois J. Math., vol. 1, pp. 591-606, Dec. 1957.
185. A. Feinstein, "A new basic theorem of information theory," IRE Trans. Inform. Theory, vol. PGIT-4, pp. 2-22, 1954.
186. T. T. Kadota, "Generalization of Feinstein's fundamental lemma," IEEE Trans. Inform. Theory, vol. IT-16, pp. 791-792, Nov. 1970.
187. I. Csiszâr and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.
188. I. Csiszar, "The method of types," this issue, pp. 2505-2523.
189. A. El-Gamal and T. M. Cover, "Multiple user information theory," Proc. IEEE, vol. 68, pp. 1466-1483, Dec. 1980.
190. C. E. Shannon, "Certain results in coding theory for noisy channels," Inform. Contr., vol. 1, pp. 6-25, Sept. 1957.
191. R. G. Gallager, "A simple derivation of the coding theorem and some applications," IEEE Trans. Inform. Theory, vol. IT-11, pp. 3-18, Jan. 1965.
192. A. I. Khinchin, "On the fundamental theorems of information theory," Usp. Mat. Nauk., vol. 11, pp. 17-75, 1956,
193. English translation in Mathematical Foundations of Information Theory. New York: Dover, 1957.
194. I. P. Tsaregradsky, "On the capacity of a stationary channel with finite memory," Theory Probab. Its Applications, vol. 3, pp. 84-96, 1958.
195. A. Feinstein, "On the coding theorem and its converse for finite-memory channels," Inform. Contr., vol. 2, pp. 2544, 1959.
196. R. L. Dobrushin, "General formulation of Shannon's main theorem in information theory," Amer. Math. Soc. Transi., vol. 33, pp. 323138, 1963.
197. V. Anantharam and S. Verdü, "Bits through queues," IEEE Trans. Inform. Theory, vol. 42, pp. 4-18, Jan. 1996.
198. A. S. Bedekar and M. Azizoglu, "The information-theoretic capacity of discrete-time queues," IEEE Trans. Inform. Theory, vol. 44, pp. 44661, Mar. 1998.
199. S. Verdü and T. S. Han, "A general formula for channel capacity," IEEE Trans. Inform. Theory, vol. 40, pp. 1147-1157, July 1994.
200. T. S. Han, Information Spectrum Methods in Information Theory. Tokyo, Japan: Baifukan, 1998, in Japanese.
201. S. Verdü, "On channel capacity per unit cost," IEEE Trans. Inform. Theory, vol. 36, pp. 1019-1030, Sept. 1990
202. R. L. Dobrushin, "Shannon's theorems for channels with synchroniza-tion errors," Probl. Inform. Transm., vol. 3, pp. 11-26, 1967.
203. S. Z. Stambler, "Memoryless channels with synchronization er-rors-General case," Probl. Inform. Transm., vol. 6, no. 3, pp. 4319, 1970.
204. K. A. S. Immink, P. Siegel, and J. K. Wolf, "Codes for digital recorders," this issue, pp. 2260-2299.
205. L. Lovâsz, "On the Shannon capacity of a graph," IEEE Trans. Inform. Theory, vol. IT-25, pp. 1-7, Jan. 1979.
206. N. Alon, "The Shannon capacity of a union," Combinatorica, to be published.
207. S. O. Rice, "Communication in the presence of noise-Probability of error for two encoding schemes," Bell Syst. Tech. J., pp. 60-93, Jan. 1950.
208. A. Feinstein, "Error bounds in noisy channels with memory," IRE Trans. Inform. Theory, vol. PGIT-1, pp. 13-14, Sept. 1955.
209. C. E. Shannon, "Probability of error for optimal codes in a Gaussian channel," Bell Syst. Tech. J., vol. 38, pp. 611-656, May 1959.
210. P. Elias, "Coding for noisy channels," in IRE Conv. Rec., Mar. 1955, vol. 4, pp. 3716.
211. R. L. Dobrushin, "Asymptotic bounds on error probability for transmis-sion over DMC with symmetric transition probabilities," Theory Probab. Applicat., vol. 7, pp. 283-311, 1962.
212. R. M. Fano, Transmission of Information. New York: Wiley, 1961.
213. R. L. Dobrushin, "Optimal binary codes for low rates of information transmission," Theory of Probab. Applicat., vol. 7, pp. 208-213, 1962.
214. C. E. Shannon, R. G. Gallager, and E. Berlekamp, "Lower bounds to error probability for coding on discrete memoryless channels, I," Inform. Contr., vol. 10, pp. 65-103, 1967.
215. "Lower bounds to error probability for coding on discrete memoryless channels, II," Inform. Contr., vol. 10, pp. 522-552, 1967.
216. E. A. Haroutunian, "Bounds on the exponent of the error probability for a semicontinuous memoryless channel," Probl. Inform. Transm., vol. 4, pp. 378, 1968.
217. R. E. Blahut, "Hypothesis testing and information theory," IEEE Trans. Inform. Theory, vol. IT-20, pp. 40517, July 1974.
218. R. J. McEliece and J. K. Omura, "An improved upper bound on the block coding error exponent for binary-input discrete memoryless channels," IEEE Trans. Inform. Theory, vol. IT-23, pp. 611-613, Sept. 1977.
219. S. Litsyn, "New upper bounds on error exponents," Tech. Rep. EE-S98-01, Tel. Aviv Univ., Ramat-Aviv., Israel, 1998.
220. I. Csiszâr and J. Körner, "Graph decomposition: A new key to coding theorems," IEEE Trans. Inform. Theory, vol. IT-27, pp. 5-11, Jan. 1981.
221. S. Kullback and R. A. Leibler, "On information and sufficiency," Ann. Math. Statist., vol. 22, pp. 79-86, 1951.
222. A. Wald, "Note on the consistency of the maximum likelihood estimate," Ann. Math. Statist., vol. 20, pp. 595-601, 1949.
223. I. M. Jacobs and E. R. Berlekamp, "A lower bound to the distribution of computation for sequential decoding," IEEE Trans. Inform. Theory, vol. IT-13, pp. 167-174, Apr. 1967.
224. E. Arikan, "An upper bound to the cutoff rate of sequential decoding," IEEE Trans. Inform. Theory, vol. 34, pp. 55-63, Jan. 1988.
225. J. L. Massey, "Coding and modulation in digital communications," in 1974 Zurich Sem. Digital Communications, 1974, pp. E2(1)-E2(4).
226. I. Csiszâr, "Generalized cutoff rates and Rényi's information measures," IEEE Trans. Inform. Theory, vol. 41, pp. 26-34, Jan. 1995.
227. P. Elias, "List decoding for noisy channels," in IRE WESCON Conv. Rec., 1957, vol. 2, pp. 94-104.
228. G. D. Forney, "Exponential error bounds for erasure list, and deci-sion feedback schemes," IEEE Trans. Inform. Theory, vol. IT-14, pp. 206-220, Mar. 1968.
229. R. Ahlswede, "Channel capacities for list codes," J. Appl. Probab., vol. 10, pp. 824-836, 1973.
230. P. Elias, "Error correcting codes for list decoding," IEEE Trans. Inform. Theory, vol. 37, pp. 5-12, Jan. 1991.
231. V. Blinovsky, "List decoding," Discr. Math., vol. 106, pp. 45-51, Sept. 1992.
232. P. Elias, "Zero error capacity under list decoding," IEEE Trans. Inform. Theory, vol. 34, pp. 1070-1074, Sept. 1988.
233. 1. E. Telatar, "Zero error list capacities of discrete memoryless channels," IEEE Trans. Inform. Theory, vol. 43, pp. 1977-1982, Nov. 1997.
234. K. S. Zigangirov, "Upper bounds on the error probability for channels with feedback," Probl. Inform. Transm., vol. 6, no. 2, pp. 87-92, 1970.
235. J. P. M. Schalkwijk and T. Kailath, "A coding scheme for additive noise channels with feedback," IEEE Trans. Inform. Theory, vol. IT-12, pp. 183-189, Apr. 1966.
236. M. S. Pinsker, "Error probability for block transmission on a Gaussian memoryless channel with feedback," Probl. Inform. Transm., vol. 4, no. 4, pp. 3-19, 1968.
237. M. S. Pinsker and A. Dyachkov, "Optimal linear transmission through a memoryless Gaussian channel with full feedback," Probl. Inform. Transm., vol. 7, pp. 123-129, 1971.
238. R. S. Liptser, "Optimal encoding and decoding for transmission of Gaussian Markov signal over a noiseless feedback channel," Probl. Inform. Transm., vol. 10, no. 4, pp. 3-15, 1974.
239. I. A. Ovseevich, "Capacity of a random channel with feedback and the matching of a source to such a channel," Probl. Inform. Transm., vol. 4, pp. 52-59, 1968.
240. M. S. Pinsker and R. L. Dobrushin, "Memory increases capacity," Probl. Inform. Transm., vol. 5, pp. 94-95, Jan. 1969.
241. P. M. Ebert, "The capacity of the Gaussian channel with feedback," Bell Syst. Tech. J., vol. 49, pp. 1705-1712, Oct. 1970.
242. S. Ihara, Information Theory for Continuous Systems. Singapore: World Scientific, 1993.
243. T. M. Cover and S. Pombra, "Gaussian feedback capacity," IEEE Trans. Inform. Theory, vol. 35, pp. 373, Jan. 1989.
244. A. Lapidoth and P. Narayan, "Reliable communication under channel uncertainty," this issue, pp. 2148-2176.
245. D. Blackwell, L. Breiman, and A. Thomasian, "The capacity of a class of channels," A««. Math. Statist., vol. 30, pp. 1229-1241, Dec. 1959.
246. R. L. Dobrushin, "Optimum information transmission through a channel with unknown parameters," Radio Eng. Electron., vol. 4, pp. 1-8, 1959.
247. J. Wolfowitz, "Simultaneous channels," Arch. Rational Mech. Anal, vol. 4, pp. 371-386, 1960.
248. W. L. Root and P. P. Varaiya, "Capacity of classes of Gaussian channels," SIAM J. Appl. Math, vol. 16, pp. 1350-1393, Nov. 1968.
249. A. Lapidoth and 1. E. Telatar, "The compound channel capacity of a class of finite-state channels," IEEE Trans. Inform. Theory, vol. 44, pp. 973-983, May 1998.
250. J. Kiefer and J. Wolfowitz, "Channels with arbitrarily varying
251. D. Blackwell, L. Breiman, and A. Thomasian, "The capacities of certain channel classes under random coding," Ann. Math. Statist., vol. 31, pp. 558-567, 1960.
252. R. Ahlswede and J. Wolfowitz, "The capacity of a channel with arbitrarily varying cpfs and binary output alphabet," Z. Wahrschein-lichkeitstheorie Verw. Gebiete, vol. 15, pp. 186-194, 1970.
253. I. Csiszär and J. Körner, "On the capacity of the arbitrarily varying chan-nel for maximum probability of error," Z. Wahrscheinlichkeitstheorie Verw. Gebiete, vol. 57, pp. 87-101, 1981.
254. R. L. Dobrushin and S. Z. Stambler, "Coding theorems for classes of arbitrarily varying discrete memoryless channels," Probl. Inform. Transm., vol. 11, no. 2, pp. 3-22, 1975.
255. R. Ahlswede, "Elimination of correlation in random codes for arbitrarily varying channels," Z. Wahrscheinlichkeitstheorie Verw. Gebiete, vol. 44, pp. 159-175, 1968.
256. T. Ericson, "Exponential error bounds for random codes in the arbitrarily varying channel," IEEE Trans. Inform. Theory, vol. 31, pp. 4218, Jan. 1985.
257. I. Csiszâr and P. Narayan, "The capacity of the arbitrarily varying channel revisited: Capacity, constraints," IEEE Trans. Inform. Theory, vol. 34, pp. 181-193, Mar. 1988.
258. "Capacity of the Gaussian arbitrarily varying channel," IEEE Trans. Inform. Theory, vol. 34, pp. 18-26, Jan. 1991.
259. B. Hughes and P. Narayan, "Gaussian arbitrarily varying channels," IEEE Trans. Inform. Theory, vol. IT-33, pp. 267-284, Mar. 1987.
260. B. L. Hughes, "The smallest list for the arbitrarily varying channel," IEEE Trans. Inform. Theory, vol. 43, pp. 803-815, May 1997.
261. V. B. Balakirsky, "Coding theorem for discrete memoryless channels with given decision rules," in Proc. 1st French-Soviet Work. Algebraic Coding, July 1991, pp. 142-150.
262. N. Merhav, G. Kaplan, A. Lapidoth, and S. Shamai, "On information rates for mismatched decoders," IEEE Trans. Inform. Theory, vol. 40, pp. 1953-1967, Nov. 1994.
263. I. Csiszâr and P. Narayan, "Channel decoding for a given decoding metric," IEEE Trans. Inform. Theory, vol. 41, pp. 3513, Jan. 1995.
264. A. Lapidoth, "Mismatched decoding and the multiple-access channel," IEEE Trans. Inform. Theory, vol. 42, pp. 1439-1452, Sept. 1996.
265. "Nearest neighbor decoding for additive non-Gaussian noise channels," IEEE Trans. Inform. Theory, vol. 42, pp. 1520-1528, Sept. 1996.
266. V. D. Goppa, "Nonprobabilistic mutual information without memory," Probl. Contr. Inform. Theory, vol. 4, pp. 97-102, 1975.
267. J. Ziv, "Universal decoding for finite-state channels," IEEE Trans. Inform. Theory, vol. IT-31, pp. 45360, July 1985.
268. A. Lapidoth and J. Ziv, "On the universality of the LZ-based decoding algorithm," IEEE Trans. Inform. Theory, vol. 44, pp. 1746-1755, Sept. 1998.
269. I. Csiszâr and P. Narayan, "Capacity and decoding rules for classes of arbitrarily varying channels," IEEE Trans. Inform. Theory, vol. 35, pp. 752-769, July 1989.
270. M. Feder and A. Lapidoth, "Universal decoding for channels with memory," IEEE Trans. Inform. Theory, vol. 44, pp. 1726-1745, Sept. 1998.
271. C. E. Shannon, "Two-way communication channels," in Proc. 4th. Berkeley Symp. Mathematical Statistics and Probability (June 20-July 30, 1960), J. Neyman, Ed. Berkeley, CA: Univ. Calif. Press, 1961, vol. 1, pp. 611-644.
272. G. Dueck, "The capacity region of the two-way channel can exceed the inner bound," Inform. Contr., vol. 40, pp. 258-266, 1979.
273. J. P. M. Schalkwijk, "The binary multiplying channel-A coding scheme that operates beyond Shannon's inner bound," IEEE Trans. Inform. Theory, vol. IT-28, pp. 107-110, Jan. 1982.
274. "On an extension of an achievable rate region for the binary multiplying channel," IEEE Trans. Inform. Theory, vol. IT-29, pp. 44548, May 1983.
275. Z. Zhang, T. Berger, and J. P. M. Schwalkwijk, "New outer bounds to capacity regions of two-way channels," IEEE Trans. Inform. Theory, vol. IT-32, pp. 383-386, 1986.
276. A. P. Hekstra and F. M. J. Willems, "Dependence balance bounds for single-output two-way channels," IEEE Trans. Inform. Theory, vol. 35, pp. 44-53, 1989.
277. R. Ahlswede, "Multi-way communication channels," in Proc. 2nd Int. Symp. Information Theory, 1971, pp. 103-135.
278. E. C. van der Meulen, "The discrete memoryless channel with two senders and one receiver," in Proc 2nd Int. Symp. Information Theory, 1971, pp. 103-135.
279. H. Liao, "Multiple access channels," Ph.D. dissertation, Univ. Hawaii, Honolulu, HI, 1972.
280. H. Liao, "A coding theorem for multiple-access communications," in 1972 Int. Symp. Information Theory, 1972.
281. R. Ahlswede, "The capacity region of a channel with two senders and two receivers," Ann. Probab., vol. 2, pp. 805-814, Oct. 1974.
282. T. M. Cover, "Some advances in broadcast channels," Adv. Commun. Syst., vol. 4, pp. 229-260, 1975.
283. B. Rimoldi and R. Urbanke, "A rate-splitting approach to the Gaussian multiple-access channel," IEEE Trans. Inform. Theory, vol. 42, pp. 364-375, Mar. 1996.
284. A. Grant, B. Rimoldi, R. Urbanke, and P. Whiting, "Rate-splitting multiple access for discrete memoryless channels," IEEE Trans. Inform. Theory, to be published.
285. R. G. Gallager, "Power limited channels: Coding, multiaccess, and spread spectrum," in 1988 Conf. Information Science and Systems, Mar. 1988. Full version published as Rep. LIDS-P-1714, Nov. 1987.
286. N. T. Gaarder and J. K. Wolf, "The capacity region of a multiple-access discrete memoryless channel can increase with feedback," IEEE Trans. Inform. Theory, vol. IT-21, pp. 100-102, Jan. 1975.
287. T. M. Cover and S. K. Leung, "A rate region for multiple access channels with feedback," IEEE Trans. Inform. Theory, vol. IT-27, pp. 292-298, May 1981.
288. L. H. Ozarow, "The capacity of the white Gaussian multiple access channel with feedback," IEEE Trans Information Theory, vol. IT-30, pp. 623-629, July 1984.
289. J. A. Thomas, "Feedback can at most double Gaussian multiple access channel capacity," IEEE Trans. Inform. Theory, vol. IT-33, pp. 711-716, Sept. 1987.
290. S. Pombra and T. Cover, "Nonwhite Gaussian multiple access channels with feedback," IEEE Trans. Inform. Theory, vol. 40, pp. 885-892, May 1994.
291. E. Ordentlich, "On the factor of two bound for Gaussian multiple-access channels with feedback," IEEE Trans. Inform. Theory, vol. 42, pp. 2231-2235, Nov. 1996.
292. S. Verdti, "Multiple-access channels with memory with and without frame-synchronism," IEEE Trans. Inform. Theory, vol. 35, pp. 605-619, May 1989.
293. R. S. Cheng and S. Verdü, "Gaussian multiple-access channels with intersymbol interference: Capacity region and multiuser water-filling," IEEE Trans. Inform. Theory, vol. 39, pp. 773-785, May 1993.
294. D. N. C. Tse and S. V. Hanly, "Multi-access fading channels: Part I: Polymatroid structure, optimal resource allocation and throughput capacities," IEEE Trans. Inform. Theory, Nov. 1998, to be published. [292]
295. G. S. Poltyrev, "Coding in an asynchronous multiple-access channel," Probl. Inform. Transm., vol. 19, pp. 12-21, July-Sept. 1983.
296. J. Y. N Hui and P. A. Humblet, "The capacity region of the totally asynchronous multiple-access channels," IEEE Trans. Inform. Theory, vol. IT-31, pp. 207-216, Mar. 1985.
297. S. Verdü, "The capacity region of the symbol-asynchronous Gaussian multiple-access channel," IEEE Trans. Inform. Theory, vol. 35, pp. 733-751, July 1989.
298. R. G. Gallager, "A perspective on multiaccess channels," IEEE Trans. Inform. Theory, vol. IT-31, pp. 124-142, Mar. 1985.
299. J. Pokorny and H. M. Wallmeier, "Random coding bound and codes produced by permutations for the multiple-access channel," IEEE Trans. Inform. Theory, vol. 31, pp. 741-750, Nov. 1985.
300. Y. S. Liu and B. L. Hughes, "A new universal random coding bound for the multiple-access channel," IEEE Trans. Inform. Theory, vol. 42, pp. 376-386, Mar. 1996.
301. D. Slepian and J. K. Wolf, "A coding theorem for multiple-access channels with correlated sources," Bell Syst. Tech. J., vol. 52, pp. 1037-1076, 1973.
302. T. M. Cover, A. E. Gamal, and M. Salehi, "Multiple-access channels with arbitrarily correlated sources," IEEE Trans. Inform. Theory, vol. IT-26, pp. 648-657, Nov. 1980.
303. E. C. van der Meulen, "Some reflections on the interference channel," in Communications and Cryptography: Two Sides of One Tapestry, R. E. Blahut, D. J. Costello, U. Maurer, and T. Mittelholzer, Eds. Boston, MA: Kluwer, 1994.
304. A. B. Carleial, "A case where interference does not reduce capacity," IEEE Trans. Inform. Theory, vol. IT-21, pp. 569-570, Sept. 1975.
305. T. S. Han and K. Kobayashi, "A new achievable rate region for the interference channel," IEEE Trans. Inform. Theory, vol. IT-27, pp. 49-60, Jan. 1981.
306. M. H. M. Costa, "On the Gaussian interference channel," IEEE Trans. Inform. Theory, vol. IT-31, pp. 607-615, Sept. 1985.
307. M. H. M. Costa and A. E. Gamal, "The capacity region of the discrete memoryless interference channel with strong interference," IEEE Trans. Inform. Theory, vol. IT-33, pp. 710-711, Sept. 1987.
308. H. Sato, "The capacity of the Gaussian interference channel under strong interference," IEEE Trans. Inform. Theory, vol. IT-27, pp. 786-788, Nov. 1981.
309. T. M. Cover, "Broadcast channels," IEEE Trans. Inform. Theory, vol. IT-18, pp. 2-4, Jan. 1972.
310. T. M. Cover, "Comments on broadcast channels," this issue, pp. 2524-2530.
311. M. Sablatash, "Transmission of all-digital advanced television-state-of-the-art and future directions," IEEE Trans. Broadcast., vol. 40, pp. 102-121, June 1994.
312. K. Ramchandran, A. Ortega, K. Uz, and M. Vetterli, "Multiresolution broadcast for digital HDTV using joint source channel coding," IEEE J. Select. Areas Commun., vol. 11, pp. 6-23, Jan. 1993.
313. A. R. Calderbank and N. Seshadri, "Multilevel codes for unequal error protection," IEEE Trans. Inform. Theory, vol. 39, pp. 1234-1248, July 1993.
314. S. Shamai (Shitz), "A broadcast strategy for the Gaussian slowly fading channel," in Proc. 1997 IEEE Int. Symp. Information Theory (Ulm, Germany, July 1997), p. 150.
315. A. D. Wyner, "The wiretap channel," Bell Syst. Tech. J., vol. 54, pp. 1355-1387, 1975
316. I. Csiszâr and J. Körner, "Broadcast channels with confidential mes-sages," IEEE Trans. Inform. Theory, vol. IT-24, pp. 339-348, May 1978
317. U. M. Maurer, "Secret key agreement by public discussion from com-mon information," IEEE Trans. Inform. Theory, vol. 39, pp. 733-742, May 1993.
318. R. Ahlswede and I. Csiszâr, "Common randomness in information theory and cryptography-Part I: Secret sharing," IEEE Trans. Inform. Theory, vol. IT-32, pp. 1121-1132, July 1993.
319. D. Haussier, "A general minimax result for relative entropy," IEEE Trans. Inform. Theory, vol. 43, pp. 1276-1280, July 1997.
320. B. Ryabko, "Encoding a source with unknown but ordered probabilities," Probl. Inform. Transm., vol. 15, pp. 134-138, 1979.
321. R. G. Gallager, "Source coding with side information and universal coding," Tech. Rep. LIDS-P-937, Lab. Inform. Decision Syst., MIT, Cambridge, MA, 1979.
322. L. D. Davisson and A. Leon-Garcia, "A source matching approach to finding minimax codes," IEEE Trans. Inform. Theory, vol. IT-26, pp. 166-174, 1980.
323. N. Merhav and M. Feder, "A strong version of the redundancy-capacity theorem of universal coding," IEEE Trans. Inform. Theory, vol. 41, pp. 714-722, May 1995.
324. R. Ahlswede and G. Dueck, "Identification via channels," IEEE Trans. Inform. Theory, vol. 35, pp. 15-29, Jan. 1989.
325. R. M. Gray and D. Neuhoff, "Quantization," IEEE Trans. Inform. Theory, this issue, pp. 2325-2383.
326. T. Berger and J. Gibson, "Lossy source coding," IEEE Trans. Inform. Theory, this issue, pp. 2693-2723.
327. D. L. Donoho, I. Daubechies, R. A. DeVore, and M. Vetterli, "Data compression and harmonic analysis," this issue, pp. 2435-2476.
328. C. E. Shannon, "Coding theorems for a discrete source with a fidelity criterion," in IRE Nat. Conv. Rec.Mai. 1959, pp. 142-163.
329. B. M. Oliver, J. R. Pierce, and C. E. Shannon, "The philosophy of PCM," Proc. IRE, vol. 36, pp. 1324-1331, 1948.
330. A. N. Kolmogorov, "On the Shannon theory of information transmission in the case of continuous signals," IEEE Trans. Inform. Theory, vol. IT-2, pp. 102-108, Sept. 1956.
331. E. Posner and E. Rodemich, "Epsilon entropy and data compression," Ann. Math. Statist., vol. 42, pp. 2079-2125, 1971.
332. A. N. Kolmogorov and V. M. Tichomirov, "e-entropy and e-capacity of sets in metric spaces," Usp. Math. Nauk., vol. 14, pp. 3-86, 1959.
333. M. S. Pinsker, "Calculation of the rate of message generation by a stationary random process and the capacity of a stationary channel," Dokl. Akad. Nauk SSSR, vol. III, pp. 753-766, 1956.
334. "Gaussian sources," Probl. Inform. Transm., vol. 14, pp. 59-100, 1963.
335. B. S. Tsybakov, "Epsilon-entropy of a vector message," Probl. Inform. Transm.. vol. 5, pp. 96-97, 1969.
336. T. J. Goblick, "Theoretical limitations on the transmission of data from analog sources," IEEE Trans. Inform. Theory, vol. IT-11, pp. 558-567, Oct. 1965.
337. J. Ziv, "The behavior of analog communication systems," IEEE Trans. Inform. Theory, vol. IT-16, pp. 587-594, Sept. 1970.
338. S. Shamai (Shitz), S. Verdü, and R. Zamir, "Systematic lossy source/ channel coding," IEEE Trans. Inform. Theory, vol. 44, pp. 564-579, Mar. 1998.
339. V. D. Erokhin, "The e-entropy of a discrete random object," Theory Probab. Appi, vol. 3, pp. 103-107, 1958.
340. Y. N. Linkov, "Epsilon-entropy of random variables when epsilon is small," Probl. Inform. Transm., vol. 1, no. 2, pp. 18-26, 1965.
341. "Epsilon-entropy of continuous random process with discrete phase space," Probl. Inform. Transm., vol. 7, no. 2, pp. 16-25, 1971.
342. F. Jelinek, "Evaluation of distortion rate functions for low distortions," Proc. IEEE, vol. 55, pp. 2067-2068, 1967
343. K. Rose, "Mapping approach to rate-distortion computation and analy-sis," IEEE Trans. Inform. Theory, vol. 40, pp. 1939-1952, Nov. 1994.
344. T. Berger, "Information rates of Wiener processes," IEEE Trans. Inform. Theory, vol. IT-16, pp. 134-139, Mar. 1970.
345. S. Verdü, "The exponential distribution in information theory," Probl. Inform. Transm., vol. 32, pp. 86-95, Jan.-Mar. 1996.
346. R. M. Gray, "Information rates of autoregressive processes," IEEE Trans. Inform. Theory, vol. IT-16, pp. 41221, July 1970.
347. T. Berger, Rate Distortion Theory. Englewood Cliffs, NJ: Prentice-Hall, 1971.
348. H. H. Tan and K. Yao, "Evaluation of rate-distortion functions for a class of independent identically distributed sources under an absolute magnitude criterion," IEEE Trans. Inform. Theory, vol. IT-21, pp. 59-64, Jan. 1975.
349. K. Yao and H. H. Tan, "Absolute error rate-distortion functions for sources with constrained magnitudes," IEEE Trans. Inform. Theory, vol. IT-24, pp. 499-503, July 1978.
350. B. Hajek and T. Berger, "A decomposition theorem for binary Markov random fields," A««. Probab., vol. 15, pp. 1112-1125, 1987.
351. L. A. Bassalygo and R. L. Dobrushin, "Rate-distortion function of the Gibbs field," Probl. Inform. Transm., vol. 23, no. 1, pp. 3-15, 1987.
352. M. Effros, P. A. Chou, and R. M. Gray, "Variable-rate source coding the-orems for stationary nonergodic sources," IEEE Trans. Inform. Theory, vol. 40, pp. 1920-1925, Nov. 1994.
353. Y. Steinberg and S. Verdü, "Simulation of random processes and rate-distortion theory," IEEE Trans. Inform. Theory, vol. 42, pp. 63-86, Jan. 1996.
354. J. C. Kieffer, "A survey of the theory of source coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. 39, pp. 1473-1490, Sept. 1993.
355. D. J. Sakrison, "The rate of a class of random processes," IEEE Trans. Inform. Theory, vol. IT-16, pp. 10-16, Jan. 1970.
356. J. Ziv, "Coding of sources with unknown statistics-Part II: Distortion relative to a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-18, pp. 389-394, May 1972.
357. D. L. Neuhoff, R. M. Gray, and L. D. Davisson, "Fixed rate universal block source coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-21, pp. 511-523, 1975.
358. D. L. Neuhoff and P. C. Shields, "Fixed-rate universal codes for Markov sources," IEEE Trans. Inform. Theory, vol. IT-24, pp. 360-367, 1978.
359. J. Ziv, "Distortion-rate theory for individual sequences," IEEE Trans. Inform. Theory, vol. IT-24, pp. 137-143, Jan. 1980.
360. R. Garcia-Munoz and D. L. Neuhoff, "Strong universal source coding subject to a rate-distortion constraint," IEEE Trans. Inform. Theory, vol. IT-28, pp. 285-295, Mar. 1982
361. D. S. Ornstein and P. C. Shields, "Universal almost sure data compres-sion," A««. Probab., vol. 18, pp. 441152, 1990.
362. Y. Steinberg and M. Gutman, "An algorithm for source coding subject to a fidelity criterion based on string matching," IEEE Trans. Inform. Theory, vol. 39, pp. 877-886, 1993.
363. B. Yu and T. P. Speed, "A rate of convergence result for a universal dsemifaithful code," IEEE Trans. Inform. Theory, vol. 39, pp. 813-820, May 1993.
364. T. Linder, G. Lugosi, and K. Zeger, "Rates of convergence in the source coding theorem, in empirical quantizer design, and in universallossy source coding," IEEE Trans. Inform. Theory, vol. 40, pp. 1728-1740, Nov. 1994.
365. "Fixed-rate universal source coding and rates of convergence for memoryless sources," IEEE Trans. Inform. Theory, vol. 41, pp. 665-676, May 1995.
366. P. A. Chou, M. Effros, and R. M. Gray, "A vector quantization approach to universal noiseless coding and quantization," IEEE Trans. Inform. Theory, vol. 42, pp. 1109-1138, July 1996.
367. Z. Zhang and E. Yang, "An on-line universal lossy data compression algorithm via continuous codebook refinement-II: Optimality for phi-mixing source models," IEEE Trans. Inform. Theory, vol. 42, pp. 822-836, May 1996.
368. Z. Zhang and V. K. Wei, "An on-line universal lossy data compression algorithm via continuous codebook refinement-Part I: Basic results," IEEE Trans. Inform. Theory, vol. 42, pp. 803-821, May 1996.
369. J. C. Kieffer and E. H. Yang, "Sequential codes, lossless compression of individual sequences, and Kolmogorov complexity," IEEE Trans. Inform. Theory, vol. 42, pp. 29-39, Jan. 1996.
370. E. H. Yang and J. C. Kieffer, "Simple universal lossy data compression schemes derived from the Lempel-Ziv algorithm," IEEE Trans. Inform. Theory, vol. 42, pp. 239-245, Jan. 1996.
371. E. H. Yang, Z. Zhang, and T. Berger, "Fixed-slope universal lossy data compression," IEEE Trans. Inform. Theory, vol. 43, pp. 1465-1476, Sept. 1997.
372. T. Luczak and W. Szpankowski, "A suboptimal lossy date compression based on approximate pattern matching," IEEE Trans. Inform. Theory, vol. 43, pp. 1439-1451, Sept. 1997.
373. D. L. Neuhoff and P. C. Shields, "Simplistic universal coding," IEEE Trans. Inform. Theory, vol. 44, pp. 778-781, Mar. 1998.
374. A. D. Wyner and J. Ziv, "The rate-distortion function for source coding with side information at the decoder," IEEE Trans. Inform. Theory, vol. IT-22, pp. 1-10, Jan. 1976.
375. A. D. Wyner, "The rate-distortion function for source coding with side information at the decoder-II: General sources," Inform. Contr., vol. 38, pp. 60-80, 1978.
376. T. Berger, K. B. Housewright, J. K. Omura, S. Tung, and J. Wolfowitz, "An upper bound to the rate distortion function for source coding with partial side information at the decoder," IEEE Trans. Inform. Theory, vol. IT-25, pp. 664-666, 1979.
377. C. Heegard and T. Berger, "Rate-distortion when side information may be absent," IEEE Trans. Inform. Theory, vol. IT-31, pp. 727-734, Nov. 1985.
378. T. Berger and R. W. Yeung, "Multiterminal source encoding with one distortion criterion," IEEE Trans. Inform. Theory, vol. 35, pp. 228-236, Jan. 1989.
379. R. Zamir, "The rate loss in the Wyner-Ziv problem," IEEE Trans. Inform. Theory, vol. 42, pp. 2073-2084, Nov. 1996.
380. T. Linder, R. Zamir, and K. Zeger, "On source coding with side in-formation dependent distortion measures," IEEE Trans. Inform. Theory, submitted for publication.
381. Z. Zhang and T. Berger, "New results in binary multiple descriptions," IEEE Trans. Inform. Theory, vol. IT-33, pp. 502-521, July 1987.
382. L. H. Ozarow, "On a source coding problem with two channels and three receivers," Bell Syst. Tech. J., vol. 59, pp. 1909-1921, Dec. 1980.
383. T. M. Cover and A. E. Gamal, "Achievable rates for multiple de-scriptions," IEEE Trans. Inform. Theory, vol. IT-28, pp. 851-857, Nov. 1982.
384. R. Ahlswede, "The rate-distortion region for multiple descriptions without excess rate," IEEE Trans. Inform. Theory, vol. IT-31, pp. 721-726, Nov. 1985.
385. V. Vaishampayan, "Design of multiple description scalar quantizers," IEEE Trans. Inform. Theory, vol. 39, pp. 821-834, May 1993.
386. Z. Zhang and T. Berger, "Multiple description source coding with no excess marginal fate," IEEE Trans. Inform. Theory, vol. 41, pp. 349-357, Mar. 1995.
387. V. Koshélev, "Estimation of mean error for a discrete successive approximation scheme," Probl. Inform. Transm., vol. 17, pp. 20-33, July-Sept. 1981
388. W. H. R. Equitz and T. M. Cover, "Successive refinement of infor-mation," IEEE Trans. Inform. Theory, vol. 37, pp. 269-274, Mar. 1991.
389. B. Rimoldi, "Successive refinement of information: Characterization of the achievable rates," IEEE Trans. Inform. Theory, vol. 40, pp. 253-259, Jan. 1994.
390. A. H. Kaspi and T. Berger, "Rate-distortion for correlated sources and partially separated encoders," IEEE Trans. Inform. Theory, vol. IT-28, pp. 828-840, Nov. 1982.
391. T. Berger, Z. Zhang, and H. Viswanathan, "The CEO problem," IEEE Trans. Inform. Theory, vol. 42, pp. 887-903, May 1996.
392. H. Viswanathan and T. Berger, "The quadratic Gaussian CEO problem," IEEE Trans. Inform. Theory, vol. 43, pp. 1549-1561, Sept. 1997.
393. A. R. Barron, "Entropy and the central limit theorem," Ann. Probab., vol. 14, pp. 336-342, 1986.
394. I. Csiszâr, "I-divergence geometry of probability distributions and minimization problems," A««. Probab., vol. 3, pp. 146-158, Feb. 1975.
395. "Sanov property, generalized I-projection and a conditional limit theorem," Ann. Probab., vol. 12, pp. 768-793, Aug. 1984.
396. A. Dembo and O. Zeitouni, Large Deviations Techniques and Applications. Boston, MA: Jones and Bartlett, 1993.
397. S. Kullback, J. C. Keegel, and J. H. Kullback, Topics in Statistical Information Theory. B_erlin, Germay: Springer, 1987.
398. K. Marton, "Bounding d-distance by information divergence: A method to prove concentration inequalities," A««. Probab., vol. 24, pp. 857-866, 1996.
399. A. Dembo, "Information inequalities and concentration of measure," Ann. Probab., vol. 25, pp. 927-939, 1997.
400. A. Ephremides and B. Hajek, "Information theory and communication networks: An unconsummated union," this issue, pp. 2416-2434.
401. S. Kullback, Information Theory and Statistics. New York: Dover, 1968.
402. I. Vajda, Theory of Statistical Inference and Information. Dordrecht, The Netherlands: Kluwer, 1989.
403. S. I. Amari and T. S. Han, "Statistical inference under multiterminal data compression," this issue, pp. 2300-2324.
404. J. Ziv and M. Zakai, "Some lower bounds on signal parameter esti-mation," IEEE Trans. Inform. Theory, vol. IT-15, pp. 386-391, May 1969.
405. L. Devroye, A Course in Density Estimation. Boston, MA: Birkhauser, 1987.
406. L. Bassalygo, S. Gelfand, G. Golubev, R. Dobrushin, V. Prelov, Y. Sinai, R. Khasminskii, and A. Yaglom, "Review of scientific achievements of M. S. Pinsker," Probl. Inform. Transm., vol. 32, pp. 3-14, 1996.
407. Y. Yang and A. Barron, "Information-theoretic determination of minimax rates of convergence," Ann. Statist., to be published.
408. B. S. Choi and T. M. Cover, "An information-theoretic proof of Burg's maximum entropy spectrum," Proc. IEEE, vol. 72, pp. 1094-1095, 1984.
409. A. R. Barron, "Information-theoretic characterization of Bayes performance and choice of priors in parametric and nonparametric problems," in Bayesian Statistics 6: Proc. 6th Valencia Int. Meet., June 1998.
410. I. Csiszâr, "Why least squares and maximum entropy? an axiomatic approach to inference for linear inverse problems," Ann. Statist., vol. 19, pp. 2032-2066, Dec. 1991.
411. S. Kulkarni, G. Lugosi, and S. Venkatesh, "Learning pattern classification-A survey," this issue, pp. 2178-2206..
412. A. R. Barron, "Approximation and estimation bounds for artificial neural networks," Mach. Learn., vol. 14, pp. 115-133, 1994.
413. Y. S. Abumostafa, "Information theory, complexity and neural networks," IEEE Commun. Mag., vol. 27, pp. 25-30, Nov. 1989.
414. F. Jelinek, Statistical Methods for Speech Recognition. Cambridge, MA: MIT Press, 1998.
415. G. J. Chaitin, Algorithmic Information Theory. Cambridge, U.K.: Cambridge Univ. Press, 1987.
416. M. Li and P. Vitânyi, An Introduction to Kolmogorov Complexity and Its Applications. Berlin, Germany: Springer, 1993.
417. H. Yamamoto, "Information theory in cryptology," IEICE Trans. Commun., Electron., Inform. Syst., vol. 74, pp. 2456-2464, Sept. 1991.
418. N. Pippenger, "Information theory and the complexity of Boolean functions," Math. Syst. Theory, vol. 10, pp. 129-167, 1977.
419. A. Steane, "Quantum computing," Repts. Progr. Phys., vol. 61, pp. 117-173, Feb. 1998.
420. P. Elias, "The efficient construction of an unbiased random sequence," Ann. Math. Statist., vol. 43, pp. 865-870, 1972.
421. D. E. Knuth and A. C. Yao, "The complexity of random number generation," in Algorithms and Complexity: Recent Results and New Directions, J. F. Traub, Ed. New York: Academic, 1976.
422. K. Visweswariah, S. Kulkarni, and S. Verdti, "Source codes as random number generators," IEEE Trans. Inform. Theory, vol. 44, pp. 462171, Mar. 1998.
423. D. Lind and B. Marcus, Symbolic Dynamics and Coding. Cambridge, U.K.: Cambridge Univ. Press, 1995.
424. A. Dembo, T. M. Cover, and J. A. Thomas, "Information theoretic inequalities," IEEE Trans. Inform. Theory, vol. 37, pp. 1501-1518, Nov. 1991.
425. H. S. Witsenhausen, "Some aspects of convexity useful in information theory," IEEE Trans. Inform. Theory, vol. IT-26, pp. 265-271, May 1980.
426. S. I. Amari and H. Nagaoka, Methods of Information Geometry. Oxford, U.K.: Oxford Univ. Press, 1999.
427. L. L. Campbell, "The relation between information theory and the differential geometry approach to statistics," Inform. Sci., vol. 35, pp. 199-210, June 1985.
428. R. Ahlswede and I. Wegener, Search Problems. New York: Wiley-Interscience, 1987.
429. S. W. Golomb, "Probability, information theory and prime number theory," Discr. Math., vol. 106, pp. 219-229, Sept. 1992.
430. O. Hijab, Stabilization of Control Systems. New York: Springer-Verlag, 1987.
431. J. K. Sengupta, Econometrics of Information and Efficiency. Dordrecht, The Netherlands: Kluwer, 1993.
432. W. T. Grandy, "Resource letter ITP-1: Information theory in physics," Amer. J. Phys., vol. 16, pp. 46676, June 1997.
433. H. S. Leff and A. F. Rex, Eds., Maxwell's Demon: Entropy, Information, Computing. Princeton, NJ: Princeton Univ. Press, 1990.
434. R. Landauer, "Information is physical," Phys. Today, vol. 44, pp. 23-29, May 1991.
435. N. Sourlas, "Statistical mechanics and error-correcting codes," in From Statistical Physics to Statistical Inference and Back, P. Grassberger and J. P. Nadal, Eds. Dordrecht, The Netherlands: Kluwer, 1994, pp. 195-204.
436. C. H. Bennett and P. Shor, "Quantum information theory," this issue, pp. 2724-2742.
437. S. Hayes, C. Grebogi, and E. Ott, "Communicating with chaos," Phys. Rev. Lett., vol. 70, no. 20, pp. 3031-3034, May 17, 1993.
438. H. P. Yockey, Information Theory and Molecular Biology. New York: Cambridge Univ. Press, 1992.
439. J. J. Atick, "Could information theory provide an ecological theory of sensory processing?," Network Comput. Neural Syst.. vol. 3, pp. 213-251, May 1992.
440. R. Linsker, "Sensory processing and information theory," in From Statistical Physics to Statistical Inference and Back. P. Grassberger and J. P. Nadal, Eds. Dordrecht, The Netherlands: Kluwer, 1994, pp. 237-248.
441. K. Eckschlager, Information Theory in Analytical Chemistry. New York: Wiley, 1994
442. C. Papadimitriou, "Information theory and computational complex-ity: The expanding interface," IEEE Inform. Theory Newslett. (Special Golden Jubilee Issue), pp. 12-13, Summer 1998.
443. T. M. Cover, "Shannon and investment," IEEE Inform. Theory Newslett. (Special Golden Jubilee Issue), pp. 10-11, Summer 1998.