Entropy and information theory

Entropy and Information Theory

Volumn , Issue , 2011, Pages 1-409

  Gray, Robert M ^a  

^a Stanford University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84892332317     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-1-4419-7970-4     Document Type: Book

References (201)

1. N. M. Abramson. Information Theory and Coding. McGraw-Hill, New York, 1963.
2. R. Adler. Ergodic and mixing properties of infinite memory channels. Proc. Amer. Math. Soc., 12:924-930, 1961.
3. R. L. Adler, D. Coppersmith, and M. Hassner. Algorithms for sliding-block codes-An application of symbolic dynamics to information theory. IEEE Trans. Inform. Theory, 29:5-22, 1983.
4. R. Ahlswede and P. Gács. Two contributions to information theory. In Topics in Information Theory, pages 17-40, Keszthely, Hungary, 1975.
5. R. Ahlswede and J. Wolfowitz. Channels without synchronization. Adv. in Appl. Probab., 3:383-403, 1971.
6. P. Algoet. Log-Optimal Investment. PhD thesis, Stanford University, 1985.
7. P. Algoet and T. Cover. A sandwich proof of the Shannon-MacMillan-Breiman theorem. Ann. Probab., 16:899-909, 1988.
8. A. R. Barron. The strong ergodic theorem for densities: generalized Shannon- MacMillan-Breiman theorem. Ann. Probab., 13:1292-1303, 1985.
9. T.W. Benjamin. Rate Distortion Functions for Discrete Sources with Continuous Reproductions. PhD thesis, Cornell University, Ithaca, New York, 1973.
10. T. Berger. Rate distortion theory for sources with abstract alphabets and memory. Inform. and Control, 13:254-273, 1968.
11. T. Berger. Rate Distortion Theory. Prentice-Hall Inc., Englewood Cliffs, New Jersey, 1971.
12. E. Berlekamp. Algebraic Coding Theory. McGraw-Hill, New York, 1968.
13. E. Berlekamp, Key Papers in the Development of Coding Theory. IEEE Press, New York, 1974.
14. James Bezdek and Richard Hathaway. Some notes on alternating optimization. In Nikhil Pal and Michio Sugeno, editors, Advances in Soft Computing AFSS 2002, volume 2275 of Lecture Notes in Computer Science, pages 187-195. Springer Berlin/Heidelberg, 2002. 10.1007/3-540-45631-7-39.
15. P. J. Bickel and D. A. Freedman. Some asymptotic theory for the bootstrap. Annals of Statistics, 9(1):196-1217, 1981.
16. P. Billingsley. Ergodic Theory and Information. Wiley, New York, 1965.
17. G. D. Birkhoff. Proof of the ergodic theorem. Proc. Nat. Acad. Sci., 17:656-660, 1931.
18. R. E. Blahut. Computation of channel capacity and rate-distortion functions. IEEE Trans. Inform. Theory, 18:460-473, 1972.
19. R. E. Blahut. Theory and Practice of Error Control Codes. Addison Wesley, Reading, Mass., 1987.
20. L. Breiman. The individual ergodic theorem of information theory. Ann. of Math. Statist., 28:809-811, 1957.
21. L. Breiman. A correction to 'The individual ergodic theorem of information theory'. Ann. of Math. Statist., 31:809-810, 1960.
22. J. R. Brown. Ergodic Theory and Topological Dynamics. Academic Press, New York, 1976.
23. T. M. Cover, P. Gacs, and R. M. Gray. Kolmogorov's contributions to information theory and algorithmic complexity. Ann. Probab., 17:840-865, 1989.
24. I. Csiszár. Information-type measures of difference of probability distributions and indirect observations. Studia Scientiarum Mathematicarum Hungarica, 2:299-318, 1967.
25. I. Csiszár. On an extremum problem of information theory. Studia Scientiarum Mathematicarum Hungarica, pages 57-70, 1974.
26. I. Csiszár. I-divergence geometry of probability distributions and minimization problems. Ann. Probab., 3(1):146-158, 1975.
27. I. Csiszár and J. Körner. Coding Theorems of Information Theory. Academic Press/Hungarian Academy of Sciences, Budapest, 1981.
28. L. D. Davisson and R.M. Gray. A simplified proof of the sliding-block source coding theorem and its universal extension. In Conf. Record 1978 Int'l. Conf. on Comm. 2, pages 34-5, Toronto, 1978.
29. L. D. Davisson and M. B. Pursley. An alternate proof of the coding theorem for stationary ergodic sources. In Proceedings of the Eighth Annual Princeton Conference on Information Sciences and Systems, 1974.
30. M. Denker, C. Grillenberger, and K. Sigmund. Ergodic Theory on Compact Spaces, volume 57 of Lecture Notes in Mathematics. Springer-Verlag, New York, 1970.
31. J.-D. Deushcel and D. W. Stroock. Large Deviations, Academic Press, Boston, 1989.
32. R. L. Dobrushin. A general formulation of the fundamental Shannon theorem in information theory. Uspehi Mat. Akad. Nauk. SSSR, 14:3-104, 1959.
33. R. L. Dobrushin. Shannon's theorems for channels with synchronization errors. Problemy Peredaci Informatsii, 3:18-36, 1967.
34. R.L. Dobrushin. Prescribing a system of random variables by conditional distributions. Theor. Prob. Appl., 15:458-486, 1970.
35. M. D. Donsker and S. R. S. Varadhan. Asymptotic evaluation of certain Markov process expectations for large time. J. Comm. Pure Appl. Math., 28:1-47, 1975.
36. J. G. Dunham. A note on the abstract alphabet block source coding with a fidelity criterion theorem. IEEE Trans. Inform. Theory, 24:760, November 1978.
37. P. Elias. Two famous papers. IRE Transactions on Information Theory, page 99, 1958.
38. R. M. Fano. Transmission of Information. Wiley, New York, 1961.
39. A. Feinstein. A new basic theorem of information theory. IRE Transactions on Information Theory, pages 2-20, 1954.
40. A. Feinstein. Foundations of Information Theory. McGraw-Hill, New York, 1958.
41. A. Feinstein. On the coding theorem and its converse for finite-memory channels. Inform. and Control, 2:25-44, 1959.
42. W. A. Finamore and W. A. Pearlman. Optimal encoding of discrete-time, continuous-Amplitude, memoryless sources with finite output alphabets. IEEE Trans. Inform. Theory, 26:144-155, Mar. 1980.
43. S.L. Fix. Rate distortion functions for continuous alphabet memoryless sources. PhD thesis, University of Michigan, Ann Arbor, Michigan, 1977.
44. G. D. Forney Jr. The Viterbi algorithm. Proc. IEEE, 61:268-278, March 1973.
45. M. Fréchet. Sur la distance de deux lois de probabilité. C. R. Acad. Sci. Paris, 244:689-692, 1957.
46. N. A. Friedman. Introduction to Ergodic Theory. Van Nostrand Reinhold Company, New York, 1970.
47. R. G. Gallager. Information Theory and Reliable Communication. John Wiley & Sons, New York, 1968.
48. R.G. Gallager. Low Density Parity Check Codes. MIT Press, 1963.
49. I. M. Gelfand, A. N. Kolmogorov, and A. M. Yaglom. On the general definitions of the quantity of information. Dokl. Akad. Nauk, 111:745-748, 1956. (In Russian.).
50. A. Gersho and R. M. Gray. Vector Quantization and Signal Compression. Kluwer Academic Publishers, Boston, 1992.
51. C. Gini. Sulla misura della concentrazione e della variabilit'a del caratteri. Atti del R. Instituto Veneto, pages 1913-1914, 1914.
52. R. M. Gray. Sliding-block source coding. IEEE Trans. Inform. Theory, 21(4):357-368, July 1975.
53. R. M. Gray. Time-invariant trellis encoding of ergodic discrete-time sources with a fidelity criterion. IEEE Trans. Inform. Theory, 23:71-83, 1977.
54. R. M. Gray. Tree-searched block source codes. In Proceedings of the 1980 Allerton Conference, Allerton IL, Oct. 1980.
55. R. M. Gray. Probability, Random Processes, and Ergodic Properties. Springer- Verlag, New York, 1988.
56. R. M. Gray. Spectral analysis of quantization noise in a single-loop sigma-delta modulator with dc input. IEEE Trans. Comm., 37:588-599, 1989.
57. R. M. Gray. Source Coding Theory. Kluwer Academic Press, Boston, 1990.
58. R. M. Gray. Probability, Random Processes, and Ergodic Properties, Second Ed. Springer, New York, 2009.
59. R. M. Gray and L. D. Davisson. Source coding without the ergodic assumption. IEEE Trans. Inform. Theory, 20:502-516, 1974.
60. R. M. Gray and L. D. Davisson. Quantizer mismatch. IEEE Trans. on Comm., 23(4):439-442, April 1975.
61. R. M. Gray and E. Karnin. Multiple local optima in vector quantizers. IEEE Trans. Inform. Theory, 28:256-261, March 1982.
62. R. M. Gray and J. C. Kieffer. Asymptotically mean stationary measures. Ann. Probab., 8:962-973, 1980.
63. R. M. Gray, D. L. Neuhoff, and J. K. Omura. Process definitions of distortion rate functions and source coding theorems. IEEE Trans. Inform. Theory, 21:524- 532, 1975.
64. R. M. Gray, D. L. Neuhoff, and J. K. Omura. Process definitions of distortion rate functions and source coding theorems. IEEE Trans. on Info. Theory, 21(5):524- 532, Sept. 1975.
65. R. M. Gray, D. L. Neuhoff, and D. Ornstein. Nonblock source coding with a fidelity criterion. Ann. Probab., 3:478-491, 1975.
66. R. M. Gray, D. L. Neuhoff, and P. C. Shields. A generalization of ornstein's dbar distance with applications to information theory. Ann. Probab., 3:315-328, April 1975.
67. R. M. Gray and D. S. Ornstein. Sliding-block joint source/noisy-channel coding theorems. IEEE Trans. Inform. Theory, 22:682-690, 1976.
68. R. M. Gray, D. S. Ornstein, and R. L. Dobrushin. Block synchronization, slidingblock coding, invulnerable sources and zero error codes for discrete noisy channels. Ann. Probab., 8:639-674, 1980.
69. R. M. Gray, M. Ostendorf, and R. Gobbi. Ergodicity of Markov channels. IEEE Trans. Inform. Theory, 33:656-664, September 1987.
70. R. M. Gray and F. Saadat. Block source coding theory for asymptotically mean stationary sources. IEEE Trans. Inform. Theory, 30:64-67, 1984.
71. R.M. Gray and D. L. Neuhoff. Quantization. IEEE Transactions on Information Theory, 44:2325-2384, October 1998.
72. Robert M. Gray and Tamás Linder. Bits in asymptotically optimal lossy source codes are asymptotically bernoulli. In J. A. Storer and M. Cohn, editors, Proceedings 2009 Data Compression Conference (DCC), pages 53-62, March 2009.
73. P. R. Halmos. Lectures on Ergodic Theory. Chelsea, New York, 1956.
74. G. H. Hardy, J. E. Littlewood, and G. Polya. Inequalities. Cambridge Univ. Press, London, 1952. Second Edition, 1959.
75. R. V. L. Hartley. Transmission of information. Bell System Tech. J., 7:535-563, 1928.
76. T. Hashimoto. A direct proof of the equality between the block definition and the process definition of distortion-rate functions for stationary ergodic sources. nform. Contr., 51:45-57, Oct. 1981.
77. E. Hoph. Ergodentheorie. Springer-Verlag, Berlin, 1937.
78. K. Jacobs. Die Übertragung diskreter Informationen durch periodishce und fastperiodische Kanale. Math. Annalen, 137:125-135, 1959.
79. K. Jacobs. Über die Struktur der mittleren Entropie. Math. Z., 78:33-43, 1962.
80. K. Jacobs. The ergodic decomposition of the Kolmogorov-Sinai invariant. In F. B. Wright and F. B. Wright, editors, Ergodic Theory. Academic Press, New York, 1963.
81. N. S. Jayant and P. Noll. Digital Coding of Waveforms. Prentice-Hall, Englewood Cliffs, New Jersey, 1984.
82. T. Kadota. Generalization of feinstein's fundamental lemma. IEEE Trans. Inform. Theory, 16:791-792, 1970.
83. S. Kakutani. Induced measure preserving transformations. In Proceedings of the Imperial Academy of Tokyo, volume 19, pages 635-641, 1943.
84. S. Kalikow and R. McCutcheon. An Outline of Ergodic Theory, Studies in Advanced Mathematics. volume 122 Cambridge University Press, Cambridge, 2010.
85. L.V. Kantorovich. On the transfer of masses. Uspekhi Mat. Nauk, 37:7-8, 1942.
86. L.V. Kantorovich, C.R. Dikl. Acad. Sci. URSS 37 (1942), 199-201.
87. L.V. Kantorovich. On a problem of monge. Dokl. Akad. Nauk, 3:225-226, 1948.
88. A. J. Khinchine. The entropy concept in probability theory. Uspekhi Matematicheskikh Nauk., 8:3-20, 1953.
89. A. J. Khinchine. On the fundamental theorems of information theory. Uspekhi Matematicheskikh Nauk., 11:17-75, 1957.
90. J. C. Kieffer. A counterexample to Perez's generalization of the Shannon- MacMillan theorem. Ann. Probab., 1:362-364, 1973.
91. J. C. Kieffer. A general formula for the capacity of stationary nonanticipatory channels. Inform. and Control, 26:381-391, 1974.
92. J. C. Kieffer. On the optimum average distortion attainable by fixed-rate coding of a nonergodic source. IEEE Trans. Inform. Theory, 21:190-193, March 1975.
93. J. C. Kieffer. A unified approach to weak universal source coding. IEEE Trans. Inform. Theory, 24:674-682, 1978.
94. J. C. Kieffer. Extension of source coding theorems for block codes to sliding block codes. IEEE Trans. Inform. Theory, 26:679-692, 1980.
95. J. C. Kieffer. Block coding for weakly continuous channels. IEEE Trans. Inform. Theory, 27:721-727, 1981.
96. J. C. Kieffer. Sliding-block coding for weakly continuous channels. IEEE Trans. Inform. Theory, 28:2-10, 1982.
97. J. C. Kieffer. An ergodic theorem for constrained sequences of functions. Bulletin American Math Society, 1989.
98. J. C. Kieffer. Elementary information theory. Unpublished manuscript, 1990.
99. J. C. Kieffer and M. Rahe. Markov channels are asymptotically mean stationary. Siam Journal of Mathematical Analysis, 12:293-305, 1980.
100. J.C. Kieffer. Sample converses in source coding theory. Information Theory, IEEE Transactions on, 37(2):263 -268, March 1991.
101. J.C. Kieffer. Strong converses in source coding relative to a fidelity criterion. Information Theory, IEEE Transactions on, 37(2):257 -262, March 1991.
102. A. N. Kolmogorov. On the Shannon theory of information in the case of continuous signals. IRE Transactions Inform, Theory, 2:102-108, 1956.
103. A. N. Kolmogorov. A new metric invariant of transitive dynamic systems and automorphisms in lebesgue spaces. Dokl. Akad. Nauk SSR, 119:861-864, 1958. (In Russian.).
104. A. N. Kolmogorov. On the entropy per unit time as a metric invariant of automorphisms. Dokl. Akad. Naud SSSR, 124:768-771, 1959. (In Russian.).
105. A. N. Kolmogorov, A. M. Yaglom, and I. M. Gelfand. Quantity of information and entropy for continuous distributions. In Proceedings 3rd All-Union Mat. Conf., volume 3, pages 300-320. Izd. Akad. Nauk. SSSR, 1956.
106. S. Kullback. A lower bound for discrimination in terms of variation. IEEE Trans. Inform. Theory, 13:126-127, 1967.
107. S. Kullback. Information Theory and Statistics. Dover, New York, 1968. Reprint of 1959 edition published by Wiley.
108. B. M. Leiner and R. M. Gray. Bounds on rate-distortion functions for stationary sources and context-dependent fidelity criteria. IEEE Trans. Inform. Theory, 19:706-708, Sept. 1973.
109. E. Levina and P. Bickel. The earth mover's distance is the mallows distance: Some insights from statistics. In Proceedings Eighth IEEE International Conference on Computer Vision (ICCV 2001), volume 2, pages 251-256, 2001.
110. S. Lin. Introduction to Error Correcting Codes. Prentice-Hall, Englewood Cliffs, NJ, 1970.
111. S. P. Lloyd. Least squares quantization in PCM. Unpublished Bell Laboratories Technical Note. Portions presented at the Institute of Mathematical Statistics Meeting Atlantic City New Jersey September 1957. Published in the March 1982 special issue on quantization of the IEEE Transactions on Information Theory, 1957.
112. K. M. MacKenthun and M. B. Pursley. Strongly and weakly universal source coding. In Proceedings of the 1977 Conference on Information Science and Systems, pages 286-291, Johns Hopkins University, 1977.
113. J. MacQueen. Some methods for classification and analysis of multivariate observations. In Proc. of the Fifth Berkeley Symposium on Math. Stat. and Prob., volume 1, pages 281-296, 1967.
114. F. J. MacWilliams and N. J. A. Sloane. The Theory of Error-Correcting Codes. North-Holland, New York, 1977.
115. A. Maitra. Integral representations of invariant measures. Transactions of the American Mathematical Society, 228:209-235, 1977.
116. J. Makhoul, S. Roucos, and H. Gish. Vector quantization in speech coding. Proc. IEEE, 73. No. 11:1551-1587, November 1985.
117. C.L. Mallows. A note on asymptotic joint normality. Ann. Math.l Stat., 43:508- 515, 1972.
118. Mark Z. Mao, Robert M. Gray, and Tamás Linder. Rate-constrained simulation and source coding iid sources. submitted for possible publication.
119. B. Marcus. Sophic systems and encoding data. IEEE Trans. Inform. Theory, 31:366-377, 1985.
120. K. Marton. On the rate distortion function of stationary sources. Problems of Control and Information Theory, 4:289-297, 1975.
121. K. Marton. A simple proof of the blowing-up lemma. TIT, 32:445-446, 1986.
122. K. Marton. Bounding d-distance by informational divergence: A method to prove measure concentration. Annals of Probability, 24(2):857-866, 1996.
123. R. McEliece. The Theory of Information and Coding. Cambridge University Press, New York, NY, 1984.
124. B. MacMillan. The basic theorems of information theory. Ann. of Math. Statist., 24:196-219, 1953.
125. L. D. Meshalkin. A case of isomorphisms of bernoulli scheme. Dokl. Akad. Nauk SSSR, 128:41-44, 1959. (In Russian.).
126. G. Monge. Mémoire sur la théorie des déblais et des remblais. 1781.
127. A. Montanari and R. Urbanke. Modern coding theory: The statistical mechanics and computer science point of view. Summer School On Complex Systems, Les Houches, pages 704-2857, 2007.
128. Shu-Teh C. Moy. Generalizations of Shannon-MacMillan theorem. Pacific Journal Math., 11:705-714, 1961.
129. N. M. Nasrabadi and R. A. King. Image coding using vector quantization: A review. IEEE Transactions on Communications, 36:957-971, August 1988.
130. J. Nedoma. On the ergodicity and r-ergodicity of stationary probability measures. Z. Wahrsch. Verw. Gebiete, 2:90-97, 1963.
131. J. Nedoma. The synchronization for ergodic channels. Transactions Third Prague Conf. Information Theory, Stat. Decision Functions, and Random Processes, pages 529-539, 1964.
132. D. L. Neuhoff and R. K. Gilbert. Causal source codes. IEEE Trans. Inform. Theory, 28:701-713, 1982.
133. D. L. Neuhoff, R. M. Gray, and L. D. Davisson. Fixed rate universal block source coding with a fidelity criterion. IEEE Trans. Inform. Theory, 21:511-523, 1975.
134. D. L. Neuhoff and P. C. Shields. Channels with almost finite memory. IEEE Trans. Inform. Theory, pages 440-447, 1979.
135. D. L. Neuhoff and P. C. Shields. Channel distances and exact representation. Inform. and Control, 55(1), 1982.
136. D. L. Neuhoff and P. C. Shields. Channel entropy and primitive approximation. Ann. Probab., 10(1):188-198, 1982.
137. D. L. Neuhoff and P. C. Shields. Indecomposable finite state channels and primitive approximation. IEEE Trans. Inform. Theory, 28:11-19, 1982.
138. David Lee Neuhoff. Source coding and distance measures on random processes. PhD thesis, Stanford University, Stanford, California, August 1974.
139. D. Ornstein. Bernoulli shifts with the same entropy are isomorphic. Advances in Math., 4:337-352, 1970.
140. D. Ornstein. An application of ergodic theory to probability theory. Ann. Probab., 1:43-58, 1973.
141. D. Ornstein. Ergodic Theory, Randomness, and Dynamical Systems. Yale University Press, New Haven, 1975.
142. D. Ornstein and B. Weiss. The Shannon-MacMillan-Breiman theorem for a class of amenable groups. Israel J. of Math, 44:53-60, 1983.
143. P. Papantoni-Kazakos and R. M. Gray. Robustness of estimators on stationary observations. Ann. Probab., 7:989-1002, Dec. 1979.
144. W. A. Pearlman. Sliding-block and random source coding with constrained size reproduction alphabets. IEEE Trans. Commun., 30(8):1859-1867, Aug. 1982.
145. W. B. Pennebaker and J. Mitchell. JPEG Still Image Data Compression Standard. Kluwer, Norwell, Massachusetts, 2004.
146. A. Perez. Notions généralisees d'incertitude, d'entropie et d'information du point de vue de la théorie des martingales. In Transactions First Prague Conf. on Information Theory, Stat. Decision Functions, and Random Processes, pages 183-208. Czech. Acad. Sci. Publishing House, 1957.
147. A. Perez. Sur la convergence des incertitudes, entropies et informations échantillon vers leurs valeurs vraies. In Transactions First Prague Conf. on Information Theory, Stat. Decision Functions, and Random Processes, pages 245-252. Czech. Acad. Sci. Publishing House, 1957.
148. A. Perez. Sur la théorie de l'information dans le cas d'un alphabet abstrait. In Transactions First Prague Conf. on Information Theory, Stat. Decision Functions, Random Processes, pages 209-244. Czech. Acad. Sci. Publishing House, 1957.
149. A. Perez. Extensions of Shannon-MacMillan's limit theorem to more general stochastic processes processes. In Third Prague Conf. on Inform. Theory, Decision Functions, and Random Processes, pages 545-574, Prague and New York, 1964. Publishing House Czech. Akad. Sci. and Academic Press.
150. K. Petersen. Ergodic Theory. Cambridge University Press, Cambridge, 1983.
151. M. S. Pinsker. Dynamical systems with completely positive or zero entropy. Soviet Math. Dokl., 1:937-938, 1960.
152. M. S. Pinsker. Information and information stability of random variables and processes. Holden Day, San Francisco, 1964. Translated by A. Feinstein from the Russian edition published in 1960 by Izd. Akad. Nauk. SSSR.
153. M. Rabbani and P. W. Jones. Digital Image Compression Techniques, Tutorial Texts in Optical Engineering. volume 7 SPIE Optical Engineering Press, Bellingham, Washington, 1991.
154. S.T. Rachev. Probability Metrics and the Stability of Stochastic Models. John Wiley & Sons Ltd, Chichester, 1991.
155. S.T. Rachev and L. Rüschendorf. Mass Transportation Problems: Applications. Probability and ITS Applications, Vol. II Springer- Verlag, New York, 1998.
156. D. Ramachandran. Perfect Measures. ISI Lecture Notes, Indian Statistical Institute, Calcutta, India, 1979.
157. T. Richardson and R. Urbanke. Modern Coding Theory. Cambridge University Press, March 2008.
158. V. A. Rohlin and Y. G. Sinai. Construction and properties of invariant measurable partitions. Soviet Math. Dokl., 2:1611-1614, 1962.
159. K. Rose. A mapping approach to rate-distortion computation and analysis. EEE Trans. Inform. Theory, 40(6):1939-1952, Nov. 1994.
160. Y. Rubner, C. Tomasi, and L.J. Guibas. A metric for distributions with applications to image databases. In Proceedings of the IEEE International Conference on Computer Vision, pages 59-66, Bombay, India, Jan 1998.
161. K. Sayood. Introduction to Data Compression. Morgan Kaufmann Publishers, San Francisco, second edition, 2000.
162. V. V. Sazanov. On perfect measures. Izv. Akad. Nauk SSSR, 26:391-414, 1962.
163. American Math. Soc. 2, No. 48, pp. 229-254, 1965.
164. C. E. Shannon. A mathematical theory of communication. Bell Syst. Tech. J., 27:379-423, 1948.
165. C. E. Shannon. Coding theorems for a discrete source with a fidelity criterion. In IRE National Convention Record, Part 4, pages 142-163, 1959.
166. P. C. Shields. The Theory of Bernoulli Shifts. The University of Chicago Press, Chicago, Ill., 1973.
167. P. C. Shields. The ergodic and entropy theorems revisited. IEEE Trans. Inform. Theory, 33:263-266, 1987.
168. P. C. Shields. The interactions between ergodic theory and information theory. IEEE Trans. Inform. Theory, 40:2079-2093, 1998.
169. P. C. Shields and D. L. Neuhoff. Block and sliding-block source coding. IEEE Trans. Inform. Theory, 23:211-215, 1977.
170. Y. G. Sinai. On the concept of entropy of a dynamical system. Dokl. Akad. Nauk. SSSR, 124:768-771, 1959. (In Russian.).
171. Y. G. Sinai. Weak isomorphism of transformations with an invariant measure. Soviet Math. Dokl., 3:1725-1729, 1962.
172. Y. G. Sinai. Introduction to Ergodic Theory, Princeton University Press, Princeton, 1976.
173. D. Slepian. A class of binary signaling alphabets. Bell Syst. Tech. J., 35:203-234, 1956.
174. D. Slepian, Key Papers in the Development of Information Theory. IEEE Press, New York, 1973.
175. M. Smorodinsky. A partition on a bernoulli shift which is not weakly bernoulli. Theory of Computing Systems, 5(3):201-203, 1971.
176. A. D. Sokai. Existence of compatible families of proper regular conditional probabilities. Z. Wahrsch. Verw. Gebiete, 56:537-548, 1981.
177. H. Steinhaus. Sur la division des corp materiels en parties. Bull. Acad. Polon. Sci., IV(III):801-804, 1956.
178. L. C. Stewart. Trellis data compression. Stanford university information systems lab technical report l905-1, Stanford University, July 1981. Stanford University Ph.D. thesis.
179. L. C. Stewart, R. M. Gray, and Y. Linde. The design of trellis waveform coders. IEEE Trans. Comm., 30:702-710, April 1982.
180. D.S. Taubman and M.W. Marcellin. JPEG2000: Image Compression Fundamentals, Standards and Practice. Kluwer, Norwell, Massachusetts, 2004.
181. A. H. Tchen. Inequalities for distributions with given marginals. The Annals of Probability, 8(4):814-827, Aug 1980.
182. I. Vajda. A synchronization method for totally ergodic channels. In Transactions of the Fourth Prague Conf. on Information Theory, Decision Functions, and Random Processes, pages 611-625, Prague, 1965.
183. S. S. Vallender. Computing the wasserstein distance between probability distributions on the line. Theory Probab. Appl., 18:824-827, 1973.
184. S. R. S. Varadhan. Large Deviations and Applications Industrial and Applied Mathematics, Philadelphia, 1984.
185. L. N. Vasershtein. Markov processes on countable product space describing large systems of automata. Problemy Peredachi Informatsii, 5:64-73, 1969.
186. S. Verdu and S. McLaughlin. Information Theory: 50 Years of Discovery. IEEE Press, Piscataway, New Jerseyu, 2000.
187. A.M. Vershik. The kantorovich metric: The initial history and little-known applications. POMI, 312:69-85, 2004.
188. C. Villani. Topics in Optimal Transportation, Graduate Studies in Mathematics. volume 58, American Mathematical Society, Providence, RI, 2003.
189. C. Villani. Optimal Transport, Old and New, volume 338. Springer, 2009.
190. A. J. Viterbi and J. K. Omura. Trellis encoding of memoryless discrete-time sources with a fidelity criterion. IEEE Trans. Inform. Theory, 20:325-332, May 1974.
191. A. J. Viterbi and J. K. Omura. Principles of Digital Communication and Coding. McGraw-Hill, New York, 1979.
192. J. vonNeumann. Zur operatorenmethode in der klassischen mechanik. Ann. of Math., 33:587-642, 1932.
193. P. Walters. Ergodic Theory-Introductory Lectures. Lecture Notes in Mathematics No. 458. Springer-Verlag, New York, 1975.
194. E. J. Weldon Jr. and W. W. Peterson. Error Correcting Codes. MIT Press, Cambridge, Mass., 1971. Second Ed.
195. S. G. Wilson and D. W. Lytle. Trellis coding of continuous-Amplitude memoryless sources. IEEE Trans. Inform. Theory, 23:404-409, May 1977.
196. K. Winkelbauer. Communication channels with finite past history. Transactions of the Second Prague Conf. on Information Theory, Decision Functions, and Random Processes, pages 685-831, 1960.
197. J. Wolfowitz. Strong converse of the coding theorem for the general discrete finite-memory channel. Inform. and Control, 3:89-93, 1960.
198. J. Wolfowitz. Coding Theorems of Information Theory. Springer-Verlag, New York, 1978. Third edition.
199. A. Wyner. A definition of conditional mutual information for arbitrary ensembles. Inform. and Control, pages 51-59, 1978.
200. J. Ziv. Distortion-rate theory for individual sequences. IEEE Trans. Inform. Theory, 26:137-143, Mar. 1980.
201. J. Ziv. Universal quantization. IEEE Trans. Inform. Theory, 31:344-347, 1985.