The method of types

IEEE Transactions on Information Theory

Volumn 44, Issue 6, 1998, Pages 2505-2523

  Csiszâr, Imre ^a,b  

^a IEEE   (Hungary)

^b ALFRÉD RÉNYI INSTITUTE OF MATHEMATICS   (Hungary)

Author keywords

Arbitrarily varying channels; Choice of decoder; Counting approach; Error exponents; Extended type concepts; Hypothesis testing; Large deviations; Multiuser problems; Universal coding

Indexed keywords

CHANNEL CAPACITY; DATA STORAGE EQUIPMENT; ENCODING (SYMBOLS); INFORMATION THEORY; PROBABILITY; SIGNAL DISTORTION;

ARBITRARILY VARYING CHANNELS; CHOICE OF DECODER; COUNTING APPROACH; ERROR EXPONENTS; EXTENDED TYPE CONCEPTS; HYPOTHESIS TESTING; LARGE DEVIATIONS; MULTIUSER PROBLEMS; SHANNON THEORY; UNIVERSAL CODING;

COMMUNICATION CHANNELS (INFORMATION THEORY);

EID: 0032183789     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.720546     Document Type: Article

References (85)

1. R. Ahlswede, Multi-way communication channels, in P roc. 2nd Int. Symp. Inform. Theory (Tsahkadzor, Armenian SSR, 1971). Budapest, Hungary: Akadémiai Kiadô, 1973, pp. 23-52.
2. Elimination of correlation in random codes for arbitrarily varying channels, Z. Wahrscheinlichkeitsth. Verw. Gebiete, vol. 44, pp. 159-175, 1978.
3. Coloring hypergraphs: A new approach to multi-user source coding I-II, J. Comb. Inform. Syst. Sci., vol. 4, pp. 76-115, 1979, and
4. vol. 5, pp. 220-268, 1980.
5. A method of coding and an application to arbitrarily varying channels, J. Comb. Inform. Systems Sci., vol. 5, pp. 10-35, 1980.
6. Extremal properties of rate distortion functions, IEEE Trans. Inform. Theory, vol. 36, pp. 166-171, Jan. 1990.
7. The maximal error capacity of arbitrarily varying channels for constant list sizes, IEEE Trans. Inform. Theory, vol. 39, pp. 1416-1417, 1993.
8. R. Ahlswede and N. Cai, Two proofs of Pinsker' s conjecture concerning arbitrarily varying channels, IEEE Trans. Inform. Theory, vol. 37, pp. 1647-1649, Nov. 1991.
9. -, Arbitrarily varying multiple-access channels, part 1. Ericson's symmetrizability is adequate, Gubner's conjecture is true, Preprint 96-068, Sonderforschungsbereich 343, Universität Bielefeld, Bielefeld, Germany.
10. R. Ahlswede, N. Cai, and Z. Zhang, Erasure, list, and detection zero-error capacities for low noise and a relation to identification, IEEE Trans. Inform. Theory, vol. 42, pp. 52-62, Jan. 1996.
11. R. Ahlswede and J. Wolfowitz, The capacity of a channel with arbitrarily varying cpfs and binary output alphabets, Z. Wahrscheinlichkeitsth. Verw. Gebiete, vol. 15, pp. 186-194, 1970.
12. R. Ahlswede, E. Yang, and Z. Zhang, Identification via compressed data, IEEE Trans. Inform. Theory, vol. 43, pp. 48-70, Jan. 1997.
13. V. B. Balakirsky, A converse coding theorem for mismatched decoding at the output of binary-input memoryless channels, IEEE Trans. Inform. Theory, vol. 41, pp. 1989-1902, Nov. 1995.
14. T. Berger, Rate Distortion Theory: A Mathematical Basis for Data Compression. Englewood Cliffs, NJ: Prentice Hall, 1971.
15. P. Billingsley, Statistical methods in Markov chains, Ann. Math. Statist., vol. 32, pp. 12-410; correction, p. 1343, 1961.
16. D. Blackwell, L. Breiman, and A. J. Thomasian, The capacities of certain channel classes under random coding, Ann. Math. Statist., vol. 31, pp. 558-567, 1960.
17. R. E. Blahut, Composition bounds for channel block codes, IEEE Trans. Inform. Theory, vol. IT-23, pp. 656-674, 1977.
18. V. Blinovsky, P. Narayan, and M. Pinsker, Capacity of the arbitrarily varying channel under list decoding, Probl. Pered. Inform., vol. 31, pp. 99-113, 1995.
19. L. Boltzmann, Beziehung zwischen dem zweiten Hauptsatze der mechanischen Wärmetheorie und der Wahrscheinlichkeitsrechnung respektive den Sätzen über das Wärmegleichgewicht, Wien. Ber., vol. 76, pp. 373-435, 1877.
20. L. B. Boza, Asymptotically optimal tests for finite Markov chains, Ann. Math. Statist., vol. 42, pp. 1992-2007, 1971.
21. H. Chernoff, A measure of asymptotic efficiency for tests of a hypothesis based on a sum of observations, Ann. Math. Statist., vol. 23, pp. 493-507, 1952.
22. G. Cohen, I. Honkala, S. Litsyn, and A. Lobstein, Covering Codes. Amsterdam, The Netherlands: North Holland, 1997.
23. I. Csiszär, Joint source-channel error exponent, Probl. Contr. Inform. Theory, vol. 9, pp. 315-328, 1980.
24. Linear codes for sources and source networks: Error exponents, universal coding, IEEE Trans. Inform. Theory, vol. IT-28, pp. 585-592, July 1982.
25. On the error exponent of source-channel transmission with a distortion threshold, IEEE Trans. Inform. Theory, vol. IT-28, pp. 823-828, Nov. 1982.
26. Arbitrarily varying channels with general alphabets and states, IEEE Trans. Inform. Theory, vol. 38, pp. 1725-1742, 1992.
27. I. Csiszär, T. M. Cover, and B. S. Choi, Conditional limit theorems under Markov conditioning, IEEE Trans. Inform. Theory, vol. IT-33, pp. 788-801, Nov. 1987.
28. I. Csiszär and J. Körner Towards a general theory of source networks, IEEE Trans. Inform. Theory, vol. IT-26, pp. 155-165, Jan. 1980.
29. -, Graph decomposition: A new key to coding theorems, IEEE Trans. Inform. Theory, vol. IT-27, pp. 5-11, Jan. 1981.
30. -, On the capacity of the arbitrarily varying channel for maximum probability of error, Z. Wahrscheinlichkeitsth. Verw. Gebiete, vol. 57, pp. 87-101, 1981.
31. -, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.
32. -, Feedback does not affect the reliability function of a DMC at rates above capacity, IEEE Trans. Inform. Theory, vol. IT-28, pp. 92-93, Jan. 1982.
33. I. Csiszär, J. Körner, and K. Marton, A new look at the error exponent of discrete memoryless channels, presented at the IEEE Int. Symp. Information Theory (Cornell Univ., Ithaca, NY, 1977), preprint.
34. I. Csiszâr and P. Narayan, The capacity of the arbitrarily varying channel revisited: Positivity, constraints, IEEE Trans. Inform. Theory, vol. 34, pp. 181-193, Mar. 1988.
35. Capacity and decoding rules for arbitrarily varying channels, IEEE Trans. Inform. Theory, vol. 35, pp. 752-769, July 1989.
36. Channel capacity for a given decoding metric, IEEE Trans. Inform. Theory, vol. 41, pp. 35-43, Jan. 1995.
37. -, Capacity of the Gaussian arbitrarily varying channel, IEEE Trans. Inform. Theory, vol. 37, pp. 18-26, Jan. 1991.
38. I. Csiszâr and P. C. Shields, Redundancy rates for renewal and other processes, IEEE Trans. Inform. Theory, vol. 42, pp. 2065-2072, Nov. 1996.
39. L. Davisson, G. Longo, and A. Sgarro, The error exponent for the noiseless encoding of finite ergodic Markov sources, IEEE Trans. Inform. Theory, vol. IT-27, pp. 431-348, 1981.
40. A. Dembo and O. Zeitouni, Large Deviations Techniques and Applica-tions. Jones and Bartlett, 1993.
41. R. L. Dobrushin and S. Z. Stambler, Coding theorems for classes of arbitrarily varying discrete memoryless channels, Probl. Pered. Inform., vol. 11, no. 2, pp. 3-22, 1975, English translation.
42. G. Dueck, Maximal error capacity regions are smaller than average error capacity regions for multi-user channels, Probl. Contr. Inform. Theory, vol. 7, pp. 11-19, 1978.
43. G. Dueck and J. Körner, Reliability function of a discrete memoryless channel at rates above capacity, IEEE Trans. Inform. Theory, vol. IT-25, pp. 82-85, Jan. 1979.
44. T. Ericson, Exponential error bounds for random codes in the arbitrarily varying channel, IEEE Trans. Inform. Theory, vol. IT-31, pp. 42-418, Jan. 1985.
45. R. M. Fano, Transmission of Information, A Statistical Theory of Communications. New York: Wiley, 1961.
46. L. Finesso, C. C. Liu, and P. Narayan, The optimal error exponent for Markov order estimation, IEEE Trans. Inform. Theory, vol. 42, pp. 1488-1497, Sept. 1996.
47. R. G. Gallager, A simple derivation of the coding theorem and some applications, IEEE Trans. Inform. Theory, vol. IT-11, pp. 3-18, Jan. 1965.
48. -, A perspective on multiaccess channels, IEEE Trans. Inform. Theory, vol. IT-31, pp. 124-142, Mar. 1985.
49. V. D. Goppa, Nonprobabilistic mutual information without memory, Probl. Contr. Inform. Theory, vol. 4, pp. 97-102, 1975.
50. P. Groeneboom, J. Oosterhoff, and F. H. Ruymgaart, Large deviation theorems for empirical probability measures, Ann. Probab., vol. 7, pp. 553-586, 1979.
51. J. A. Gubner, On the deterministic-code capacity of the multiple-access arbitrarily varying channel, IEEE Trans. Inform. Theory, vol. 36, pp. 262-275, Mar. 1990.
52. M. Gutman, Asymptotically optimal classification for multiple tests with empirically observed statistics, IEEE Trans. Inform. Theory, vol. 35, pp. 401-408, Mar. 1989.
53. E. A. Haroutunian, Bounds on the error probability exponent for semicontinuous memoryless channels, Probl. Pered. Inform., vol. 4, no. 4, pp. 37-48, 1968, in Russian.
54. W. Hoeffding, Asymptotically optimal tests for multinominal distributions, Ann. Math. Statist., vol. 36, pp. 1916-1921, 1956.
55. B. L. Hughes, The smallest list for the arbitrarily varying channel, IEEE Trans. Inform. Theory, vol. 43, pp. 803-815, May 1997.
56. J. Y. N. Hui, Fundamental issues of multiple accessing, Ph.D. disser-tation, MIT, Cambridge, MA, 1983.
57. J. H. Jahn, Coding for arbitrarily varying multiuser channels, IEEE Trans. Inform. Theory, vol. IT-27, pp. 212-226, Mar. 1981.
58. J. Kiefer and J. Wolfowitz, Channels with arbitrarily varying channel probability functions, Inform. Contr., vol. 5, pp. 44-54, 1962.
59. J. Körner and A. Sgarro, Universally attainable error exponents for broadcast channels with degraded message sets, IEEE Trans. Inform. Theory, vol. IT-26, pp. 670-679, 1980.
60. A. Lapidoth, Mismatched decoding and the multiple-access channel, IEEE Trans. Inform. Theory, vol. 42, pp. 1439-1452, Sept. 1996.
61. H. J. Liao, Multiple access channels, Ph.D. dissertation, Univ. Hawaii, Honolulu, 1972.
62. T. Linder, G. Lugosi, and K. Zeger, Fixed-rate universal lossy source coding and rates of convergence for memoryless sources, IEEE Trans. Inform. Theory, vol. 41, pp. 665-676, May 1995.
63. Y. S. Liu and B. L. Hughes, A new universal random coding bound for the multiple-access channel, IEEE Trans. Inform. Theory, vol. 42, pp. 376-386, 1996.
64. G. Longo and A. Sgarro, The source coding theorem revisited: A combinatorial approach, IEEE Trans. Inform. Theory, vol. IT-25, pp. 544-548, May 1979.
65. K. Marton, Error exponent for source coding with a fidelity criterion, IEEE Trans. Inform. Theory, vol. IT-20, pp. 197-199, Mar. 1974.
66. N. Merhav, A comment on 'A rate of convergence result for a universal
67. N. Merhav, M. Gutman, and J. Ziv, On the estimation of the order of a Markov chain and universal data compression, IEEE Trans. Inform. Theory, vol. 35, pp. 1014-1019, Sept. 1989.
68. N. Merhav, G. Kaplan, A. Lapidoth, and S. Shamai, On information rates for mismatched decoders, IEEE Trans. Inform. Theory, vol. 40, pp. 1953-1967, Nov. 1994.
69. S. Natarajan, Large deviations, hypothesis testing, and source coding for finite Markov sources, IEEE Trans. Inform. Theory, vol. IT-31, pp. 360-365, 1985.
70. Y. Oohama, Universal coding for correlated sources with linked encoders, IEEE Trans. Inform. Theory, vol. 42, pp. 837-847, May 1996.
71. Y. Oohama and T. S. Han, Universal coding for the Slepian-Wolf data compression system and the strong converse theorem, IEEE Trans. Inform. Theory, vol. 40, pp. 1908-1919, 1994.
72. J. Pokorny and H. M. Wallmeier, Random coding bound and codes produced by permutations for the multiple-access channel, IEEE Trans. Inform. Theory, vol. IT-31, pp. 741-750, Nov. 1985.
73. I. N. Sanov, On the probability of large deviations of random variables, Mat. Sbornik, vol. 42, pp. 11-414, 1957, in Russian;
74. English translation in Select. Transi. Math. Statist. Probab., vol. 1, pp. 213-244, 1961.
75. E. Schrödinger, Über die Umkehrung der Naturgesetze, Sitzungsber. Preuss. Akad. Wiss. Berlin Phys. Math. Klass., vols. 8/9, pp. 144-153, 1931.
76. C. E. Shannon, R. G. Gallager, and E. R. Berlekamp, Lower bounds to error probability for coding on discrete memoryless channels, Inform. Contr., vol. 10, pp. 65-104 and 522-552, 1967.
77. C. E. Shannon and W. Weaver, The Mathematical Theory of Communication. Urbana, IL: Univ. Illinois Press, 1949.
78. D. Slepian and J. K. Wolf, Noiseless coding of correlated information sources, IEEE Trans. Inform. Theory, vol. IT-19, pp. 471-4180, 1973.
79. 1. E. Telatar and R. G. Gallager, New exponential upper bounds to error and erasure probabilities, in Proc. IEEE Int. Symp. Information Theory (Trondheim, Norway, 1994), p. 379.
80. G. Tusnâdy, On asymptotically optimal tests, Ann. Statist., vol. 5, pp. 358-393, 1977.
81. M. J. Weinberger, N. Merhav, and M. Feder, Optimal sequential probability assignment for individual sequences, IEEE Trans. Inform. Theory, vol. 40, pp. 384-396, Mar. 1994.
82. P. Whittle, Some distributions and moment formulae for the Markov chain, J. Roy. Stat. Soc., ser. B, vol. 17, pp. 235-242, 1955.
83. J. Wolfowitz, Coding Theorems of Information Theory. Berlin, Germany: Springer, 1961.
84. B. Yu and T. P. Speed, A rate of convergence result for a universal dsemifaithful code, IEEE Trans. Inform. Theory, vol. 39, pp. 513-820, May 1993.
85. Z. Zhang, E. Yang, and V. K. Wei, The redundancy of source coding with a fidelity criterion-Part one: Known statistics, IEEE Trans. Inform. Theory, vol. 43, pp. 71-91, Jan. 1997.