On universal simulation of information sources using training data

IEEE Transactions on Information Theory

Volumn 50, Issue 1, 2004, Pages 5-20

  Merhav, Neri ^a   Weinberger, Marcelo J ^b  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^b HEWLETT PACKARD LABORATORIES   (United States)

Author keywords

Mutual information; Random number generators; Random process simulation; Typical sequences

Indexed keywords

ALGORITHMS; COMMUNICATION CHANNELS (INFORMATION THEORY); FUNCTIONS; INFORMATION THEORY; PROBABILITY DISTRIBUTIONS; RANDOM NUMBER GENERATION; SET THEORY; SPEECH SYNTHESIS; SPURIOUS SIGNAL NOISE;

INFORMATION SOURCES; RANDOM NUMBER GENERATORS; RANDOM PROCESS SIMULATION; UNIVERSAL SIMULATION;

RANDOM PROCESSES;

EID: 1242263526     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2003.821993     Document Type: Article

References (23)

1. S. M. Ali and S. D. Silvey, "A general class of coefficients of divergence of one distribution from another," J. Roy. Statist. Soc. Ser. B, vol. 28, pp. 131-142, 1966.
2. K. Atteson, "The asymptotic redundancy of Bayes rules for Markov chains," IEEE Trans. Inform. Theory, vol. 45, pp. 2104-2109, Sept. 1999.
3. E. A. Bender and L. B. Richmond, "Central and local limit theorems applied to asymptotic enumeration II: Multivariate generating functions," J. Combin. Theory Ser. A, vol. 34, pp. 255-265, 1983.
4. R. N. Bhattacharya and R. R. Rao, Normal Approximation and Asymptotic Expansions. New York: Wiley, 1976.
5. A. A. Borovkov and A. A. Mogulskii, "Integro-local limit theorems including large deviations for sums of random vectors," Theory Probab. Appl., vol. 43, pp. 1-12, 1999.
6. B. S. Clarke and A. R. Barron, "Information-theoretic asymptotics of Bayes methods," IEEE Trans. Inform. Theory, vol. 36, pp. 453-471, May 1990.
7. T. M. Cover, "Enumerative source encoding," IEEE Trans. Inform. Theory, vol. IT-19, pp. 73-77, Jan. 1973.
8. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
9. I. Csiszár, "Eine Informationstheoretische Ungleichung und ihre Anwendung auf den Beweis der Ergodizität von Markoffschen Ketten," Publ. Math. Inst. Hungar. Acad. Sci., vol. 8, pp. 85-108, 1963.
10. I. Csiszár, "On topological properties of f-divergences," Studia Scient. Hung., vol. 2, pp. 329-339, 1967.
11. I. Csiszár and J. Körner, Information Theory: Coding Theorems for Discrete Memoryless Systems. New York: Academic, 1981.
12. W. Feller, An Introduction to Probability Theory and its Applications. New York: Wiley, 1971.
13. R. G. Gallager, "Variations on a theme by Huffman," IEEE Trans. Inform. Theory, vol. IT-24, pp. 668-674, Nov. 1978.
14. T. S. Han and M. Hoshi, "Interval algorithm for random number generation," IEEE Trans. Inform. Theory, vol. 43, pp. 599-611, Mar. 1997.
15. T. S. Han and S. Verdú, "Approximation theory of output statistics," IEEE Trans. Inform. Theory, vol. 39, 752-772, May 1993.
16. J. Rissanen, "Universal coding, information, prediction, and estimation," IEEE Trans. Inform. Theory, vol. IT-30, pp. 629-636, July 1984.
17. Y. Steinberg and S. Verdú, "Channel simulation and coding with side information," IEEE Trans. Inform. Theory, vol. 40, pp. 634-646, May 1994.
18. Y. Steinberg and S. Verdú, "Simulation of random processes and rate-distortion theory," IEEE Trans. Inform. Theory, vol. 42, pp. 63-86, Jan. 1996.
19. T. Uyematsu and F. Kanaya, "Channel simulation by interval algorithm: A performance analysis of interval algorithm," IEEE Trans. Inform. Theory, vol. 45, pp. 2121-2129, Sept. 1999.
20. K. Visweswariah, S. R. Kulkarni, and S. Verdú, "Separation of random number generation and resolvability," IEEE Trans. Inform. Theory, vol. 46, pp. 2237-2241, Sept. 2000.
21. P. Whittle, "Some distributions and moment formulae for the Markov chain," J. Roy. Stat. Soc., Ser. B, vol. 17, pp. 235-242, 1955.
22. M. Zakai and J. Ziv, "A generalization of the rate-distortion theory and applications," in Information Theory New Trends and Open Problems, G. Longo, Ed. Berlin, Germany: Springer-Verlag, 1975, pp. 87-123.
23. J. Ziv and M. Zakai, "On functionals satisfying a data-processing theorem," IEEE Trans. Inform. Theory, vol. IT-19, pp. 275-283, May 1973.