A Measure of Relative Entropy Between Individual Sequences with Application to Universal Classification

IEEE Transactions on Information Theory

Volumn 39, Issue 4, 1993, Pages 1270-1279

  Ziv, Jacob ^a   Merhav, Neri ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; BINARY SEQUENCES; COMPUTATIONAL METHODS; DATA COMPRESSION; FINITE AUTOMATA; PARAMETER ESTIMATION; PROBABILITY; RANDOM PROCESSES;

FINITE ALPHABET; INFORMATION DIVERGENCE; LEMPEL-ZIV ALGORITHM; MARKOV PROCESSES; RELATIVE ENTROPY;

SIGNAL PROCESSING;

EID: 0027632406     PISSN: 00189448     EISSN: 15579654     Source Type: Journal    
DOI: 10.1109/18.243444     Document Type: Article

References (15)

1. J. Ziv and A. Lempel, “Compression of individual sequences via variable-rate coding,” IEEE Trans. Inform. Theory, vol. IT-24, no. 5, pp. 530–536, Sept. 1978.
2. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
3. A. D. Wyner and J. Ziv. “Fixed-data-base version of the Lempel—Ziv data compression algorithm,” IEEE Trans. Inform. Theory, vol. 37, no. 3, pp. 878–880, May 1991.
4. H. Van Trees, Detection Estimation, and Modulation Theory. New York: Wiley, 1968.
5. R. E. Blahut, Principles and Practice of Information Theory. New York:Addison-Wesley, 1987.
6. J. Ziv, “On classification with empirically observed statistics and universal data compression,” IEEE Trans. Inform. Theory, vol. 34, no. 2, pp. 278–286, Mar. 1988.
7. M. Gutman, “Asymptotically optimal classification for multiple tests with empirically observed statistics,” IEEE Trans. Inform. Theory, vol. 35, no. 2, pp. 401–408, Mar. 1989.
8. M. Gutman, “On tests for independence, tests for randomness and universal data compression,” submitted for publication.
9. N. Merhav, M. Gutman, and J. Ziv, “On the estimation of the order of a Markov chain and universal data compression,” IEEE Trans. Inform. Theory, vol. 35, no. 5, pp. 1014–1019, Sept. 1989.
10. O. Zeitouni and M. Gutman, “On universal hypotheses testing via large deviations,” IEEE Trans. Inform. Theory, vol. 37, no. 2, pp. 285–290, Mar. 1991.
11. O. Zeitouni, J. Ziv, and N. Merhav, “When is the generalized likelihood ratio test optimal?” IEEE Trans. Inform. Theory, vol. 38, no. 5, pp. 1597–1602, Sept. 1992.
12. E. Plotnik, M. J. Weinberger, and J. Ziv, “Upper bounds on the probability of sequences emitted by finite-state sources and on the redundancy of the Lempel—Ziv algorithm,” IEEE Trans. Inform. Theory, vol. 38, no. 1, pp. 66–72, Jan. 1992.
13. I. Csiszár and J. Korner, Information Theory: Coding Theorems for Discrete Mernoryless Systems. New York: Academic Press, 1981.
14. M. Rodeh, V. R. Pratt, and S. Even “Linear algorithm for data compression via string matching,” J. Assoc. Comput. Mach., vol. 28, sec. 3, pp. 16–24, Jan. 1981.
15. L. D. Davisson, G. Longo, and A. Sgarro, “The error exponent for the noiseless encoding of finite ergodic Markov sources,” IEEE Trans. Inform. Theory, vol. IT-27, no. 4, pp. 431–438, July 1981.