On Competitive Prediction and Its Relation to Rate-Distortion Theory

IEEE Transactions on Information Theory

Volumn 49, Issue 12, 2003, Pages 3185-3194+3355

  Weissman, Tsachy ^a   Merhav, Neri ^b  

^a Stanford University   (United States)

^b TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Competitive prediction; Error exponents; Prediction with expert advice; Rate distortion theory; Redundancy; Scandiction; Strong converse

Indexed keywords

ASYMPTOTIC STABILITY; COMPUTATIONAL METHODS; INTEGRAL EQUATIONS; INTEGRATION; OPTIMIZATION; PROBABILITY; PROBLEM SOLVING; RANDOM PROCESSES; SIGNAL FILTERING AND PREDICTION;

COMPETITIVE PREDICTION; RATE DISTORTION THEORY;

INFORMATION THEORY;

EID: 0346076614     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2003.820014     Document Type: Article

References (26)

1. P. H. Algoet, "The strong law of large numbers for sequential decisions under uncertainty," IEEE Trans. Inform. Theory, vol. 40, pp. 609-633, May 1994.
2. _, "Universal schemes for learning the best nonlinear predictor given the infinite past and side information," IEEE Trans. Inform. Theory, vol. 45, pp. 1165-1185, May 1999.
3. E. Arikan and N. Merhav, "Guessing subject to distortion," IEEE Trans. Inform. Theory, vol. IT-44, pp. 1041-1056, May 1998.
4. A. Barron, J. Rissanen, and B. Yu, "The minimum description length principle in coding and modeling," IEEE Trans. Inform. Theory, vol. 44, pp. 2743-2760, Oct. 1998.
5. T. Berger, Rate-Distortion Theory: A Mathematical Basis for Data Compression. Englewood Cliffs, N.J: Prentice-Hall, 1971.
6. T. Berger and J. D. Gibson, "Lossy source coding," IEEE Trans. Inform. Theory, vol. 44, pp. 2693-2723, Oct. 1998.
7. N. Cesa-Bianchi, Y. Freund, D. P. Helmbold, D. Haussler, R. Schapire, and M. K. Warmuth, "How to use expert advice," J. ACM, vol. 44, no. 3, pp. 427-485, 1997.
8. N. Cesa-Bianchi and G. Lugosi, "Minimax regret under log loss for general classes of experts," in Proc. 12th Annu. Workshop on Computational Learning Theory, New York, 1999, pp. 12-18.
9. _, "On sequential prediction of individual sequences relative to a set of experts," Ann. Statist., vol. 27, no. 6, pp. 1865-1895, 1999.
10. _, "Potential-based algorithms in on-line prediction and game theory," in Proc. 5th Europ. Conf. Computational Learning Theory (EuroCOLT 2001), Amsterdam, The Netherlands, July 2001.
11. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
12. M. Feder, N. Merhav, and M. Gutman, "Universal prediction of individual sequences," IEEE Trans. Inform. Theory, vol. 38, pp. 1258-1270, July 1992.
13. R. G. Gallager, Information Theory and Reliable Communication. New York: Wiley, 1968.
14. T. S. Han, "An information-spectrum approach to source coding theorems with a fidelity criterion," IEEE Trans. Inform. Theory, vol. 43, pp. 1145-1164, July 1997.
15. D. Haussler, J. Kivinen, and M. K. Warmuth, "Sequential prediction of individual sequences under general loss functions," IEEE Trans. Inform. Theory, vol. 44, pp. 1906-1925, Sept. 1998.
16. K. Marton, "Error exponent for source coding with a fidelity criterion," IEEE Trans. Inform. Theory, vol. IT-20, pp. 197-199, Mar. 1974.
17. N. Merhav and M. Feder, "Universal prediction," IEEE Trans. Inform. Theory, vol. IT-44, pp. 2124-2147, Oct. 1998.
18. N. Merhav and T. Weissman, "Scanning and prediction in multi-dimensional data arrays," IEEE Trans. Inform. Theory, vol. 49, pp. 65-82, Jan. 2003.
19. A. Puhalskii and V. Spokoiny, "On large deviation efficiency in statistical inference," Bernoulli, vol. 4, no. 2, pp. 203-272, 1998.
20. J. Rissanen, "Universal coding, information, prediction, and estimation," IEEE Trans. Inform. Theory, vol. IT-30, pp. 629-636, July 1984.
21. K. Rose, "A mapping approach to rate-distortion computation and analysis," IEEE Trans. Inform. Theory, vol. 40, pp. 1939-1952, Nov. 1994.
22. C. E. Shannon, "The zero-error capacity of a noisy channel," IRE Trans. Inform. Theory, vol. IT-2, pp. 8-19, Sept. 1956.
23. V. Vovk, "Aggregating strategies," in Proc. 3rd Annu. Workshop on Computational Learning Theory, San Mateo, CA, 1990, pp. 371-383.
24. _, "A game of prediction with expert advice," in Proc 8th Annu. Workshop on Computational Learning Theory., New York, 1995, pp. 51-60.
25. M. J. Weinberger, N. Merhav, and M. Feder, "Optimal sequential probability assignment for individual sequences," IEEE Trans. Inform. Theory, vol. 40, pp. 384-396, Mar. 1994.
26. T. Weissman and N. Merhav, "Universal prediction of individual binary sequences in the presence of noise," IEEE Trans. Inform. Theory, vol. 47, pp. 2151-2173, Sept. 2001.