Asymptotic minimax regret for data compression, gambling, and prediction

IEEE Transactions on Information Theory

Volumn 46, Issue 2, 2000, Pages 431-445

  Xie, Qun ^b   Barron, Andrew R ^a,c  

^a IEEE   (United States)

^b GENERAL ELECTRIC COMPANY   (United States)

^c YALE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ASYMPTOTIC MINIMAX REGRET METHOD; DIRICHLET METHOD; JEFFREYS' PRIOR METHOD;

APPROXIMATION THEORY; ASYMPTOTIC STABILITY; DATA COMPRESSION; OPTIMIZATION; PROBABILITY DENSITY FUNCTION; REDUNDANCY;

SIGNAL ENCODING;

EID: 0033901631     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.825803     Document Type: Article

References (40)

1. A. R. Barron, "Logically smooth density estimation," Ph.D. dissertation, Stanford Univ., Stanford, CA, 1985.
2. _, "Are Bayes rules consistent in information," in Open Problems in Communication and Computing, T. M. Cover and B. Gopinath, Eds., 1987, pp. 85-91.
3. _, "Exponential convergence of posterior probabilities with implications for Bayes estimations of density functions," Dept. Stat., Univ. Illinois, Urbana-Champaign, IL, Tech. Rep. 7, 1987.
4. A. R. Barron, J. Rissanen, and B. Yu, "The minimum description length principle in coding and modeling," IEEE Trans. Inform. Theory, vol. 44, pp. 2743-2760, Oct. 1998.
5. B. S. Clarke and A. R. Barron, "Information-theoreetic asymptotics of Bayes methods," IEEE Trans. Inform. Theory, vol. 36, pp. 453-471, May 1990.
6. _, "Jeffreys' prior is asymptotically least favorable under entropy risk," J. Statist. Planning Inference, vol. 41, pp. 37-60, Aug. 1994.
7. T. M. Cover and E. Ordentlich, "Universal portfolios with side information," IEEE Trans. Inform. Theory, vol. 42, pp. 348-363, Mar. 1996.
8. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
9. L. D. Davisson and A. Leon-Garcia, "A source matching approach to finding minimax codes," IEEE Trans. Inform. Theory, vol. IT-26, pp. 166-174, Mar. 1980.
10. _, "A source matching approach to finding minimax codes," IEEE Trans. Inform. Theory, vol. IT-26, pp. 166-174, Mar. 1980.
11. L. D. Davisson, R. J. McEliece, M. B. Pursley, and M. S. Wallace, "Efficient universal noiseless source codes," IEEE Trans. Inform. Theory, vol. IT-27, pp. 269-279, May 1981.
12. M. Feder, N. Merhav, and M. Gutman, "Universal prediction of individual sequences," IEEE Trans. Inform. Theory, vol. 38, pp. 1258-1268, July 1992.
13. W. Feller, An Introduction to Probability Theory and Its Applications. New York: Wiley , 1968, vol. I, 3rd ed..
14. D. P. Foster, "Prediction in the worst case," Ann. Statist. , vol. 19, no. 2, pp. 1084-1090, 1991.
15. Y. Freund, "Predicting a binary sequence almost as well as the optimal biased coin," in Proc. 9th Annu. Workshop Computational Learning Theory, 1996, pp. 89-98.
16. D. Haussler, "A general minimax result for relative entropy," IEEE Trans. Inform. Theory, vol. 43, pp. 1276-1280, 1997.
17. D. Haussler and A. R. Barron, "How well do Bayes methods work for on-line prediction of {±1} values?," in Proc. NEC Symp. Computation and Cognition, 1992.
18. D. Haussler, J. Kivinen, and M. K. Warmuth, "Sequential prediction of individual sequences under general loss functions," IEEE Trans. Inform. Theory, vol. 44, pp. 1906-1925, Sept. 1998.
19. D. Haussler and M. Opper, "Mutual information, metric entropy and cumulative relative entropy risk," Ann. Statist., vol. 25, pp. 2451-2492, Dec. 1997.
20. H. Jeffreys, Theory of Probability. Oxford, U.K.: Oxford Univ. Press, 1961.
21. T. Kløve, "Bounds on the worst case probability of undetected error," IEEE Trans. Inform. Theory, vol. 41, pp. 298-300, Jan. 1995.
22. R. E. Krichevsky and V. K. Trofimov, "The performance of universal encoding," IEEE Trans. Inform. Theory, vol. IT-27, pp. 199-207, Jan., 1981.
23. N. Merhav and M. Feder, "Universal Sequential Decision Schemes from Individual Sequences," IEEE Trans. Inform. Theory, vol. IT-39, pp. 1280-1292, July 1993.
24. E. Ordentlich and T. M. Cover, "The cost of achieving the best portfolio in hindsight," Math. Oper. Res., 1998.
25. M. Opper and D. Haussler, "Worst case prediction over sequences under log loss," in The Mathematics of Information Coding, Extraction, and Distribution. New York: Springer Verlag, 1997.
26. J. Rissanen, "Universal coding, information, prediction and estimation," IEEE Trans. Inform. Theory, vol. IT-30, pp. 629-636, 1984.
27. _, "Complexity of strings in the class of Markov sources," IEEE Trans. Inform. Theory, vol. IT-32, pp. 526-532, July 1986.
28. _, "Fisher information and stochastic complexity," IEEE Trans. Inform. Theory, vol. 40, pp. 40-47, Jan 1996.
29. Yu. M. Shtarkov, "Encoding of Discrete Sources Under Conditions of Real Restrictions and Data Compression," Speciality 05.13.01, Technical cybernetics and information theory, Ph.D. dissertation, Moscow, USSR, 1980.
30. _, "Universal sequential coding of single messages," Probl. Inform. Transm. , vol. 23, pp. 3-17, July 1988.
31. Yu. M. Shtarkov, Tj. J. Tjalkens, and F. M. J. Willems, "Multi-alphabet university coding of memoryless sources," Probl. Inform. Transm. , vol. 31, pp. 114-127, 1995.
32. J. Suzuki, "Some notes on universal noiseless coding," IEICE Trans. Fundamentals, vol. E78-A, , no. 12, Dec. 1995.
33. W. Szpankowski, "On asymptotics of certain sums arising in coding theory," IEEE Trans. Inform. Theory, vol. 41, no. 6, pp. 2087-2090, Nov. 1995.
34. V. Vovk, "Aggregating strategies," in Proc. 3rd Annu. Workshop Computer Learning Theory, 1990, pp. 371-383.
35. M. J. Weinberger, N. Merhav, and M. Feder, "Optimal sequential probability assignment for individual sequences," IEEE Trans. Inform. Theory, vol. 40, pp. 384-396, Mar. 1994.
36. M. J. Weinberger, N. Rissanen, and M. Feder, "A universal finite memory sources," IEEE Trans. Inform. Theory, vol. 41, pp. 643-652, May 1995.
37. E. T. Whittaker and G. N. Watson, A Course of Modern Analysis, 4 ed. Cambridge, U.K.: Cambridge Univ. Press, 1963.
38. F. M. Willems, Y. M. Shtarkov, and T. J. Tjalkens, "The context tree weighting method: Basic properties," IEEE Trans. Inform. Theory, vol. 41, pp. 653-664, May 1995.
39. Q. Xie and A. R. Barron, "Minimax redundancy for the class of memoryless sources," IEEE Trans. Inform. Theory, vol. 43, pp. 646-657, May 1997.
40. Q. Xie, "Minimax Coding and Prediction," Ph.D. dissertation, Yale Univ., New Haven, CT, May 1997.