The Strong Law of Large Numbers for Sequential Decisions Under Uncertainty

IEEE Transactions on Information Theory

Volumn 40, Issue 3, 1994, Pages 609-633

  Algoet, Paul H ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

asymptotic optimality; ergodic theory; learning from experience; Pattern classification; Sequential decisions; universal decision schemes

Indexed keywords

CLASSIFICATION (OF INFORMATION); INFORMATION THEORY; MATHEMATICAL MODELS; NUMBER THEORY; OPTIMIZATION; PATTERN RECOGNITION; RANDOM PROCESSES;

ASYMPTOTIC OPTIMALITY; ERGODIC THEORY; PATTERN CLASSIFICATION; SEQUENTIAL DECISIONS; UNIVERSAL DECISION SCHEMES;

DECISION THEORY;

EID: 0028428283     PISSN: 00189448     EISSN: 15579654     Source Type: Journal    
DOI: 10.1109/18.335876     Document Type: Article

References (107)

1. P. H. Algoet, “Universal schemes for prediction, gambling, and portfolio selection,” Ann. Prob., vol. 20, 901-941, Apr. 1992.
2. P. H. Algoet and T. M. Cover, “Asymptotic optimality and asymptotic equipartition properties of log-optimum investment,” Ann. Prob., vol. 16, pp. 876-898, Apr. 1988
3. ——, “A sandwich proof of the Shannon-McMillan-Breiman theorem,” Ann. Prob., vol. 16, pp. 899-909, Apr. 1988.
4. T. Ando and I. Amemiya, “Almost everywhere convergence of prediction sequences in Lp(1 < p < ∞),” Z. Wahrscheinlichkeits-theorie verw. Gebiete, vol. 4, pp. 113-120, 1965.
5. A. R. Barron, “The strong ergodic theorem for densities: generalized Shannon-McMillan-Breiman theorem,” Ann. Prob., vol. 13, pp. 1292-1303, Nov. 1985.
6. R. Bell and T. M. Cover, “Game-theoretic optimal portfolios,” Management Sci., vol. 34, pp. 724-733, June 1988.
7. D. P. Bertsekas and S. E. Shreve, Stochastic Optimal Control, The Discrete Time Case. New York: Academic, 1978.
8. D. Blackwell, “Controlled random walks,” in Proc. Int. Congr. Math. 1954. Amsterdam: North-Holland, 1956, vol. III, pp. 336-338.
9. ——, “An analog of the minimax theorem for vector payoffs,” Pacific J. Math., vol. 6, pp. 1-8, 1956.
10. D. Blackwell and L. E. Dubins, “A converse to the dominated convergence theorem,” Illinois J. Math., vol. 7, 508-514, 1963.
11. L. Breiman, “The individual ergodic theorem of information theory,” Ann. Math. Statist., vol. 28, pp. 809-811, “Correction,” Ann. Math. Statist., vol. 31, pp. 809-810, 1960.
12. —,”Optimal gambling systems for favorable games,” in Proc. 4th Berkeley Symp. Math. Statist. Prob. Berkeley, CA: Univ. California Press, 1961, vol. 1, pp. 65-78.
13. D. L. Burkholder, “Successive conditional expectations of an integrable function,” Ann. Math. Statist., vol. 33, pp. 887-893, 1962.
14. K. L. Chung, “Note on some strong laws of large numbers,” Amer. J. Math., vol. 69, pp. 189-192, 1947.
15. Y. S. Chow, “Local convergence of martingales and the law of large numbers,” Ann. Math. Statist., vol. 36, pp. 552-558, 1965.
16. —, “On a strong law of large numbers for martingales,” Ann. Math. Statist., vol. 38, p. 610, 1967.
17. Y. S. Chow and H. Teicher, Probability Theory, 2nd ed. New York: Springer-Verlag, 1988.
18. J. P. Copas, “Compound decisions and empirical Bayes (with discussion),” J. Roy. Statist. Soc., Ser. B, vol. 31, pp. 397-425, 1969.
19. T. M. Cover, “Behavior of sequential predictors of binary sequences,” in Trans 4th Prague Conf. Inform. Theory, Statist. Decision Functions, Random Processes, Prague, 1965. Prague: Czechoslovak Acad. Sci., 1967, pp. 263-272.
20. —, personal communication.
21. —, “Ergodic process selection" in Open Problems in Communication and Computation, T. M. Cover and B. Gopinath, Ed. New York: Springer-Verlag, 1987, p. 153.
22. —, “Universal portfolios,” Math. Finance, vol. 1, pp. 1-29, Jan. 1991.
23. T. M. Cover and D. H. Gluss, “Empirical Bayes stock market portfolios,” Adv. Appl. Math., vol. 7, pp. 170-181, June 1986.
24. T. M. Cover and A. Shenhar,”Compound Bayes predictors for sequences with apparent Markov structure,” IEEE Trans. Syst., Man, Cybern., vol. SMC-7, pp. 421-424, June 1977.
25. J. A. Cuesta and C. Matran, “The strong law of large numbers for k-means and best possible nets of Banach valued random variables,” Prob. Theory Related Fields, vol. 78, pp. 523-534, 1988.
26. A. P. Dawid, “Present position and potential developments: Some personal views, statistical theory, the prequential approach,” J. Roy. Statist. Soc., ser. A, vol. 147, pp. 278-292, 1984.
27. M. DeGroot and M. M. Rao, “Multidimensional information equalities and prediction,” in Proc. Int. Symp. Multivariate Anal., P. R. Krishnaiah, Ed. New York: Academic, 1966, pp. 287-313.
28. J. L. Doob, Stochastic Processes. New York: Wiley, 1953.
29. M. Driml and A. Spacek, “Continuous random decision processes controlled by experience,” in Trans. 1st Prague Conf. Inform. Theory, Statist. Decision Functions, Random Processes, Prague, 1956. Prague: Czechoslovak Acad. Sci,, 1957, pp. 43-60.
30. M. Driml and O. Hanš, “On experience theory problems,” in Trans. 2nd Prague Conf. Inform. Theory, Statist. Decision Functions, Random Processes, Prague, 1959. Prague: Czechoslovak Acad. Sci., 1960, pp. 93-111.
31. J. Dupačová and R. Wets, “Asymptotic behavior of statistical estimators and of optimal solutions of stochastic optimization problems,” Ann. Statist., vol. 16, pp. 1517-1549, Dec. 1988.
32. W. Esty, R. Gillette, M. Hamilton, and D. Taylor, “Asymptotic distribution theory of statistical functionals,” Ann. Inst. Statist. Math., ser. A, vol. 37, pp. 109-129, 1985.
33. V. Fabian and A. Spacek, “Experience in statistical decision problems,” Czechoslovak Math. J., vol. 6, pp. 190-194, 1956, “Correction,” Czechoslovak Math. J., vol. 6, p. 434, 1956.
34. L. T. Femholtz, von Mises Calculus for Statistical Functionals. New York: Springer-Verlag, 1983.
35. A. Gersho and R. M. Gray, Vector Quantization and Signal Compression. Boston, MA: Kluwer Academic, 1991.
36. R. D. Gill, “Non- and semi-parametric maximum likelihood estimators and the von Mises method (Part 1),” Scand. J. Statist., vol. 16, pp. 97-128, 1989.
37. D. C. Gilliland, “Sequential compound estimation,” Ann. Math. Statist., vol. 39, pp. 1890-1905, 1968.
38. —, “Approximation to Bayes risk in sequences of non-finite games,” Ann. Math. Statist., vol. 40, pp. 467-474, 1969.
39. —, “Asymptotic risk stability resulting from play against the past in a sequence of decision problems,” IEEE Trans. Inform. Theory, vol. IT-18, pp. 614-617, Sept. 1972.
40. D. C. Gilliland and J. F. Hannan, “On an extended compound decision problem,” Ann. Math. Statist., vol. 40, pp. 1536-1541, 1969.
41. D. C. Gilliland and M. K. Helmers, “On continuity of the Bayes response,” IEEE Trans. Inform. Theory, vol. IT-24, pp. 506-508, July 1978.
42. C. Gouriéroux, A. Montfort, and A. Trognon, “Pseudo maximum likelihood methods: theory,” Econometrica, vol. 52, pp. 681-700, May 1984.
43. R. M. Gray, Source Coding Theory. Boston, MA: Kluwer Academic, 1990.
44. S. J. Haberman, “Concavity and estimation,” Ann. Statist., vol. 17, pp. 1631-1661, Dec. 1989.
45. B. Hajek, “Ergodic process selection,” in Open Problems in Communication and Computation, T. M. Cover and B. Gopinath, Ed. New York: Springer-Verlag, 1987, pp. 199-203.
46. P. Hall and C. C. Hyde, Martingale Limit Theory and Its Applications. New York: Academic, 1980.
47. J. F. Hannan, “Approximation to Bayes risk in repeated play,” in Contributions to the Theory of Games III, Ann. Math. Studies. Princeton, NJ: Princeton Univ. Press, 1957, vol. 39, pp. 97-139.
48. J. F. Hannan and H. Robbins, “Asymptotic solutions for the compound sequential decision problem for two completely specified distributions,” Ann. Math. Statist., vol. 26, pp. 37-51, 1955.
49. J. F. Hannan and J. R. Van Ryzin, “Rate of convergence in the compound decision problem for two completely specified distributions,” Ann. Math. Statist., vol. 36, pp. 1743-1752, 1965.
50. O. Hanš, “On conditional Bayes loss,” in Trans 3rd Prague Conf. Inform. Theory, Statist. Decision Functions, Random Processes, Prague, 1962. Prague: Czechoslovak Acad. Sci., 1964, pp. 207-221.
51. N. Hermdorf, “Counterexamples to results of M. M. Rao,” Z. Wahrscheinlichkeitstheorie verw. Gebiete, vol. 53, pp. 283-290, 1980.
52. ——, “Best Φ-and NΦ-approaximants in Orlicz spaces of vector valued functions,” Z. Wahrscheinlichkeitstheorie verw. Gebiete, vol. 58, pp. 309-329, 1981.
53. P. J. Huber, “Robust estimation of a location parameter,” Ann. Math. Statist., vol. 35, pp. 73-101, 1964.
54. ——, “The behavior of maximum likelihood estimates under standardized conditions,” in Proc. 5th Berkeley Symp. Math. Statist. Prob. Berkeley, CA: Univ. California Press, 1967, vol. I, pp. 221-233.
55. ——, Robust Statistics. New York: Wiley, 1981.
56. S. Jilovec, “On the experience problem,” in Trans. 6th Prague Conf. Inform. Theory, Statist. Decision Functions, Random Processes, Prague, 1971. Prague: Czechoslovak Acad. Sci., 1973, pp. 207-221.
57. S. Jilovec and B. Subert, “Repetitive play of a game against nature,” Aplikace Matematiky, vol. 12, pp. 383-397, 1967.
58. M. V. Johns, “Two-action compound decision problems,” in Proc. 5th Berkeley Symp. Math Statist. Prob. Berkeley, CA: Univ. California Press, 1967, vol. I, pp. 463-478.
59. J. L. Kelly, Jr., “A new interpretation of information rate,” Bell Syst. Tech. J., vol. 35, pp. 917-926, 1956.
60. J. C. Kieffer, “An ergodic theorem for constrained sequences of functions,” Bull. Amer. Math. Soc., vol. 21, pp. 249-254, Oct. 1989.
61. ——, “An almost sure convergence theorem for sequences of random variables selected from log-convex sets,” in Almost Sure Convergence II, A. Bellow and R. L. Jones, Ed. San Diego, CA: Academic, 1991, pp. 151-166.
62. S. J. Kilmer, “Best approximants in Musielak-Orlicz spaces,” in Approximation Theory, G. A. Anastassiou, Ed., Lecture Notes on Pure and Appl. Math. New York: Marcel Dekker, 1992, vol. 138, pp. 363-376.
63. T. L. Lai, “Information bounds, certainty equivalence and learning in asymptotically efficient adaptive control of time-invariant control of time-invariant stochastic systems,” Tech. Rep., Dep. Statist., Stanford Univ., 1991.
64. D. Landers and L. Rogge, “Best approximants in LΦ spaces,” Z. Wahrscheinlichkeitstheorie verw. Gebiete, vol. 51, pp. 215-237, 1980.
65. P. Levy, Theorie de l'Addition des Variables Aléatories, 2nd Ed. Paris: Gauthier-Villars, 1954.
66. M. Loéve, Probability Theory II, 4th ed. New York: Springer-Verlag, 1978.
67. J. MacQueen, “Some methods for classification and analysis of multivariate observations,” in Proc. 5th Berkeley Symp. Math. Statist. Prob. Berkeley, CA: Univ. California Press, 1967, vol. I, 281-297.
68. N. Merhav, M. Feder, and M. Gutman, “Some properties of sequential predictors of binary Markov sources,” in IEEE Trans. Inform. Theory, May 1993, vol. 39, pp. 887-892.
69. J. Neyman, “Two breakthroughs in the theory of statistical decision making,” Rev. Inst. Int. Statist., vol. 30, pp. 11-27, 1962.
70. W. Niemiro, “Asymptotics for M-estimators defined by convex minimization,” Ann. Statist., vol. 20, pp. 1514-1533, Sept. 1992.
71. S. Orey, “On the Shannon-Pérez-Moy theorem,” Contemporary Math., vol. 41, pp. 319-327, 1985.
72. D. Ornstein, “Guessing the next output of a stationary process,” Israel J. Math., vol. 30, pp. 292-296, 1978.
73. A. Perez, “L'expérience et Pinformation puisée dans elle a l'aide des lois limites de la theorie des probability,” in Trans. 2nd Prague Conf. Inform. Theory, Statist. Decision Functions, Random Processes, Prague, 1959. Prague: Czechoslovak Acad. Sci., 1960, pp. 499-519.
74. K. E. Petersen, Ergodic Theory. Cambridge: Cambridge Univ. Press, 1983.
75. D. Pollard, “Strong consistency of k-means clustering,” Ann. Statist., vol. 9, pp. 135-140, Jan. 1981.
76. —, “Quantization and the method of k-means,” IEEE Trans. Inform. Theory, vol. IT-28, pp. 199-205, Mar. 1982.
77. —, “A central limit theorem for k-means clustering,” Ann. Prob., vol. 10, pp. 919-926, Nov. 1982.
78. M. M. Rao, “Inference in stochastic processes III,” Z. Wahrscheinlichkeitstheorie verw. Gebiete, vol. 8, 49-72, 1967.
79. —, “Abstract nonlinear prediction and operator martingales,” Multivariate Anal., vol. 1, pp. 129-159, 1971. “Correction,” Multivariate Anal., vol. 9, p. 615, 1979.
80. J. Rissanen, “Stochastic complexity and modeling,” Ann Statist., vol. 14, pp. 1080-1100, Sept. 1986.
81. H. Robbins, “Asymptotically subminimax solutions of compound statistical decision problems,” in Proc. 2nd Berkeley Symp. Math. Statist. Prob. Berkeley, CA: Univ. California Press, 1950, pp. 131-148.
82. —, “An empirical Bayes approach to statistics,” in Proc. 3rd Berkeley Symp. Math. Statist. Prob. Berkeley, CA: Univ. California Press, 1955, vol. I, pp. 157-164.
83. —, “The empirical Bayes approach to statistical decision problems,” Ann. Math. Statist., vol. 35, pp. 1-20, 1964.
84. E. Samuel, “Asymptotic solutions of the sequential compound decision problem,” Ann. Math. Statist., vol. 34, pp. 1079-1094, 1963.
85. —, “Convergence of the losses of certain decision rules for the sequential compounded decision problem,” Ann. Math. Statist., vol. 35, 1606-1621, 1964.
86. —, “Sequential compounded estimators,” Ann. Math. Statist., vol. 36, 879-889, 1965.
87. —, “The compound statistical decision problem,” Sankhya: Indian J. Statist., ser. A, vol. 29, pp. 123-140, 1967.
88. P. S. Shields, “Universal redundancy rates do not exist,” IEEE Trans. Inform. Theory, vol. 39, pp. 520-524, Mar. 1993.
89. T. Shintani and T. Ando, “Best approximants in L1 space,” Z. Wahrscheinlichkeitstheorie verw. Gebiete, vol. 33, pp. 33-39, 1975.
90. A. Spacek, “An elementary experience problem,” in Trans. 1st Prague Conf. Inform. Theory. Statist. Decision Functions, Random Processes, Prague, 1956. Prague: Czechoslovak Acad. Sci., 1957, 253-258.
91. —, “Processus aleatoires de decision statistique conditionnee. Le calcul des probability et ses applications,” in Colloques Inter-national: du Centre National de la Recherche Scientifique LXXXVII, Paris, 1959, pp. 157-164.
92. W. F. Stout, Almost Sure Convergence. New York: Academic, 1974.
93. B. Subert, “An asymptotically optimal decision procedure,” in Trans. 4th Prague Conf. Inform. Theory, Statist. Decision Functions, Random Processes, Prague, 1965. Prague: Czechoslovak Acad. Sci., 1967, 573-587.
94. —, “Bayesian model of decision-making as a result of learning from experience,” Ann. Math. Statist., vol. 40, pp. 2127-2142, 1969.
95. V. Susarla, “Empirical Bayes Theory,” Encyclopedia of Statistical Sciences, Vol. 2, S. Kotz and N. L. Johnson, Ed. Wiley-Interscience, 1982, pp. 490-503.
96. H. Sverdrup-Thygeson, “Strong law of large numbers for measures of central tendency and dispersion of random variables in compact metric spaces,” Ann. Statist., vol. 9, pp. 141-145, Jan. 1981.
97. D. D. Swain, “Bounds and rates of convergence for the extended compound estimation problems,” Tech. Rep. Dep. Statist., Stanford Univ., 1965.
98. D. C. Taylor, “Asymptotic distribution theory for general statistical functionals,” Ann. Inst. Statist. Math., ser. A, vol, 37, pp. 131-138, 1985.
99. J. R. Van Ryzin, “The compound decision problem with m × n finite loss matrix,” Ann. Math. Statist., vol. 37, pp. 412-424, 1966.
100. —, “The sequential compound decision problem with m X n finite loss matrix,” Ann. Math. Statist., vol. 37, pp. 954-975, 1966.
101. —, “Repetitive play in finite statistical games with unknown distributions,” Ann. Math. Statist., vol. 37, pp. 976-994, 1966.
102. —, “Non-parametric Bayesian decision procedures for (pattern) classification with stochastic learning,” in Trans. 4th Prague Conf. Inform. Theory, Statist. Decision Functions, Random Processes, Prague, 1965. Prague: Czechoslovak Acad. Sci., 1967, pp. 479-494.
103. A. Wald, “A note on the consistency of the maximum likelihood estimate,” Ann. Math. Statist., vol. 60, pp. 595-603, 1943.
104. —, Statistical Decision Functions. New York: Wiley, 1950.
105. H. White, “Maximum likelihood estimation of misspecified models,” Econometrica, vol. 50, pp. 1-25, Jan. 1982.
106. N. Wiener, “The ergodic theorem,” Duke Math. J., vol. 5, pp. 1-18, 1978.
107. K. Winkelbauer, “Experience in games of strategy and in statistical decision,” in Trans. 1st Prague Conf. Inform. Theory, Statist. Decision Functions, Random Processes, Prague, 1956. Prague: Czechoslovak Acad. Sci., 1957, pp. 297-354.