On optimal sequential prediction for general processes

IEEE Transactions on Information Theory

Volumn 49, Issue 1, 2003, Pages 83-98

  Nobel, Andrew B ^a  

^a The University of North Carolina at Chapel Hill   (United States)

Author keywords

Bayes prediction scheme; Nonstationary process; pth power loss; Sequential prediction; Stochastic process

Indexed keywords

BINARY SEQUENCES; MATHEMATICAL MODELS; PROBABILITY; PROBLEM SOLVING; THEOREM PROVING;

BAYES PREDICTION SCHEMES;

RANDOM PROCESSES;

EID: 0037268871     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2002.806141     Document Type: Article

References (43)

1. p," Z. Wahrscheinlichkeitstheorie verw. Geb., vol. 4, pp. 113-120, 1965.
2. P. H. Algoet, "The strong law of large numbers for sequential decisions under uncertainty," IEEE Trans. Inform. Theory, vol. 40, pp. 609-633, May 1994.
3. _, "Universal schemes for prediction, gambling and portfolio selection," Ann. Probab., vol. 20, pp. 901-941, 1992. Correction: Ann. Probab., vol. 23, pp. 474-478, 1995.
4. _, "Universal schemes for learning the best nonlinear predictor given the infinite past and side information," IEEE Trans. Inform. Theory, vol. 45, pp. 1165-1185, May 1999.
5. K. Azuma, "Weighted sums of certain dependent random variables," Tohoku Math. J., vol. 68, pp. 357-367, 1967.
6. D. H. Bailey, "Sequential schemes for classifying and predicting ergodic processes," Ph.D. dissertation, Dept. Math., Stanford Univ., Stanford, CA, 1976.
7. L. Breiman, Probability. Philadelphia, PA: SIAM, 1992.
8. N. Cesa-Bianchi, "Analysis of two gradient-based algorithms for on-line regression," in Proc. 12th Annu. Conf. Computational Learning Theory. New York: ACM Press, 1999, pp. 163-170.
9. N. Cesa-Bianchi, Y. Freund, D. P. Helmbold, and M. K. Warmuth, "On-line prediction and conversion strategies," Machine Learning, vol. 25, pp. 71-110, 1996.
10. N. Cesa-Bianchi, Y. Freund, D. Haussler, D. P. Helmbold, R. E. Schapire, and M. K. Warmuth, "How to use expert advice," J. Assoc. Comput. Mach., vol. 44, pp. 427-485, 1997.
11. A. P. Dawid, "The well-calibrated Bayesian (with discussion)," J. Amer. Statist. Assoc., vol. 77, pp. 605-613, 1982.
12. _, "Statistical theory. The prequential approach (with discussion)," J. Roy. Statist. Soc. A, vol. 147, pp. 278-292, 1984.
13. _, "Calibration-based empirical probability (with discussion)," Ann. Statist., vol. 13, pp. 1251-1285, 1985.
14. A. P. Dawid and V. G. Vovk, "Prequential probability: Principles and properties," Bernoulli, vol. 5, pp. 125-162, 1999.
15. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
16. J. L. Doob, Stochastic Processes. New York: Wiley, 1953.
17. M. Feder, N. Merhav, and M. Gutman, "Universal prediction of individual sequences," IEEE Trans. Inform. Theory, vol. 38, pp. 1258-1270, July 1992.
18. D. P. Foster, "Prediction in the worst case," Ann. Statist., vol. 19, pp. 1084-1090, 1991.
19. D. P. Foster and R. Vohra, "Regret in the on-line decision problem," Games and Econ. Behav., vol. 29, pp. 1084-1090, 1999.
20. T. S. Ferguson, Mathematical Statistics. San Diego, CA: Academic, 1967.
21. R. M. Gray, Probability, Random Processes, and Ergodic Properties. New York: Springer-Verlag, 1988.
22. L. Györfi and G. Lugosi, "Strategies for sequential prediction of stationary time series," in Modeling Uncertainty: An Examination of Its Theory, Methods, and Applications, M. Dror, P. L'Ecuyer, and F. Szidarovszky, Eds. Norwell, MA: Kluwer, 2001.
23. L. Györfi, G. Morvai, and S. J. Yakowitz, "Limits to consistent on-line forecasting for ergodic time series," IEEE Trans. Inform. Theory, vol. 44, pp. 886-892, Mar. 1998.
24. L. Györfi, G. Lugosi, and G. Morvai, "A simple randomized algorithm for consistent sequential prediction of ergodic time series," IEEE Trans. Inform. Theory, vol. 45, pp. 2642-2650, Nov. 1999.
25. D. H. Haussler, J. Kivenen, and M. K. Warmuth, "Sequential prediction of individual sequences under general loss functions," IEEE Trans. Inform. Theory, vol. 44, pp. 1906-1925, Sept. 1998.
26. W. Hoeffding, "Probability inequalities for sums of bounded random variables," J. Amer. Statist. Assoc., vol. 58, pp. 13-30, 1963.
27. N. Littlestone and M. K. Warmuth, "The weighted majority algorithm," Inform. and Comput., vol. 108, pp. 212-261, 1994.
28. N. Merhav and M. Feder, "Universal prediction," IEEE Trans. Inform. Theory, vol. 44, pp. 2124-2147, Oct. 1998.
29. D. S. Modha and E. Masry, "Memory-universal prediction of stationary random processes," IEEE Trans. Inform. Theory, vol. 44, pp. 117-133, Jan. 1998.
30. G. Morvai, S. Yakowitz, and L. Györfi, "Nonparametric inference for ergodic, stationary time series," Ann. Statist., vol. 24, pp. 370-379, 1996.
31. G. Morvai, S. Yakowitz, and P. Algoet, "Weakly convergent nonparametric forecasting of stationary time series," IEEE Trans. Inform. Theory, vol. 43, pp. 483-498, Mar. 1997.
32. A. H. Murphy and R. L. Winkler, "Reliability of subjective probability forecasts of precipitation and temperature," JRSS Set C, vol. 26, pp. 41-47, 1977.
33. D. S. Ornstein, "Guessing the next output of a stationary process," Israel J. Math, vol. 30, pp. 292-296, 1978.
34. K. R. Parthasarathy, Probability Measures on Metric Spaces. New York: Academic, 1967.
35. R. T. Rockafellar, Convex Analysis. Princeton, NJ: Princeton Univ. Press, 1970.
36. H. L. Royden, Real Analysis, 3rd ed. Englewood Cliffs, NJ: Prentice-Hall, 1988.
37. B. Y. Ryabko, "Prediction of random sequences and universal coding," Probl. Inform. Transm., vol. 24, pp. 87-96, 1998.
38. K. Skouras and A. P. Dawid, "On efficient point prediction systems," J. Roy. Statist. Soc. B, vol. 60, pp. 765-780, 1998.
39. _, "On efficient probability forecasting systems," Biometrika, vol. 86, pp. 765-784, 1999.
40. W. F. Stout, Almost Sure Convergence. New York: Academic, 1974.
41. V. Vovk, "Aggregating strategies," in Proc. 3rd Annu. Workshop Computational Learning Theory. San Mateo, CA: Morgan Kaufman, 1990, pp. 371-383.
42. T. Weissman and N. Merhav, "Universal prediction of random binary sequences in a noisy environment," preprint, 2000.
43. _, "Universal prediction of individual binary sequences in the presence of noise," IEEE Trans. Inform. Theory, vol. 47, pp. 2151-2173, Sept. 2001.