Competitive on-line statistics

International Statistical Review

Volumn 69, Issue 2, 2001, Pages 213-248

  Vovk, Volodya ^a  

^a UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Bayes's rule; Competitive on line algorithms; Linear regression; Prequential statistics; Worst case analysis

Indexed keywords

EID: 0035413537     PISSN: 03067734     EISSN: None     Source Type: Journal    
DOI: 10.1111/j.1751-5823.2001.tb00457.x     Document Type: Review

References (73)

1. Auer, P., Cesa-Bianchi, N., Freund, Y. & Schapire, R.E. (1995). Gambling in a rigged casino: The adversarial multi-armed bandit problem. In Proceedings of the 36th Annual Symposium on Foundations of Computer Science, pp. 322-331.
2. Auer, P. & Warmuth, M.K. (1998). Tracking the best disjunction. Machine Learning, 32, 127-150.
3. Azoury, K.S. & Warmuth, M.K. (1999). Relative loss bounds for on-line density estimation with the exponential family of distributions. To appear in Machine Learning. An extended abstract appeared in Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence, pp. 31-40, San Francisco, CA: Morgan Kaufmann.
4. Barron, A.R. & Xie, Q. (2000). Asymptotic minimax regret for data compression, gambling, and prediction. IEEE Transactions on Information Theory, 46, 431-445.
5. Beckenbach, E.F. & Bellman, R. (1965). Inequalities. Berlin: Springer.
6. Blum, A. & Burch, C. (2000). On-line learning and the metrical task system problem. Machine Learning, 39, 35-38.
7. Blum, A. & Kalai, A. (1999). Universal portfolios with and without transaction costs. Machine Learning, 35, 193-205.
8. Borovkov, A. (1984). Mathematical Statistics (in Russian). Moscow: Nauka.
9. Cesa-Bianchi, N. & Fischer, P. (1998). Finite-time regret bounds for the multiarmed bandit problem. In 5th International Conference on Machine Learning, pp. 100-108. Morgan Kaufmann.
10. Cesa-Bianchi, N., Freund, Y., Haussler, D., Helmbold, D.P., Schapire, R.E. & Warmuth, M.K. (1997). How to use expert advice. Journal of the Association for Computing Machinery, 44, 427-485.
11. Cesa-Bianchi, N., Helmbold, D.P. & Sandra Panizza, S. (1998). On Bayes methods for on-line boolean prediction. Algorithmica, 22, 112-137.
12. Cesa-Bianchi, N., Long, P.M. & Warmuth, M.K. (1996). Worst-case quadratic loss bounds for on-line prediction of linear functions by gradient descent. IEEE Transactions on Neural Networks, 7, 604-619.
13. Clarke, B. & Barron, A.R. (1990). Information theoretic asymptotics of Bayes methods. IEEE Transactions on Information Theory, 36, 453-471.
14. Clarke, B. & Barron, A.R. (1994). Jeffreys' prior is asymptotically least favourable under entropy risk. Journal of Statistical Planning and Inference, 41, 37-60.
15. Clarke, B. & Dawid, A.P. (1999) On-line prediction with experts under a log-scoring rule. Manuscript.
16. Cohen, W.W. & Singer, Y. (1999). Context-sensitive learning methods for text categorization. ACM Transactions on Information Systems, 17, 141-173.
17. Cover, T. (1991). Universal portfolios. Mathematical Finance, 1, 1-29.
18. Cover, T. & Ordentlich, E. (1996). Universal portfolios with side information. IEEE Transactions on Information Theory, 42, 348-363.
19. Dawid, A.P. (1984). Statistical theory: the prequential approach. Journal of the Royal Statistical Society A, 147, 278-292.
20. Dawid, A.P. (1986). Probability forecasting. In Encyclopedia of Statistical Sciences, volume 7. Eds. S. Kotz, N.L. Johnson and C.B. Read, pp. 210-218. New York: Wiley-Interscience.
21. Dawid, A.P. (1991). Fisherian inference in likelihood and prequential frames of reference (with discussion). Journal of the Royal Statistical Society B, 53, 79-109.
22. DeSantis, A., Markowsky, G. & Wegman, M.N. (1988). Learning probabilistic prediction functions. In Proceedings of the 29th Annual IEEE Symposium on Foundations of Computer Science, pp. 110-119. Los Alamitos, CA: IEEE Computer Society.
23. Draper, N.R. & Smith, H. (1998). Applied Regression Analysis. New York: Wiley, 3nd edition.
24. Feder, M. & Singer, A.C. (1998). Universal data compression and linear prediction. In Proceedings of the 1998 IEEE Data Compression Conference, pp. 511-520.
25. Forster, J. (1999). On relative loss bounds in generalized linear regression. In Fundamentals of Computation Theory, 12th International Symposium, Eds. C. Ciobanu and G. Paun. Lecture Notes in Computer Science, vol. 1684, 269-280, Springer.
26. Foster, D.P. (1991). Prediction in the worst case. Annals of Statistics, 19, 1084-1090.
27. Freund, Y. (1996). Predicting a binary sequence almost as well as the optimal biased coin. In Proceedings of the 9th Annual Conference on Computational Learning Theory, pp. 89-98. New York: Association for Computing Machinery.
28. Freund, Y. & Schapire, R.E. (1996). Game theory, on-line prediction and boosting. In Proceedings, 9th Annual Conference on Computational Learning Theory, pp. 325-332. New York: Association for Computing Machinery.
29. Freund, Y. & Schapire, R.E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55, 119-139.
30. Freund, Y. & Schapire, R.E. (1999). Adaptive game playing using multiplicative weights. Games and Economic Behavior, 29, 79-103.
31. Freund, Y., Schapire, R.E., Singer, Y. & Warmuth, M.K. (1997). Using and combining predictors that specialize. In Proceedings of the 29th Annual ACM Symposium on Theory of Computing. New York: Association for Computing Machinery, pp. 334-343.
32. Grünwald, P. (1998). The Minimum Description Length Principle and Reasoning under Uncertainty. PhD thesis, Institute for Logic, Language and Computation, Universiteit van Amsterdam.
33. Haussler, D., Kivinen, J. & Warmuth, M.K. (1998). Sequential prediction of individual sequences under general loss functions. IEEE Transactions on Information Theory, 44, 1906-1925.
34. Helmbold, D.P., Long, D.D.E. & Sherrod, B. (1996). A dynamic disk spin-down technique for mobile computing. In Proceedings of the 2nd Annual ACM International Conference on Mobile Computing and Networking.
35. Helmbold, D.P. & Schapire, R.E. (1997). Predicting nearly as well as the best pruning of a decision tree. Machine Learning, 27, 51-68.
36. Herbster, M. & Warmuth, M.K. (1998). Tracking the best expert. Machine Learning, 32, 151-178.
37. Herbster, M. & Warmuth, M.K. (1998). Tracking the best regressor. In Proceedings of the 11th Annual Conference on Computational Learning Theory, pp. 24-31. Association for Computing Machinery.
38. Kalnishkan, Y. (1999). General linear relations among different types of predictive complexity. In Proceedings of the 10th International Conference on Algorithmic Learning Theory, volume 1720 of Lecture Notes in Artificial Intelligence, pp. 323-334. Accepted for publication in Theoretical Computer Science.
39. Kalnishkan, Y. (1999). Linear relations between square-loss and Kolmogorov complexity. In Proceedings of the 12th Annual Conference on Computational Learning Theory, pp. 226-232. New York: Association for Computing Machinery.
40. Kalnishkan, Y. (2000). Complexity Approximation Principle and Rissanen's approach to real-valued parameters. In 11th European Conference on Machine Learning, vol. 1810 of Lecture Notes in Artificial Intelligence, Springer.
41. Kalnishkan, Y. & Vovk, V. (2000). The existence of predictive complexity and the Legendre transformation. Technical Report CLRC-TR-00-04, Computer Learning Research Centre, Royal Holloway, University of London, March 2000.
42. Kivinen, J. & Warmuth, M.K. (1997). Exponential Gradient versus Gradient Descent for linear predictors. Information and Computation, 132, 1-63.
43. Kivinen, J. & Warmuth, M.K. (1999). Averaging expert predictions. In Computational Learning Theory: 4th European Conference, volume 1572 of Lecture Notes in Artificial Intelligence, Eds. P. Fischer and H.U. Simon, pp. 153-167. Berlin: Springer.
44. Li, Ming & Vitányi, P. (1997). An Introduction to Kolmogorov Complexity and Its Applications. New York: Springer, 2nd edition.
45. Littlestone, N. & Warmuth, M.K. (1994). The Weighted Majority Algorithm. Information and Computation, 108, 212-261.
46. Pereira, F. & Singer, Y. (1997). An efficient extension to mixture techniques for prediction and decision trees. In Proceedings, 10th Annual Conference on Computational Learning Theory, pp. 114-121. Association for Computing Machinery.
47. Press, W.H., Flannery, B.P., Teukolsky, S.A. & Vetterling, W.T. (1992). Numerical Recipes in C. Cambridge: Cambridge University Press.
48. Raghavan, P. (Ed.) (1994). Special issue on competitive on-line algorithms. Algorithmica, 11, 1-91.
49. Rissanen, J. (1983). A universal prior for integers and estimation by minimum description length. Annals of Statistics, 11, 416-431.
50. Rissanen, J. (1987). Stochastic complexity (with discussion). Journal of Royal Statistical Society B, 49, 223-239 and 252-265.
51. Shafer, G. & Vovk, V. (2001). Probability and Finance: It's only a Game! New York: Wiley.
52. Solomonoff, R.J. (1964). A formal theory of inductive inference. Parts I and II. Information and Control, 7, 1-22 and 224-254.
53. Takimoto, E., Maruoka, A. & Vovk, V. (1998). Predicting nearly as well as the best pruning of a decision tree through dynamic programming scheme. To appear in Theoretical Computer Science.
54. Vovk, V. (1998). Probability theory for the Brier game. Theoretical Computer Science. Accepted for publication. Preliminary version in Ming Li and Akira Maruoka, editors, Algorithmic Learning Theory, Lecture Notes in Computer Science, volume 1316, pp. 323-338, 1997. Full version: Technical Report CSD-TR-97-09, Department of Computer Science, Royal Holloway, University of London, revised February 1998.
55. Vovk, V. (1987). On a randomness criterion. Soviet Mathematics Doklady, 35, 656-660.
56. Vovk, V. (1990). Aggregating strategies. In Proceedings of the 3rd Annual Workshop on Computational Learning Theory, Eds. M. Fulk and J. Case, pp. 371-383. San Mateo, CA: Morgan Kaufmann.
57. Vovk, V. (1992). Universal forecasting algorithms. Information and Computation, 96, 245-277.
58. Vovk, V. (1993). A logic of probability, with application to the foundations of statistics (with discussion). Journal of the Royal Statistical Society B, 55,317-351.
59. Vovk, V. (1998). Competitive on-line linear regression. In Advances in Neural Information Processing Systems, Eds. M.I. Jordan, M.J. Kearns and S.A. Solla, pp. 364-370. Cambridge, MA: MIT Press.
60. Vovk, V. (1998). A game of prediction with expert advice. Journal of Computer and System Sciences, 56, 153-173.
61. Vovk, V. (1999). Competitive on-line statistics. In Bulletin of the International Statistical Institute. The 52nd Session, Proceedings, volume LVIII, book 1, pp. 231-234.
62. Vovk, V. (1999). Derandomizing stochastic prediction strategies. Machine Learning, 35(3), 247-282.
63. Vovk, V. & Gammerman, A. (1999). Complexity Approximation Principle. Computer Journal, 42, 318-322.
64. Vovk, V. & Gammerman, A. (1999). Statistical applications of algorithmic randomness. In Bulletin of the International Statistical Institute. The 52nd Session, Contributed Papers, volume LVIII, book 3, pp. 469-470.
65. Vovk, V. & Watkins, C.J.H.C. (1998). Universal portfolio selection. In Proceedings of the 11th Annual Conference on Computational Learning Theory, pp. 12-23. New York: Association for Computing Machinery.
66. V'yugin, V.V. (1999). Does snooping help? Technical Report CLRC-TR-99-06, Computer Learning Research Centre, Royal Holloway, University of London. Can be downloaded from http://www.clrc.rhul.ac.uk.
67. V'yugin, V.V. (1999). Most sequences are predictable. Technical Report CLRC-TR-99-01, Computer Learning Research Centre, Royal Holloway, University of London. Can be downloaded from http://www.clrc.rhul.ac.uk.
68. V'yugin, V.V. (2000). Sub-optimal measures of predictive complexity. Technical Report CLRC-TR-00-05, Computer Learning Research Centre, Royal Holloway, University of London. Can be downloaded from http://www.clrc.rhul.ac.uk.
69. Wallace, C.S. & Boulton, D.M. (1968). An information measure for classification. Computer Journal, 11, 185-195.
70. Wallace, C.S. & Freeman, P.R. (1987). Estimation and inference by compact coding (with discussion). Journal of the Royal Statistical Society B, 49, 240-265.
71. Yamanishi, K. (1994). Generalized stochastic complexity and its applications to learning. In Proceedings of the 1994 Conference on Information Science and Systems, volume 2, pp. 763-768.
72. Yamanishi, K. (1998). A decision-theoretic extension of stochastic complexity and its applications to learning. IEEE Transactions on Information Theory, 20, 100-114.
73. Yamanishi, K. (1998). Minimax relative loss analysis for sequential prediction algorithms using parametric hypotheses. In Proceedings of the 11th Annual Conference on Computational Learning Theory, pp. 32-43. New York: Association for Computing Machinery.