Regret minimization under partial monitoring

Mathematics of Operations Research

Volumn 31, Issue 3, 2006, Pages 562-580

  Cesa Bianchi, Nicolò ^a   Lugosi, Gábor ^b   Stoltz, Gilles ^c  

^a UNIVERSITY OF MILAN   (Italy)

^b UNIVERSITAT POMPEU FABRA   (Spain)

^c ECOLE NORMALE SUPÉRIEURE   (France)

Author keywords

Hannan consistency; Imperfect monitoring; Internal regret; Repeated games

Indexed keywords

CONDITION MONITORING; CONVERGENCE OF NUMERICAL METHODS; FEEDBACK; GAME THEORY; PROBABILITY;

HANNAN CONSISTENCY; IMPERFECT MONITORING; INTERNAL REGRET; REPEATED GAMES;

PROFESSIONAL ASPECTS;

EID: 33748442333     PISSN: 0364765X     EISSN: 15265471     Source Type: Journal    
DOI: 10.1287/moor.1060.0206     Document Type: Article

References (44)

1. Auer, P. 2002. Using confidence bounds for exploitation-exploration trade-offs. J. Machine Learn. Res. 3 397-422.
2. A preliminary version appeared in Proc. 41st Annual Sympos. Foundations Comput. Sci.
3. Auer, P., N. Cesa-Bianchi, C. Gentile. 2002. Adaptive and self-confident on-line learning algorithms. J. Comput. System Sci. 64 48-75.
4. Auer, P., N. Cesa-Bianchi, Y. Freund, R. E. Schapire. 2002. The nonstochastic multiarmed bandit problem. SIAM J. Comput. 32 48-77.
5. Azuma, K. 1967. Weighted sums of certain dependent random variables. Tohoku Math. J. 68 357-367.
6. Baños, A. 1968. On pseudo-games. Ann. Math. Statist. 39 1932-1945.
7. Blackwell, D. 1956. An analog of the minimax theorem for vector payoffs. Pacific J. Math. 6 1-8.
8. Blum, A., J. Hartline. 2005. Near-optimal online auctions. Proc. 16th ACM-SIAM Sympos. Discrete Algorithms, 1156-1163.
9. Blum, A., Y. Mansour. 2005. From external to internal regret. Proc. 18th Annual Conf. Comput. Learn. Theory, Springer, 621-636.
10. Blum, A., V. Kumar, A. Rudra, F. Wu. 2004. Online learning in online auctions. Theoret. Comput. Sci. 324 137-146.
11. Cesa-Bianchi, N., G. Lugosi. 1999. On prediction of individual sequences. Ann. Statist. 27 1865-1895.
12. Cesa-Bianchi, N., G. Lugosi. 2003. Potential-based algorithms in on-line prediction and game theory. Machine Learn. 51 239-261.
13. Cesa-Bianchi, N., G. Lugosi. 2006. Prediction, Learning, and Games. Cambridge University Press, Cambridge, UK.
14. Cesa-Bianchi, N., G. Lugosi, G. Stoltz. 2005. Minimizing regret with label efficient prediction. IEEE Trans. Inform. Theory 51 2152-2162.
15. Cesa-Bianchi, N., Y. Freund, D. P. Helmbold, D. Haussler, R. Schapire, M. K. Warmuth. 1997. How to use expert advice. J. ACM 44 427-485.
16. Cover, T. M., J. A. Thomas. 1991. Elements of Information Theory. John Wiley, New York.
17. Feder, M., N. Merhav, M. Gutman. 1992. Universal prediction of individual sequences. IEEE Trans. Inform. Theory 38 1258-1270.
18. Foster, D., R. Vohra. 1997. Calibrated learning and correlated equilibrium. Games Econom. Behav. 21 40-55.
19. Foster, D., R. Vohra. 1998. Asymptotic calibration. Biometrika 85 379-390.
20. Foster, D., R. Vohra. 1999. Regret in the on-line decision problem. Games Econom. Behav. 29 7-36.
21. Freedman, D. A. 1975. On tail probabilities for martingales. Ann. Probab. 3 100-118.
22. Fudenberg, D., D. K. Levine. 1995. Universal consistency and cautious fictitious play. J. Econom. Dynam. Control 19 1065-1089.
23. Fudenberg, D., D. K. Levine. 1998. The Theory of Learning in Games. MIT Press, Boston, MA.
24. Hannan, J. 1957. Approximation to Bayes risk in repeated play. M. Dresher, A. W. Tucker, P. Wolfe, eds. Contributions to the Theory of Games, Vol. 3. Princeton University Press, Princeton, NJ, 97-139.
25. Hart, S., A. Mas-Colell. 2000. A simple adaptive procedure leading to correlated equilibrium. Econometrica 68 1127-1150.
26. Hart, S., A. Mas-Colell. 2001. A general class of adaptive strategies. J. Econom. Theory 98 26-54.
27. Hart, S., A. Mas-Colell. 2002. A reinforcement procedure leading to correlated equilibrium. G. Debreu, W. Neuefeind, W. Trockel, eds. Economic Essays: A Festschrift for Werner Hildenbrand. Springer, New York, 181-200.
28. Helmbold, D. P., S. Panizza. 1997. Some label efficient learning results. Proc. 10th Annual Conf. Comput. Learn. Theory, ACM Press, New York, 218-230.
29. Helmbold, D. P., N. Littlestone, P. M. Long. 2000. Apple tasting. Inform. Comput. 161 85-139.
30. Hoeffding, W. 1963. Probability inequalities for sums of bounded random variables. J. Amer. Statist. Assoc. 58 13-30.
31. Kleinberg, R., T. Leighton. 2003. The value of knowing a demand curve: Bounds on regret for on-line posted-price auctions. Proc. 44th Annual IEEE Sympos. Foundations Comput. Sci. IEEE Press, Piscataway, NJ, 594-605.
32. Littlestone, N., M. K. Warmuth. 1994. The weighted majority algorithm. Inform. Comput. 108 212-261.
33. Mannor, S., N. Shimkin. 2003. On-line learning with imperfect monitoring. Proc. 16th Annual Conf. Learn. Theory, Springer, New York, 552-567.
34. Massart, P. 2003. Concentration inequalities and model selection. Lectures on Probability Theory and Statistics (Saint-Flour, 2003), Lecture Notes in Mathematics. Springer, New-York.
35. Megiddo, N. 1980. On repeated games with incomplete information played by non-Bayesian players. Internat. J. Game Theory 9 157-167.
36. Merhav, N., M. Feder. 1998. Universal prediction. IEEE Trans. Inform. Theory 44 2124-2147.
37. Mertens, J.-F., S. Sorin, S. Zamir. 1994. Repeated games. Discussion Paper 9420, 9421, 9422, CORE, Louvain-la-Neuve, Belgium.
38. Piccolboni, A., C. Schindelhauer. 2001. Discrete prediction games with arbitrary feedback and loss. Proc. 14th Annual Conf. Comput. Learn. Theory, 208-223.
39. Rustichini, A. 1999. Minimizing regret: The general case. Games Econom. Behav. 29 224-243.
40. Stoltz, G., G. Lugosi. 2005. Internal regret in on-line portfolio selection. Machine Learn. 59 125-159.
41. Vovk, V. G. 1990. Aggregating strategies. Proc. 3rd Annual Workshop Comput. Learn. Theory, 372-383.
42. Vovk, V. G. 2001. Competitive on-line statistics. Internat. Statist. Rev. 69 213-248.
43. Weissman, T., N. Merhav. 2001. Universal prediction of binary individual sequences in the presence of noise. IEEE Trans. Inform. Theory 47 2151-2173.
44. Weissman, T., N. Merhav, A. Somekh-Baruch. 2001. Twofold universal prediction schemes for achieving the finite state predictability of a noisy individual binary sequence. IEEE Trans. Inform. Theory 47 1849-1866.