Combining expert advice in reactive environments

Journal of the ACM

Volumn 53, Issue 5, 2006, Pages 762-799

  De Farias, Daniela Pucci ^a,c   Megiddo, Nimrod ^b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b IBM ALMADEN RESEARCH CENTER   (United States)

^c NONE   (United States)

Author keywords

Complexity and performance bounds; Experts algorithms; Exploration exploitation tradeoffs; Reactive environments; Sequential decision making

Indexed keywords

ASYMPTOTIC STABILITY; COMPUTATIONAL COMPLEXITY; DECISION MAKING; LEARNING SYSTEMS; POLYNOMIALS;

COMPLEXITY AND PERFORMANCE BOUNDS; EXPERTS ALGORITHMS; EXPLORATION-EXPLOITATION TRADEOFFS; REACTIVE ENVIRONMENTS; SEQUENTIAL DECISION MAKING;

ALGORITHMS;

EID: 33845302015     PISSN: 00045411     EISSN: None     Source Type: Journal    
DOI: 10.1145/1183907.1183911     Document Type: Article

References (19)

1. AUER, P., CESA-BIANCHI, N., FREUND, Y., AND SCHAPIRE, R. E. 2002. The nonstochastic multiarmed bandit problem. SIAM J. Comput. 32, 1.
2. CESA-BIANCHI, N., FREUND, Y., HAUSSLER, D., HELMBOLD, D. P., SCHAPIRE, R. E., AND WARMUTH, M. K. 1997. How to use expert advice. J. ACM 44, 427-485.
3. CHERNOFF, H. 1952. A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations. Ann. Math. Stat. 23, 493-507.
4. DE FARIAS, D., AND MEGIDDO, N. 2004. How to combine expert (or novice) advice when actions impact the environment. In Advances in Neural Information Processing Systems, Vol. 16.
5. FELLER, W. 1971. Probability Theory and its Applications. Wiley, New York.
6. FOSTER, D. P. AND VOHRA, R. V. 1993. A randomization rule for selecting forecasts. Oper. Res. 41, 704-709.
7. FOSTER, D. AND VOHRA, R. 1999. Regret and the on-line decision problem. Games Econ. Behav. 29, 7-35.
8. FREUND, Y., AND SCHAPIRE, R. E. 1995. A decision-theoretic generalization of on-line learning and an application to boosting. In Computational Learning Theory, (P. Vitányi, Ed.), Lecture Notes in Computer Science, vol. 904. Springer-Verlag, New York, 23-37.
9. FREUND, Y., AND SCHAPIRE, R. E. 1999. Adaptive game playing using multiplicative weights. Games Econ. Behav. 29, 79-103.
10. FUDENBERG, D., AND LEVINE, D. 1997. The Theory of Learning in Games. The MIT Press, Cambridge, MA.
11. HOEFFDING, W. 1963. Probability inequalities for sums of bounded random variables. J. ASA 58, 13-30.
12. KAKADE, S. 2003. On the sample complexity of reinforcement learning. Ph.D. dissertation, Gatsby Computational Neuroscience Unit, University College, London, England.
13. KEARNS, M., AND SINGH, S. 1999. Finite-sample convergence rates for Q-learning and indirect algorithms. In Neural Information Processing Systems 12. MIT Press.
14. KEARNS, M., AND SINGH, S. 2002. Near-optimal reinforcement learning in polynomial time. Mach. Learn. 49, 2, 209-232.
15. LAI, T.-L., AND YAKOWITZ, S. 1995. Machine learning and nonparametric bandit theory. IEEE Trans. Automat. Cont. 40, 7, 1199-1209.
16. LITTLESTONE, N., AND WARMUTH, M. 1994. The weighted majority algorithm. Inf. Comput. 108, 2, 212-261.
17. VOVK, V. 1998. A game of prediction with expert advice. J. Compu. Syst. Sci. 56, 153-173.
18. WATKINS, C., AND DAYAN, P. 1992. Q-learning. Mach. Learn. 8, 279-292.
19. WILLIAMS, D. 1991. Probability with Martingales. Cambridge University Press, Cambridge, UK.