Solving two-armed Bernoulli bandit problems using a Bayesian learning automaton

International Journal of Intelligent Computing and Cybernetics

Volumn 3, Issue 2, 2010, Pages 207-234

  Granmo, Ole Christoffer ^a  

^a UNIVERSITY OF AGDER   (Norway)

Author keywords

Automata theory; Learning processes; Programming and algorithm theory; Stochastic processes

Indexed keywords

BANDIT PROBLEMS; BAYESIAN; BAYESIAN APPROACHES; BAYESIAN COMPUTATION; BAYESIAN LEARNING; BAYESIAN METHODOLOGY; BAYESIAN METHODS; BAYESIAN PERSPECTIVE; BERNOULLI; BETA DISTRIBUTIONS; CLASSICAL OPTIMIZATION; CONJUGATE PRIOR; DESIGN/METHODOLOGY/APPROACH; DISTRIBUTED APPLICATIONS; LEARNING PROCESS; OPTIMAL DECISION MAKING; PROGRAMMING AND ALGORITHM THEORY; RANDOM SAMPLING; STOCHASTIC PROCESS; STOCHASTIC PROCESSES;

AUTOMATA THEORY; COMPUTATIONAL COMPLEXITY; DECISION MAKING; LEARNING ALGORITHMS; OPTIMIZATION; RANDOM PROCESSES; ROBOTS; STOCHASTIC SYSTEMS; TRANSLATION (LANGUAGES);

BAYESIAN NETWORKS;

EID: 78549244167     PISSN: 1756378X     EISSN: 17563798     Source Type: Journal    
DOI: 10.1108/17563781011049179     Document Type: Article

References (35)

1. Agache, M. and Oommen, B.J. (2002), "Generalized pursuit learning schemes: new families of continuous and discretized learning automata" in IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, Vol. 32, No. 6, pp. 738-49.
2. Audibert, J.-Y., Munos, R. and Szepesvarri, C. (2007), "Tuning bandit algorithms in stochastic environments" in Proceedings of the 18th International Conference of Algorithmic Learning Theory, Sendai, Japan, Springer, Berlin, pp. 150-65.
3. Auer, P., Cesa-Bianchi, N. and Fischer, P. (2002), "Finite-time analysis of the multiarmed bandit problem" in Machine Learning, Vol. 47, pp. 235-56.
4. Bhulai, S. and Koole, G. (2000), "On the value of learning for Bernoulli bandits with unknown parameters" in IEEE Transactions on Automatic Control, Vol. 45, No. 11, pp. 2135-40.
5. Blum, A., Even-Dar, E. and Ligett, K. (2006), "Routing without regret: on convergence to Nash equilibria of regret-minimizing algorithms in routing games" in Proceedings of the Twenty-Fifth Annual ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing (PODC 2006), ACM, New York, NY, pp. 45-52.
6. Cao, Y.U., Fukunaga, A.S. and Kahng, A. (1997), "Cooperative mobile robotics: antecedents and directions" in Autonomous Robots, Vol. 4, No. 1, pp. 7-27.
7. Chen, D. and Varshney, P.K. (2004), "QoS support in wireless sensor networks: a survey", paper presented at The 2004 International Conference on Wireless Networks (ICWN 20 04), Las Vegas, NV.
8. Dimitrakakis, C. (2006), "Nearly optimal exploration-exploitation decision thresholds" in Proceedings of the 16th International Conference on Artificial Neural Networks (ICANN 2006), Athens, Greece, Springer, Berlin, p. 850, Lecture Notes in Computer Science.
9. Duda, R., Hart, P. and Stork, D. (2000), Pattern Classification, 2nd ed., Wiley, New York, NY.
10. Gelly, S. and Wang, Y. (2006), "Exploration exploitation in go: UCT for Monte-Carlo go", Proceedings of NIPS-2006.
11. Granmo, O.-C. and Bouhmala, N. (2007), "Solving the satisfiability problem using finite learning automata" in International Journal of Computer Science and Applications, Vol. 4, No. 3, pp. 15-29.
12. Granmo, O.-C., Oommen, B.J., Myrer, S.A. and Olsen, M.G. (2007), "Learning automata-based solutions to the nonlinear fractional knapsack problem with applications to optimal resource allocation" in IEEE Transactions on Systems, Man, and Cybernetics, Part B, Vol. 37, No. 1, pp. 166-75.
13. Iyer, R. and Kleinrock, L. (2003), "QoS control for sensor networks" in IEEE International Conference on Communications, Vol. 1, pp. 517-21.
14. Kaelbling, L.P. (1993), Learning in Embedded Systems, Stanford University, Stanford, CA, PhD thesis.
15. Kocsis, L. and Szepesvari, C. (2006), "Bandit based Monte-Carlo planning" in Proceedings of the 17th European Conference on Machine Learning (ECML 2006), Springer, Berlin, pp. 282-93.
16. Lakshmivarahan, S. (1981), Learning Algorithms Theory and Applications, Springer, Berlin.
17. Lanctôt, J.K. (1989), "Discrete estimator algorithms: a mathematical model of computer learning", Department of Mathematics and Statistics, Carleton University, Ottawa, Master's thesis.
18. Lanctôt, J.K. and Oommen, B.J. (1992), "Discretized estimator learning automata" in IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-22, No. 6, pp. 1473-83.
19. Misra, S., Oommen, B.J. and Granmo, O.-C. (2007), "Routing bandwidth guaranteed paths in MPLS traffic engineering: a multiple race track learning approach" in IEEE Transactions on Computers, Vol. 56, No. 7, pp. 959-76.
20. Mitchell, T.M. (1997), Machine Learning, McGraw-Hill, New York, NY.
21. Najim, K. and Poznyak, A.S. (1994), Learning Automata: Theory and Applications, Pergamon Press, Oxford.
22. Narendra, K.S. and Thathachar, M.A.L. (1989), Learning Automata: An Introduction, Prentice-Hall, Englewood Cliffs, NJ.
23. Obaidat, M.S., Papadimitriou, G.I. and Pomportsis, A.S. (2002), "Learning automata: theory, paradigms and applications" in IEEE Transactions on Systems Man and Cybernetics, Vol. SMC-32, pp. 706-9.
24. Oommen, B.J. and Agache, M. (2001), "Continuous and discretized pursuit learning schemes: various algorithms and their comparison" in IEEE Transactions on Systems, Man, and Cybernetics-Part B: Cybernetics, Vol. 31, pp. 277-87.
25. Oommen, B.J. and Lanctôt, J.K. (1990), "Discretized pursuit learning automata" in IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-20, No. 4, pp. 931-8.
26. Poznyak, A.S. and Najim, K. (1997), Learning Automata and Stochastic Optimization, Springer, Berlin.
27. Sastry, P.S. (1985), Systems of Learning Automata: Estimator Algorithms Applications, Dept of Electrical Engineering, Indian Institute of Science, Bangalore, PhD thesis.
28. Sutton, R.S. and Barto, A.G. (1998), Reinforcement Learning: An Introduction, MIT Press, Cambridge, MA.
29. Thathachar, M.A.L. and Sastry, P.S. (2004), Networks of Learning Automata: Techniques for Online Stochastic Optimization, Kluwer Academic Publishers, Dordrecht.
30. Thompson, W.R. (1933), "On the likelihood that one unknown probability exceeds another in view of the evidence of two samples" in Biometrika, Vol. 25, pp. 285-94.
31. Tsetlin, M.L. (1973), Automaton Theory and Modeling of Biological Systems, Academic Press, New York, NY.
32. Tung, B. and Kleinrock, L. (1996), "Using finite state automata to produce self-optimization and self-control" in IEEE Transactions on Parallel and Distributed Systems, Vol. 7, No. 4, pp. 47-61.
33. Vermorel, J. and Mohri, M. (2005), "Multi-armed bandit algorithms and empirical evaluation" in Proceedings of the 16th European Conference on Machine Learning (ECML 2005), Porto, Portugal, Springer, Heidelberg, pp. 437-48.
34. Wang, T., Lizotte, D., Bowling, M. and Scuurmans, D. (2005), "Bayesian sparse sampling for on-line reward optimization" in Proceedings of the 22nd International Conference on Machine Learning, Bonn, Germany, pp. 956-63.
35. Wyatt, J. (1997), "Exploration and inference in learning from reinforcement", University of Edinburgh, Edinburgh, PhD thesis.