Distributed online learning via cooperative contextual bandits

IEEE Transactions on Signal Processing

Volumn 63, Issue 14, 2015, Pages 3740-3754

  Tekin, Cem ^a   Van Der Schaar, Mihaela ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Contextual bandits; cooperative learning; distributed learning; multi user bandits; multi user learning; online learning

Indexed keywords

ALGORITHMS; BIG DATA; DATA MINING; E-LEARNING; LEARNING ALGORITHMS; ONLINE SYSTEMS; SENSOR NETWORKS; SOCIAL NETWORKING (ONLINE);

CONTEXTUAL BANDITS; COOPERATIVE LEARNING; DISTRIBUTED LEARNING; MULTI-USER; ONLINE LEARNING;

LEARNING SYSTEMS;

EID: 84933576393     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2015.2430837     Document Type: Article

References (23)

1. K. Liu and Q. Zhao, "Distributed learning in multi-Armed bandit with multiple players," IEEE Trans. Signal Process., vol. 58, no. 11, pp. 5667-5681, 2010.
2. C. Tekin and M. Liu, "Online learning in decentralized multi-user spectrum access with synchronized explorations," in Proc. IEEEMILCOM, 2012, pp. 1-6.
3. R. Kleinberg, A. Slivkins, and E. Upfal, "Multi-Armed bandits in metric spaces," in Proc. 40th Annu. ACM Symp. Theory Comput., 2008, pp. 681-690.
4. S. Bubeck, R. Munos, G. Stoltz, and C. Szepesvari, "X-Armed bandits," J. Mach. Learn. Res., vol. 12, pp. 1655-1695, 2011.
5. A. Slivkins, "Contextual bandits with similarity information," in Proc. 24th Annu. Conf. Learn. Theory (COLT), Jun. 2011, vol. 19, pp. 679-702.
6. M. Dudik, D. Hsu, S. Kale, N. Karampatziakis, J. Langford, L. Reyzin, and T. Zhang, "Efficient optimal learning for contextual bandits," 2011, ArXiv preprint arXiv:1106.2369 [Online]. Available: http://arxiv.org/abs/1106.2369
7. J. Langford and T. Zhang, "The epoch-greedy algorithm for contextual multi-Armed bandits," Adv. Neural Inf. Process. Syst., vol. 20, pp. 1096-1103, 2007.
8. W. Chu, L. Li, L. Reyzin, and R. E. Schapire, "Contextual bandits with linear payoff functions," in Proc. 14th Int. Conf. Artif. Intell. Statist. (AISTATS), Apr. 2011, vol. 15, pp. 208-214.
9. L. Li, W. Chu, J. Langford, and R. E. Schapire, "A contextual-bandit approach to personalized news article recommendation," in Proc. 19th Int. Conf. World Wide Web, 2010, pp. 661-670.
10. P. Auer, N. Cesa-Bianchi, and P. Fischer, "Finite-time analysis of the multiarmed bandit problem," Mach. Learn., vol. 47, pp. 235-256, 2002.
11. K. Crammer and C. Gentile, "Multiclass classification with bandit feedback using adaptive regularization," Mach. Learn., vol. 90, no. 3, pp. 347-383, 2013.
12. A. Anandkumar, N. Michael, and A. Tang, "Opportunistic spectrum access with multiple players: Learning under competition," in Proc. IEEE INFOCOM, Mar. 2010.
13. C. Tekin and M. Liu, "Online learning of rested and restless bandits," IEEE Trans. Inf. Theory, vol. 58, no. 8, pp. 5588-5611, 2012.
14. H. Liu, K. Liu, and Q. Zhao, "Learning in a changing world: Restless multiarmed bandit with unknown dynamics," IEEE Trans. Inf. Theory, vol. 59, no. 3, pp. 1902-1916, 2013.
15. R. Stranders, L. Tran-Thanh, F. M. D. Fave, A. Rogers, and N. R. Jennings, "DCOPs and bandits: exploration and exploitation in decentralised coordination," in Proc. 11th Int. Conf. Autonom. Agents Multiagent Syst.-Volume 1, 2012, pp. 289-296.
16. Y. Gai, B. Krishnamachari, and R. Jain, "Combinatorial network optimization with unknown variables: multi-Armed bandits with linear rewards and individual observations," IEEE/ACM Trans. Netw., vol. 20, no. 5, pp. 1466-1478, 2012.
17. S. S. Ram, A. Nedic, and V. V. Veeravalli, "Distributed stochastic subgradient projection algorithms for convex optimization," J. Optim. Theory Appl., vol. 147, no. 3, pp. 516-545, 2010.
18. F. Yan, S. Sundaram, S. Vishwanathan, and Y. Qi, "Distributed autonomous online learning: regrets and intrinsic privacy-preserving properties," IEEE Trans. Knowl. Data Eng., vol. 25, no. 11, pp. 2483-2493, 2013.
19. M. Raginsky, N. Kiarashi, and R. Willett, "Decentralized online convex programming with local information," in Proc. Amer. Control Conf. (ACC), 2011, pp. 5363-5369.
20. C. Tekin, S. Zhang, and M. van der Schaar, "Distributed online learning in social recommender systems," IEEE J. Sel. Topics Signal Process, vol. 8, no. 4, pp. 638-652, Aug. 2014.
21. H. Liu, K. Liu, and Q. Zhao, "Learning in a changing world: Non-Bayesian restless multi-Armed bandit," Univ. of California- Davis, Tech. Rep., 2010.
22. R. Ortner, "Exploiting similarity information in reinforcement learning," in Proc. 2nd ICAART, 2010, pp. 203-210.
23. E. Chlebus, "An approximate formula for a partial sum of the divergent p-series," Appl. Math. Lett., vol. 22, no. 5, pp. 732-737, 2009.