Multi-armed bandit formulation for autonomous mobile acoustic relay adaptive positioning

Proceedings - IEEE International Conference on Robotics and Automation

Volumn , Issue , 2013, Pages 4165-4170

  Cheung, Mei Yi ^a   Leighton, Joshua ^a   Hover, Franz S ^a  

^a Massachusetts Institute of Technology   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE POSITIONING; AUTONOMOUS SURFACE VEHICLES; EXPLORATION VS. EXPLOITATIONS; MULTI ARMED BANDIT; OPTIMAL RELAY POSITION; PRIOR INFORMATION; UNDERWATER ACOUSTIC DATA; VARIABLE PERFORMANCE;

COMPLEX NETWORKS; DATA COMMUNICATION SYSTEMS; UNDERWATER ACOUSTICS;

ROBOTICS;

EID: 84887299428     PISSN: 10504729     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICRA.2013.6631165     Document Type: Conference Paper

References (24)

1. M. Chitre, S. Shahabudeen, and M. Stojanovic. Underwater acoustic communications and networking: Recent advances and future challenges. Marine Technology Society Journal, 42(1):103-116, 2008.
2. J. C. Preisig. Performance analysis of adaptive equalization for coherent acoustic communications in the time-varying ocean environment. The Journal of the Acoustical Society of America, 118(1):263-278, 2005.
3. M. Stojanovic and J. Preisig. Underwater acoustic communication channels: Propagation models and statistical characterization. Communications Magazine, IEEE, 47(1):84-89, Jan. 2009.
4. J. B. Bowlin, J. L. Spiesberger, T. F. Duda, and L. E. Freitag. Ocean acoustical ray-tracing : Software Ray. 1992.
5. B. Tomasi, G. Zappa, K. McCoy, P. Casari, and M. Zorzi. Experimental study of the space-time properties of acoustic channels for underwater communications. In OCEANS 2010 IEEE-Sydney, pages 1-9, May 2010.
6. L. Freitag, M. Grund, S. Singh, J. Partan, P. Koski, and K. Ball. The WHOI micro-modem: an acoustic communications and navigation system for multiple platforms. In OCEANS, 2005. Proceedings of MTS/IEEE, volume 2, pages 1086-1092, Sept. 2005.
7. C. Detweiler, M. Doniec, I. Vasilescu, E. Basha, and D. Rus. Autonomous depth adjustment for underwater sensor networks. In Proceedings of the Fifth ACM International Workshop on Underwater Networks, WUWNet '10, pages 12:1-12:4, New York, NY, USA, 2010. ACM.
8. I. F. Akyildiz, D. Pompili, and T. Melodia. State-of-The-art in protocol research for underwater acoustic sensor networks. In Proceedings of the 1st ACM international workshop on Underwater networks, WUWNet '06, pages 7-16, New York, NY, USA, 2006. ACM.
9. J. C. Gittins. Bandit Processes and Dynamic Allocation Indices. Journal of the Royal Statistical Society. Series B (Methodological), 41(2):148-177, 1979.
10. P. Whittle. Multi-armed Bandits and the Gittins Index. Journal of the Royal Statistical Society. Series B (Methodological), 42(2):143-149, 1980.
11. V. Krishnamurthy and R. J. Evans. Hidden markov model multiarm bandits: a methodology for beam scheduling in multitarget tracking. Signal Processing, IEEE Transactions on, 49(12):2893-2908, Dec. 2001.
12. Y. Gai, B. Krishnamachari, and R. Jain. Learning multiuser channel allocations in cognitive radio networks: A combinatorial multi-armed bandit formulation. In New Frontiers in Dynamic Spectrum, 2010 IEEE Symposium on, pages 1-9, Apr. 2010.
13. F. Radlinski, R. Kleinberg, and T. Joachims. Learning diverse rankings with multi-armed bandits. In Proceedings of the 25th International Conference on Machine Learning, ICML '08, pages 784-791, New York, NY, USA, 2008. ACM.
14. P. Stone and S. Kraus. To Teach or not to Teach?: Decision making under uncertainty in ad hoc teams. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: Volume 1, pages 117-124. International Foundation for Autonomous Agents and Multiagent Systems, 2010.
15. J. Le Ny, M. Dahleh, and E. Feron. Multi-agent task assignment in the bandit framework. In Decision and Control, 2006 45th IEEE Conference on, pages 5281-5286, Dec. 2006.
16. D. Bertsimas and J. Niño Mora. Restless bandits, linear programming relaxations, and a primal-dual index heuristic. Operations Research, 48(1):80-90, 2000.
17. I. M. Sonin. A generalized Gittins index for a Markov chain and its recursive calculation. Statistics and Probability Letters, 78(12):1526-1533, 2008.
18. J. Niño Mora. A (2/3)n3 Fast-Pivoting Algorithm for the Gittins Index and Optimal Stopping of a Markov Chain. INFORMS J. on Computing, 19(4):596-606, Oct. 2007.
19. W. B. Powell. Approximate Dynamic Programming: Solving the curses of dimensionality, volume 703. Wiley-Blackwell, 2007.
20. J. C. Gittins, R. Weber, and K. D. Glazebrook. Multi-armed bandit allocation indices, volume 25. Wiley Online Library, 1989.
21. J. S. Banks and R. K. Sundaram. Switching costs and the Gittins Index. Econometrica, 62(3):687-694, 1994.
22. R. Agrawal, M. V. Hedge, and D. Teneketzis. Asymptotically efficient adaptive allocation rules for the multiarmed bandit problem with switching cost. Automatic Control, IEEE Transactions on, 33(10):899-906, 1988.
23. M. Asawa and D. Teneketzis. Multi-armed bandits with switching penalties. Automatic Control, IEEE Transactions on, 41(3):328-348, 1996.
24. B. Tomasi, J. Preisig, G. B. Deane, and M. Zorzi. A study on the widesense stationarity of the underwater acoustic channel for non-coherent communication systems. Wireless Conference 2011-Sustainable Wireless Technologies (European Wireless), 11th European, pages 1-6, Apr. 2011.