Optimal sensing via multi-armed bandit relaxations in mixed observability domains

Proceedings - IEEE International Conference on Robotics and Automation

Volumn 2015-June, Issue June, 2015, Pages 4807-4812

  Lauri, Mikko ^a   Ritala, Risto ^a  

^a TAMPERE UNIVERSITY OF TECHNOLOGY   (Finland)

Author keywords

[No Author keywords available]

Indexed keywords

AGRICULTURAL ROBOTS; DECISION MAKING; RANDOM PROCESSES; ROBOTICS; STOCHASTIC SYSTEMS;

AMOUNT OF INFORMATION; MIXED OBSERVABILITIES; MULTI ARMED BANDIT; OPTIMAL VALUE FUNCTIONS; RELAXING CONSTRAINTS; SEQUENTIAL DECISION MAKING UNDER UNCERTAINTIES; SOLUTION QUALITY; TARGET MONITORING;

OBSERVABILITY;

EID: 84938230088     PISSN: 10504729     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICRA.2015.7139867     Document Type: Conference Paper

References (23)

1. L. Kaelbling, M. Littman, and A. Cassandra, "Planning and acting in partially observable stochastic domains," Artificial Intelligence, vol. 101, no. 1-2, pp. 99-134, 1998.
2. A. O. Hero, D. Castañón, D. Cochran, and K. Kastella, Eds., Foundations and Applications of Sensor Management. New York, NY: Springer, 2007.
3. E. K. P. Chong, C. M. Kreucher, and A. O. Hero, "Partially Observable Markov Decision Process Approximations for Adaptive Sensing," Discrete Event Dynamic Systems, vol. 19, no. 3, pp. 377-422, May 2009.
4. B. Charrow, V. Kumar, and N. Michael, "Approximate representations for multi-robot control policies that maximize mutual information," Autonomous Robots, vol. 37, no. 4, pp. 383-400, Aug. 2014.
5. N. Atanasov, J. Le Ny, K. Daniilidis, and G. Pappas, "Information Acquisition with Sensing Robots: Algorithms and Error Bounds," in IEEE Int. Conf. on Robotics and Automation (ICRA), Hong Kong, China, June 2014, pp. 6447-6454.
6. M. Lauri and R. Ritala, "Stochastic control for maximizing mutual information in active sensing," in ICRA 2014 Workshop on Robots in Homes and Industry: Where to Look First, Hong Kong, China, June 2014.
7. E. A. Hansen, "Indefinite-horizon POMDPs with action-based termination," in Proceedings of the National Conference on Artificial Intelligence, Vancouver, Canada, July 2007, pp. 1237-1242.
8. C. H. Papadimitriou and J. N. Tsitsiklis, "The Complexity of Markov Decision Processes," Mathematics of Operations Research, vol. 12, pp. 441-450, 1987.
9. M. T. Spaan and N. A. Vlassis, "Perseus: Randomized point-based value iteration for POMDPs," Journal of Artificial Intelligence Research, vol. 24, pp. 195-220, 2005.
10. J. Pineau, G. Gordon, and S. Thrun, "Anytime point-based approximations for large POMDPs," Journal of Artificial Intelligence Research, vol. 27, no. 1, pp. 335-380, 2006.
11. S. Ross, J. Pineau, S. Paquet, and B. Chaib-Draa, "Online planning algorithms for POMDPs," Journal of Artificial Intelligence Research, vol. 32, pp. 663-704, 2008.
12. D. Silver and J. Veness, "Monte-Carlo Planning in Large POMDPs," in Advances in Neural Information Processing Systems 23, Vancouver, Canada, Dec. 2010, pp. 2164-2172.
13. S. C. Ong, S. W. Png, D. Hsu, and W. S. Lee, "Planning under uncertainty for robotic tasks with mixed observability," International Journal of Robotics Research, vol. 29, no. 8, pp. 1053-1068, 2010.
14. A. O. Hero and D. Cochran, "Sensor management: Past, present, and future," IEEE Sensors Journal, vol. 11, no. 12, pp. 3064-3075, 2011.
15. R. Bellman, Dynamic Programming. Princeton, New Jersey: Princeton University Press, 1957.
16. R. D. Smallwood and E. J. Sondik, "The optimal control of partially observable Markov processes over a finite horizon," Operations Research, vol. 21, no. 5, pp. 1071-1088, 1973.
17. W. S. Lovejoy, "A survey of algorithmic methods for partially observed Markov decision processes," Annals of Operations Research, vol. 28, no. 1, pp. 47-65, 1991.
18. M. Hauskrecht, "Value-function approximations for partially observable Markov decision processes," Journal of Artificial Intelligence Research, vol. 13, no. 1, pp. 33-94, 2000.
19. V. Krishnamurthy and D. V. Djonin, "Structured Threshold Policies for Dynamic Sensor Scheduling-A Partially Observed Markov Decision Process Approach," IEEE Transactions on Signal Processing, vol. 55, no. 10, pp. 4938-4957, Oct. 2007.
20. M. Araya, O. Buffet, V. Thomas, and F. Charpillet, "A POMDP Extension with Belief-dependent Rewards," in Advances in Neural Information Processing Systems 23, Vancouver, Canada, Dec. 2010, pp. 64-72.
21. J. C. Gittins, "Bandit processes and dynamic allocation indices," Journal of the Royal Statistical Society. Series B (Methodological), pp. 148-177, 1979.
22. S. Paquet, B. Chaib-draa, and S. Ross, "Hybrid POMDP algorithms," in Proceedings of The AAMAS Workshop on Multi-Agent Sequential Decision Making in Uncertain Domains, Hakodate, Japan, May 2006, pp. 133-147.
23. S. H. A. Ahmad, M. Liu, T. Javidi, Q. Zhao, and B. Krishnamachari, "Optimality of myopic sensing in multichannel opportunistic access," IEEE Transactions on Information Theory, vol. 55, no. 9, pp. 4040-4050, 2009.