Monte-carlo-based partially observable markov decision process approximations for adaptive sensing

Proceedings - 9th International Workshop on Discrete Event Systems, WODES' 08

Volumn , Issue , 2008, Pages 173-180

  Chong, Edwin K P ^a   Kreucher, Christopher M ^b   Hero III, Alfred O ^c  

^a Department of Ecosystem Science and Sustainability

^b INTEGRITY APPLICATIONS INCORPORATED   (United States)

^c UNIVERSITY OF MICHIGAN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE SENSING; APPROXIMATION METHODS; DISCRETE EVENT SYSTEMS; DISCRETE-EVENT SYSTEM; EXACT COMPUTATIONS; MONTE-CARLO; NEW APPLICATIONS; OBJECT DETECTION; OPTIMAL SOLUTIONS; PARTIALLY OBSERVABLE MARKOV DECISION PROCESS; PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES; SENSOR RESOURCES; STATE SPACES;

DISCRETE EVENT SIMULATION; MARKOV PROCESSES; SENSOR NETWORKS; STATE SPACE METHODS; STOCHASTIC CONTROL SYSTEMS; TECHNICAL PRESENTATIONS;

MONTE CARLO METHODS;

EID: 53149143884     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/WODES.2008.4605941     Document Type: Conference Paper

References (19)

1. D. P. Bertsekas, "Dynamic Programming and Suboptimal Control: A Survey from ADP to MPC," in Proc. of the Joint 44th IEEE Conference on Decision and Control and European Control Conference, Seville, Spain, December 12-15, 2005.
2. D. P. Bertsekas, Dynamic Programming and Optimal Control. Belmont, MA: Athena Scientific, Vol. I, 3rd Ed., 2005; Vol. II, 2nd Ed., 2001.
3. D. P. Bertsekas and D. A. Castanon, "Rollout algorithms for stochastic scheduling problems," Journal of Heuristics, vol. 5, pp. 89-108, 1999.
4. D. Castanon, "Approximate dynamic programming for sensor management," Proceedings of the 36th IEEE Conference on Decision and Control, San Diego, December 1997, pp. 1202-1207.
5. E. K. P. Chong, R. L. Givan, and H. S. Chang, "A framework for simulation-based network control via hindsight optimization," Proc. of the 39th IEEE Conf. on Decision and Control, Sydney, Australia, Dec. 12-15, 2000, pp. 1433-1438.
6. H. S. Chang, R. L. Givan, and E. K. P. Chong, "Parallel rollout for online solution of partially observable Markov decision processes," Discrete Event Dynamic Systems, vol. 14, no. 3, pp. 309-341, 2004.
7. Y. He and E. K. P. Chong, "Sensor scheduling for target tracking in sensor networks," in Proceedings of the 43rd IEEE Conference on Decision and Control (CDC'04), December 14-17, 2004, pp. 743-748.
8. Y. He and E. K. P. Chong, "Sensor scheduling for target tracking: A Monte Carlo sampling approach," Digital Signal Processing, vol. 16, no. 5, pp. 533-545, September 2006.
9. A. Hero, D. Castanon, D. Cochran, and K. Kastella, Eds., Foundations and Applications of Sensor Management, Springer, 2007.
10. L. W. Krakow, Y. Li, E. K. P. Chong, K. N. Groom, J. Harrington, and B. Rigdon, "Control of perimeter surveillance wireless sensor networks via partially observable Markov decision process," in Proceedings of the 2006 IEEE International Carnahan Conference on Security Technology (ICCST), Lexington, Kentucky, October 17-20, 2006.
11. M. J. Kearns, Y. Mansour, and A. Y. Ng, "A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes," Proc. of the Sixteenth International Joint Conference on Artificial Intelligence, 1999, pp. 1324-1331.
12. C. M. Kreucher, A. O. Hero, K. Kastella, and D. Chang, "Efficient Methods of Non-myopic Sensor Management for Multitarget Tracking," in Proceedings of the 43rd IEEE Conference on Decision and Control (CDC'04), December 14-17, 2004.
13. C. M. Kreucher, K. Kastella, and A. O. Hero III, "Sensor Management Using An Active Sensing Approach," Signal Processing, vol. 85, no. 3, pp. 607-624, March 2005.
14. C. M. Kreucher, K. Kastella, and A. O. Hero III, "Multitarget Tracking using the Joint Multitarget Probability Density," IEEE Transactions on Aerospace and Electronic Systems, vol. 41, no. 4, pp. 1396-1414, October 2005.
15. V. Krishnamurthy and R. J. Evans, "Hidden Markov Model Multiarm Bandits: A Methodology for Beam Scheduling in Multitarget Tracking," IEEE Transactions on Signal Processing, vol. 49, no. 12, pp. 2893-2908, December 2001.
16. Y. Li, L. W. Krakow, E. K. P. Chong, and K. N. Groom, "Approximate stochastic dynamic programming for sensor scheduling to track multiple targets," Digital Signal Processing, 2008, to appear.
17. L. Shi and C.-H. Chen, "A new algorithm for stochastic discrete resource allocation optimization," Discrete Event Dynamic Systems, vol. 10, pp. 271-294, 2000.
18. R. D. Smallwood and E. J. Sondik, "The optimal control of partially observable Markov processes over a finite horizon," Operations Research, vol. 21, pp. 1071-1088, no. 5, 1973.
19. G. Wu, E. K. P. Chong, and R. L. Givan, "Burst-level congestion control using hindsight optimization," IEEE Transactions on Automatic Control, vol. 47, no. 6, pp. 979-991, June 2002.