Nonmyopic multiaspect sensing with partially observable Markov decision processes

IEEE Transactions on Signal Processing

Volumn 55, Issue 6 I, 2007, Pages 2720-2730

  Ji, Shihao ^a   Parr, Ronald ^b   Carin, Lawrence ^a  

^a Duke University   (United States)

^b Duke University   (United States)

Author keywords

Hidden Markov models (HMMs); Multiaspect sensing; Nonmyopic algorithms; Partially observable Markov decision processes (POMDPs)

Indexed keywords

ACOUSTIC WAVE SCATTERING; ALGORITHMS; DATA PROCESSING; HIDDEN MARKOV MODELS;

MULTIASPECT SENSING; NONMYOPIC ALGORITHMS; PARTIALLY OBSERVABLE MARKOV DECISION PROCESSES;

SENSOR DATA FUSION;

EID: 34249808524     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2007.893747     Document Type: Article

References (28)

1. L. J. Guibas, "Sensing, tracking and reasoning with relations," IEEE Signal Process. Mag., vol, 19, pp. 73-85, Mar. 2002.
2. V. V. Fedorov, Theory of Optimal Experiments. New York: Academic, 1972.
3. Y. Zhang, X. Liao, and L. Carin, "Detection of buried targets via active selection of labeled data: Application to sensing subsurface uxo," IEEE Trans. Geosci. Remote Sens., vol. 42, pp, 2535-2543, Nov. 2004.
4. X. Liao and L. Carin, "Application of the theory of optimal experiments to adaptive ele ctromagnetic-induction sensing of buried targets," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, pp. 961-972, Aug. 2004.
5. P. Whaite and F. P. Ferrie, "Autonomous exploration: Driven by uncertainty," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 3, pp. 193-205, Mar. 1997.
6. E. J. Sondik, "The optimal control of partially observable Markov processes," Ph.D. dissertation, Standford Univ., Stanford, 1971.
7. L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, "Planning and acting in partially observable stochastic domains," Artif. Intell. vol. 101, pp. 99-134, 1998.
8. C. Kreucher, K. Kastella, and A. Hero, "Sensor management using an active sensing approach," Signal Process., vol. 85, no. 3, pp, 607-624, Mar. 2005.
9. J. Pineau, G. Gordon, and S. Thrun, "Point-based value iteration: An anytime algorithm for POMDPs," in Proc. Int. Joint Conf. Artif. Intell. (IJCAI), 2003, pp. 1025-1032.
10. P. Runkle, P. Bharadwaj, and L. Carin, "Hidden Markov model multi-aspect target classification," IEEE Trans. Signal Process., vol. 47, pp. 2035-2040, Jul. 1999.
11. J. M. Bernardo and A. Smith, Bayesian Theory. New York: Wiley, 2001.
12. A. R. Cassandra, M. Littman, and N. Zhang, "Incremental pruning: A simple, fast, exact method for partially observable Markov decision processes," in Proc. 13th Ann. Conf. Uncertainty in Artif. Intell. (UAI), 1997, pp. 54-61.
13. V. Krishnamurthy, "De centralized emission management for low probability of intercept sensor platforms in network centric warfare - A multi-armed bandit approach," IEEE Drans. Aerosp. Electron. Syst. vol. 41, no. 1, pp. 133-152, Jan. 2005.
14. V. Krishnamurthy and R. J. Evans, "Hidden Markov model multiarm bandits: A methodology for beam scheduling in multitarget tracking," IEEE Trans. Signal Process., vol. 49, pp. 2893-2908, Dec. 2001.
15. D. A. Castañon, "Approximate dynamic programming for sensor management," in Proc. 36th IEEE Conf. Decision Contr., 1997, pp. 1202-1207.
16. D. A. Castañon, "Stochastic control bounds on sensor network performance," in IEEE Conf. Decision Contr., 2005.
17. V. Krishnamurthy, "Algorithms for optimal scheduling and management of hidden Markov model sensors," IEEE Trans. Signal Process., vol. 50, pp. 1382-1397, Jun. 2002.
18. R. Evans, V. Krishnamurthy, G, Nair, and L. Sciacca, "Networked sensor management and data rate control for tracking maneuvering targets." IEEE Trans. Signal Process., vol. 53, pp, 1979-1991, Jun. 2005.
19. M. L. Littman, A.R. Cassandra, and L.P. Kaelbling, "Learning policies for partially observable environments: Scaling up," in Proc. 12th Int. Conf. Mach. Learn. (ICML), 1995, pp. 362-370.
20. W. S. Lovejoy, "Computationally feasible bounds for partially observed Markov decision processes," Operat. Res., vol. 39, pp. 162-175, 1991.
21. P. Runkle, L. Carin, L. Couchman, T. Yoder, and J. Bucaro, "Multi-aspect identification of submerged elastic targets via wave-based matching pursuits and hidden Markov models," J. Acoust. Soc. Amer., vol. 106, pp. 605-616, Aug. 1999.
22. M. McClure and L. Carin, "Matched pursuits with a wave-based dictionary," IEEE Trans. Signal Process., vol. 45, pp. 2912-2927, Dec. 1997.
23. R. M. Gray, "Vector quantization," IEEE ASSP Mag., pp. 4-29, Apr. 1984.
24. M. J. Beal and Z. Ghahramani, "The variational Bayesian EM algorithm for incomplete data: With application to scoring graphical model structures." in Bayesian Statistics 7. Oxford, UX.: Oxford Univ. Press, 2003.
25. Y. Ephraim and N. Mehrav, "Hidden Markov processes," IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1518-1569, Jun. 2002.
26. A. Viterbi, "Error bounds for convolutional codes and an asymptotically optimum decoding algorithm," IEEE Trans. Inf. Theory, vol. 13, pp. 260-269, Apr. 1967.
27. S. Mallat, A Wavelet Tour of Signal Processing. New York: Academic, 1998.
28. R. I. Brafman and M. Tennenholz, "R-max - A general polynomial time algorithm for near-optimal reinforcement learning," J. Mach. Learn. Res., vol. 3, pp. 213-231, 2002.