Structured threshold policies for dynamic sensor scheduling - A partially observed Markov decision process approach

IEEE Transactions on Signal Processing

Volumn 55, Issue 10, 2007, Pages 4938-4957

  Krishnamurthy, Vikram ^a   Djonin, Dejan V ^a,b  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

^b Dyaptive Systems   (Canada)

Author keywords

Bayesian filtering; Monotone likelihood ratio (MLR) ordering; Partially observed Markov decision processes (POMDPs); Sensor scheduling; Stochastic approximation algorithms; Stochastic dynamic programming; Threshold policies

Indexed keywords

APPROXIMATION ALGORITHMS; BAYESIAN NETWORKS; COMPUTATIONAL COMPLEXITY; DYNAMIC PROGRAMMING; ERROR ANALYSIS; SCHEDULING ALGORITHMS;

MONOTONE LIKELIHOOD RATIO (MLR) ORDERING; PARTIALLY OBSERVED MARKOV DECISION PROCESSES; SENSOR SCHEDULING; THRESHOLD POLICIES;

MARKOV PROCESSES;

EID: 35148894375     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2007.897908     Document Type: Article

References (23)

1. R. Atar and O. Zeitouni, "Lyapunov exponents for finite state nonlinear filtering," SIAM J. Control Optimiz., vol. 35, no. 1, pp. 36-55, 1997.
2. D. Bertsekas, Dynamic Programming and Optimal Control. Belmont, MA: Athena Scientific, 1995, vol. 1.
3. S. Blackman and R. Popoli, Design and Analysis of Modern Tracking Systems. Boston, MA: Artech House, 1999.
4. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
5. A. R. Cassandra, "Exact and approximate algorithms for POMDPs," Ph.D. dissertation, Comp. Sci. Dept., Brown Univ., Providence, RI, 1998.
6. Y. Ephraim and N. Merhav, "Hidden Markov processes," IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1518-1569, Jun. 2002.
7. R. Evans, V. Krishnamurthy, and G. Nair, "Networked sensor management and data rate control for tracking maneuvering targets," IEEE Trans. Signal Process., vol. 53, no. 6, pp. 1979-1991, Jun. 2005.
8. F. R. Gantmacher, Matrix Theory. New York: Chelsea, 1960, vol. 2.
9. S. Ji, R. Parr, and L. Carin, "Non-myopic multi-aspect sensing with POMDPs," IEEE Trans. Signal Process., 2005, accepted for publication.
10. C. Kreucher, A. Hero, and K. Kastella, "A comparison of task driven and information driven sensor management for target tracking," presented at the IEEE Conf. Decision Control (CDC), Seville, Spain, Dec. 2005.
11. V. Krishnamurthy, "Algorithms for optimal scheduling and management of hidden Markov model sensors," IEEE Trans. Signal Process., vol. 50, no. 6, pp. 1382-1397, Jun. 2002.
12. V. Krishnamurthy, X. Wang, and G. Yin, "Spreading code optimization and adaptation in CDMA via discrete stochastic approximation," IEEE Trans. Inf. Theory, vol. 50, no. 9, pp. 1927-1949, Sep. 2004.
13. F. LeGland and L. Mevel, "Exponential forgetting and geometric ergodicity in hidden Markov models," Math. Control Signals Syst., vol. 13, no. 1, pp. 63-93, Feb. 2000.
14. W. S. Lovejoy, "Some monotonicity results for partially observed Markov decision processes," Oper. Res., vol. 35, no. 5, pp. 736-743, Sep.-Oct. 1987.
15. W. S. Lovejoy, "A survey of algorithmic methods for POMPDPs," Ann. Oper. Res., vol. 28, pp. 47-66, 1991.
16. A. Muller and D. Stoyan, Comparison Methods for Stochastic Models and Risk. New York. Wiley, 2002.
17. C. H. Papadimitrou and J. N. Tsitsiklis, "The compexity of Markov decision processes," Math. Oper. Res., vol. 12, no. 3, pp. 441-450, 1987.
18. G. Pflug, Optimization of Stochastic Models: The Interface Between Simulution and Optimization. Norwell, MA: Kluwer Academic. 1996.
19. U. Rieder, "Structural results for partially observed control models," Methods Models Oper. Res., vol. 35, pp. 473-490, 1991.
20. J. Spall, Introduction to Stochastic Search and Optimization. New York: Wiley, 2003.
21. D.M. Topkis, Supermodularity and Complementarity. Princeton, NJ: Princeton Univ. Press, 1998.
22. J. Wintenby and V. Krishnamurthy, "Hierarchical resource management in adaptive airborne surveillance radars - A stochastic discrete event system formulation," IEEE Trans. Aerosp. Electron. Syst., vol. 20, no. 2, pp. 401-420, Apr. 2006.
23. Q. Zhao, L. Tong, and A. Swami, "A cross layer approach to cognitive MAC for spectrum agility," presented at the 39th Asilomar Conf. Signals, Syst. Comput., Pacific Grove, CA, Oct. 2005.