How to schedule measurements of a noisy Markov chain in decision making?

IEEE Transactions on Information Theory

Volumn 59, Issue 7, 2013, Pages 4440-4461

  Krishnamurthy, Vikram ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Active sensing; Bayesian filtering; Change detection; Decision making; Optimal sequential sampling; Partially observed Markov decision process; Quickest state estimation; Stochastic dominance; Stochastic dynamic programming; Submodularity

Indexed keywords

BAYESIAN NETWORKS; COST BENEFIT ANALYSIS; COSTS; DECISION MAKING; ERRORS; OPTIMIZATION; SENSITIVITY ANALYSIS;

ACTIVE SENSING; BAYESIAN FILTERING; CHANGE DETECTION; OPTIMAL SEQUENTIAL; PARTIALLY OBSERVED MARKOV DECISION PROCESS; STOCHASTIC DOMINANCE; STOCHASTIC DYNAMIC PROGRAMMING; SUBMODULARITY;

MARKOV PROCESSES;

EID: 84897695042     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2013.2253352     Document Type: Article

References (35)

1. S. C. Albright, "Structural results for partially observed Markov decision processes," Oper. Res., vol. 27, no. 5, pp. 1041-1053, Sep./Oct. 1979. (Pubitemid 10422991)
2. R. Amir, "Supermodularity and complementarity in economics: An elementary survey," South. Econ. J., vol. 71, no. 3, pp. 636-660, 2005.
3. S. Athey, "Monotone comparative statics under uncertainty," Quart. J. Econ., vol. 117, no. 1, pp. 187-223, 2002.
4. T. Banerjee and V. Veeravalli, "Data-efficient quickest change detection with on-off observation control," Seq. Anal., vol. 31, pp. 40-77, 2012.
5. D. P. Bertsekas, Dynamic Programming and Optimal Control. Belmont, MA, USA: Athena Scientific, 2000, vol. 1 and 2.
6. S. Blackman and R. Popoli, Design and Analysis of Modern Tracking Systems. Norwood, MA, USA: Artech House, 1999.
7. D. Blackwell, "Discounted dynamic programming," Anna. Math. Statist., vol. 36, pp. 226-235, 1965.
8. J. Boger, P. Poupart, and J. Hoey, "A decision-theoretic approach to task assistance for persons with dementia," in Proc. Int. Joint Conf. Artif. Intell., 2005, pp. 1293-1299.
9. A. R. Cassandra, "Exact and approximate algorithms for partially observed Markov decision process," Ph.D. dissertation, Brown Univ., Providence, RI, USA, 1998.
10. T. M. Cover and J.A. Thomas, Elements of Information Theory. New York, NY, USA: Wiley-Interscience, 2006.
11. S. Dayanik and C. Goulding, "Sequential detection and identification of an unobservable change in the distribution of a Markov-modulated random sequence," IEEE Trans. Inf. Theory, vol. 55, no. 7, pp. 3323-3345, Jul. 2009.
12. G. Fellouris and G. Moustakides, "Decentralized sequential change detection," ArXiv Oct. 2012.
13. O. Hernández-Lerma and J. Bernard Laserre, Discrete-Time Markov Control Processes: Basic Optimality Criteria. NewYork, NY, USA: Springer-Verlag, 1996.
14. D. P. Heyman and M. J. Sobel, Stochastic Models in Operations Research. New York, NY, USA: McGraw-Hill, 1984, vol. 2.
15. S. Karlin and Y. Rinott, "Classes of orderings of measures and related correlation inequalities. I. Multivariate totally positive distributions," J. Multivariate Anal., vol. 10, no. 4, pp. 467-498, Dec. 1980.
16. M. Kijima, Markov Processes for Stochastic Modelling. London, U.K.: Chapman & Hall, 1997.
17. V. Krishnamurthy, "Algorithms for optimal scheduling and management of hidden Markov model sensors," IEEE Trans. Signal Process., vol. 50, no. 6, pp. 1382-1397, Jun. 2002. (Pubitemid 34822549)
18. V. Krishnamurthy, "Bayesian sequential detection with phase-distributed change time and nonlinear penalty-A lattice programming POMDP approach," IEEE Trans. Inf. Theory, vol. 57, no. 3, pp. 7096-7124, Oct. 2011.
19. V. Krishnamurthy, "Quickest detection POMDPs with social learning: Interaction of local and global decision makers," IEEE Trans. Inf. Theory, vol. 58, no. 8, pp. 5563-5587, Aug. 2012.
20. V. Krishnamurthy, R. Bitmead, M. Gevers, and E. Miehling, "Sequential detection with mutual information stopping cost: Application in GMTI radar," IEEE Trans. Signal Process., vol. 60, no. 2, pp. 700-714, Feb. 2012.
21. V. Krishnamurthy and D. Djonin, "Structured threshold policies for dynamic sensor scheduling-A partially observed Markov decision process approach," IEEE Trans. Signal Process., vol. 55, no. 10, pp. 4938-4957, Oct. 2007. (Pubitemid 47534660)
22. W. S. Lovejoy, "On the convexity of policy regions in partially observed systems," Oper. Res., vol. 35, no. 4, pp. 619-621, Jul./Aug. 1987.
23. W. S. Lovejoy, "Some monotonicity results for partially observed Markov decision processes," Oper. Res., vol. 35, no. 5, pp. 736-743, Sep./Oct. 1987.
24. A. Muller and D. Stoyan, Comparison Methods for Stochastic Models and Risk. New York, NY, USA: Wiley, 2002.
25. M. F. Neuts, Structured Stochastic Matrices of M/G/1 Type and their Applications. New York, NY, USA: Marcel Dekker, 1989.
26. C. H. Papadimitriou and J. N. Tsitsiklis, "The complexity of Markov decision processes," Math. Oper. Res., vol. 12, no. 3, pp. 441-450, 1987.
27. M. Pollack, L. Brown, and D. Colbry, "Autominder: An intelligent cognitive orthotic system for people with memory impairment," Robot. Autonom. Syst., vol. 44, pp. 273-282, 2003.
28. H. V. Poor and O. Hadjiliadis, Quickest Detection. Cambridge, U.K.: Cambridge Univ. Press, 2008.
29. U. Rieder, "Structural results for partially observed control models," Methods Models Oper. Res., vol. 35, no. 6, pp. 473-490, 1991.
30. A. N. Shiryaev, "On optimum methods in quickest detection problems," Theory Probabil. Appl., vol. 8, no. 1, pp. 22-46, 1963.
31. A. N. Shiryayev, Optimal Stopping Rules. New York, NY, USA: Springer-Verlag, 1978.
32. A. G. Tartakovsky and V. V. Veeravalli, "General asymptotic Bayesian theory of quickest change detection," Theory Probabil. Appl., vol. 49, no. 3, pp. 458-497, 2005.
33. D. M. Topkis, Supermodularity and Complementarity. Princeton, NJ, USA: Princeton Univ. Press, 1998.
34. W. Whitt, "Multivariate monotone likelihood ratio and uniform conditional stochastic order," J. Appl. Probabil., vol. 19, pp. 695-701, 1982.
35. Y. Yilmaz, G. Moustakides, and X. Wang, "Cooperative sequential spectrum sensing based on level-triggered sampling," IEEE Trans. Signal Process., vol. 60, no. 9, pp. 4509-4524, Sep. 2012.