Sequential decision-making in healthcare IoT: Real-time health monitoring, treatments and interventions

2016 IEEE 3rd World Forum on Internet of Things, WF-IoT 2016

Volumn , Issue , 2016, Pages 24-29

  Zois, Daphney Stavroula ^a  

^a University of Illinois at Urbana Champaign   (United States)

Author keywords

constrained Markov decision processes; dynamic programming; e health; Markov decision processes; multi armed bandits; partially observable Markov decision processes; partially observable semi Markov decision processes; semi Markov decision processes; stochastic optimal control

Indexed keywords

BEHAVIORAL RESEARCH; DECISION MAKING; HEALTH CARE; INTERNET OF THINGS; LEARNING ALGORITHMS; MARKOV PROCESSES; STOCHASTIC CONTROL SYSTEMS; STOCHASTIC MODELS; STOCHASTIC SYSTEMS;

CONSTRAINED MARKOV DECISION PROCESS; E HEALTH; EHEALTH; MARKOV DECISION PROCESSES; MULTIARMED BANDITS (MABS); PARTIALLY OBSERVABLE MARKOV DECISION PROCESS; PARTIALLY OBSERVABLE SEMI-MARKOV DECISION PROCESS; SEMI-MARKOV DECISION PROCESS; STOCHASTIC OPTIMAL CONTROL;

DYNAMIC PROGRAMMING;

EID: 85015191857     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/WF-IoT.2016.7845446     Document Type: Conference Paper

References (25)

1. D. P. Bertsekas, Dynamic Programming and Optimal Control. Athena Scientific, 2005, vol. 1.
2. A. Mahajan and D. Teneketzis, Multi-Armed Bandit Problems. Springer US, 2008.
3. U. Mitra, B. A. Emken, S. Lee, M. Li, V. Rozgic, G. Thatte, H. Vathsangam, D.-S. Zois, M. Annavaram, S. Narayanan, M. Levorato, D. Spruijt-Metz, and G. Sukhatme, "KNOWME: a case study in wireless body area sensor network design, " IEEE Comm. Magazine, vol. 50, no. 5, pp. 116-125, 2012.
4. J. Hoey, C. Boutilier, P. Poupart, P. Olivier, A. Monk, and A. Mihailidis, "People, sensors, decisions: Customizable and adaptive technologies for assistance in healthcare, " ACM Transactions on Interactive Intelligent Systems, vol. 2, no. 4, pp. 1-36, Dec. 2012.
5. P. Paredes, R. Gilad-Bachrach, M. Czerwinski, A. Roseway, K. Rowan, and J. Hernandez, "PopTherapy: Coping with Stress Through Pop-culture, " in Proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare, 2014, pp. 109-117.
6. P. Magni, S. Quaglini, M. Marchetti, and G. Barosi, "Deciding when to intervene: a Markov decision process approach, " International Journal of Medical Informatics, vol. 60, pp. 237-253, 2000.
7. M. Hauskrecht and H. Fraser, "Planning treatment of ischemic heart disease with partially observable Markov decision processes, " Artificial Intelligence in Medicine, vol. 18, no. 3, pp. 221-244, 2000.
8. J. Mason, "Using Electronic Health Records to Monitor and Improve Adherence to Medication, " 2013. [Online]. Available: http://people. virginia. edu/-jem4yb/Papers/Mason 2013. pdf
9. D.-S. Zois, M. Levorato, and U. Mitra, "Energy-Efficient, Heterogeneous Sensor Selection for Physical Activity Detection in Wireless Body Area Networks, " IEEE Trans. Signal Process., vol. 61, no. 7, pp. 1581-1594, Apr. 2013.
10. R. E. Bellman, Dynamic Programming. Princeton University Press, 1957.
11. R. A. Howard, Dynamic Programming and Markov Processes. MIT Press, 1960.
12. A. Eitan, Constrained Markov decision processes. CRC Press, 1999, vol. 7.
13. E. J. Sondik, "The optimal control of partially observable Markov decision processes, " Ph. D. dissertation, Stanford University, 1971.
14. D. Aberdeen, "A (Revised) Survey of Approximate Methods for Solving Partially Observable Markov Decision Processes, " National ICT Australia, Tech. Rep., 2003.
15. J. Pineau, G. J. Gordon, and S. Thrun, "Anytime Point-Based Approximations for Large POMDPs, " Journal of Artificial Intelligence Research, vol. 27, pp. 335-380, 2006.
16. P. Poupart and C. Boutilier, "VDCBPI: an Approximate Scalable Algorithm for Large POMDPs, " in NIPS, 2004, pp. 1081-1088.
17. H. Yu, "Approximate Solution Methods for Partially Observable Markov and Semi-Markov Decision Processes, " Ph. D. dissertation, Massachusetts Institute of Technology, 2006.
18. T. Lai and H. Robbins, "Asymptotically efficient adaptive allocation rules, " Advances in Applied Mathematics, vol. 6, no. 1, pp. 4-22, 1985.
19. J. C. Gittins, "Bandit processes and dynamic allocation indices, " Journal of the Royal Statistical Society, Series B, pp. 148-177, 1979.
20. J. Langford and T. Zhang, "The Epoch-Greedy Algorithm for Multi-armed Bandits with Side Information, " in NIPS, 2008, pp. 817-824.
21. J. Mickova, "Stochastic scheduling with multi-armed bandits, " Ph. D. dissertation, University of Melbourne, 2000.
22. M. Rabbi, M. H. Aung, M. Zhang, and T. Choudhury, "MyBehavior: Automatic Personalized Health Feedback from User Behaviors and Preferences Using Smartphones, " in Proc. of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, 2015, pp. 707-718.
23. M. T. J. Spaan and N. Vlassis, "Perseus: Randomized Point-based Value Iteration for POMDPs, " Journal of Artificial Intelligence Research, vol. 24, pp. 195-220, 2005.
24. S. Bubeck and N. Cesa-Bianchi, "Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, " Foundations and Trends in Machine Learning, vol. 5, no. 1, pp. 1-122, 2012.
25. P. Auer, N. Cesa-Bianchi, and P. Fischer, "Finite-time Analysis of the Multiarmed Bandit Problem, " Machine Learning, vol. 47, no. 2, pp. 235-256, 2002.