Reinforcement learning for closed-loop propofol anesthesia: A human volunteer study

Proceedings of the National Conference on Artificial Intelligence

Volumn 3, Issue , 2010, Pages 1807-1813

  Moore, Brett L ^a   Panousis, Periklis ^b   Kulkarni, Vivek ^b   Pyeatt, Larry D ^a   Doufas, Anthony G ^b  

^a Texas Tech University   (United States)

^b STANFORD UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE CONTROL SYSTEMS; ANESTHESIOLOGY; ARTIFICIAL INTELLIGENCE; REINFORCEMENT LEARNING;

BISPECTRAL INDEX; CLOSED-LOOP CONTROL; CONTROLLED VARIABLES; HUMAN VOLUNTEERS; MODEL BASED CONTROLS; MODEL-BASED OPC; PATIENT SPECIFIC; PERFORMANCE METRICS;

CLOSED LOOP CONTROL SYSTEMS;

EID: 77958566186     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (40)

1. Absalom, A., and Kenny, G. 2003. Closed-loop control of propofol anaesthesia using bispectral index(tm): performance assessment in patients receiving computer-controlled propofol and manually controlled remifentanil infusions for minor surgery. British Journal of Anaesthesia 90(6):737-41.
2. Absalom, A.; Sutcliffe, N.; and Kenny, G. 2002. Closed-loop control of anesthesia using bispectral index performance assessment in patients undergoing major orthopedic surgery under combined general and regional anesthesia. Anesthesiology 96(1):67-73.
3. Ausems, M; Jr., C. H.; Stanski, D.; and Burm, A. 1986. Plasma concentrations of alfentanil required to supplement nitrous oxide anesthesia for general surgery. Anesthesiology 65(4):362-73.
4. Avidan, M.; Zhang, L.; Burnside, B.; Finkel, K.; Searleman, A.; Selvidge, J.; Saager, L.; Turner, M.; Rao, S.; Bottros, M.; Hantler, C; Jacobsohn, E.; and Evers, A. 2008. Anesthesia awareness and the bispectral index. New Eng J Med 11(358): 1097-1108.
5. Baird, L. 1995. Residual algorithms: reinforcement learning with function approximation. In Proc. 12th International Conference on Machine Learning, 30-37. Morgan Kauf-mann.
6. Barvais, L.; Rausin, I.; Glen, J.; Hunter, S.; D'Hulster, D.; Cantraine, E; and d'Hollander, A. 1996. Administration of propofol by target-controlled infusion in patients undergoing coronary artery surgery. J Cardiothorac Vase Anesth 10(7):877-83.
7. Carregal, A.; Lorenzo, A.; Taboada, J.; and Barreiro, J. 2000. Intraoperative control of mean arterial pressure and heart rate with alfentanyl with fuzzy logic. Rev Esp Aneste-siol Reanim 47(3): 108-113.
8. Dayan, P. 1992. The convergence of TD(A) for general A. Machine Learning 8:341-362.
9. De Smet, T; Struys, M; Neckebroek, M.; den Hauwe, K. V.; Bonte, S.; and Mortier, E. 2008. The accuracy and clinical feasibility of a new bayesian-based closed-loop control system for propofol administration using the bispectral index as a controlled variable. Anesthesia and Analgesia 107:1200-1210.
10. Doufas, A.; Bakhshandeh, M.; Bjorksten, A.; Shafer, S.; and Sessler, D. 2004. Induction speed is not a determinant of propofol pharmacodynamics. Anesthesiology 101:1112-21.
11. Esmaeili, V.; Assareh, A.; Shamsollahi, M. B.; Moradi, M. H.; and Arefian, N. M. 2008. Estimating the depth of anesthesia using fuzzy soft computation applied to EEG features, lntell. Data Anal. 12(4):393-407.
12. Glass, P.; Bloom, M.; Kearse, L.; Rosow, C; Sebel, P.; and Manberg, P. 1997. Bispectral analysis measures sedation and memory effects of propofol, midazolam, isoflurane, and alfentanil in healthy volunteers. Anesthesiology 86(4):836-847.
13. Gullapalli, V. 1993. Learning control under extreme uncertainty. In Hanson, S. J.; Cowan, J. D.; and Giles, C. L., eds., Advances in Neural Information Processing Systems, volume 5, 327-334. Morgan Kaufmann, San Mateo, CA.
14. Hu, C; Lovejoy, W.; and Shafer, S. 1994. Comparison of some control strategies for three-compartment PK7PD models. Journal of Pharmacokinetics and Biopharmaceutics 22(6):525-550.
15. Leslie, K.; Absalom, A.; and Kenny, G. 2002. Closed loop control of sedation for colonoscopy using the bispectral index. Anaesthesia 57(7):690-709.
16. Lindholm, M.; Traff, S.; Granath, F.; Greenwald, S.; Ek-bom, A.; Lennmarken, C; and Sandin, R. 2009. Mortality within 2 years after surgery in relation to low intraoperative bispectral index values and preexisting malignant disease. Anesthesia and Analgesia 108(2):508-512.
17. Liu, N.; Chazot, T.; A, G.; A Landais, A Restoux, K. M.; PA Laloe, B. T.; Barvais, L.; and Fischler, M. 2006. Titration of propofol for anesthetic induction and maintenance guided by the bispectral index: closed-loop versus manual control: a prospective, randomized, multicenter study. Anesthesiology 104(4):686-695.
18. Moore, B.; Sinzinger, E.; Quasny, X; and Pyeatt, L. 2004. Intelligent control of closed-loop sedation in simulated ICU patients. In FLAIRS 2004. AAAI Press.
19. Moore, B.; Pyeatt, L.; and Doufas, A. 2009. Fuzzy control for closed-loop, patient-specific hypnosis in intraoperative patients: A simulation study. In Conf Proc IEEE Eng Med Biol Soc, volume 1.
20. Myles, P.; Leslie, K.; McNeil, J.; Forbes, A.; and Chan, M. 2000. Bispectral index monitoring to prevent awareness during anaesthesia: the B-Aware randomised controlled trial. The Lancet 363(9423): 1757-1763.
21. Pilge, S.; Zanner, R.; Schneider, G.; Blum, J.; and Kreuzer, M. 2006. Analysis of cerebral state, bispectral, and nar-cotrend indices. Anesthesiology 104(3):488-494.
22. Rampil, I. 1997. A primer for EEG signal processing in anesthesia. Anesthesiology 89(4):980-1002.
23. Ropcke, H.; Rehberg, B.; Koenen-Bergmann, M.; Bouillon, T.; Bruhn, J.; and Hoeft, A. 2001. Surgical stimulation shifts EEG concentration-response relationship of des-flurane. Anesthesiology 94(3):255-l13.
24. Sakai, T.; Matsuki, A.; White, P.; and Giesecke, A. 2000. Use of an EEG-bispectral closed-loop delivery system for administering propofol. Acta Anesthesiologica Scandinav-ica 44:1007-1010.
25. Sandin, R.; Enlund, G.; Samuelsson, P.; and Lennmarken, C. 2000. Awareness during anaesthesia : a prospective case study. The Lancet 355(9205):707-711.
26. Schaublin, J.; Derighetti, M.; Feigenwinter, P.; Petersen-Felix, S.; and Zbinden, A. M. 1996. Fuzzy logic control of mechanical ventilation during anaesthesia. British Journal Anaesthesia 77(5):636-41.
27. Schnider, T.; Minto, C; Gambus, P.; Andresen, C; Goodale D. ; Shafer, S.; and Youngs, E. 1998. The influence of method of administration and covariates on the pharmacokinetics of propofol in adult volunteers. Anesthesiology 88(5): 1170-1182.
28. Sebel, P.; Bowdle, T.; Ghoneim, M.; Rampil, I.; Padilla, R. Gan, T.; and Domino, K. 2004. The incidence of aware ness during anesthesia: A multicenter United States study Anesthesia and Analgesia 99:833-839.
29. Servin, F. 1998. TCI compared with manually controllec infusion of propofol: a multicentre study. Anaesthesic 53(Suppl 1):82-86.
30. Struys, M.; De Smet, T.; GreenWald, S.; Abasalom, A. Bingé, S.; and Mortier, E. 2001. Closed-loop controllec administration of propofol using bispectral analysis. Anes thesiology 95(1):6-17.
31. Struys, M.; De Smet, T.; Greenwald, S.; Absalom, A. Binge, S.; and Mortier, E. 2004. Performance evaluatior of two published closed-loop control systems using bispec tral index monitoring: a simulation study. Anesthesiology 100(3):640-7.
32. Struys, M.; Coppens, M.; Neve, N. D.; Mortier, E.; Doufas A.; Van Bocxlaer, J.; and Shafer, S. 2007. Influence o administration rate on propofol plasma-effect site equilibra tion. Anesthesiology 07(3):386-396.
33. Sutton, R., and Barto, A. 1998. Reinforcement Learning. An Introduction. MIT Press.
34. Theil, D.; 3rd, T. S.; White, W.; Goodman, D.; Glass, P. Bai, S.; Jacobs, J.; and Reves, J. 1993. Midazolam anc fentanyl continuous infusion anesthesia for cardiac surgery a comparison of computer-assisted versus manual infusior systems. J Cardiothorac Vase Anesth 7(3):300-6.
35. Tsitsiklas, J., and Van Roy, B. 1996. An analysis of tempo ral difference learning with function approximation. Tech nical Report LIDS-P-2322, Massachusetts Institute of Tech nology.
36. Varvel, J.; Donoho, D.; and Shafer, S. 1992. Measuring the predictive performance of computer-controlled infusior pumps. J Pharmacokinet Biopharm 20:63-94.
37. Vuyk, J.; Engbers, F; Burm, A.; Vletter, A.; and Bovill J. 1995. Performance of computer-controlled infusion o propofol: an evaluation of five pharmacokinetic parametei sets. Anesth Analg 81(6): 1275-1282.
38. Watkins, C. 1989. Learning from Delayed Rewards. PhE dissertation, Cambridge University, Computer Science De partment.
39. Wood, W. 1989. Variability of human drug response. Anes thesiology 71 (4):631-634.
40. Zadeh, L. 1965. Fuzzy sets. Information and Control 8:338-353.