Treating epilepsy via adaptive neurostimulation: A reinforcement learning approach

International Journal of Neural Systems

Volumn 19, Issue 4, 2009, Pages 227-240

  Pineau, Joelle ^a   Guez, Arthur ^a   Vincent, Robert ^a   Panuccio, Gabriella ^b   Avoli, Massimo ^b  

^a MCGILL UNIVERSITY   (Canada)

^b MCGILL UNIVERSITY   (Canada)

Author keywords

Epilepsy; Neurostimulation; Reinforcement learning

Indexed keywords

ARTIFICIAL INTELLIGENCE; LEARNING ALGORITHMS; LEARNING SYSTEMS; NEUROLOGY; OPTIMIZATION; PATIENT TREATMENT;

EPILEPSY; LABELED TRAINING DATA; MACHINE LEARNING LITERATURE; NEUROSTIMULATION; OPTIMIZATION PROBLEMS; REINFORCEMENT LEARNING APPROACH; STIMULATION STRATEGIES; TECHNICAL CHALLENGES;

REINFORCEMENT LEARNING;

4 AMINOPYRIDINE; POTASSIUM CHANNEL BLOCKING AGENT;

ALGORITHM; ANIMAL; ARTICLE; BIOPHYSICS; CHEMICALLY INDUCED DISORDER; DISEASE MODEL; ELECTROENCEPHALOGRAPHY; ELECTROSTIMULATION THERAPY; ENTORHINAL CORTEX; EPILEPSY; IN VITRO STUDY; LEARNING; MALE; MAN MACHINE INTERACTION; METHODOLOGY; PATHOLOGY; PATHOPHYSIOLOGY; PHYSIOLOGY; RAT; REINFORCEMENT; SPRAGUE DAWLEY RAT;

4-AMINOPYRIDINE; ALGORITHMS; ANIMALS; BIOPHYSICS; DISEASE MODELS, ANIMAL; ELECTRIC STIMULATION THERAPY; ELECTROENCEPHALOGRAPHY; ENTORHINAL CORTEX; EPILEPSY; LEARNING; MALE; MAN-MACHINE SYSTEMS; POTASSIUM CHANNEL BLOCKERS; RATS; RATS, SPRAGUE-DAWLEY; REINFORCEMENT (PSYCHOLOGY);

EID: 70149084425     PISSN: 01290657     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0129065709001987     Document Type: Article

References (47)

1. H. Adeli, Z. Zhou and N. Dadmehr, Analysis of EEG records in an epileptic patient using wavelet transform, Journal of Neuroscience Methods 123(1) (2003) 69-87.
2. N. Agopyan and M. Avoli, Synaptic and non-synaptic mechanisms underlying low calcium bursts in the in vitro hippocampal slice, Exp. Brain Res. 73 (3) (1988) 533-540
3. M. Avoli, M. G. Biagini, R. Pumain, G. DArcangelo and V. Tan-credi, Network and pharmacological mechanisms leading to epileptiform synchronization in the lim-bic system in vitro, Prog. Neurobiol. 68(3) (2002) 167-207.
4. M. Barbarosie and M. Avoli, Ca3-driven hippocampal-entorhinal loop controls rather than sustains in vitro limbic seizures, J. Neurosci. 17(23) (1997) 9308-9314
5. R. Bellman, A markovian decision process, Journal of Mathematics and Mechanics 6 (1957).
6. M. Bikson, J. Lian, P. J. Hahn, W. C. Stacey, C. Sciortino and D. M. Durand, Suppression of epileptiform activity by high frequency sinusoidal fields in rat hippocampal slices, Journal of Physiol. 531(1) (2001) 181-191.
7. P. Boon, K. Vonck, V. De Herdt, A. Van Dycke, M. Goethals, L. Goossens, M. Van Zandijcke, T. De Smedt, I. Dewaele, R. Achten, W. Wadman, F. Dewaele, J. Caemaert and D. Van Roost, Deep brain stimulation in patients with refractory temporal lobe epilepsy, Epilepsia 48(8) (2007) 1551-1560.
8. N. Chakravarthy, S. Sabesan, L. Iasemidis and K. Tsakalis, Controlling synchronization in a neuron-level population model, International Journal of Neural Systems 17(2) (2007) 123-138.
9. M. D'Antuono, G. Biagini and M. Avoli, Unpublished data.
10. G. D'Arcangelo, G. Panuccio, B. Tancredi and M. Avoli, Repetitive low-frequency stimulation reduces epileptiform synchronization in limbic neuronal networks, Neurobiology of Disease 19(1-2) (2005) 119-128.
11. D. Durand and M. Bikson, Suppression and control of epileptiform activity by electrical stimulation: A review, Proceedings of the IEEE 89(7) (2001) 1065-1082.
12. D. Ernst, P. Geurts and L. Wehenkel, Tree-based batch mode reinforcement learning, Journal of Machine Learning Research 6 (2005) 503-556.
13. D. Ernst, G.-B. Stan, J. Gonçalves and L. Wehenkel, Clinical data based optimal STI strategies for HIV: A reinforcement learning approach, in 15th Machine Learning Conference of Belgium and The Netherlands, Ghent, Belgium (May 2006).
14. P. Geurts, D. Ernst and L. Wehenkel, Extremely randomized trees, Machine Learning 63(1) (2006) 3-42.
15. S. Ghosh-Dastidar, H. Adeli and N. Dadmehr, Mixed-band wavelet-chaos-neural network methodology for epilepsy and epileptic seizure detection, IEEE Transactions on Biomedical Engineering 54(9) (2007) 1545-1551.
16. B. J. Gluckman, H. Nguyen, S. L. Weinstein and S. J. Schiff, Adaptive electric field control of epileptic seizures, Journal of Neuroscience 21(2) (2001) 590-600.
17. A. Guez, R. Vincent, M. Avoli and J. Pineau, Adaptive treatment of epilepsy via batch-mode reinforcement learning, in Innovative Applications of Artificial Intelligence (2008).
18. A. Handforth, C. M. DeGiorgio, S. C. Schachter et al., Vagus nerve stimulation therapy for partial-onset seizures. a randomized active-control trial, Neurology 51 (1998) 48-55.
19. M. Hauskrecht and H. Fraser, Planning treatment of ischemic heart disease with partially observable markov decision processes, Artificial Intelligence in Medicine 18 (2000) 221-244.
20. L. Iasemidis, Epileptic seizure prediction and control, IEEE Trans Biomed Eng. 50(5) (2003) 549-558.
21. K. Jerger and S.J. Schiff, Periodic pacing an in vitro epileptic focus, J of Neurophysiol 73 (1995) 876-879.
22. L. P. Kaelbling, M. L. Littman and A. W. Moore, Reinforcement learning: A survey, Journal of Artifi-cial Intelligence Research 4 (1996) 237-285.
23. H. Khosravani, P. L. Carlen and J. L. Perez Velazquez, The control of seizure-like activity in the rat hippocampal slice, Biophysical Journal 84 (2003) 687-695.
24. E. H. Kossoff, E. A. Ritzl, J. M. Politsky, A. M. Murro, J. R. Smith, R. B. Duckrow, D. D. Spencer and G. K. Bergey, Effect of an external responsive neurostimulator on seizures and electrographic discharges during subdural electrode monitoring, Epilepsia 45(12) (2004) 1560-1567.
25. M. Le Van Quyen, J. Martinerie, V. Navarro, P. Boon, M. and M. Baulac, Anticipation of epileptic seizures from standard EEG recordings, The Lancet 3567 (2001).
26. B. Litt and J. Echauz, Prediction of epileptic seizures, The Lancet Neurology 1(1) (2002) l22-30.
27. I. Mody, J.D. Lambert and U. Heinemann, Low extracellular magnesium induces epileptiform activity and spreading depression in rat hippocampal slices, J. Neurophysiol. 57(3) (1987) 869-888
28. F. Mormann, T. Kreuz, C. Rieke, R. G. Andrzejak, A. Kraskov, P. David, C. E. Elger and K. Lehnertz, On the predictability of epileptic seizures, Clinical Neuropsychology 116 (2005) 569-587.
29. T. I. Netoff, R. Clewley, Arno. S., T. Keck and J. A. White, Epilepsy in small world networks, Journal of Neuroscience 24(37) (2004) 8075-8083.
30. I. Osorio, J. Overman, J. Giftakis and S. B. Wilkinson, High frequency thalamic stimulation for inoperable mesial temporal epilepsy, Epilepsia 48(8) (2007) 1561-1571.
31. H. Osterhage, F. Mormann, T. Wagner and K. Lehn-ertz, Measuring the directionality of coupling: Phase versus state space dynamics and application to EEG time series, International Journal of Neural Systems 17 (3) (2007) 139-148.
32. J. Pineau, M.G. Bellemare, A. J. Rush, A. Ghizaru and S.A. Murphy, Constructing evidence-based treatment strategies using methods from computer science, Drug and Alcohol Dependence 88S (2007) S52-S60.
33. M. L. Puterman, Markov Decision Processes (Wiley, 1994).
34. H. Qu and J. Gotman, A patient-specific algorithm for the detection of seizure onset in long-term EEG monitoring: Possible use as a warning device, IEEE Trans Biomed Engineering 44 (1997) 115-112.
35. G. Regesta and P. Tanganelli, Clinical aspects and biological bases of drug-resistant epilepsies, Epilepsy Res 34 (1999) 109-122.
36. Ghosh-Dastidar S. and H. Adeli, A new supervised learning algorithm for multiple spiking neural networks with application in epilepsy and seizure detection, Neural Networks 22 (2009).
37. Sutton R. S. and A. G. Barto, Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA, 1998).
38. S.J. Schiff, K. Jerger, D.H. Duong et al., Controlling chaos in the brain, Nature 370 (1994) 615-620.
39. S.F. Traynelis and R. Dingledine, Potassium-induced spontaneous electrographic seizures in the rap hip-pocampal slice, J. Neurophysiol. 59 (1988) 259-276.
40. K. Tsakalis, N. Chakravarthy and L. Iasemidis, Control of epileptic seizures: Models of chaotic oscillator networks, in 44th IEEE Conference on Decision and Control, Vols. 12-15 (2005), pp. 2975-2981.
41. K. Tsakalis and L. Iasemidis, Control aspects of a theoretical model for epileptic seizures, International Journal of Bifurcation and Chaos 16(7) (2006) 2013- 2027.
42. B. M. Uthman, A. M. Reichl, J. C. Dean, S. Eisen-schenk, R. Gilmore, S. A. Reid, S. N. Roper and B. J. Wilder, Effectiveness of vagus nerve stimulation in epilepsy patients: A 12 year observation, Neurology 63 (2004) 1124-1126.
43. A. L. Velasco, F. Velasco, M. Velasco, D. Trejo, G. Castro and J. D. Carrillo-Ruiz, Electrical stimulation of the hippocampal epileptic FOCI for seizure control: A double-blind, long-term follow-up study, Epilepsia 48(10) (2007) 1895-1903.
44. J. Vesper, B. Steinhoff, S. Rona, C. Wille, S. Bilic, G. Nikkhah and C. Ostertag, Chronic high-frequency deep brain stimulation of the STN/SNR for progressive myoclonic epilepsy, Epilepsia 48(8) (2007) 1984-1989.
45. K. Vonck, P. Boon, E. Achten, J. De Reuck and J. Caemaert, Long-term amygdalohippocampal stimulation for refractory temporal lobe epilepsy, Ann Neurol 52(5) (2002) 556-565.
46. R.J. Warren and E. Durand, Effects of applied currents on spontaneous epileptiform activity induced by low calcium in the rat hippocampus, Brain Res 806 (1998) 186-195.
47. D. Zumsteg, A. M. Lozano, H. G. Wieser and R. A. Wennberg, Cortical activation with deep brain stimulation of the anterior thalamus for epilepsy, Clinical Neurophysiology 117 (2006) 192-207. (Pubitemid 43099964)