Sleep-spindle detection: Crowdsourcing and evaluating performance of experts, non-experts and automated methods

Nature Methods

Volumn 11, Issue 4, 2014, Pages 385-392

  Warby, Simon C ^a   Wendt, Sabrina L ^a,b   Welinder, Peter ^c   Munk, Emil G S ^a,b   Carrillo, Oscar ^a   Sorensen, Helge B D ^d   Jennum, Poul ^b   Peppard, Paul E ^e   Perona, Pietro ^c   Mignot, Emmanuel ^a  

^a STANFORD UNIVERSITY   (United States)

^b GLOSTRUP UNIVERSITY HOSPITAL   (Denmark)

^c CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^d TECHNICAL UNIVERSITY OF DENMARK   (Denmark)

^e UNIVERSITY OF WISCONSIN SCHOOL OF MEDICINE AND PUBLIC HEALTH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ALGORITHM; ARTICLE; AUTOMATION; CONSENSUS; CONTROLLED STUDY; CROWDSOURCING; ELECTROENCEPHALOGRAPHY; FEMALE; GOLD STANDARD; HUMAN; INTERNET; MALE; MIDDLE AGED; NORMAL HUMAN; POLYSOMNOGRAPHY; PRIORITY JOURNAL; SCORING SYSTEM; SLEEP SPINDLE;

AGED; ALGORITHMS; AUTOMATION; CROWDSOURCING; ELECTROENCEPHALOGRAPHY; HUMANS; INTERNET; MIDDLE AGED; SLEEP STAGES;

EID: 84897954667     PISSN: 15487091     EISSN: 15487105     Source Type: Journal    
DOI: 10.1038/nmeth.2855     Document Type: Article

References (51)

1. Iber, C., Ancoli-Israel, S., Chesson, A. & Quan, S.F. AASM Manual for the Scoring of Sleep and Associated Events 1st edn. (American Academy of Sle ep Medicine, 2007
2. Silverstein, L.D. & Levy, C.M. The stability of the sigma sleep spindle. Electroencephalogr. Clin. Neurophysiol. 40, 666-670 (1976
3. Tan, X., Ca mpbell, I.G. & Feinberg, I. Internight reliability and benchmark values for computer analyses of non-rapid eye movement (NREM) and REM EEG in normal young adult and elderly subjects. Clin. Neurophysiol. 112, 1540-1552 (2001
4. Werth, E., Achermann, P., Dijk, D.J. & Borbly, A.A. Spindle frequency activity in the sleep EEG: Individual differences and topographic distribution. Electroencephalogr. Clin. Neurophysiol. 103, 535-542 (1997
5. De Gennaro, L., Ferrara, M., Vecchio, F., Curcio, G. & Bertini, M. An electroencephalographic fingerprint of human sleep. Neuroimage 26, 114-122 (2005
6. De Gennaro, L. & Ferrara, M. Sleep spindles: An overview. Sleep Med. Rev. 7, 423-440 (2003
7. Shibagaki, M., Kiyono, S. & Watanabe, K. Spindle evolution in normal and mentally retarded children: A review. Sleep 5, 47-57 (1982
8. Crowley, K., Trinder, J., Kim, Y., Car rington, M. & Colrain, I.M. The effects of normal aging on sleep spindle and K-complex production. Clin. Neurophysiol. 113, 1615-1622 (2002
9. Nicolas, A., Petit, D., Rompr, S. & Montpl aisir, J. Sleep spindle characteristics in healthy subjects of different age groups. Clin. Neurophysiol. 112, 521-527 (2001
10. Martin, N. et al. Topography of age-related changes in sleep spindles. Neurobiol. Aging 34, 468- 476 (2013
11. De Gennaro, L. et al. The electroencephalographic fingerprint of sleep is genetically determined: A twin study. Ann. Neurol. 64, 455-460 (200 8
12. Ambrosius, U. et al. Heritability of sleep electroencephalogram. Biol. Psychiatry 64, 344-348 (2008
13. Fogel, S.M. & Smith, C.T. The function of the sleep s pindle: A physiological index of intelligence and a mechanism for sleep-dependent memory consolidation. Neurosci. Biobehav. Rev. 35, 1154-1165 (2011
14. Walker, M.P. The role of sleep in cognition and emotion. Ann. NY Acad. Sci. 1156, 168-197 (2009
15. Diekelmann, S. & Born, J. The memory function of sleep. Nat. Rev. Neurosci. 11, 114- 126 (2010
16. Barakat, M. et al. Fast and slow spindle involvement in the consolidation of a new motor sequence. Behav. Brain Res. 217, 117 -121 (2011
17. Ferrarelli, F. et al. Reduced sleep spindle activity in schizophrenia patients. Am. J. Psychiatry 164, 483-492 (2007
18. Wamsley, E.J. et al. Reduced sleep spindles and spindle coherence in schizophrenia: Mechanisms of impaired memory consolidation Biol. Psychiatry 71, 154-161 (2012
19. Limoges, E., Mottron, L., Bolduc, C., Berthiaume, C. & Godbout, R. Atypical sleep architecture and the autism phenotype. Brain 128, 1049-1061 (2005
20. Myatchin, I. & Lagae, L. Sleep spindle abnormalities in children with generalized spike-wave discharges. Pediatr. Neurol. 36, 106-111 (2007
21. Montagna, P., Gambetti, P., Cortelli, P. & Lugaresi, E. Familial and sporadic fatal insomnia. Lancet Neurol. 2, 167-176 (2003
22. Espa, F., Ondze, B., Deglise, P ., Billiard, M. & Besset, A. Sleep architecture, slow wave activity, and sleep spindles in adult patients with sleepwalking and sleep terrors. Clin. Neurophysiol. 111, 929-939 (2000
23. Himanen, S.- L., Virkkala, J., Huupponen, E. & Hasan, J. Spindle frequency remains slow in sleep apnea patients throughout the night. Sleep Med. 4, 229-234 (2003
24. Petit, D., Gagnon, J.-F., Fa ntini, M.L., Ferini-Strambi, L. & Montplaisir, J. Sleep and quantitative EEG in neurodegenerative disorders. J. Psychosom. Res. 56, 487-496 (2004
25. Ferrara, M., Moroni, F., De Gennaro, L. & Nobili, L. Hippocampal sleep features: Relations to human memory function. Front. Neurol. 3, 57 (2012
26. Steriade, M. Grouping of brain rhythms in corticothalamic systems. Neuroscience 137, 1087-1106 (2006
27. Vukadinovic, Z. Sleep abnormalities in schizophrenia may suggest impaired trans-thalamic cortico-cortical communication: Towards a dynamic model of the illness. Eur. J. Neurosci. 34, 1031- 1039 (2011
28. Schimicek, P., Zeitlhofer, J., Anderer, P. & Saletu, B. Automatic sleep-spindle detection procedure: Aspects of reliability and validity. Clin. Electroencephalogr. 25, 26-29 (1994
29. Huupponen, E. et al. Optimization of sigma amplitude threshold in sleep spindle detection. J. Sleep Res. 9, 327-334 (2000
30. Gais, S., Mulle, M ., Helms, K. & Born, J. Learning-dependent increases in sleep spindle density. J. Neurosci. 22, 6830-6834 (2002
31. Mulle, M., Marshall, L., Gais, S. & Born, J. Grouping of spindle activity during slow oscillations in human non-rapid eye movement sleep. J. Neurosci. 22, 10941-10947 (2002
32. Anderer, P. et al. An E-health solution for automatic sleep classification accor ding to Rechtschaffen and Kales: Validation study of the Somnolyzer 24 x 7 utilizing the Siesta database. Neuropsychobiology 51, 115-133 (2005
33. Schabus, M. et al. Sleep spindle-related activity in t he human EEG and its relation to general cognitive and learning abilities. Eur. J. Neurosci. 23, 1738-1746 (2006
34. Huupponen, E. et al. Development and comparison of four sleep spindle detection methods. Artif. Intell. Med. 40, 157-170 (2007
35. Devuyst, S. et al. Automatic sleep spindle detection in patients with sleep disorders. Conf. Proc. IEEE Eng. Med. Biol. Soc. 1, 3883-3886 (2006
36. Barakat, M. et al. Sleep spindles predict neural and behavioral changes in motor sequence consolidation. Hum. Brain Mapp. 34, 2918 -2928 (2013
37. Bergmann, T.O., Molle, M., Diedrichs, J., Born, J. & Siebner, H.R. Sleep spindle-related reactivation of category-specific cortical regions after learning face-scene associations. Neuroimage 59, 2733 -2742 (2012
38. Ayoub, A. et al. Differential effects on fast and slow spindle activity, and the sleep slow oscillation in humans with carbamazepine and flunarizine to antagonize voltage-dependent Na+ and C a2+ channel activity. Sleep 36, 905-911 (2013
39. Ray, L.B., Fogel, S.M., Smith, C.T. & Peters, K.R. Validating an automated sleep spindle detection algorithm using an individualized approach. J. Sleep Res. 19, 374-378 (2010
40. Schabus, M. et al. Hemodynamic cerebral correlates of sleep spindles during human non-rapid eye movement sleep. Proc. Natl. Acad. Sci. USA 104 , 13164-13169 (2007
41. Budizs, R., Krmendi, J., Rig, P. & Lzr, A.S. The individual adjustment method of sleep spindle analysis: Methodological improvements and roots in the fingerprint paradigm. J. Neurosci. Methods 178, 205-213 (2009
42. Ruch, S. et al. Sleep stage II contributes to the consolidation of declarative memories. Neuropsychologia 50, 2389 -2396 (2012
43. Bdizs, R., Gombos, F. & Kov cs, I. Sleep EEG fingerprints reveal accelerated thalamocortical oscillatory dynamics in Williams syndrome. Res. Dev. Disabil. 33, 153-164 (2012
44. Sitnikova, E., Hramov, A.E., Koronovsky, A.A. & van Luijtelaar, G. Sleep spindles and spike-wave discharges in EEG: Their generic features, similarities and distinctions disclosed with Fourier transform and continuous wav elet analysis. J. Neurosci. Methods 180, 304-316 (2009
45. Wendt, S.L. et al. Validation of a novel automatic sleep spindle detector with high performance during sleep in middle aged subjects. Conf. Pr oc. IEEE Eng. Med. Biol. Soc. 2012, 4250-4253 (2012
46. Plante, D.T. et al. Topographic and sex-related differences in sleep spindles in major depressive disorder: A high-density EEG investigati on. J. Affect. Disord. 146, 120-125 (2013
47. Peppard, P.E. et al. Increased prevalence of sleep-disordered breathing in adults. Am. J. Epidemiol. 177, 1006- 1014 (2013
48. Feinberg, I., Koresko, R.L. & Heller, N. EEG sleep patterns as a function of normal and pathological aging in man. J. Psychiatr. Res. 5, 107-144 (1967
49. Nir, Y. et al. Regional slow waves and spindles in human sleep. Neuron 70, 153- 169 (2011
50. McCormick, L., Nielsen, T., Nicolas, A., Ptito, M. & Montplai sir, J. Topographical distribution of spindles and K-complexes in normal subjects. Sleep 20, 939-941 (1997). 2
51. Donoho, D.L. An invitation to reproducible computational research. Biostatistics 11, 385- 388 (2010). 3