Medium-term speaker states - A review on intoxication, sleepiness and the first challenge

Computer Speech and Language

Volumn 28, Issue 2, 2014, Pages 346-374

  Schuller, Björn ^a,b   Steidl, Stefan ^c,d   Batliner, Anton ^a,d   Schiel, Florian ^e   Krajewski, Jarek ^f   Weninger, Felix ^a   Eyben, Florian ^a  

^a TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^b JOANNEUM RESEARCH   (Austria)

^c INTERNATIONAL COMPUTER SCIENCE INSTITUTE   (United States)

^d UNIVERSITY OF ERLANGEN NUREMBERG   (Germany)

^e UNIVERSITY OF MUNICH   (Germany)

^f UNIVERSITY OF WÜRZBURG   (Germany)

Author keywords

Challenge; Computational paralinguistics; Intoxication; Sleepiness; Survey

Indexed keywords

ALCOHOL INTOXICATION; ALGORITHMIC ANALYSIS; BINARY CLASSIFICATION; CHALLENGE; HIGH RISK ENVIRONMENT; INTOXICATION; PARALINGUISTICS; SLEEPINESS;

LINGUISTICS; SURVEYING;

RISK ASSESSMENT;

EID: 84890561011     PISSN: 08852308     EISSN: 10958363     Source Type: Journal    
DOI: 10.1016/j.csl.2012.12.002     Document Type: Article

References (94)

1. A. Afifi, and S. Azen Statistical Analysis. A Computer Oriented Approach 1979 Academic Press New York
2. T. Ananthapadmanabha Aerodynamic and acoustic theory of voice production A. Neustein, H.A. Patil, Forensic Speaker Recognition, Law Enforcement and Counter-terrorism 2011 Springer New York 309 363
3. E.G. Bard, C. Sotillo, A.H. Anderson, H.S. Thompson, and M.M. Taylor The DCIEM Map Task Corpus: spontaneous dialogue under sleep deprivation and drug treatment Speech Communication 20 1996 71 84
4. A. Batliner, D. Seppi, S. Steidl, and B. Schuller Segmenting into adequate units for automatic recognition of emotion-related episodes: a speech-based approach Advances in Human-Computer Interaction 2010 2010 15 Article ID 782802
5. D.M. Behne, S.M. Rivera, and D.B. Pisoni Effects of alcohol on speech: durations of isolated words, sentences and passages Research on Speech Perception 17 1991 285 301
6. F. Biadsy, W.Y. Wang, A. Rosenberg, and J. Hirschberg Intoxication detection using phonetic, phonotactic and prosodic cues Proc. of INTERSPEECH Florence, Italy 2011 3209 3212
7. T. Bocklet, K. Riedhammer, and E. Nöth Drink and speak: on the automatic classification of alcohol intoxication by acoustic, prosodic and text-based features Proc. of INTERSPEECH Florence, Italy 2011 3213 3216
8. D. Bone, M.P. Black, M. Li, A. Metallinou, S. Lee, and S. Narayanan Intoxicated speech detection by fusion of speaker normalized hierarchical features and GMM supervectors Proc. of INTERSPEECH Florence, Italy 2011 3217 3220
9. E. Bozkurt, E. Erzin, C. Eroǧlu Erdem, and A.T. Erdem RANSAC-based training data selection for speaker state recognition Proc. of INTERSPEECH Florence, Italy 2011 3293 3296
10. D. Bratzke, B. Rolke, R. Ulrich, and M. Peters Central slowing during the night Psychological Science 18 2007 456 461
11. A. Braun Speaking while intoxicated: phonetic and forensic aspects Proc. of International Congress of Phonetic Sciences (ICPhS) Aix-en-Provence 1991 146 149
12. M. Brenner, and J. Cash Speech analysis as an index of alcohol intoxication - the Exxon Valdez accident. Aviation Space, and Environmental Medicine 62 1991 893 898
13. N.V. Chawla, K.W. Bowyer, L.O. Hall, and W.P. Kegelmeyer Synthetic minority over-sampling technique Journal of Artificial Intelligence Research 16 2002 321 357
14. S.B. Chin, and D.B. Pisoni Alcohol and Speech 1997 Academic Press Inc San Diego, CA
15. O.M. Cooney, K. McGuigan, P. Murphy, and R. Conroy Acoustic analysis of the effects of alcohol on the human voice Journal of the Acoustical Society of America 103 5 1998 2895
16. L. Dhupati, S. Kar, A. Rajaguru, and A. Routray A novel drowsiness detection scheme based on speech analysis with validation using simultaneous EEG recordings Proc. IEEE Conference on Automation Science and Engineering (CASE) Toronto, ON 2010 917 921
17. D. Dinges, and N. Kribbs Performing while sleepy: effects of experimentally-induced sleepiness T. Monk, Sleep, Sleepiness and Performance 1991 Wiley Chister, England 97 128
18. G.R. Doddington Sheep, goats, lambs and wolves - an analysis of individual differences in speaker recognition performance. Proc. of ICSLP Sydney, Australia 1998
19. E. Echeburúa, R.B. de Medina, and J. Aizpiri Comorbidity of alcohol dependence and personality disorders: a comparative study Alcohol & Alcoholism 42 2007 618 622
20. F. Eyben, M. Wöllmer, and B. Schuller openEAR - introducing the munich open-source emotion and affect recognition toolkit Proc. ACII Amsterdam 2009 576 581
21. F. Eyben, M. Wöllmer, and B. Schuller openSMILE - the Munich versatile and fast open-source audio feature extractor Proc. ACM Multimedia Florence, Italy 2010 1459 1462
22. Flatley, D.; Reyner, L.A.; Horne, J.A.; 2004. Sleep-related crashes on sections of different road types in the UK (1995-2001). In: Road Safety Research Report, Vol. 52. Department for Transport, London, pp. 4-132.
23. S. Fulda, and R. Popp Measurement of daytime sleepiness in the elderly Somnology 15 2011 154 159
24. R. Gajšek, S. Dobrišek, and F. Mihelič University of Ljubljana system for interspeech 2011 speaker state challenge Proc. of INTERSPEECH Florence, Italy 2011 3297 3300
25. M. Golz, D. Sommer, and D. Mandic Microsleep detection in electrophysiological signals K. Dinesh, G. Hugo, Proc. of the 1st International Workshop on Biosignal Processing and Classification (BPC) Barcelona, Spain 2005 102 109
26. M. Golz, D. Sommer, U. Trutschel, B. Sirois, and D. Edwards Evaluation of fatigue monitoring technologies Somnology 14 2010 187 189
27. H.P. Greeley, J. Berg, E. Friets, J. Wilson, G. Greenough, J. Picone, J. Whitmore, and T. Nesthus Sleepiness estimation using voice analysis Behaviour Research Methods 39 2007 610 619
28. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. Witten The WEKA data mining software: an update SIGKDD Explorations 11 1 2009 10 18
29. Y. Harrison, and J.A. Horne Sleep deprivation affects speech Sleep 20 1997 871 877
30. C. Heinze, U. Trutschel, T. Schnupp, D. Sommer, A. Schenka, J. Krajewski, and M. Golz Operator fatigue estimation using heart rate measures World Congress on Medical Physics and Biomedical Engineering, IFMBE Proceedings, Vol. 25 2009 930 934
31. H. Hollien, G. De Jong, C.A. Martin, R. Schwartz, and K. Liljegren Effects of ethanol intoxication on speech suprasegmentals Journal of the Acoustical Society of America 110 6 2001 3198 3206
32. F. Hönig, A. Batliner, and E. Nöth Does it groove or does it stumble - automatic classification of alcoholic intoxication using prosodic features Proc. of INTERSPEECH Florence, Italy 2011 3225 3228
33. T. Horberry, R. Hutchins, and R. Tong Motorcycle rider fatigue: a review Road Safety Research Report, Vol. 78 Department for Transport, London 2008 4 63
34. D.-Y. Huang, S.S. Ge, and Z. Zhang Speaker state classification based on fusion of asymmetric SIMPLS and support vector machines. Proc. of INTERSPEECH Florence, Italy 2011 3301 3304
35. M. Ingre, T. Åkerstedt, B. Peters, A. Anund, G. Kecklund, and A. Pickles Subjective sleepiness and accident risk: avoiding the ecological fallacy Journal of Sleep Research 15 2006 142 148
36. B. Johannes, V.P. Salnitski, H.-C. Gunga, and K. Kirsch Voice stress monitoring in space - possibilities and limits Aviation, Space, and Environmental Medicine 71 2000 A58 65
37. K. Johnson, D.B. Pisoni, and R.H. Bernacki Do voice recordings reveal whether a person is intoxicated? A case study Phonetica 41 1990 215 237
38. P.W. Kalivas Predisposition to addiction: pharmacokinetics, pharmacodynamics, and brain circuitry American Journal of Psychiatry 160 2003 1 3
39. F. Klingholz, R. Penning, and E. Liebhardt Recognition of low-level alcohol intoxication from speech signal Journal of the Acoustical Society of America 84 3 1988 929 935
40. B. Kostyk, and A. Rochet Laryngeal airway resistance in teachers with vocal fatigue: a preliminary study Journal of Voice 12 1998 287 299
41. J. Krajewski, A. Batliner, and M. Golz Acoustic sleepiness detection: framework and validation of a speech-adapted pattern recognition approach Behavior Research Methods 41 2009 795 804
42. J. Krajewski, M. Golz, S. Schnieder, T. Schnupp, C. Heinze, and D. Sommer Detecting fatigue from steering behaviour applying continuous wavelet transform Proceedings Measuring Behaviour, Vol. 7 2010 326 329
43. J. Krajewski, and B. Kroeger Using prosodic and spectral characteristics for sleepiness detection Proc. of Interspeech Antwerp 2007 1841 1844
44. J. Krajewski, S. Schnieder, D. Sommer, A. Batliner, and B. Schuller Applying multiple classifiers and non-linear dynamics features for detecting sleepiness from speech. Neurocomputing 84 2012 65 75 Special Issue "From neuron to behavior: evidence from behavioral measurements"
45. L.I. Kuncheva, and C.J. Whitaker Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy Machine Learning 51 2 2003 181 207
46. H.J. Künzel, and A. Braun The effect of alcohol on speech prosody Proc. of International Congress of Phonetic Sciences (ICPhS) Barcelona 2003 2645 2648
47. H.J. Künzel, A. Braun, and U. Eysholdt Einfluß von Alkohol auf Sprache und Stimme 1992 Kriminalistik Verlag Heidelberg
48. W.J.M. Levelt, A. Roelofs, and A.S. Meyer A theory of lexical access in speech production Journal of Behavioral and Brain Sciences 22 1999 1 75
49. M. Levit, R. Huber, A. Batliner, and E. Noeth Use of prosodic speech characteristics for automated detection of alcohol intoxination M. Bacchiani, J. Hirschberg, D. Litman, M. Ostendorf, Proc. of Workshop on Prosody and Speech Recognition 2001 Red Bank NJ, USA 103 106
50. P. Lieberman, B.G. Kanki, and A. Protopapas Speech production and cognitive decrements on Mount Everest Aviation, Space, and Environmental Medicine 66 1995 857 864
51. A. Loukas, J.L. Krull, L. Chassin, and A.C. Carle The relation of personality to alcohol abuse/dependence in a high-risk sample Journal of Personality 68 2000 1153 1175
52. C.S. Martin, and M. Yuchtman Using speech as an index of alcohol-intoxication Research on Speech Perception 12 1986 413 426
53. S. Melamed, and A. Oksenberg Excessive daytime sleepiness and risk of occupational injuries in non-shift daytime workers Sleep 25 2002 315 322
54. C. Montacié, and M.-J. Caraty Combining multiple phoneme-based classifiers with audio feature-based classifier for the detection of alcohol intoxication Proc. of INTERSPEECH Florence, Italy 2011 3205 3208
55. G.O. Morris, H.L. Williams, and A. Lubin Misperception and disorientation during sleep deprivation Archives of General Psychiatry 2 1960 247 252
56. A. Nogueiras An HMM-based approach to the INTERSPEECH 2011 speaker state challenge Proc. of INTERSPEECH Florence, Italy 2011 3289 3292
57. T.L. Nwe, H. Li, and M. Dong Analysis and detection of speech under sleep deprivation Proc. of Interspeech 2006 17 21
58. D. O'Shaughnessy Speech Communications: Human and Machine 2000 IEEE Press New York, USA
59. T. Rahman, S. Mariooryad, S. Keshavamurthy, G. Liu, J.H.L. Hansen, and C. Busso Detecting sleepiness by fusing classifiers trained with novel acoustic features Proc. of INTERSPEECH Florence, Italy 2011 3285 3288
60. L. Read Road Safety Part 1: Alcohol, Drugs and Fatigue 2006 Department for Transport London 1-12
61. W. Revelle, and K. Scherer Personality and emotion Oxford Companion to the Affective Sciences 2009 Oxford University Press Oxford pp. 1-4
62. A. Rosenberg AuToBI - a tool for automatic ToBI annotation Proc. of INTERSPEECH Makuhari, Japan 2010 146 149
63. R. Ruiz, P. Plantin De Hugues, and C. Legros Advanced voice analysis of pilots to detect fatigue and sleep inertia Acta Acustica United with Acustica 96 2010 567 579
64. C. Schenka, T. Schnupp, C. Heinze, J. Krajewski, and M. Golz The compensatory tracking task: a pattern recognition based approach for classifying vigilance Proceedings Measuring Behaviour, Vol. 7 2010 470 472
65. K.R. Scherer Vocal communication of emotion: a review of research paradigms Speech Communication 40 2003 227 256
66. F. Schiel Perception of alcoholic intoxication in speech Proc. of Interspeech Florence, Italy 2011 3281 3284
67. F. Schiel, and C. Heinrich Laying the foundation for in-car alcohol detection by speech Proc. INTERSPEECH 2009 Brighton, UK 2009 983 986
68. F. Schiel, C. Heinrich, and S. Barfüßer Alcohol language corpus Language Resources and Evaluation 46 3 2012 503 521 10.1007/s10579-011- 9139-y Springer, Berlin-New York
69. F. Schiel, C. Heinrich, and V. Neumeyer Rhythm and formant features for automatic alcohol detection Proc. of Interspeech Chiba, Japan 2010 458 461
70. R. Schleicher, N. Galley, S. Briest, and L. Galley Blinks and saccades as indicators of fatigue in sleepiness warnings: looking tired? Ergonomics 51 2008 982
71. S. Schnieder, J. Krajewski, T. Esch, B. Baluch, and B. Wilhelm Just valid or even accurate: determine the measurement accuracy of the pupillographic sleepiness test by applying self- and observer ratings Somnology, Sleep Research and Sleep Medicine 1 2012 1 15
72. T.A. Schnupp, S. Edwards, D. Krajewski, and J.M. Golz Is posturography a candidate for a vigilance test? Proc. World Congress on Medical Physics and Biomedical Engineering, Vol. 25 2009 388 392
73. B. Schuller, S. Steidl, and A. Batliner The INTERSPEECH 2009 emotion challenge Proc. of Interspeech Brighton, UK 2009 312 315
74. B. Schuller, S. Steidl, A. Batliner, F. Burkhardt, L. Devillers, C. Müller, and S. Narayanan The INTERSPEECH 2010 paralinguistic challenge - age, gender, and affect Proc. of Interspeech Makuhari, Japan 2010 2794 2797
75. B. Schuller, S. Steidl, A. Batliner, E. Nth, A. Vinciarelli, F. Burkhardt, R. van Son, F. Weninger, F. Eyben, T. Bocklet, G. Mohammadi, and B. Weiss The INTERSPEECH 2012 speaker trait challenge Proc. of Interspeech Portland, OR 2012
76. B. Schuller, S. Steidl, A. Batliner, F. Schiel, and J. Krajewski The Interspeech 2011 speaker state challenge. Proc. of Interspeech Florence, Italy 2011 3201 3204
77. B. Schuller, F. Weninger, M. Wöllmer, Y. Sun, and G. Rigoll Non-negative matrix factorization as noise-robust feature extractor for speech recognition. Proc. of ICASSP Dallas, TX, USA 2010 March 4562 4565
78. E. Sforza, Z. de Saint Hilaire, A. Pelissolo, T. Rochat, and V. Ibanez Personality anxiety and mood traits in patients with sleep-related breathing disorders: effect of reduced daytime alertness Sleep Medicine 3 2 2002 139 145
79. P. Shahidi, S.C. Southward, and M. Ahmadian Estimating crew alertness from speech Proceedings of the American Society of Mechanical Engineers Joint Rail Conference 2010 51 59
80. M. Sigmund, A. Prokes, and P. Zelinka Detection of alcohol in speech signal using LF model Proc. of International Conference on Artificial Intelligence and Applications Innsbruck 2010 193 196
81. L.C. Sobell, M.B. Sobell, and R.F. Coleman Alcohol-inducted disfluency in non-alcoholics Folia Phoniatrica 34 1982 316 323
82. D. Sommer, M. Golz, T. Schnupp, J. Krajewski, U. Trutschel, and D. Edwards A measure of strong driver fatigue Proc. of International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design, Vol. 4 2009 9 15
83. B. Story An overview of the physiology, physics and modeling of the sound source for vowels Acoustical Science and Technology 23 2002 195 206
84. H. Traunmüller Evidence for demodulation in speech perception Proc. of ICSLP Beijing, China 2000 790 793
85. F. Trojan, and K. Kryspin-Exner The decay of articulation under the influence of alcohol and paraldehyde Folia Phoniatrica 20 1968 217 238
86. S. Ultes, A. Schmitt, and W. Minker Attention, sobriety checkpoint! Can humans determine by means of voice, if someone is drunk. and can automatic classifiers compete? Proc. of INTERSPEECH Florence, Italy 2011 3221 3224
87. A.P. Vogel, J. Fletcher, and P. Maruff Acoustic analysis of the effects of sustained wakefulness on speech Journal of the Acoustical Society of America 128 2010 3747 3756
88. M. Vollrath Mikropausen im Sprechen 1993 Peter Lang Frankfurt, Germany
89. H. Watanabe, T. Shin, H. Matsuo, F. Okuno, T. Tsuji, M. Matsuoka, J. Fukaura, and H. Matsunaga Studies on vocal fold injection and changes in pitch associated with alcohol intake Journal of Voice 1994 340 346
90. F. Weninger, and B. Schuller Fusing utterance-level classifiers for robust intoxication recognition from speech Proc. Workshop on Inferring Cognitive and Emotional States from Multimodal Measures (MMCogEmS) held in conjunction with the 13th International Conference on Multimodal Interaction (ICMI 2011) Alicante, Spain 2011
91. J. Whitmore, and S. Fisher Speech during sustained operations Speech Communication 20 1996 55 70
92. N. Wright, and A. McGown Vigilance on the civil flight deck: Incidence of sleepiness and sleep during long-haul flights and associated changes in physiological parameters Ergonomics 44 2001 82 106
93. G. Yule On the association of attributes in statistics Philosophical Transactions of the Royal Society A 194 1900 257 319
94. X.-J. Zhang, J.H. Gu, and Z. Tao Research of detecting fatigue from speech by PNN Proc. of 2010 International Conference on Information Networking and Automation (ICINA). Vol. 2 2010 278 281