The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing

IEEE Transactions on Affective Computing

Volumn 7, Issue 2, 2016, Pages 190-202

  Eyben, Florian ^a,b,c   Scherer, Klaus R ^c,d   Schuller, Bjorn W ^a,e,f   Sundberg, Johan ^g   Andre, Elisabeth ^h   Busso, Carlos ⁱ   Devillers, Laurence Y ^j   Epps, Julien ^k,l   Laukka, Petri ^m   Narayanan, Shrikanth S ⁿ   Truong, Khiet P ^o  

^a AUDEERING GMBH   (Germany)

^b TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^c UNIVERSITY OF GENEVA   (Switzerland)

^d UNIVERSITY OF MUNICH   (Germany)

^e UNIVERSITY OF PASSAU   (Germany)

^f IMPERIAL COLLEGE LONDON   (United Kingdom)

^g ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

^h UNIVERSITY OF AUGSBURG   (Germany)

ⁱ The University of Texas at Dallas   (United States)

^j UFR 919 Laboratoire d'Informatique Pour la Mécanique et les Sciences de l'Ingénieur   (France)

^k UNIVERSITY OF NEW SOUTH WALES   (Australia)

^l CSIRO   (Australia)

^m STOCKHOLM UNIVERSITY   (Sweden)

ⁿ UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^o UNIVERSITY OF TWENTE   (Netherlands)

more..

Author keywords

Acoustic Features; Affective Computing; Emotion Recognition; Geneva Minimalistic Parameter Set; Speech Analysis; Standard

Indexed keywords

ELASTIC WAVES; PARAMETER ESTIMATION; REGULATORY COMPLIANCE; SPEECH ANALYSIS; STANDARDS;

ACOUSTIC FEATURES; ACOUSTIC PARAMETERS; AFFECTIVE COMPUTING; COMPARATIVE EVALUATIONS; EMOTION RECOGNITION; PARAMETER SET; RECOGNITION SYSTEMS; STATE-OF-THE-ART METHODS;

SPEECH RECOGNITION;

EID: 84973513831     PISSN: 19493045     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAFFC.2015.2457417     Document Type: Article

References (68)

1. K. R. Scherer, "Vocal affect expression: A review and a model for future research, " Psychol. Bull., vol. 99, pp. 143-165, 1986.
2. R. Banse and K. R. Scherer, "Acoustic profiles in vocal emotion expression, " J. Personality Soc. Psychol., vol. 70, no. 3, pp. 614-636, 1996.
3. P. N. Juslin and P. Laukka, "Impact of intended emotion intensity on cue utilization and decoding accuracy in vocal expression of emotion, " Emotion, vol. 1, pp. 381-412, 2001.
4. S. Yildirim, M. Bulut, C. Lee, A. Kazemzadeh, C. Busso, Z. Deng, S. Lee, S. Narayanan, "An acoustic study of emotions expressed in speech, " in Proc. 8th Int. Conf. Spoken Language Process., Jeju Island, Korea, Oct. 2004, pp. 2193-2196.
5. E. Moore, M. Clements, J. Peifer, L. Weisser, "Critical analysis of the impact of glottal features in the classification of clinical depression in speech, " IEEE Trans. Biomed. Eng., vol. 55, no. 1, pp. 96-107, Jan. 2008.
6. C. Busso, S. Lee, S. Narayanan, "Analysis of emotionally salient aspects of fundamental frequency for emotion detection, " IEEE Trans. Audio, Speech Language Process., vol. 17, no. 4, pp. 582-596, May 2009.
7. J. Sundberg, S. Patel, E. Bjorkner, K. R. Scherer, "Interdependencies among voice source parameters in emotional speech, " IEEE Trans. Affective Comput., vol. 2, no. 3, pp. 162-174, Jul.-Sep. 2011.
8. T. F. Yap, "Production under cognitive load: Effects and classification, " Ph. D. dissertation, The Univ. New South Wales, Sydney, Australia, 2012.
9. P. N. Juslin and P. Laukka, "Communication of emotions in vocal expression and music performance: Different channels, same code" Psychol. Bull., vol. 129, no. 5, pp. 770-814, Sep. 2003.
10. S. Patel and K. R. Scherer, Vocal Behaviour. Berlin, Germany: Mouton-DeGruyter, 2013, pp. 167-204.
11. P. Boersma, "Praat, a system for doing phonetics by computer, " Glot Int., vol. 5, nos. 9/10, pp. 341-345, 2001.
12. B. Schuller, S. Steidl, A. Batliner, A. Vinciarelli, K. Scherer, F. Ringeval, M. Chetouani, "The INTERSPEECH 2013 computational paralinguistics challenge: Social signal, conflict, emotion, autism, " in Proc. 14th Annu. Conf. Int. Speech Commun. Assoc, 2013, pp. 148-152.
13. M. Schroder, Speech and Emotion Research. An Overview of Research Frameworks and a Dimensional Approach to Emotional Speech Synthesis (Reports in Phonetics, Univ. of the Saarland), vol. 7. Inst. for Phonetics, Univ. of Saarbrucken, Saarbrucken, Germany, 2004.
14. M. Schroder, F. Burkhardt, S. Krstulovic, "Synthesis of emotional speech, " in Blueprint for Affective Computing, K. R. Scherer, T. Banziger, E. Roesch, Eds. Oxford, U. K. : Oxford Univ. Press, 2010, pp. 222-231.
15. K. R. Scherer, "Vocal communication of emotion: A review of research paradigms, " Speech Commun., vol. 40, pp. 227-256, 2003.
16. K. Hammerschmidt and U. Jurgens, "Acoustical correlates of affective prosody, " J. Voice, vol. 21, pp. 531-540, 2007.
17. M. Goudbeek and K. R. Scherer, "Beyond arousal: Valence and potency/control cues in the vocal expression of emotion, " J. Acoust. Soc. Amer., vol. 128, pp. 1322-1336, 2010.
18. D. A. Sauter, F. Eisner, A. J. Calder, S. K. Scott, "Perceptual cues in nonverbal vocal expressions of emotion, " Quart. J. Experimental Psychol., vol. 63, pp. 2251-2272, 2010.
19. P. Laukka and H. A. Elfenbein, "Emotion appraisal dimensions can be inferred from vocal expressions, " Soc. Psychol. Personality Sci., vol. 3, pp. 529-536, 2012.
20. B. Schuller, B. Vlasenko, F. Eyben, M. Wollmer, A. Stuhlsatz, A. Wendemuth, G. Rigoll, "Cross-corpus acoustic emotion recognition: Variances and strategies, " IEEE Trans. Affective Comput., vol. 1, no. 2, pp. 119-131, Jul.-Dec. 2010.
21. A. Batliner, S. Steidl, B. Schuller, D. Seppi, K. Laskowski, T. Vogt, L. Devillers, L. Vidrascu, N. Amir, L. Kessous, V. Aharonson, "Combining efforts for improving automatic classification of emotional user states, " in Proc. 5th Slovenian 1st Int. Language Technol. Conf., Oct. 2006, pp. 240-245.
22. D. Bone, C.-C. Lee, S. Narayanan, "Robust unsupervised arousal rating: A rule-based framework with knowledge-inspired vocal features, " IEEE Trans. Affective Comput., vol. 5, no. 2, pp. 201-213, Apr.-Jun. 2014.
23. B. Schuller, A. Batliner, D. Seppi, S. Steidl, T. Vogt, J. Wagner, L. Devillers, L. Vidrascu, N. Amir, L. Kessous, V. Aharonson, "The relevance of feature type for the automatic classification of emotional user states: Low level descriptors and functionals, " in Proc. 8th Annu. Conf. Int. Speech Commun. Assoc., Aug. 2007, pp. 2253-2256.
24. F. Weninger, F. Eyben, B. W. Schuller, M. Mortillaro, K. R. Scherer, "On the acoustics of emotion in audio: What speech, music and sound have in common, " Frontiers Psychol., vol. 4, no. Article ID 292, pp. 1-12, May 2013.
25. F. Eyben, F. Weninger, B. Schuller, "Affect recognition in Real-life acoustic conditions-A new perspective on feature selection, " in Proc. Annu. Conf. Int. Speech Commun. Assoc., Aug. 2013, pp. 2044-2048.
26. M. Tahon and L. Devillers, "Acoustic measures characterizing anger across corpora collected in artificial or natural context, " in Proc. Speech Prosody, Chicago, IL, USA, 2010.
27. T. F. Yap, J. Epps, E. Ambikairajah, E. H. C. Choi, "Formant frequencies under cognitive load: Effects and classification, " EURASIP J. Adv. Signal Process., vol. 2011, pp. 1:1-1:11, Jan. 2011.
28. A. C. Trevino, T. F. Quatieri, N. Malyska, "Phonologicallybased biomarkers for major depressive disorder, " EURASIP J. Adv. Signal Process., vol. 2011, no. 42, pp. 1-18, 2011.
29. L. Tamarit, M. Goudbeek, K. R. Scherer, "Spectral slope measurements in emotionally expressive speech, " in Proc. Speech Anal. Process. Knowl. Discovery, 2008, p. 007.
30. P. Le, E. Ambikairajah, J. Epps, V. Sethu, E. H. C. Choi, "Investigation of spectral centroid features for cognitive load classification, " Speech Commun., vol. 54, no. 4, pp. 540-551, 2011.
31. N. Cummins, J. Epps, M. Breakspear, R. Goecke, "An investigation of depressed speech detection: Features and normalizatio, " in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2011, pp. 2997-3000.
32. D. Ververidis and C. Kotropoulos, "Emotional speech recognition: Resources, features, methods, " Speech Commun., vol. 48, no. 9, pp. 1162-1181, Sep. 2006.
33. B. Schuller, M. Wimmer, L. Mosenlechner, C. Kern, D. Arsi-c, G. Rigoll, "Brute-forcing hierarchical functionals for paralinguistics: A waste of feature space" in Proc. 33rd IEEE Int. Conf. Acoust., Speech, Signal Process., Apr. 2008, pp. 4501-4504.
34. F. Eyben, A. Batliner, B. Schuller, "Towards a standard set of acoustic features for the processing of emotion in speech, " Proc. Meetings Acoust., vol. 9, no. 1, pp. 1-12, Jul. 2012.
35. T. F. Yap, J. Epps, E. Ambikairajah, E. H. C. Choi, "Voice source features for cognitive load classification, " in Proc. Int. Conf. Acoust., Speech Signal Process., 2011, pp. 5700-5703.
36. B. Schuller, S. Steidl, A. Batliner, F. Burkhardt, L. Devillers, C. Muller, S. Narayanan, "The INTERSPEECH 2010 paralinguistic challenge, " in Proc. 11th Annu. Conf. Int. Speech Commun. Assoc., Sep. 2010 pp. 2794-2797.
37. B. Schuller, S. Steidl, A. Batliner, J. Epps, F. Eyben, F. Ringeval, E. Marchi, Y. Zhang, "The INTERSPEECH 2014 computational paralinguistics challenge: Cognitive and physical load, " in Proc. 11th Annu. Conf. Int. Speech Commun. Assoc., 2014, pp. 427-431.
38. J. R. Williamson, T. F. Quatieri, B. S. Helfer, R. Horwitz, B. Yu, D. D. Mehta, "Vocal biomarkers of depression based on motor incoordination, " in Proc. 3rd ACM Int. Workshop Audio/Vis. Emotion Challenge, 2013, pp. 41-48.
39. V. Sethu, E. Ambikairajah, J. Epps, "On the use of speech parameter contours for emotion recognition, " EURASIP J. Audio, Speech Music Process., vol. 2013, no. 1, pp. 1-14, 2013.
40. B. Schuller and G. Rigoll, "Recognising Interest in Conversational Speech-Comparing bag of frames and supra-segmental features, " in Proc. 10th Annu. Conf. Int. Speech Commun. Assoc., Sep. 2009, pp. 1999-2002.
41. E. Marchi, A. Batliner, B. Schuller, S. Fridenzon, S. Tal, O. Golan, "Speech, emotion, age, language, task, typicality: Trying to disentangle performance and feature relevance, " in Proc. 1st Int. Workshop Wide Spectrum Social Signal Process. ASE/IEEE Int. Conf. Social Comput., Sep. 2012, pp. 961-968.
42. S. Steidl, Automatic Classification of Emotion-Related User States in Spontaneous Children's Speech. Berlin, Germany: Logos Verlag, 2009.
43. B. Schuller, S. Steidl, A. Batliner, F. Jurcicek, "The INTERSPEECH 2009 emotion challenge, " in Proc. 10th Annu. Conf. Int. Speech Commun. Assoc., Brighton, U. K., Sep. 2009, pp. 312-315.
44. B. Schuller, A. Batliner, S. Steidl, F. Schiel, J. Krajewski, "The INTERSPEECH 2011 speaker state challenge, " in Proc. 10th Annu. Conf. Int. Speech Commun. Assoc., Florence, Italy, Aug. 2011, pp. 3201-3204.
45. B. Schuller, S. Steidl, A. Batliner, E. Noth, A. Vinciarelli, F. Burkhardt, R. van Son, F. Weninger, F. Eyben, T. Bocklet, G. Mohammadi, B. Weiss, "The INTERSPEECH 2012 speaker trait challenge, " in Proc. 13th Annu. Conf. Int. Speech Commun. Assoc., Sep. 2012.
46. B. Schuller, R. Muller, F. Eyben, J. Gast, B. Hornler, M. Wollmer, G. Rigoll, A. Hothker, H. Konosu, "Being bored recognising natural interest by extensive audiovisual integration for Real-life application, " Image Vis. Comput., Special Issue Vis. Multimodal Anal. Human Spontaneous Behavior, vol. 27, no. 12, pp. 1760-1774, Nov. 2009.
47. B. Schuller, B. Vlasenko, F. Eyben, G. Rigoll, A. Wendemuth, "Acoustic emotion recognition: A benchmark comparison of performances, " in Proc. IEEE Workshop Automatic Speech Recognit. Understanding, Nov. 2009, pp. 552-557.
48. F. Burkhardt, A. Paeschke, M. Rolfes, W. Sendlmeier, B. Weiss, "A database of German emotional speech, " in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2005, pp. 1517-1520.
49. T. Banziger, M. Mortillaro, K. R. Scherer, "Introducing the Geneva multimodal expression corpus for experimental research on emotion perception, " Emotion, vol. 12, no. 5, pp. 1161-1179, 2012.
50. K. R. Scherer, J. Sundberg, L. Tamarit, G. L. Salom~ao, "Comparing the acoustic expression of emotion in the speaking and the singing voice, " Comput. Speech Language, vol. 29, no. 1, pp. 218-235, Jan. 2015.
51. M. Grimm, K. Kroschel, S. Narayanan, "The Vera am Mittag German audio-visual emotional speech database, " in Proc. IEEE Int. Conf. Multimedia Expo, Hannover, Germany, 2008, pp. 865-868.
52. M. Grimm, E. Mower, K. Kroschel, S. Narayanan, "Primitives based estimation and evaluation of emotions in speech, " Speech Commun., vol. 49, pp. 787-800, 2007.
53. H. Schlosberg, "Three dimensions of emotion, " Psychol. Rev., vol. 61, pp. 81-88, 1954.
54. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, I. H. Witten, "The WEKA data mining software: An update, " ACM SIGKDD Explorations Newsletter, vol. 11, no. 1, pp. 10-18, 2009.
55. V. Sethu, E. Ambikairajah, J. Epps, "Speaker normalization of speech-based emotion recognition, " in Proc. IEEE Int. Conf. Digital Signal Process., Cardiff, U. K., 2007, pp. 611-614.
56. I. Fonagy, "La vive voix : essais de psychophonetique, " prefazione di Roman Jakobson, Parigi, Payot, 1983.
57. M. Airas, H. Pulakka, T. Backstrom, P. Alku, "A toolkit for voice inverse filtering and parametrisation, " in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2005, pp. 2145-2148.
58. S. Patel, K. R. Scherer, J. Sundberg, E. Bjorkner, "Mapping emotions into acoustic space: The role of voice production, " Biol. Psychol., vol. 87, pp. 93-98, 2011.
59. F. Eyben, F. Weninger, F. Gross, B. Schuller, "Recent developments in openSMILE, the Munich open-source multimedia feature extractor, " in Proc. 21st ACMInt. Conf. Multimedia, 2013, pp. 835-838.
60. D. J. Hermes, "Measurement of pitch by subharmonic summation, " J. Acoust. Soc. Amer., vol. 83, no. 1, pp. 257-264, 1988.
61. B. Schuller, Intelligent Audio Analysis (Signals and Communication Technology). New York, Ny, USA: Springer, 2013.
62. E. Zwicker and H. Fastl, Psychoacoustics-Facts and Models. New York, NY, USA: Springer, 1999.
63. H. Hermansky, "Perceptual linear predictive (PLP) analysis for speech, " J. Acoust. Soc. Amer., vol. 87, pp. 1738-1752, 1990.
64. J. Makhoul and L. Cosell, "LPCW: An LPC vocoder with linear predictive spectral warping, " in Proc. Int. Conf. Acoust., Speech Signal Process., 1976, pp. 466-469.
65. B. Hammarberg, B. Fritzell, J. Gauffin, J. Sundberg, L. Wedin, "Perceptual and acoustic correlates of abnormal voice qualities, " Acta Otolaryngologica, vol. 90, pp. 441-451, 1980.
66. S. Patel, K. R. Scherer, J. Sundberg, E. Bjorkner, "Acoustic markers of emotions based on voice physiology, " in Proc. Speech Prosody, Chicago, IL, USA, 2010, pp. 1-4.
67. J. Makhoul, "Linear prediction: A tutorial review, " Proc. IEEE, vol. 63, no. 5, pp. 561-580, Apr. 1975.
68. S. Young, G. Evermann, M. Gales, T. Hain, D., X. Liu, G. Moore, J. Odell, D. Ollason, D. Povey, V. Valtchev, P. Woodland, The HTK Book for HTK Version 3. 4. Cambridge, U. K. : Cambridge Univ. Eng. Dept., 2006.