Emotion Recognition in Naturalistic Speech and Language-A Survey

Emotion Recognition: A Pattern Analysis Approach

Volumn , Issue , 2015, Pages 237-267

  Weninger, Felix ^a   Wöllmer, Martin ^a   Schuller, Björn ^a  

^a TECHNICAL UNIVERSITY OF MUNICH   (Germany)

Author keywords

Emotional speech data; Naturalistic emotion recognition; Quality of life; Real time processing; Unsupervised learning

Indexed keywords

LINGUISTICS; QUALITY OF SERVICE; SPEECH ANALYSIS; UNSUPERVISED LEARNING;

APPLICATION SCENARIO; EMOTIONAL SPEECH; EVALUATION MEASURES; INCREMENTAL PROCESSING; NATURALISTIC EMOTIONS; QUALITY OF LIFE; REAL-LIFE APPLICATIONS; REALTIME PROCESSING;

SPEECH RECOGNITION;

EID: 85016363297     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781118910566.ch10     Document Type: Chapter

References (127)

1. B. Schuller, M. Wimmer, L. Mösenlechner, C. Kern, D. Arsic, and G. Rigoll, "Bruteforcing hierarchical functionals for paralinguistics: a waste of feature space?" in Proceedings of the ICASSP, Las Vegas, NV, 2008, pp. 4501-4504.
2. L. Devillers, L. Vidrascu, and L. Lamel, "Challenges in real-life emotion annotation and machine learning based detection," Neural Netw., 18: 407-422, 2005.
3. B. Schuller, R.Müller, F. Eyben, J. Gast, B. Hörnler, M.Wöllmer, G. Rigoll, A. Höthker, and H. Konosu, Being bored? Recognising natural interest by extensive audiovisual integration for real-life application. Image Vis. Comput. J., (Special Issue on Visual and Multimodal Analysis of Human Spontaneous Behavior), 27: 1760-1774, 2009.
4. A. Batliner, S. Steidl, and Nöth E., "Releasing a thoroughly annotated and processed spontaneous emotional database: the FAU Aibo Emotion Corpus," in Proceedings of a SatelliteWorkshop of LREC 2008 on Corpora for Research on Emotion and Affect (eds L. Devillers, J.-C. Martin, R. Cowie, E. Douglas-Cowie, and A. Batliner), Marrakesh, 2008, pp. 28-31.
5. F. Burkhardt, A. Paeschke, M. Rolfes, W. Sendlmeier, and B. Weiss, "A database of German emotional speech," in Proceedings of the Interspeech, Lisbon, 2005, pp. 1517-1520.
6. B. Schuller, A. Batliner, S. Steidl, and D. Seppi, "Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge," Speech Commun., (Special Issue on Sensing Emotion and Affect-Facing Realism in Speech Processing), 2011.
7. S. Steidl, B. Schuller, A. Batliner, and D. Seppi, "The hinterland of emotions: facing the open-microphone challenge," in Proceedings of the ACII, Amsterdam, 2009, pp. 690-697.
8. A. Batliner, F. Burkhardt, Van Ballegooy M., and Nöth E., "A taxonomy of applications that utilize emotional awareness," in Proceedings of the IS-LTC 2006, Ljubliana, 2006, pp. 246-250.
9. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. Taylor, "Emotion recognition in human-computer interaction," IEEE Signal Processing Magazine, 18(1): 32-80, 2001.
10. R. Picard, "Affective computing: challenges," J. Hum.-Comput. Stud., 59: 55-64, 2003.
11. F. Burkhardt, Van Ballegooy M., R. Englert, and R. Huber, "An emotion-aware voice portal," in Proceedings of the Electronic Speech Signal Processing (ESSP), 2005, pp. 123-131.
12. G. Mishne, D. Carmel, R. Hoory, A. Roytman, and A. Soffer, "Automatic analysis of call-center conversations," in Proceedings of the CIKM'05, Bremen, Germany, 2005, pp. 453-459.
13. D. Litman and K. Forbes, "Recognizing emotions from student speech in tutoring dialogues," in Proceedings of the ASRU, Virgin Island, 2003, pp. 25-30.
14. L. Price, J. T. E. Richardson, and A. Jelfs, "Face-to-face versus online tutoring support in distance education," Stud. High. Educ., 32(1): 1-20, 2007.
15. A. Delaborde and L. Devillers, "Use of non-verbal speech cues in social interaction between human and robot: emotional and interactional markers," in AFFINE'10- Proceedings of the Third ACM Workshop on Affective Interaction in Natural Environments, Co-located with ACM Multimedia 2010, 2010, pp. 75-80.
16. A. Batliner, S. Steidl, and Nöth E., "Associating children's non-verbal and verbal behaviour: body movements, emotions, and laughter in a human-robot interaction," in Proceedings of ICASSP, Prague, 2011, pp. 5828-5831.
17. Schröder M., R. Cowie, D. Heylen, M. Pantic, C. Pelachaud, and B. Schuller, "Towards responsive sensitive artificial listeners," in Proceedings of the Fourth InternationalWorkshop on Human-Computer Conversation, Bellagio, 2008.
18. B. Schuller, F. Eyben, S. Can, and H. Feussner, "Speech in minimal invasive surgery- towards an affective language resource of real-life medical operations," in Proceedings of the Third ELRA InternationalWorkshop on EMOTION (satellite of LREC): Corpora for Research on Emotion and Affect, Valetta, 2010, pp. 5-9.
19. B. Schuller, M. Wimmer, D. Arsic, T. Moosmayr, and G. Rigoll, "Detection of security related affect and behaviour in passenger transport," in Proceedings of the Interspeech, Brisbane, 2008, pp. 265-268.
20. C. Clavel, I. Vasilescu, L. Devillers, G. Richard, and T. Ehrette, "Fear-type emotion recognition for future audio-based surveillance systems," Speech Commun., 50(6): 487-503, 2008.
21. K. Laskowski, "Contrasting emotion-bearing laughter types in multiparticipant vocal activity detection for meetings," in Proceedings of the ICASSP, IEEE, Taipei, Taiwan, 2009, pp. 4765-4768.
22. Z. Massida, P. Belin, C. James, J. Rouger, B. Fraysse, P. Barone, and O. Deguine, "Voice discrimination in cochlear-implanted deaf subjects," Hear. Res., 275: 120-129, 2011.
23. J. Demouy, M. Plaza, J. Xavier, F. Ringeval, M. Chetouani, D. Périsse, D. Chauvin, S. Viaux, B. Golse, D. Cohen, and L. Robel, "Differential language markers of pathology in autism, pervasive developmental disorder not otherwise specified and specific language impairment," Res. Autism Spectr. Dis., 5(4): 1402-1412, 2011.
24. C. Biever, "You have three happy messages," New Scientist, 185: 21, 2005.
25. T. Athanaselis, S. Bakamidis, I. Dologlu, R. Cowie, E. Douglas-Cowie, and C. Cox, "ASR for emotional speech: clarifying the issues and enhancing performance," Neural Netw., 18: 437-444, 2005.
26. R. Fernandez and R. W. Picard, "Modeling drivers' speech under stress," Speech Commun., 40: 145-159, 2003.
27. S. Steidl, A. Batliner, D. Seppi, and B. Schuller, "On the impact of children's emotional speech on acoustic and language models," EURASIP J. Audio Speech Music Process., 1-14, 2010.
28. H. Boril, A. Sangwan, T. Hasan, and J. H. L. Hansen, "Automatic excitement-level detection for sports highlights generation," in Proceedings of the Interspeech 2010, Makuhari, Japan, 2011, pp. 2202-2205.
29. F. Burkhardt, Van Ballegooy M., K.-P. Engelbrecht, T. Polzehl, and J. Stegmann, "Emotion detection in dialog systems: applications, strategies and challenges," in Proceedings of the ACII, Amsterdam, The Netherlands, 2009, pp. 1-6.
30. D. Erickson, K. Yoshida, C. Menezes, A. Fujino, T. Mochida, and Y. Shibuya, "Exploratory study of some acoustic and articulatory characteristics of sad speech," Phonetica, 63: 1-25, 2004.
31. R. Cowie, E. Douglas-Cowie, and C. Cox, "Beyond emotion archetypes: databases for emotion modelling using neural networks," Neural Netw., 18(3): 388, 2005.
32. A. Stuhlsatz, C. Meyer, F. Eyben, T. Zielke, G. Meier, and B. Schuller, "Deep neural networks for acoustic emotion recognition: raising the benchmarks," in Proceedings of ICASSP, IEEE, Prague, Czech Republic, 2011, pp. 5688-5691.
33. I. S. Engberg, A. V. Hansen, O. Andersen, and P. Dalsgaard, "Design, recording and verification of a Danish emotional speech database," in Proceedings of the Eurospeech, Rhodes, Greece, 1997, pp. 1695-1698.
34. J. H. L. Hansen and S. Bou-Ghazale, "Getting started with susas: a speech under simulated and actual stress database," in Proceedings of EUROSPEECH-97, vol. 4, Rhodes, Greece, 1997, pp. 1743-1746.
35. J. Hirschberg, J. Liscombe, and J. Venditti, "Experiments in emotional speech," in Proceedings of the ISCA & IEEE Workshop on Spontaneous Speech Processing and Recognition, Tokyo, 2003, pp. 1-7.
36. O. Martin, I. Kotsia, B. Macq, and I. Pitas, "The enterface'05 audio-visual emotion database," in IEEE Workshop on Multimedia Database Management, 2006.
37. B. Schuller, M. Wimmer, D. Arsic, G. Rigoll, and B. Radig, "Audiovisual behaviour modeling by combined feature spaces," in Proceedings of ICASSP, 2007, pp. 733-736.
38. M. Grimm, K. Kroschel, and S. Narayanan, "The Vera am Mittag German audio-visual emotional speech database," in Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), Hannover, Germany, 2008, pp. 865-868.
39. J. Liscombe, G. Riccardi, and D. Hakkani-Tür, "Using context to improve emotion detection in spoken dialog systems," in Proceedings of Interspeech, Lisbon, 2005, pp. 1845-1848.
40. J. Ang, R. Dhillon, E. Shriberg, and A. Stolcke, "Prosody-based automatic detection of annoyance and frustration in human-computer dialog," in Proceedings of Interspeech, Denver, CO, 2002, pp. 2037-2040.
41. J. Liscombe, J. Hirschberg, and J. J. Venditti, "Detecting certainness in spoken tutorial dialogues," in Proceedings of INTERSPEECH, Lisbon, Portugal, 2005, pp. 1837-1840.
42. A. Batliner, V. Zeissler, C. Frank, J. Adelhardt, R. P. Shi, and Nöth E., "We are not amused-but how do you know? User states in a multi-modal dialogue system," in Proceedings of Interspeech, Geneva, 2003, pp. 733-736.
43. A. Batliner, S. Steidl, C. Hacker, and E.Nöth, "Private emotions vs. social interaction-a data-driven approach towards analysing emotions in speech," User Model. User-Adapt. Interact., 18: 175-206, 2008.
44. S. Steininger, F. Schiel, O. Dioubina, and S. Raubold, "Development of user-state conventions for the multimodal corpus in smartkom," in Proceedings of the Workshop on Multimodal Resources and Multimodal Systems Evaluation, Las Palmas, 2002, pp. 33-37.
45. C. P. Said, C. D. Moore, K. A. Norman, J. V. Haxby, and A. Todorov, "Graded representations of emotional expressions in the left superior temporal sulcus," Front. Syst. Neurosci., 4(6): 2010. doi:10.3389/fnsys.2010.00006.
46. A. Vinciarelli, M. Pantic, H. Bourlard, and A. Pentland, "Social signals, their function, and automatic analysis: a survey," in IMCI '08: Proceedings of the 10th International Conference on Multimodal Interfaces, ACM, New York, 2008, pp. 61-68.
47. B. Schuller and G. Rigoll, "Timing levels in segment-based speech emotion recognition," in Proceedings of Interspeech, Pittsburgh, PA, 2006, pp. 1818-1821.
48. M. Shami and W. Verhelst, "Automatic classification of expressiveness in speech: a multi-corpus study," in Speaker Classification II, volume 4441 of Lecture Notes in Computer Science/Artificial Intelligence (ed. C. Müller), Springer, Heidelberg, 2007, pp. 43-56.
49. A. Batliner, D. Seppi, S. Steidl, and B. Schuller, "Segmenting into adequate units for automatic recognition of emotion-related episodes: a speech-based approach," Adv. Hum.-Comput. Interact., vol. 2010, Article ID 782802, 2010, 15 p.
50. D. Seppi, A. Batliner, S. Steidl, B. Schuller, and Nöth E., "Word accent and emotion," in Proceedings of Speech Prosody 2010, Chicago, IL, 2010.
51. B. Schuller, R. Jiménez Villar, G. Rigoll, and M. Lang, "Meta-classifiers in acoustic and linguistic feature fusion-based affect recognition," in Proceedings of ICASSP, Philadelphia, PA, 2005, pp. 325-328.
52. F. Eyben, Wöllmer M., and B. Schuller, "openSMILE-the Munich versatile and fast open-source audio feature extractor," in Proceedings of ACM Multimedia, Florence, Italy, 2010, pp. 1459-1462.
53. B. Schuller, R.Müller, M. Lang, and G. Rigoll, "Speaker independent emotion recognition by early fusion of acoustic and linguistic features within ensemble," in Proceedings of Interspeech, Lisbon, 2005, pp. 805-808.
54. T. Joachims, "Text categorization with support vector machines: learning with many relevant features," in Proceedings of ECML-98, 10th European Conference on Machine Learning, Chemnitz (eds C. Nédellec and C. Rouveirol), Springer, Heidelberg, 1998, pp. 137-142.
55. B. Schuller and T. Knaup, "Learning and knowledge-based sentiment analysis in movie review key excerpts," in Toward Autonomous, Adaptive, and Context-Aware Multimodal Interfaces: Theoretical and Practical Issues, Volume 6456 of Lecture Notes on Computer Science (LNCS) (eds A. Esposito, A. M. Esposito, R. Martone, V. Müller, and G. Scarpetta), Springer, Heidelberg, 2010, pp. 448-472.
56. B. Schuller, F. Metze, S. Steidl, A. Batliner, F. Eyben, and T. Polzehl, "Late fusion of individual engines for improved recognition of negative emotions in speech-learning vs. democratic vote," in Proceedings of the 35th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Dallas, TX, 2010, pp. 5230-5233.
57. M.Wöllmer, F. Eyben, A. Graves, B. Schuller, and G. Rigoll, "Bidirectional LSTM networks for context-sensitive keyword detection in a cognitive virtual agent framework," Cogn. Comput., 2(3): 180-190, 2010.
58. S. Steidl, Automatic Classification of Emotion-Related User States in Spontaneous Children's Speech, Logos Verlag, Berlin, 2009 (PhD thesis, FAU Erlangen-Nuremberg).
59. A. Batliner, S. Steidl, B. Schuller, D. Seppi, T.Vogt, J.Wagner, L. Devillers, L.Vidrascu, V. Aharonson, and N. Amir, "Whodunnit-searching for the most important feature types signalling emotional user states in speech," Comput. Speech Lang., 25: 4-28, 2011.
60. P. Pudil, J. Novovicova, and J. Kittler, "Floating search methods in feature selection," Pattern Recognit. Lett., 15: 1119-1125, 1994.
61. N. Landwehr, M. Hall, and E. Frank, "Logistic model trees," Mach. Learn., 161-205, 2005.
62. I. T. Jolliffe, Principal Component Analysis, Springer, Berlin, 2002.
63. A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis, John Wiley & Sons, New York, 2001.
64. A. Cichocki, R. Zdunek, A. H. Phan, and S.-I. Amari, Nonnegative Matrix and Tensor Factorizations, John Wiley & Sons, 2009.
65. P. Smaragdis, "Approximate nearest-subspace representations for sound mixtures," in Proceedings of ICASSP, Special Session on "Innovative Representations of Audio," Prague, Czech Republic, 2011, pp. 5892-5895.
66. F. Eyben, A. Batliner, B. Schuller, D. Seppi, and S. Steidl, "Cross-corpus classification of realistic emotions-some pilot experiments," in Proceedings of the Third International Workshop on EMOTION (Satellite of LREC): Corpora for Research on Emotion and Affect, Valetta, 2010, pp. 77-82.
67. J.-C. Martin, R. Niewiadomski, L. Devillers, S. Buisine, and C. Pelachaud, "Multimodal complex emotions: gesture expressivity and blended facial expressions," Int. J. Hum. Robot., 3(3): 1-23, 2006.
68. D. Seppi, A. Batliner, B. Schuller, S. Steidl, T.Vogt, J.Wagner, L. Devillers, L.Vidrascu, N. Amir, and V. Aharonson, "Patterns, prototypes, performance: classifying emotional user states," in Proceedings of Interspeech, Brisbane, 2008, pp. 601-604.
69. E. Mower, A. Metallinou, C.-C. Lee, A. Kazemzadeh, C. Busso, S. Lee, and S. Narayanan, "Interpreting Ambiguous Emotional Expressions," in Proceedings of ACII, Amsterdam, 2009, pp. 662-669.
70. C. E. Erdem, E. Bozkurt, E. Erzin, and A. T. Erdem, "RANSAC-based training data selection for emotion recognition from spontaneous speech," in AFFINE'10- Proceedings of the 3rd ACM Workshop on Affective Interaction in Natural Environments, Co-located with ACM Multimedia 2010, 2010, pp. 9-14.
71. B. Schuller and L. Devillers, "Incremental acoustic valence recognition: an inter-corpus perspective on features, matching, and performance in a gating paradigm," in Proceedings of INTERSPEECH, Makuhari, Japan, 2010, pp. 801-804.
72. B. Schuller and G. Rigoll, "Recognising interest in conversational speech-comparing bag of frames and supra-segmental features," in Proceedings of Interspeech, Brighton, 2009, pp. 1999-2002.
73. S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Comput., 9(8): 1735-1780, 1997.
74. M.Wöllmer, B. Schuller, F. Eyben, and G. Rigoll, "Combining long short-term memory and dynamic Bayesian networks for incremental emotion-sensitive artificial listening," IEEE J. Sel. Top. Signal Process., 4(5): 867-881, 2010.
75. Wöllmer M., F. Weninger, F. Eyben, and B. Schuller, "Computational assessment of interest in speech-facing the real-life challenge," Künstl. Intell., (Special Issue on Emotion and Computing), 2011.
76. T. K. Ho, "The random subspace method for constructing decision forests," IEEE Transactions on Pattern Analysis and Machine Intelligence, 20: 832-844, 1998.
77. I. H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn, Morgan Kaufmann, San Francisco, CA, 2005.
78. T. Fawcett, "An introduction to ROC analysis," Pattern Recognit. Lett., 27(8): 861-874, 2006.
79. B. Schuller, S. Steidl, A. Batliner, F. Burkhardt, L. Devillers, C. Müller, and S. Narayanan, "The INTERSPEECH 2010 paralinguistic challenge-age, gender, and affect," in Proceedings of Interspeech, Makuhari, Japan, 2010, pp. 2794-2797.
80. T. G. Dietterich, "Approximate statistical tests for comparing supervised classification learning algorithms," Neural Comput., 10: 1895-1923, 1998.
81. L. Gillick and S. J. Cox, "Some statistical issues in the comparison of speech recognition algorithms," in Proceedings of ICASSP, IEEE, 1989, pp. 532-535.
82. R. S. Nickerson, "Null hypothesis significance testing: a review of an old and continuing controversy," Psychol. Methods, 5: 241-301, 2000.
83. S. Young, G. Evermann, M. Gales, T. Hain, D. Kershaw, X. Liu, G. Moore, J. Odell, D. Ollason, D. Povey, V. Valtchev, and P.Woodland, The HTK Book, Cambridge University Engineering Department, 2006. For HTK Version 3.4.
84. F. Eyben, Wöllmer M., and B. Schuller, "openEAR-introducing the Munich opensource emotion and affect recognition toolkit," in Proceedings of ACII, Amsterdam, 2009, pp. 576-581.
85. T. Vogt, E. André, and N. Bee, Emovoice-a framework for online recognition of emotions from voice," in Proceedings of an IEEE Tutorial and Research Workshop on Perception and Interactive Technologies for Speech-Based Systems (PIT 2008), volume 5078 of Lecture Notes in Computer Science, Springer, Kloster Irsee, 2008, pp. 188-199.
86. Schröder M., L. Devillers, K. Karpouzis, J.-C. Martin, C. Pelachaud, C. Peter, H. Pirker, B. Schuller, J. Tao, and I.Wilson, "What should a generic emotion markup language be able to represent?" in Affective Computing and Intelligent Interaction (eds A. Paiva, R. Prada, and R. W. Picard), Springer, Berlin, 2007, pp. 440-451.
87. P. Baggia, D. C. Burnett, J. Carter, D. A. Dahl, G. McCobb, and D. Raggett, EMMA: Extensible MultiModal Annotation Markup Language, 2007.
88. B. Fehr and J. A. Russel, "Concept of emotion viewed from a prototype perspective," J. Exp. Psychol.: Gen., 113: 464-486, 1984.
89. P. R. Shaver, S. Wu, and J. C. Schwartz, Cross-cultural similarities and differences in emotion and its representation: a prototype approach, Emotion, 175-212, 1992.
90. A. Batliner, S. Steidl, C. Hacker, Nöth E., and H. Niemann, "Tales of tuning- prototyping for automatic classification of emotional user states," in Proceedings of Interspeech, Lisbon, 2005, pp. 489-492.
91. K. Krippendorff, Content Analysis, An Introduction to Its Methodology, 2nd edn, Sage Publications, Thousand Oaks, CA, 2004.
92. M. Grimm and K. Kroschel, "Evaluation of natural emotions using self assessment manikins," in Proceedings of ASRU, IEEE, 2005, pp. 381-385.
93. M. J. Frank and E. D. Claus, "Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal," Psychol. Rev., 113(2): 300-326, 2006.
94. B. Schuller, B. Vlasenko, F. Eyben, Wöllmer M., A. Stuhlsatz, A. Wendemuth, and G. Rigoll, "Cross-corpus acoustic emotion recognition: variances and strategies," IEEE Transactions on Affective Computing, 1(2): 119-131, 2010.
95. I. Lefter, L. J. M. Rothkrantz, P.Wiggers, and D. A. van Leeuwen, "Emotion recognition from speech by combining databases and fusion of classifiers," in Proceedings of Text and Speech and Dialogue, Berlin, Germany, 2010.
96. B. Schuller, Z. Zhang, F. Weninger, and G. Rigoll, "Using multiple databases for training in emotion recognition: to unite or to vote?" in Proceedings of INTERSPEECH, Florence, Italy, 2011.
97. H. A. Elfenbein, M. K. Mandal, N. Ambady, S. Harizuka, and S. Kumar, "On the universality and cultural specificity of emotion recognition: a meta-analysis," Psychol. Bull., 128(2): 236-242, 2002.
98. A. Abelin, "Cross-cultural multimodal interpretation of emotional expressions-an experimental study of Spanish and Swedish," in Proceedings of Speech Prosody, ISCA, 2004.
99. K. R. Scherer, R. Banse, and H. G.Wallbott, "Emotion inferences from vocal expression correlate across languages and cultures," J. Cross-Cult. Psychol., 32(1): 76-92, 2001.
100. T. Polzehl, A. Schmitt, and F. Metze, "Approaching multi-lingual emotion recognition from speech-on language dependency of acoustic/prosodic features for anger detection," in Proceedings of Speech Prosody, ISCA, 2010.
101. A. Chen, "Perception of paralinguistic intonational meaning in a second language," Lang. Learn., 59(2): 367-409, 2009.
102. A. Esposito and M. T. Riviello, "The cross-modal and cross-cultural processing of affective information," Front. Intell. Appl., 226: 301-310, 2010.
103. Wöllmer M., F. Eyben, S. Reiter, B. Schuller, C. Cox, E. Douglas-Cowie, and R. Cowie, "Abandoning emotion classes-towards continuous emotion recognition with modelling of long-range dependencies," in Proceedings of Interspeech, Brisbane, 2008, pp. 597-600.
104. N. Ward and W. Tsukahara, "Prosodic features which cue backchannel responses in English and Japanese," J. Pragmat., 32: 1177-1207, 2000.
105. B. Schuller, M.Wöllmer, T. Moosmayr, and G. Rigoll, "Recognition of noisy speech: a comparative survey of robust model architectures and feature enhancement," EURASIP J. Audio Speech Music Process., 2009. Article ID 942617, 17 p.
106. B. Schuller, D. Arsić, F. Wallhoff, and G. Rigoll, "Emotion recognition in the noise applying large acoustic feature sets," in Proceedings of Speech Prosody 2006, Dresden, 2006.
107. A. Tawari and M. Trivedi, "Speech emotion analysis in noisy real world environment," in Proceedings of ICPR, Istanbul, Turkey, August 2010, pp. 4605-4608.
108. M. Grimm, K. Kroschel, H. Harris, C. Nass, B. Schuller, G. Rigoll, and T. Moosmayr, "On the necessity and feasibility of detecting a driver's emotional state while driving," in Affective Computing and Intelligent Interaction (eds A. Paiva, R. Prada, and R. W. Picard), Springer, Berlin, 2007, pp. 126-138.
109. M. Lugger, B. Yang, and W. Wokurek, "Robust estimation of voice quality parameters under real world disturbances," in Proceedings of ICASSP, Toulouse, 2006, pp. 1097-1100.
110. B. Schuller, "Affective speaker state analysis in the presence of reverberation," Int. J. Speech Technol., 1-11, 2011.
111. F.Weninger, B. Schuller, A. Batliner, S. Steidl, and D. Seppi, "Recognition of nonprototypical emotions in reverberated and noisy speech by nonnegative matrix factorization," EURASIP J. Adv. Signal Process., (Special Issue on Emotion and Mental State Recognition from Speech), 2011. Article ID 838790, 16 p.
112. J. F. Gemmeke and T. Virtanen, "Noise robust exemplar-based connected digit recognition," in Proceedings of ICASSP, Dallas, TX, 2010, pp. 4546-4549.
113. Wöllmer M., F. Weninger, S. Steidl, A. Batliner, and B. Schuller, "Speech-based nonprototypical affect recognition for child-robot interaction in reverberated environments," in Proceedings of INTERSPEECH, Florence, Italy, 2011.
114. B. Schuller and F. Burkhardt, "Learning with synthesized speech for automatic emotion recognition," in Proceedings of the 35th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Dallas, TX, 2010, pp. 5150-5153.
115. R. Reisenzein and H.Weber, "Personality and emotion," in The Cambridge Handbook of Personality Psychology (eds P. J. Corr and G. Matthews), Cambridge University Press, Cambridge, 2009, pp. 54-71.
116. W. Revelle and K. R. Scherer, "Personality and emotion," in Oxford Companion to the Affective Sciences, Oxford University Press, Oxford, 2009.
117. T.Vogt and E. André, "Improving automatic emotion recognition from speech via gender differentiation," in Proceedings of Language Resources and Evaluation Conference (LREC 2006), 2006.
118. J. Stadermann, W. Koska, and G. Rigoll, "Multi-task learning strategies for a recurrent neural net in a hybrid tied-posteriors acoustic mode," in Proceedings of Interspeech 2005, Lisbon, Portugal, 2005, ISCA, pp. 2993-2996.
119. M. Pantic and L. J. M. Rothkrantz, "Toward an affect-sensitive multimodal humancomputer interaction," Proceedings of IEEE, 91(9): 1370-1390, 2003.
120. Z. Zeng, M. Pantic, G. I. Roisman, and T. S. Huang, "A survey of affect recognition methods: audio, visual, and spontaneous expressions," IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(1): 39-58, 2009.
121. K. H. Kim, S. W. Bang, and S. R. Kim, "Emotion recognition system using short-term monitoring of physiological signals," Med. Biol. Eng. Comput., 42(3): 419-427, 2004.
122. F. Nasoz, K. Alvarez, C. L. Lisetti, and N. Finkelstein, "Emotion recognition from physiological signals using wireless sensors for presence technologies," Cogn. Technol. Work, 6(1): 4-14, 2004.
123. R.W. Picard, E. Vyzas, and J. Healey, "Toward machine emotional intelligence: analysis of affective physiological state," IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(10): 1175-1191, 2001.
124. S. Bengio, "An asynchronous hidden Markov model for audio-visual speech recognition," Advances in NIPS 15, 2003.
125. H. Gunes and M. Piccardi, "Affect recognition from face and body: early fusion vs. late fusion," IEEE International Conference on Systems, Man and Cybernetics, 4: 3437-3443, 2005.
126. W. Lizhong, S. Oviatt, and P. R. Cohen, "Multimodal integration-a statistical view," IEEE Transactions on Multimedia, 1: 334-341, 1999.
127. Wöllmer M., M. Al-Hames, F. Eyben, B. Schuller, and G. Rigoll, "A multidimensional dynamic time warping algorithm for efficient multimodal fusion of asynchronous data streams," Neurocomputing, 73: 366-380, 2009.