Evaluating evaluators: A case study in understanding the benefits and pitfalls of multi-evaluator modeling

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn , Issue , 2009, Pages 1583-1586

  Mower, Emily ^a   Matarić, Maja J ^a   Narayanan, Shrikanth ^a  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

Author keywords

Emotion; Hidden Markov model; Perception

Indexed keywords

CLASSIFICATION PERFORMANCE; CLASSIFICATION RESULTS; COMPLEX PROCESSES; EMOTION; EMOTIONAL SPEECH; MULTIPLE USER;

ACOUSTIC PROPERTIES; ACOUSTICS; HIDDEN MARKOV MODELS;

SPEECH COMMUNICATION;

EID: 70450172154     PISSN: None     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (17)

1. K. Dautenhahn, A. Bond, L. Cañamero, and B. Edmonds, Socially intelligent agents, vol. 3, chapter 1, pp. 1-20, Springer, 2002.
2. M. Pantic, N. Sebe, J.F. Cohn, and T. Huang, "Affective multi-modal human-computer interaction," in Proceedings of ACM Int. Conf. on Multimedia. New York, NY, USA, 2005, pp. 669-676.
3. C. Busso, S. Lee, and S. Narayanan, "Using neutral speech models for emotional speech analysis," in InterSpeech ICSLP, Antwerp, Belgium, Aug. 2007, pp. 2225-2228.
4. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J.G. Taylor, "Emotion recognition in human-computer interaction," IEEE Signal Processing Magazine, pp. 32-80, Jan. 2001.
5. M. Grimm, K. Kroschel, E. Mower, and S. Narayanan, "Primitives-based evaluation and estimation of emotions in speech," Speech Communication, vol. 49, pp. 787-800, 2007.
6. Z. Zeng, J. Tu, BM Pianfetti, and TS Huang, "Audio-Visual Affective Expression Recognition Through Multistream Fused HMM," IEEE Transactions on Multimedia, vol. 10, no. 4, pp. 570-577, 2008.
7. S. Biersack and V. Kempe, "Tracing vocal emotion expression through the speech chain: do listeners perceive what speakers feel," in ISCA Workshop on Plasticity in Speech Perception, London, UK, June 2005, pp. 211-214.
8. C. Busso and S.S. Narayanan, "The expression and perception of emotions: Comparing assessments of self versus others," in Interspeech 2008 - Eurospeech, Brisbane, Australia, Sept. 2008, pp. 257-260.
9. K.P. Truong, M.A. Neerincx, and D.A. van Leeuwen, "Assessing agreement of observer- and self-annotations in spontaneous multimodal emotion data," in Interspeech 2008, Brisbane, Australia, Sept. 2008, pp. 318-321.
10. S. Steidl, M. Levit, A. Batliner, E. Noth, and H. Niemann, ""Of all things the measure is man": Automatic classification of emotions and inter-labeler consistency," in ICASSP, 2005, vol. 1, pp. 317-320.
11. J.A. Russell and L.F. Barrett, "Core affect, prototypical emotional episodes, and other things called emotion: Dissecting the elephant," Journal of Personality and Social Psychology, vol. 76, no. 5, pp. 805-819, 1999.
12. C. Busso, M. Bulut, C.C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J.N. Chang, S. Lee, and S.S. Narayanan, "IEMOCAP: Interactive emotional dyadic motion capture database," Journal of Language Resources and Evaluation, pp. 335-359, Nov. 5 2008.
13. T. Vogt and E. Andre, "Comparing feature sets for acted and spontaneous speech in view of automatic emotion recognition," in IEEE Int. Conf. on Multimedia and Expo (ICME), July 2005, pp. 474-477.
14. R. Cowie and R.R. Cornelius, "Describing the emotional states that are expressed in speech," Speech Communication, vol. 40, no. 1-2, pp. 5-32, 2003.
15. E. Mower, M. Matarić, and S. Narayanan, "Human perception of audio-visual synthetic character emotion expression in the presence of ambiguous and conflicting information," IEEE Transactions on Multimedia, Accepted for Publication.
16. S. Young, J. Odell, D. Ollason, V. Valtchev, and P.Woodland, The HTK book, Entropic Cambridge Research Laboratory Cambridge, England, 1997.
17. R.P.W. Duin, P. Juszczak, P. Paclik, E. Pekalska, D. De Ridder, and DMJ Tax, "PRTools, a matlab toolbox for pattern recognition," Delft University of Technology, 2004.