Robust unsupervised arousal rating: A rule-based framework withknowledge-inspired vocal features

IEEE Transactions on Affective Computing

Volumn 5, Issue 2, 2014, Pages 201-213

  Bone, Daniel ^a   Lee, Chi Chun ^b   Narayanan, Shrikanth ^a  

^a Department of Electrical Engineering University of Southern California ^*  (United States)

^b NATIONAL TSING HUA UNIVERSITY   (Taiwan)

Author keywords

activation; Arousal; continuous affect tracking; cross corpora classification; knowledge inspired features; rule based rating

Indexed keywords

CHEMICAL ACTIVATION;

AFFECT RECOGNITION; AROUSAL; BASELINE MODELS; BEHAVIORAL STUDIES; HIGH-DIMENSIONAL FEATURES; KNOWLEDGE-INSPIRED FEATURES; RULE BASED; SOCIAL CONTEXT;

RATING;

EID: 84905716338     PISSN: 19493045     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAFFC.2014.2326393     Document Type: Article

References (51)

1. W. Schramm, "How communication works," in Schramm, W. (Ed.), The Process and Effects of Communication, University of Illinois Press, Urbana, IL, Mass Media Soc., p. 51, 1954.
2. M. L. Patterson, "An arousal model of interpersonal intimacy," Psychol. Rev., vol. 83, no. 3, p. 235, 1976.
3. E. Duffy, "The psychological significance of the concept of arousal or activation," Psychol. Rev., vol. 64, no. 5, p. 265, 1957.
4. P. L. Harris, Children and Emotion: The Development of Psychological Understanding. Oxford, U.K.: Blackwell, 1989.
5. M. Hernandez-Reif, T. Field, M. Diego, and M. Ruddock, "Greater arousal and less attentiveness to face/voice stimuli by neonates of depressed mothers on the brazelton neonatal behavioral assessment scale," Infant Behav. Develop., vol. 29, no. 4, pp. 594-598, 2006. (Pubitemid 44774533)
6. R. P. Hobson, J. Ouston, and A. Lee, "Emotion recognition in autism: Coordinating faces and voices," Psychol. Med., vol. 18, no. 4, pp. 911-923, 1988. (Pubitemid 19032580)
7. G. Dawson, S. J. Webb, L. Carver, H. Panagiotides, and J. McPartland, "Young children with autism show atypical brain responses to fearful versus neutral facial expressions of emotion," Develop. Sci., vol. 7, no. 3, pp. 340-359, 2004. (Pubitemid 38750669)
8. J.-A. Bachorowski, "Vocal expression and perception of emotion," Current Directions Psychol. Sci., vol. 8, no. 2, pp. 53-57, 1999.
9. B. Schuller, Z. Zhang, F. Weninger, and G. Rigoll, "Using multiple databases for training in emotion recognition: To unite or to vote?" in Proc. 12th Annu. Conf. Int. Speech Commun. Assoc., 2011, pp. 1553-1556.
10. B. Schuller, B. Vlasenko, F. Eyben, M. Wollmer, A. Stuhlsatz, A. Wendemuth, and G. Rigoll, "Cross-corpus acoustic emotion recognition: Variances and strategies," IEEE Trans. Affect. Comput., vol. 1, no. 2, pp. 119-131, Jul.-Dec. 2010.
11. B. Z. Pollermann, "A place for prosody in a unified model of cognition and emotion," in Proc. 1st Int. Conf. Speech Prosody, 2002, pp. 17-22.
12. D. Watson, L. A. Clark, and A. Tellegen, "Development and validation of brief measures of positive and negative affect: The Panas scales," J. Personality Soc. Psychol., vol. 54, no. 6, p. 1063, 1988.
13. C. S. Carver and T. L. White, "Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS scales," J. Personality Soc. Psychol., vol. 67, no. 2, p. 319, 1994. (Pubitemid 24813236)
14. M. M. Bradley, L. Miccoli, M. A. Escrig, and P. J. Lang, "The pupil as a measure of emotional arousal and autonomic activation," Psychophysiology, vol. 45, no. 4, pp. 602-607, 2008. (Pubitemid 351832085)
15. J. F. Cohn, T. S. Kruez, I. Matthews, Y. Yang, M. H. Nguyen, M. T. Padilla, F. Zhou, and F. D. la Torre, "Detecting depression from facial actions and vocal prosody," in Proc. 3rd Int. Conf. Affect. Comput. Intell. Interaction Workshops, Sep. 2009, pp. 1-7.
16. M.Nowak, J. Kim,N.W.Kim,andC. Nass, "Social visualization and negotiation: Effects of feedback configuration and status," in Proc. ACMConf. Comput. Supported Cooperative Work, 2012, pp. 1081-1090.
17. C. M. Lee, S. Narayanan, and R. Pieraccini, "Recognition of negative emotions from the speech signal," in Proc. IEEE Workshop Autom. Speech Recog. Understanding, 2001, pp. 240-243.
18. B. Schuller, G. Rigoll, and M. Lang, "Hidden Markov model-based speech emotion recognition," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Process., 2003, vol. 2, pp. II-1-II-4.
19. C. Busso, Z. Deng, S. Yildirim, M. Bulut, C. M. Lee, A. Kazemzadeh, S. Lee, U. Neumann, and S. Narayanan, "Analysis of emotion recognition using facial expressions, speech and multimodal information," in Proc. 6th Int. Conf. Multimodal Interfaces, 2004, pp. 205-211. (Pubitemid 40371911)
20. C. M. Lee and S. Narayanan, "Towards detecting emotions in spoken dialogs," IEEE Trans. Speech Audio Process., vol. 13, no. 2, pp. 293-302, Mar. 2005.
21. C.-C. Lee, E. Mower, C. Busso, S. Lee, and S. Narayanan, "Emotion recognition using a hierarchical binary decision tree approach," Speech Commun., vol. 53, no. 9, pp. 1162-1171, 2011.
22. B. Schuller, A. Batliner, S. Steidl, and D. Seppi, "Recognising realistic emotions and affect in speech: State of the art and lessons learnt from the first challenge," Speech Commun., vol. 53, pp. 1062-1087, 2011.
23. H. J. Steeneken and J. H. Hansen, "Speech under stress conditions: Overview of the effect on speech production and on system performance," in Proc. Int. Conf. Acoustics, Speech, Signal Process., 1999, vol. 4, pp. 2079-2082.
24. F. Eyben, A. Batliner, B. Schuller, D. Seppi, and S. Steidl, "Crosscorpus classification of realistic emotions-some pilot experiments," in Proc. 3rd Inter. Workshop Emotion, 2010, pp. 77-82.
25. P. Juslin and K. Scherer, "Vocal expression of affect," in The New Handbook of Methods in Nonverbal Behavior Research. Oxford, U.K.: Oxford Univ. Press., 2005, ch. 3, pp. 65-135.
26. B. Vlasenko, B. Schuller, A. Wendemuth, and G. Rigoll, "On the influence of phonetic content variation for acoustic emotion recognition," in Proc. Perception Multimodal Dialogue Syst.: 4th IEEE Tutorial Res. Workshop Perception Interactive Technol. Speech-Based Syst., 2008, pp. 217-220.
27. D. Bone, C.-C. Lee, and S. S. Narayanan, "A robust unsupervised arousal rating framework using prosody with cross-corpora evaluation," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2012, pp. 1175-1178.
28. M. Grimm and K. Kroschel, "Evaluation of natural emotions using self-assessment manikins," in Proc. IEEE Workshop Autom. Speech Recog. Understanding, 2005, pp. 381-385.
29. C. Busso, M. Bulut, C.-C. Lee, A. Kazemzadeh, E. Mower, S. Kim, J. N. Chang, S. Lee, and S. S. Narayanan, "IEMOCAP: Interactive emotional dyadic motion capture database," J. Lang. Resources Eval., vol. 42, pp. 335-359, 2008.
30. S. Lee, S. Yildirim, A. Kazemzadeh, and S. Narayanan, "An articulatory study of emotional speech production," in Proc. 9th Eur. Conf. Speech Commun. Technol., 2005, pp. 497-500. (Pubitemid 43908108)
31. J. Kim, S. Lee, and S. S. Narayanan, "An exploratory study of the relations between perceived emotion strength and articulatory kinematics," in Proc. 12th Annu. Conf. Int. Speech Commun. Assoc., 2011, pp. 2961-2964.
32. F. Burkhardt, A. Paeschke, M. Rolfes, W. Sendlmeier, and B. Weiss, "A database of german emotional speech," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2005, pp. 1517-1520. (Pubitemid 43908362)
33. A. Metallinou, C.-C. Lee, C. Busso, S. Carnicke, S. Narayanan, and D. Tx, "The USC creativeit database: A multimodal database of theatrical improvisation," in Proc. Multimodal Corpora Workshop: Adv. Capturing, Coding Anal., Multimodality, 2010, pp. 64-68.
34. M. Grimm, K. Kroschel, and S. Narayanan, "The Vera am Mittag German audio-visual emotional speech database," in Proc. IEEE Int. Conf. Multimedia Expo, 2008, pp. 865-868.
35. K. Scherer, "Vocal affect expression: A review and a model for future research," Psychol. Bull., vol. 99, no. 2, p. 143, 1986.
36. N. H. De Jong and T. Wempe, "Praat script to detect syllable nuclei and measure speech rate automatically," Behav. Res. Methods, vol. 41, no. 2, pp. 385-390, 2009.
37. M. Young, M. Landy, and L. Maloney, "A perturbation analysis of depth perception from combinations of texture and motion cues," Vis. Res., vol. 33, pp. 2685-2696, 1993. (Pubitemid 23354566)
38. F. Eyben, M. Wöllmer, and B. Schuller, "Opensmile: The Munich versatile and fast open-source audio feature extractor," in Proc. Int. Conf. Multimedia, 2010, pp. 1459-1462.
39. B. Schuller, S. Steidl, and A. Batliner, "The INTERSPEECH 2009 emotion challenge," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2009, pp. 312-315.
40. B. Schuller, S. Steidl, A. Batliner, F. Schiel, and J. Krajewski, "The INTERSPEECH 2011 speaker state challenge," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2011, pp. 3201-3204.
41. B. Schuller, S. Steidl, A. Batliner, A. Vinciarelli, K. Scherer, F. Ringeval, M. Chetouani, F. Weninger, F. Eyben, E. Marchi, M. Mortillaro, H. Salamin, A. Polychroniou, F. Valente, and S. Kim, "The INTERSPEECH 2013 computational paralinguistics challenge: Social signals, conflict, emotion, autism," in Proc. Interspeech, 2013, pp. 148-152.
42. R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin, "Liblinear: A library for large linear classification," J. Mach. Learn. Res., vol. 9, pp. 1871-1874, 2008.
43. Z. Zhang, F. Weninger, M. Wollmer, and B. Schuller, "Unsupervised learning in cross-corpus acoustic emotion recognition," in Proc. IEEE Workshop Autom. Speech Recognit. Understanding, 2011, pp. 523-528.
44. R. Fernandez, "A computational model for the automatic recognition of affect in speech," Ph.D. dissertation, Dept. Archit., Massachusetts Institute of Technology, Cambridge, MA, USA, 2004.
45. E. Zwicker, H. Fastl, U. Widnmann, K. Kurakata, S. Kuwano, and S. Namba, "Program for calculating loudness according to din 45631 (iso 532b)," J. Acoust. Soc. Japan (E), vol. 12, no. 1, pp. 39-42, 1991.
46. S. Mariooryad and C. Busso, "Analysis and compensation of the reaction lag of evaluators in continuous emotional annotations," in Proc. Humaine Assoc. Conf. Affect. Comput. Intell. Interaction, 2013, pp. 85-90.
47. C. Busso and T. Rahman, "Unveiling the acoustic properties that describe the valence dimension," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2012, pp. 1179-1182.
48. M. P. Black, A. Katsamanis, B. R. Baucom, C.-C. Lee, A. C. Lammert, A. Christensen, P. G. Georgiou, and S. S. Narayanan, "Toward automating a human behavioral coding system for married couples' interactions using speech acoustic features," Speech Commun., vol. 55, no. 1, pp. 1-21, 2013.
49. M. P. Black, D. Bone, M. E. Williams, P. Gorrindo, P. Levitt, and S. S. Narayanan, "The USC care corpus: Child-psychologist interactions of children with autism spectrum disorders," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2011, pp. 1497-1500.
50. D. Bone, M. P. Black, C.-C. Lee, M. E. Williams, P. Levitt, S. Lee, and S. Narayanan, "The psychologist as an interlocutor in autism spectrum disorder assessment: Insights from a study of spontaneous prosody," J. Speech, Lang., Hearing Res., (in press) 2014.
51. D. Bone, M. P. Black, C.-C. Lee, M. E. Williams, P. Levitt, S. Lee, and S. Narayanan, "Spontaneous-speech acoustic-prosodic features of children with autism and the interacting psychologist," in Proc. Annu. Conf. Int. Speech Commun. Assoc., 2012, pp. 1043-1046.