Constructing a spoken dialogue corpus for studying paralinguistic information in expressive conversation and analyzing its statistical/acoustic characteristics

Speech Communication

Volumn 53, Issue 1, 2011, Pages 36-50

  Mori, Hiroki ^a   Satake, Tomoyuki ^a   Nakamura, Makoto ^a   Kasuya, Hideki ^a  

^a UTSUNOMIYA UNIVERSITY   (Japan)

Author keywords

Abstract dimensions; Annotation; Emotional state; Expressive speech; Spoken dialogue; Spontaneous speech

Indexed keywords

ABSTRACT DIMENSIONS; ANNOTATION; EMOTIONAL STATE; EXPRESSIVE SPEECH; SPOKEN DIALOGUE; SPONTANEOUS SPEECH;

ABSTRACTING; BEHAVIORAL RESEARCH; DATABASE SYSTEMS; LINGUISTICS;

SPEECH RECOGNITION;

EID: 78649306850     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2010.08.002     Document Type: Article

References (48)

1. A.H. Anderson, M. Bader, E.G. Bard, E.H. Boyle, G.M. Doherty, S.C. Garrod, S.D. Isard, J.C. Kowtko, J.M. McAllister, J. Miller, C.F. Sotillo, H.S. Thompson, and R. Weinert The HCRC map task corpus Lang. Speech 34 4 1991 351 366
2. M. Argyle, and J. Dean Eye contact, distance and affiliation Sociometry 28 3 1965 289 304
3. M. Argyle, F. Alkema, and R. Gilmour The communication of friendly and hostile attitudes by verbal and non-verbal signals Eur. J. Soc. Psychol. 1 3 1972 385 402
4. Arimoto, Y., Kawatsu, H., Ohno, S., Iida, H., 2008. Emotion recognition in spontaneous emotional speech for anonymity-protected voice chat systems. In: Proceedings of the Interspeech 2008, pp. 322-325.
5. Aubergé, V., Audibert, N., Rilliard, A., 2003. Why and how to control the authentic emotional speech corpora. In: Proceedings of the Eurospeech 2003, pp. 185-188.
6. J.K. Burgoon, D.B. Buller, and W.G. Woodall Nonverbal Communication: The Unspoken Dialogue 1989 Harper and Row New York
7. Campbell, N., 2003. The JST/CREST ESP project - a mid-term progress report. In: First JST/CREST International Workshop on Expressive Speech Processing, pp. 61-70.
8. N. Campbell, and D. Erickson What do people hear?. A study of the perception of non-verbal affective information in conversational speech J. Phonet. Soc. Jpn. 8 1 2004 9 28
9. R. Cowie, and R.R. Cornelius Describing the emotional states that are expressed in speech Speech Commun. 40 1-2 2003 5 32
10. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J.G. Taylor Emotion recognition in human-computer interaction IEEE Signal Process. Mag. 18 1 2001 32 80
11. Devillers, L., Vidrascu, L., 2006. Real-life emotions detection with lexical and paralinguistic cues on human-human call center dialogues. In: Proceedings of the Interspeech 2006, pp. 801-804.
12. Douglas-Cowie, E., Cowie, R., Schröder, M., 2000. A new emotion database: considerations, sources and scope. In: Proceedings of the ISCA Workshop on Speech and Emotion, pp. 39-44.
13. E. Douglas-Cowie, N. Campbell, R. Cowie, and P. Roach Emotional speech: towards a new generation of databases Speech Commun. 40 1-2 2003 33 60
14. S.W. Duck, Social Context and Relationships Understanding Relationship Processes vol. 3 1993 Sage Newbury Park, CA
15. D. Erickson Expressive speech: production, perception and application to speech synthesis Acoust. Sci. Tech. 26 4 2005 317 325
16. Greasley, P., Setter, J., Waterman, M., Sherrard, C., Roach, P., Arnfield, S., Horton, D., 1995. Representation of prosodic and emotional features in a spoken language database. In: Proceedings of the 13th ICPhS, pp. 242-245.
17. Grimm, M., Kroschel, K., Narayanan, S., 2008. The Vera am Mittag German audio-visual emotional speech database. In: Proceedings of the ICME 2008, pp. 865-868.
18. Y. Horiuchi, Y. Nakano, H. Koiso, M. Ishizaki, H. Suzuki, M. Okada, M. Naka, S. Tutiya, and A. Ichikawa The design and statistical characterization of the Japanese map task dialogue corpus J. JSAI 14 2 1999 261 272
19. G.L. Huttar Prosodic variables and emotions in normal American English utterances J. Speech Hear. Res. 11 3 1968 481 487
20. Kasuya, H., Maekawa, K., Kiritani, S., 1999. Joint estimation of voice source and vocal tract parameters as applied to the study of voice source dynamics. In: Proceedings of the 14th ICPhS, pp. 2505-2512.
21. Kasuya, H., Yoshizawa, M., Maekawa, K., 2000. Roles of voice source dynamics as a conveyer of paralinguistic features. In: Proceedings of the ICSLP 2000, pp. 345-348.
22. M. Keeley, and A. Hart Nonverbal behaviors in dyadic interaction S.W. Duck, Dynamics of Relationships Understanding Relationship Processes vol. 4 1994 Sage Newbury Park, CA 35 162
23. A. Mehrabian, and S. Ksionzky A Theory of Affiliation 1974 Lexington Books Lexington, MA
24. Meteer, M., Taylor, A., MacIntyre, R., Iver, R., 1995. Dysfluency annotation stylebook for the switchboard corpus. The Linguistic Data Consortium.
25. G.R. Miller, and J.B. Stiff Deceptive Communication Interpersonal Communication vol. 14 1993 Sage Newbury Park, CA
26. Mori, H., 2007. Basic Unit in Annotating Paralinguistic Information for Conversational Speech. Technical Report, SIG-SLUD-A603-03, JSAI.
27. H. Mori An analysis of switching pause duration in expressive dialogues as a paralinguistic feature Acoust. Sci. Tech. 30 5 2009 376 378
28. Mori, H., Kasuya, H., 2007. Voice source and vocal tract variations as cues to emotional states perceived from expressive conversational speech. In: Proceedings of the Interspeech 2007, pp. 102-105.
29. H. Mori, and K. Ohshima Facial expression generation from speaker's emotional states in daily conversation IEICE Trans. Inf. Syst. E91-D 6 2008 1628 1633
30. H. Mori, H. Aizawa, and H. Kasuya Consistency and agreement of paralinguistic information annotation for conversational speech J. Acoust. Soc. Jpn. 61 2005 690 697
31. Morimoto, T., Uratani, N., Takezawa, T., Furuse, O., Sobashima, Y., Iida, H., Nakamura, A., Sagisaka, Y., Higuchi, N., Yamazaki, Y., 1994. A speech and language database for speech translation research. In: Proceedings of the ICSLP 1994, pp. 1791-1794.
32. I.R. Murray, and J.L. Arnott Toward the simulation of emotion in synthetic speech: a review of the literature on human vocal emotion J. Acoust. Soc. Am. 93 2 1993 1097 1108
33. Ohtsuka, T., Kasuya, H., 2001. Aperiodicity control in ARX-based speech analysis-synthesis method. In: Proceedings of the Eurospeech 2001, vol. 3, pp. 2267-2270.
34. M.T. Palmer Controlling conversations: turns, topics and interpersonal control Commun. Monogr. 56 1 1989 1 18
35. M.L. Patterson A functional approach to nonverbal exchange R.S. Feldman, B. Rime, Fundamentals of Nonverbal Behavior 1991 Cambridge University Press New York 58 495
36. R. Rosenthal, J.A. Hall, M.R. DiMatteo, P.L. Rogers, and D. Archer Sensitivity to Nonverbal Communication: The PONS Test 1979 Johns Hopkins University Press Baltimore, MD
37. J.A. Russell, and M. Bullock Multidimensional scaling of emotional facial expressions: similarity from preschoolers to adults J. Pers. Soc. Psychol. 48 1985 1290 1298
38. J.A. Russell, A. Weiss, and G.A. Mendelsohn Affect grid: a single-item scale of pleasure and arousal J. Pers. Soc. Psychol. 57 3 1989 493 502
39. Satake, T., Mori, H., 2009. Estimation of Speaker's Emotional States Based on Discourse Analysis of Conversational Dialogue. Technical Report, SIG-SLUD-A803-13, JSAI.
40. K.R. Scherer Vocal correlates of emotional arousal and affective disturbance H.L. Wagner, A.S.R. Manstead, Handbook of Social Psychophysiology 1989 Wiley 165 197
41. K.R. Scherer, and J.S. Oshinsky Cue utilization in emotion attribution from auditory stimuli Motiv. Emotion 1 1977 331 346
42. H. Schlosberg The description of facial expressions in terms of two dimensions J. Exp. Psychol. 44 4 1952 229 237
43. Schröder, M., 2004. Speech and Emotion Research: An Overview of Research Frameworks and A Dimensional Approach to Emotional Speech Synthesis. Ph.D. Thesis, Saarland University.
44. D.J. Stang Effect of interaction rate on ratings of leadership and liking J. Pers. Soc. Psychol. 27 3 1973 405 408
45. The Japanese Discourse Research Initiative, 2000. Japanese dialogue corpus of multi-level annotation. In: Proceedings of the First SIGdial, pp. 1-8.
46. Truong, K.P., Neerincx, M.A., van Leeuwen, D.A., 2008. Assessing agreement of observer- and self-annotations in spontaneous multimodal emotion data. In: Proceedings of the Interspeech 2008, pp. 318-321.
47. R.M. Warner, D. Malloy, K. Schneider, R. Knoth, and B. Wilder Rhythmic organization of social interaction and observer ratings of positive affect and involvement J. Nonverbal Behav. 11 2 1987 57 74
48. I.H. Witten, and E. Frank Data Mining second ed. 2005 Morgan Kaufmann San Francisco