Multimodal Indices to Japanese and French Prosodically Expressed Social Affects

Language and Speech

Volumn 52, Issue 2-3, 2009, Pages 223-243

  Rilliard, Albert ^a   Shochi, Takaaki ^b   Martin, Jean Claude ^a   Erickson, Donna ^c   Aubergé, Véronique ^d  

^a CNRS   (France)

^b KUMAMOTO UNIVERSITY   (Japan)

^c Showa Music University   (Japan)

^d GIPSA LAB   (France)

Author keywords

Attitudes; Audovisual prosody; French; Japanese; Perception

Indexed keywords

EID: 67949090929     PISSN: 00238309     EISSN: None     Source Type: Journal    
DOI: 10.1177/0023830909103171     Document Type: Article

References (58)

1. Allerton, D.J., & Cruttenden, A. (1978). Syntactic, illocutionary, thematic and attitudinal factors in the intonation of adverbials. Journal of Pragmatics, 2, 155-188.
2. Aubergé, V. (2002). A Gestalt morphology of prosody directed by functions: The example of a step by step model developed at ICP. In B. Bel & I. Vincent-Marlien (Eds.), Proceedings of Speech Prosody 2002 (pp. 151-154). Aix-en Provence: Laboratoire Parole et Langage.
3. Aubergé, V., Audibert, N., & Rilliard, A. (2004). Acoustic morphology of expressive speech: What about contours? Proceedings of Speech Prosody 2004 (pp. 201-204). Nara, Japan.
4. Aubergé, V., & Cathiard M. (2003). Can we hear the prosody of smile ? Speech Communication, 40 (1). 87-97.
5. Audibert, N., Aubergé, V., & Rilliard, A. (2008). How we are not equally competent for discriminating acted from spontaneous expressive speech. Proceedings of Speech Prosody 2008 (pp.693-696). Campinhas, Brasil.
6. Banse, R., & Scherer, K.R. (1996). Acoustic profiles in vocal emotion expression. Journal of Personality and Social Psychology, 70, 614-636. (Pubitemid 126420699)
7. Bänziger, T., & Scherer, K.R. (2005). The role of intonation in emotional expressions. Speech Communication, 46, 252-267. (Pubitemid 40952515)
8. Barkhuysen, P., Krahmer, E., & Swerts, M. (2006). How auditory and visual prosody is used in end-of-utterance detection. In Proceedings of Interspeech 2006 - ICSLP, Ninth International Conference on Spoken Language Processing (pp.1276-1279). Retrieved February 2009, from http://www.isca-speech. org/archive/interspeech-2006
9. Barkhuysen, P., Krahmer, E., & Swerts, M. (2007 a). Cross-modal perception of emotional speech. In J. Trouvain & W. J. Barry (Eds.), Proceedings of the 16th International Congress of PhoneticSciences (pp.2133-2136). Retrieved February 2009, from http://www.icphs2007.de/ conference/Papers/1482/index.html
10. Barkhuysen, P., Krahmer, E., & Swerts, M. (2007 b). Incremental perception of acted and real emotional speech. In Proceedings of Interspeech 2007, 8th Annual Conference of the International Speech Communication Association (pp.1262-1265). Retrieved February 2009, from http://www.isca- speech.org/archive/interspeech-2007
11. Benzecri, J.P. (1973). L'analyse des données. Paris: Bordas.
12. Brandt, P.A. (2008). Thinking and language. A view from cognitive semio-linguistics. In P. A. Barbosa, S. Madureira, & C. Reis (Eds.), Proceedings of Speech Prosody 2008 (pp.6 49-654). Campinhas, Brazil: Editora RG / CNPq.
13. Buisine, S., Abrilian, S., Niewiadomski, R., Martin, J.-C., Devillers, L., & Pelachaud, C. (2006). Perception of blended emotions: From video corpus to expressive agent. In J. Gratch, M. Young, R. Aylett, D. Ballin, & P. Olivier (Eds.), Proceedings of 6th International Conference on Intelligent Virtual Agents (pp. 93-106). Berlin / Heidelberg: Springer. (Pubitemid 44532606)
14. Calbris, G., & Montredon, J. (1981). Oh là là. Expression intonative et mimique. Paris: CLE international.
15. Calbris, G., & Porcher, L. (1989). Geste et communication. Paris: Hatier-Crédif.
16. Callamand, M. (1973). L'intonation expressive. Collection le franĉais dans le monde, B.E.L.C. Paris: Librairies Hachette et Larousse.
17. Campbell, N. (2005). Getting to the heart of the matter: Speech as the expression of affect. rather than just text or language. Language Resources and Evaluation, 39 (1). 111-120.
18. Cooper, C. (2002). Individual differences (2 nd ed.). London: Arnold.
19. Dahlbäck, N., Jönsson, A., & Ahrenberg, L. (1993). Wizard of Oz studies - Why and how. Paper presented at the Workshop on Intelligent User Interfaces, Orlando, FL, U.S.A.
20. d'Alessandro, C. (2006). Voice source parameters and prosodic analysis. In S. Sudhoff, D. Leternova, R. Meyer, S. Pappert, P. Augurzky, I. Mleinek, et al. (Eds.), Methods in Empirical Prosody Research (pp.6 3-87). Berlin / New York: Walter de Gruyter.
21. d'Alessandro, C., Rilliard, A., & Le Beux, S. (2007). Computerized chironomy: evaluation of hand-controlled intonation reiteration. In Proceedings of Interspeech 2007 (pp.1270-1273). Retrieved February 2009, from http://www.isca-speech.org/archive/interspeech-2007
22. Damasio, A.R. (1994). Descartes' error. Emotion, reason, and the human brain. New York: G. P. Putnam.
23. Daneš F. (1994). Involvement with language and in language. Journal of Pragmatics, 22, 251-264.
24. de Cheveigné, A., & Kawahara, H. (2002). YIN, a fundamental frequency estimator for speech and music. Journal of the Acoustical Society of America, 111, 1917-1930. (Pubitemid 34297247)
25. de Moraes, J.A. (2008). The pitch accents in Brazilian Portuguese: Analysis by synthesis. In P. A. Barbosa, S. Madureira, & C. Reis (Eds.), Proceedings of Speech Prosody 2008 (pp.389-397). Campinhas, Brazil: Editora RG/ CNPq.
26. Ekman, P. (1999). Facial expressions. In T. Dalgleish & T. Power (Eds.), The Handbook of cognition and emotion (pp. 301-320). Sussex, U.K.: Wiley.
27. Ekman, P., Friesen, W.C., & Hager, J.C. (2002). Facial action coding system. Salt Lake City, UT: The Manual on CD ROM., Research Nexus division of Network Information Research Corporation.
28. Erickson, D., Ohashi, S., Makita, S., Kajimoto, N., & Mokhtari, P. (2003). Perception of naturally-spoken expressive speech by American English and Japanese listeners. In N. Campbell (Ed.), Proceedings of CREST International Workshop on Expressive Speech Processing (pp.31-36). Kobe, Japan: Kobe University.
29. Fónagy, I. (2003). Des fonctions de l'intonation: essai de synthése. Flambeau, 29, 1-20.
30. Fónagy, I., Bérard, E., & Fónagy, J. (1984). Clichés mélodiques. Folia Linguistica, 17, 153-185.
31. Granström, B., & House, D. (2005). Audiovisual representation of prosody in expressive speech communication. Speech Communication, 46, 473-484. (Pubitemid 40952529)
32. Grichkovtsova, I., Lacheret, A., Morel, M., Beaucousin, V., & Tzourio-Mazoyer, N. (2007). Affective speech gating. In J. Trouvain & W. J. Barry (Eds.), Proceedings of the 16th International Congress of Phonetic Sciences (pp.805-808). Retrieved February 2009, from http://www.icphs2007.de/ conference/Papers/1539/1539.pdf
33. Loveday, L. (1981). Pitch, politeness and sexual role: An exploratory investigation into the pitch correlates to English and Japanese politeness formulae. Language and Speech, 24 (1). 71-89.
34. Maekawa, K. (1998). Phonetic and phonological characteristics of paralinguistic information in spoken Japanese. Proceedings of the 5th International Conference on Spoken Language Processing (pp.635-638), Sydney.
35. Martin, J.-C., d'Alessandro, C., Jacquemin, C., Katz, B., Max, A., Pointal, L., ET AL. (2007). 3D audiovisual rendering and real-time interactive control of expressivity in a talking head. In C. Pelachaud, J.-C. Martin, E. André, G. Chollet, K. Karpouzis, & D. Pelé (Eds.), 7th International Conference on Intelligent Virtual Agents (IVA'2007) (pp.1 7-19). Berlin / Heidelberg: Springer.
36. Martin, J.-C., Niewiadomski, R., Devillers, L., Buisine, S., & Pelachaud, C. (2006). Multimodal complex emotions: Gesture expressivity and blended facial expressions. [Special issue on "Achieving Human-like Qualities in Interactive Virtual and Physical Humanoids, " C. Pelachaud & L. Canamero (Eds.)] Journal of Humanoid Robotics, 3 (3). 269-291.
37. Matthews, G., Zeidner, M., & Roberts, R.D. (2002). Emotional intelligence-science and myth. Cambridge, MA: MIT Press.
38. Mizutani, O., & Mizutani N. (1979). Aural comprehension practice in Japanese. Tokyo: The Japan Times.
39. Morlec, Y., Bailly, G., & Aubergé, V. (2001). Generating prosodic attitudes in French: Data, model and evaluation. Speech Communication, 33 (4). 357-371. (Pubitemid 32160799)
40. Mozziconacci, S.J.L. (1998). Speech variability and emotion: Production and perception. Unpublished Ph.D. thesis, Eindhoven, The Netherlands.
41. Ohala, J.J. (1996). Ethological theory and the expression of emotion in the voice. In Proceedings of the 4th International Conference on Spoken Language Processing (pp.1812-1815). Retrieved February 2009, from http://www.isca- speech.org/archive/icslp-1996
42. Paulmann, S., Schmidt, P., Pell, M., & Kotz, S. (2008). Rapid processing of emotional and voice information as evidenced by ERPs. In P. A. Barbosa, S. Madureira, & C. Reis (Eds.), Proceedings of Speech Prosody 2008 (pp.2 05-209). Campinhas, Brazil: Editora RG / CNPq.
43. Pavlenko, A. (2005). Emotions and multilingualism. Cambridge, U.K.: Cambridge University Press.
44. Rilliard, A., Martin, J.C., Aubergé, V., & Shochi, T. (2008). Perception of French audio-visual prosodic attitudes. In P.A. Barbosa, S. Madureira, & C. Reis (Eds.), Speech prosody 2008 (pp. 685-688). Campinas, Brazil: Editoria RG / CNPq.
45. Rossi, M., DI Cristo, A., Hirst, D., Martin, P., & Nishinuma, Y. (1981). L'intonation: de l'acoustique la sémantique. Paris: Klincksieck.
46. Sadanobu, T. (2004). A natural history of Japanese pressed voice. Journal of the Phonetic Society of Japan, 8 (1). 29-44.
47. Sagisaka, Y., Yamashita, T., & Kokenawa, Y. (2004). Speech synthesis with attitude. In Proceedings of Speech Prosody 2004 (pp.401-404). Nara, Japan.
48. Scherer, K.R. (2003). Vocal communication of emotion: A review of research paradigms. Speech Communication, 40 (1-2). 227-256.
49. Scherer, K.R., Banse, R., & Wallbott, H.G. (2001). Emotion inferences from vocal expression correlate across languages and cultures. Journal of Cross-cultural Psychology, 32 (1). 76-92.
50. Scherer, K.R., & Ellgring, H. (2007). Multimodal expression of emotion: Affect programs or componential appraisal patterns ? Emotion, 7 (1). 158-171. (Pubitemid 46433172)
51. Shigeno, S. (1998). Cultural similarities and differences in the recognition of audiovisual speech stimuli. In Proceedings of 5th International Conference on Spoken Language Processing (pp.149-152). Sydney, Australia.
52. Shochi, T. (2008). Prosodie des affects socioculturels en japonais, franĉais et anglais: la recherche des vrais et faux-amis pour le parcours de l'apprenant. Unpublished Ph.D. thesis, University Grenoble 3, France.
53. Shochi, T., Aubergé, V., & Rilliard, A. (2006). How prosodic attitudes can be false friends: Japanese vs. French social affects. In R. Hoffmann & H. Mixdorff (Eds.), Speech Prosody 2006 Abstract Book and CD-ROM Proceedings (pp.6 92-696). Dresden: TUDpress Verlag der Wissenschaften.
54. Shochi, T., Aubergé, V., & Rilliard, A. (2007). Cross-listening of Japanese, English and French social affect: About universals, false friends and unknown attitudes. In J. Trouvain & W. J. Barry (Eds.), Proceedings of the 16th International Congress of Phonetic Sciences (pp.2097-2100). Retrieved February 2009, from http://www.icphs2007.de/ conference/Papers/1435/1435.pdf
55. Shochi, T., Erickson, D., Rilliard, A., Aubergé, V., &. Martin, J.C. (2008). Recognition of Japanese attitudes in Audio-Visual speech. P. A. Barbosa, S. Madureira, & C. Reis (Eds.), Speech prosody 2008 (pp. 689-692). Campinas, Brazil: Editora RG / CNPq.
56. Swerts, M., & Krahmer, E. (2005). Audiovisual prosody and feeling of knowing. Journal of Memory and Language, 53 (1). 81-94. (Pubitemid 40801104)
57. van Heuven, V.J., Haan, J., Janse, E., & van der Torre, E.J. (1997). Perceptual identification of sentence type and the time distribution of prosodic interrogativity marker in Dutch. In A. Botinis, G. Kouroupetroglou, & G. Carayiannis (Eds.), Intonation: Theory, models and applications (Proceedings of an ESCA Workshop, September 18-20, 1997, Athens, Greece) (pp.317-320). Athens: ESCA and University of Athens Department of Informatics.
58. Wilting, J., Krahmer, E., & Swerts, M. (2006). Real vs. acted emotional speech. In Proceedings of the Interspeech 2006 - ICSLP, Ninth International Conference on Spoken Language Processing. Retrieved February 2009, from http://www.isca-speech.org/archive/interspeech-2006