Prosody off the top of the head: Prosodic contrasts can be discriminated by head motion

Speech Communication

Volumn 52, Issue 6, 2010, Pages 555-564

  Cvejic, Erin ^a   Kim, Jeesun ^a   Davis, Chris ^a  

^a UNIVERSITY OF WESTERN SYDNEY   (Australia)

Author keywords

Prosody; Rigid head motion; Suprasegmental; Upper head; Visual speech

Indexed keywords

AUDITORY-VISUAL; CROSS-MODAL; HEAD MOTION; HIGH DEGREE OF ACCURACY; NON-RIGID MOTION; RIGID MOTIONS; UNDERLYING FACTORS; VISUAL MATCHING; VISUAL SPEECH;

FEATURE EXTRACTION;

EXPERIMENTS;

EID: 77951138429     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2010.02.006     Document Type: Article

References (35)

1. Benoît C., and Le Goff B. Audio-visual speech synthesis from French text: eight years of models, designs and evaluation at the ICP. Speech Commun. 26 (1998) 117-129
2. Bernstein L.E., Eberhardt S.P., and Demorest M.E. Single-channel vibrotactile supplements to visual perception of intonation and stress. J. Acoust. Soc. Am. 85 (1989) 397-405
3. Boersma, P., Weenink, D., 2008. Praat: Doing Phonetics by Computer (Version 5.0.33) [Software]. .
4. Bolinger D. Accent is predictable (if you're a mind reader). Language 48 (1972) 633-644
5. Bolinger D. Intonation and its Uses (1989), Edward Arnold, London
6. Burnham D., Reynolds J., Vignali G., Bollwerk S., and Jones C. Rigid vs non-rigid face and head motion in phone and tone perception. Interspeech 698 (2007) 701
7. Cavé, C., Guaïtella, I., Bertrand, R., Santi, S., Harley, F., Espesser, R., 1996. About the relationship between eyebrow movements and F0 variations. In: International Conference on Spoken Language Processing, pp. 2175-2178.
8. Cutler A., Dahan D., and van Donselaar W. Prosody in the comprehension of spoken language: a literature review. Language Speech 40 (1997) 141-201
9. Davis C., and Kim J. Audiovisual interactions with intact clearly audible speech. Quart. J. Exp. Psychol. 57A (2004) 1103-1121
10. Davis C., and Kim J. Audio-visual speech perception off the top of the head. Cognition 100 (2006) B21-B31
11. Dohen M., Lœvenbruck H., Cathiard M.A., and Schwartz J.L. Visual perception of contrastive focus in reiterant French speech. Speech Commun. 44 (2004) 155-172
12. Dohen M., and Lœvenbruck H. Audiovisual production and perception of contrastive focus in French: a multispeaker study. Interspeech 2413 (2005) 2416
13. Dohen M., Lœvenbruck H., and Hill H. Visual correlates of prosodic contrastive focus in French: description and inter-speaker variability. Speech Prosody 221 (2006) 224
14. Eady S.J., and Cooper W.E. Speech intonation and focus location in matched statements and questions. J. Acoust. Soc. Am. 80 (1986) 402-415
15. Erickson D., Fujimura O., and Pardo B. Articulatory correlates of prosodic control: emotion and emphasis. Language Speech 41 (1998) 399-417
16. Flecha-Garcia, M.L., 2006. Eyebrow Raising in Dialogue: Discourse Structure, Utterance Function, and Pitch Accents, Ph.D. Thesis. Theoretical & Applied Linguistics, University of Edinburgh.
17. Forster K.I., and Forster J.C. DMDX: a windows display program with millisecond accuracy. Behav. Res. Methods: Instrum. Comput. 35 (2003) 116-124
18. Guaïtella I., Santi S., Lagrue B., and Cavé C. Are eyebrow movements linked to voice variations and turn-taking in dialogue? An experimental investigation. Language Speech 52 (2009) 207-222
19. IEEE Subcommittee on Subjective Measurements. IEEE recommended practices for speech quality measurements. IEEE Trans. Audio Electroacoust. 17 (1969) 227-246
20. Ishi, C.T., Haas, J., Wilbers, F.P., Ishiguro, H., Hagita, N., 2007. Analysis of head motions and speech, and head motion control in an android. In: IEEE/RSJ International Conference on Intelligent Robots & Systems, pp. 548-553.
21. Krahmer E., and Swerts M. On the alleged existence of contrastive accents. Speech Commun. 34 (2001) 391-405
22. Krahmer E., and Swerts M. More about brows: a cross-linguistic study via analysis-by-synthesis. In: Ruttkay Z., and Pelachaud C. (Eds). From Brows to Trust: Evaluating Embodied Conversational Agents (2004), Kluwer Academic Press, Dordrecht 191-216
23. Lansing C.R., and McConkie G.W. Attention to facial regions in segmental and prosodic speech perception tasks. J. Speech Language Hearing Res. 42 (1999) 526-539
24. Lappin J.S., Tadin D., Nyquist J.B., and Corn A.L. Spatial and temporal limits of motion perception across variations in speed, eccentricity, and low vision. J. Vision 9 (2009) 1-14
25. Lee, A., 2008. Virtual Dub (Version 1.8.6) [Software]. .
26. McKee S.P., and Nakayama K. The detection of motion in the peripheral visual field. Vision Res. 24 (1984) 25-32
27. Nooteboom S. The prosody of speech: melody and rhythm. In: Hardcastle W.J., and Laver J. (Eds). The Handbook of Phonetic Science (1997), Blackwell, London 640-673
28. Paré M., Richler R.C., and ten Hove M. Gaze behavior in audiovisual speech perception: the influence of ocular fixations on the McGurk effect. Percept. Psychophys. 65 (2003) 553-567
29. Scarborough R., Keating P., Mattys S., Cho T., and Alwan A. Optical phonetics and visual perception of lexical and phrasal stress in English. Language Speech 52 (2009) 135-175
30. Srinivasan R.J., and Massaro D.W. Perceiving prosody from face and voice: distinguishing statements from echoic questions in English. Language Speech 46 (2003) 1-22
31. Sumby W.H., and Pollack I. Visual contribution to speech intelligibility in noise. J. Acoust. Soc. Am. 26 (1954) 212-215
32. Summerfield Q. Lipreading and audio-visual speech perception. Phil. Trans. Biosci. 335 (1992) 71-78
33. Swerts M., and Krahmer E. Facial expression and prosodic prominence: effects of modality and facial area. J. Phonet. 36 (2008) 219-238
34. Vatikiotis-Bateson E., Eigsti I., Yano S., and Munhall K.G. Eye movement of perceivers during audiovisual speech perception. Percept. Psychophys. 60 (1998) 926-940
35. Yehia H.C., Kuratate T., and Vatikiotis-Bateson E. Linking facial animation, head motion and speech acoustics. J. Phonet. 30 (2002) 555-568