Interaction of Audition and Vision for the Perception of Prosodic Contrastive Focus

Language and Speech

Volumn 52, Issue 2-3, 2009, Pages 177-206

  Dohen, Marion ^a   LAœvenbruck, Hélène ^a  

^a GIPSA LAB   (France)

Author keywords

Auditory visual perception; Perception; Prosodic focus; Speech; Whispered speech

Indexed keywords

EID: 67949115459     PISSN: 00238309     EISSN: None     Source Type: Journal    
DOI: 10.1177/0023830909103166     Document Type: Article

References (64)

1. Audouy, M. (2000). Traitement d'images vidéo pour la capture des mouvements labiaux. Engineering Master Thesis, Institut National Polytechnique de Grenoble, France.
2. Bailly, G., Elisei, F., & Raidt, S., (2007). Virtual talking heads and ambiant face-to-face communication. In A. Esposito, E. Keller, M. Marinaro, & M. Bratanic (Eds.), The fundamentals of verbal and non-verbal communication and the biometrical issue (pp. 302-316). Amsterdam: IOS Press.
3. Bartels, C., & Kingston, J. (1994). Salient pitch cues in the perception of contrastive focus. In P. Bosch & R. Van Der Sandt (Eds.), Proceedings of the Conference on Focus and Natural Language Processing, Working Papers of the Institute for Logic and Linguistics (pp. 94-106). Heidelberg: IBM.
4. Baum, S.R., Kelsch Daniloff, J., Daniloff, R., & Lewis, J. (1982). Sentence comprehension by Broca's aphasics: Effects of some suprasegmental variables. Brain and Language, 17, 261-271. (Pubitemid 13148645)
5. Bernstein, L.E., Eberhardt, S.P., & Demorest, M.E. (1989). Single-channel vibrotactile supplements to visual perception of intonation and stress. The Journal of the Acoustical Society of America, 85 (1). 397-405. (Pubitemid 19041135)
6. Beskow, J., Granström, B., & House, D. (2006). Visual correlates to prominence in several expressive modes. In Proceedings of Interspeech 2006 - ICSLP (pp. 1272-1275). Pittsburgh, PA.
7. Birch, S., & Clifton, C. Jr. (1995). Focus, accent, and argument structure: effects on language comprehension. Language and speech, 38, 365-391.
8. Brådvik, B., Dravins, C., Holtås, S., Rosén, I., Ryding, E., & Ingvar, D. (1991). Disturbances of speech prosody following right hemisphere infarcts. Acta Neurologica Scandinavica, 84, 114-126.
9. Bryan, K. (1989). Language prosody and the right hemisphere. Aphasiology, 3, 285-299. (Pubitemid 19139174)
10. Cavé, C., Guaïtella, I., Bertrand, R., Santi, S., Harlay, F., & Espesser, R. (1996). About the relationship between eyebrow movements and F0 variations. Proceedings of the ICSLP 1996, 4, 2175-2179.
11. Cho, T. (2005). Prosodic strengthening and featural enhancement: Evidence from acoustic and articulatory realizations of /a,i/ in English. The Journal of the Acoustical Society of America, 117 (6). 3867-3878. (Pubitemid 40807422)
12. Christophe, A., Gout, A., Peperkamp, S., & Morgan, J. (2003). Discovering words in the continuous speech stream: The role of prosody. Journal of Phonetics, 31, 585-598. (Pubitemid 37495931)
13. Dahan, D., & Bernard, J.-M. (1996). Interspeaker variability in emphatic accent production in French. Language and Speech, 39 (4). 341-374. (Pubitemid 126421035)
14. De Jong, K. (1995). The supraglottal articulation of prominence in English: Linguistic stress as localized hyperarticulation. The Journal of the Acoustical Society of America, 97 (1). 491-504.
15. Di Cristo, A. (2000). Vers une modélisation de l'accentuation du franĉais (deuxiéme partie). Journal of French Language Studies, 10, 27-44.
16. D'Imperio, M. (2001). Focus and tonal structure in Neapolitan Italian. Speech Communication, 33 (4). 339-356. (Pubitemid 32160798)
17. Dohen, M., & Lœvenbruck, H. (2005). Audiovisual production and perception of contrastive focus in French: A multispeaker study. In Proceedings of Interspeech 2005 - EUROSPEECH (pp.2413-2416). Lisbon, Portugal.
18. Dohen, M., Lœvenbruck, H., Cathiard, M.-A., & Schwartz, J.-L. (2004). Visual perception of contrastive focus in reiterant French speech. Speech Communication, 44, 155-172.
19. Dohen, M., Lœvenbruck, H., & Hill, H. (2006). Visual correlates of prosodic contrastive focus in French: Description and inter-speaker variabilities. Proceedings of Speech Prosody 2006, I, 221-224.
20. Erickson, D. (2002). Articulation of extreme formant patterns for emphasized vowels. Phonetica, 59, 134-149.
21. Erickson, D., Maekawa, K., Hashi, M., & Dang, J. (2000). Some articulatory and acoustic changes associated with emphasis in spoken English. Proceedings of the ICSLP 2000, 3, 247-250.
22. Graf, H.P., Cosatto, E., Strom, V., & Huang, F.J. (2002). Visual prosody: Facial movements accompanying speech. In Proceedings of the 5th IEEE International Conference on Automatic Face and Gesture Recognition (FGR'02) (pp.381-386). Washington, DC.
23. Granström, B., & House, D. (2005). Audiovisual representation of prosody in expressive speech communication. Speech Communication, 46, 473-484. (Pubitemid 40952529)
24. Gussenhoven, C. (1983). Testing the reality of focus domains. Language and Speech, 26 (1). 61-80.
25. Hadar, U., Steiner, T.J., Grant, E.C., & Rose, F.C. (1983). Head movement correlates of juncture and stress at sentence level. Language and Speech, 26, 117-129.
26. Halliday, M.A.K. (1967). Intonation and grammar in British English. The Hague: Mouton.
27. Harrington, J., Fletcher, J., & Roberts, C. (1995). Coarticulation and the accented/ unaccented distinction: Evidence from jaw movement data. Journal of Phonetics, 23, 305-322.
28. Higashikawa, M., & Minifie, F.D. (1999). Acoustical perceptual correlates of 'whispered pitch' in synthetically generated vowels. Journal of Speech, Language, and Hearing Research, 42, 583-591. (Pubitemid 29287937)
29. Keating, P., Baroni, M., Mattys, S., Scarborough, R., Alwan, A., Auer, E.T., ET AL. (2003). Optical phonetics and visual perception of lexical and phrasal stress in English. In M. J. Solé, D. Recasens, & J. Romero (Eds.), Proceedings of the ICPhS 2003 (pp. 2071-2074). Barcelona.
30. Kehoe, M., Stoel-Gammon, C., & Buder, E.H. (1995). Acoustic correlates of stress in young children's speech. Journal of Speech, Language, and Hearing Research, 38, 338-350.
31. Kelso, J.A.S., Vatikiotis-Bateson, E., Saltzman, E., & Kay, B.A. (1985). A qualitative dynamic analysis of reiterant speech production: Phase portraits, kinematics, and dynamic modeling. The Journal of the Acoustical Society of America, 77 (1). 266-280. (Pubitemid 15136365)
32. Konopczinsky, G. (1986). Du prélangage au langage: acquisition de la structuration prosodique. PhD thesis, Strasbourg University, France.
33. Krahmer, E., Ruttkay, Z., Swerts, M., & Wesselink, W. (2002 a). Perceptual evaluation of audiovisual cues for prominence. Proceedings of the ICSLP 2002 (pp. 1933-1936). Denver, CO.
34. Krahmer, E., Ruttkay, Z., Swerts, M., & Wesselink, W. (2002 b). Pitch, eyebrows and the perception of focus. In Proceedings of Speech Prosody 2002 (pp. 443-446). Aixen-Provence, France.
35. Krahmer, E., & Swerts, M. (2006). Perceiving focus. In C.-M. Lee (Ed.), Topic and focus: A cross-linguistic perspective (pp. 121-137). Dordrecht, The Netherlands: Kluwer Academic Publishers.
36. Ladd, R.D. (1996). Intonational phonology. Cambridge: Cambridge University Press (Cambridge Studies in Linguistics).
37. Lallouache, M.-T. (1991). Un poste visage-parole couleur. Acquisition et traitement automatique des contours de lévres. PhD thesis, Institut National Polytechnique de Grenoble, France.
38. LAœvenbruck, H. (1996). Pistes pour le contrǒ'le d'un robot parlant capable de réduction vocalique. PhD thesis, Institut National Polytechnique de Grenoble, France.
39. LAœvenbruck, H. (1999). An investigation of articulatory correlates of the accentual phrase in French. Proceedings of the 14th ICPhS, 1, 667-670.
40. LAœvenbruck, H. (2000). Effets articulatoires de l'emphase contrastive sur la phrase accentuelle en franĉais. In Proceedings of the XXIIIth JEP (pp. 165-168). Aussois, France.
41. MacLeod, A., & Summerfield, A.Q. (1987). Quantifying the contribution of vision to speech perception in noise. British Journal of Audiology, 21, 131-141.
42. McGurk, H., & MacDonald, J. (1976). Hearing lips and seeing voices. Nature, 264, 746-748.
43. Miller, G.A., & Nicely, P. (1955). An analysis of perceptual confusions among some English consonants. The Journal of the Acoustical Society of America, 27 (2). 338-352.
44. Morel, M.-A., & Danon-Boileau, L. (1998). Grammaire de l'intonation. L'exemple du franĉais oral. Paris / Gap France: Ophrys, Bibliothéque de Faits de Langues.
45. Munhall, K.G., Jones, J.A., Callan, D.E., Kuratate, T., & Vatikiotis-Bateson, E. (2004). Visual prosody and speech intelligibility - head movement improves auditory speech perception. Psychological Science, 15 (2). 133-137. (Pubitemid 38361306)
46. Nlke, H. (1994). Linguistique modulaire: de la forme au sens. Louvain / Paris: Peeters.
47. Pierrehumbert, J., & Hirshberg, J. (1990). The meaning of intonational contours in discourse. In P. Cohen, J. Morgan, & M. Pollack (Eds.), Intentions in communication (pp. 271-311). Cambridge, MA: The MIT Press.
48. Reisberg, D., Mclean, J., & Goldfield, A. (1987). Easy to hear but hard to understand: A lip-reading advantage with intact auditory stimuli. In B. Dodd & R. Campbell (Eds.), Hearing by eye: The psychology of lip-reading (pp. 97-114). Hillsdale, NJ: Lawrence Erlbaum Associates.
49. Risberg, A., & Agelfors, E. (1978). On the identification of intonation contours by hearing impaired listeners. Speech Transmission Laboratory - Quarterly Progress Report and Status Report, 19 (2-3). 51-61.
50. Risberg, A., & Lubker, J. (1978). Prosody and speechreading. Speech Transmission Laboratory - Quarterly Progress Report and Status Report, 19 (4). 1-16.
51. Rossi, M. (1999). La focalisation. In L'intonation, le systéme du franĉais: description et modélisation (pp.1 16-128). Paris: Ophrys.
52. Séguinot, A. (1976). L'accent d'insistance en franĉais standard. Studia Phonetica, 12, 1-58.
53. Selkirk, E.O. (1984). The grammar of intonation. In E. O. Selkirk (Ed.), Phonology and syntax: The relation between sound and structure (pp. 197-296). Cambridge, MA: The MIT Press.
54. Sumby, W.H., & Pollack, I. (1954). Visual contribution to speech intelligibility in noise. The Journal of the Acoustical Society of America, 26 (2). 212-215.
55. Summers, W.V. (1987). Effects of stress and final-consonant voicing on vowel production: Articulatory and acoustic analyses. The Journal of the Acoustical Society of America, 82 (3). 847-863.
56. Swerts, M., & Krahmer, E. (2004). Congruent and incongruent audiovisual cues to prominence. Proceedings of Speech Prosody 2004 (pp.69-72). Nara, Japan.
57. Swerts, M., & Krahmer, E. (2008). Facial expression and prosodic prominence: Effects of modality and facial area. Journal of Phonetics, 36 (2). 219-238.
58. Thompson, D.M. (1934). On the detection of emphasis in spoken sentences by means of visual, tactual, and visual-tactual cues. Journal of General Psychology, 11, 160-172.
59. Touati, P. (1987). Structures prosodiques du suédois et du franĉais. Sweden Lund University Press.
60. Touati, P. (1989). De la prosodie franĉaise du dialogue. Rapport du projet KIPROS. Working Papers in Linguistics, 35, 203-214.
61. Touratier, C. (2000). La sémantique. Paris: Armand Collin.
62. Vatikiotis-Bateson, E., & Kelso, J.A.S. (1993). Rhythm type and articulatory dynamics in English, French and Japanese. Journal of Phonetics, 21, 231-265.
63. Vihman, M.M. (1996). Phonological development: The origins of language in the child. Oxford: Blackwell.
64. Weintraub, S., Mesulam, M.-M., & Krahmer, L. (1981). Disturbances in prosody: A right-hemisphere contribution to language. Archives of Neurology, 38, 742-744. (Pubitemid 12235676)