Optical Phonetics and Visual Perception of Lexical and Phrasal Stress in English

Language and Speech

Volumn 52, Issue 2-3, 2009, Pages 135-175

  Scarborough, Rebecca ^a   Keating, Patricia ^b   Mattys, Sven L ^c   Cho, Taehong ^d   Alwan, Abeer ^b  

^a UNIVERSITY OF COLORADO   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c UNIVERSITY OF BRISTOL   (United Kingdom)

^d Hanyang University   (South Korea)

Author keywords

Audiovisual speech; Lexical stress; Phrasal stress; Visual prosody; Visual speech perception

Indexed keywords

EID: 67949095663     PISSN: 00238309     EISSN: None     Source Type: Journal    
DOI: 10.1177/0023830909103165     Document Type: Article

References (59)

1. Alexandre, M.-A., & Gérard, C. (2002, April). Perception of emphatic stress: Multiple regression analyses. Paper presented at Temporal Integration in Perception of Speech, Aix-en-Provence, France.
2. Auer, E.T., JR., Bernstein, L.E., & Coulter, D.C. (1998). Temporal and spatiotemporal vibrotactile displays for voice fundamental frequency: An initial evaluation of a new vibrotactile speech perception aid with normal-hearing and hearing-impaired individuals. Journal of the Acoustical Society of America, 104, 2477-2489. (Pubitemid 28463143)
3. Beckman, M., & Elam, G.A. (1997). Guidelines for ToBI labelling (Version 3). Unpublished manuscript, Ohio State University, USA.
4. Beckman, M.E. (1986). Stress and non-stress accent. Dordrecht: Foris.
5. Beckman, M.E., & Edwards, J. (1994). Articulatory evidence for differentiating stress categories. In P. A. Keating (Ed.), Papers in laboratory phonology III: Phonological structure and phonetic form (pp. 7-33). Cambridge: Cambridge University Press.
6. Bernstein, L.E., Auer, E.T., Chaney, B., Alwan, A., & Keating, P.A. (2000). Development of a facility for simultaneous recordings of acoustic, optical (3-D motion and video), and physiological speech data [abstract]. Journal of the Acoustical Society of America, 107, 2887.
7. Bernstein, L.E., Eberhardt, S.P., & Demorest, M.E. (1989). Single-channel vibrotactile supplements to visual perception of intonation and stress. Journal of the Acoustical Society of America, 85, 397-405. (Pubitemid 19041135)
8. Bolinger, D.L. (1958). A theory of pitch accent in English. Word, 14, 109-149.
9. Campbell, W.N. (1993). Automatic detection of prosodic boundaries in speech. Speech Communication, 13, 343-354.
10. Cavé, C., Guaitella, I., Bertrand, R., Santi, S., Harlay, F., & Espesser, R. (1996). About the relationship between eyebrow movements and F0 variation. ICSLP 96, 4, 2175-2179.
11. Cho, T. (2005). Prosodic strengthening and featural enhancement: Evidence from acoustic and articulatory realizations of /a,i/ in English. Journal of the Acoustical Society of America, 117, 3867-3878. (Pubitemid 40807422)
12. Cho, T. (2006). Manifestation of prosodic structure in articulation: Evidence from lip kinematics in English. In L. M. Goldstein, D. H. Whalen, & C. T. Best (Eds.), Laboratory phonology 8: Varieties of phonological competence (pp. 519-548). Berlin / New York: Mouton de Gruyter.
13. Choi, J-Y., Hasegawa-Johnson, M., & Cole, J. (2005). Finding intonational boundaries using acoustic cues related to the voice source. Journal of the Acoustical Society of America, 118, 2579-2587. (Pubitemid 41463968)
14. Condon, W. (1976). An analysis of behavioral organization. Sign Language Studies, 13, 285-318.
15. de Jong, K. (1995). The supraglottal articulation of prominence in English: Linguistic stress as localized hyperarticulation. Journal of the Acoustical Society of America, 97, 491-504.
16. Dohen, M., Loevenbruck, H., Cathiard, M.-A., & Schwartz, J.-L. (2004 a). Visual perception of contrastive focus in reiterant French speech. Speech Communication, 44, 155-172.
17. Dohen, M., Loevenbruck, H., Cathiard, M.-A., & Schwartz, J.-L. (2004 b). Identification of the possible visible correlates of contrastive focus in French. In B. Bel and I. Marlien (Eds.), Speech prosody 2004 (pp. 77-80). Retrieved February 2009, from http://www.isca-speech.org/archive/sp2004
18. Dohen, M., Loevenbruck, H., Cathiard, M.-A., & Schwartz, J.-L. (2004 c). Can we see focus? A visual perception study of contrastive focus in French. In B. Bel & I. Marlien (Eds.), Speech prosody 2004 (pp. 73-76). Retrieved February 2009, from http://www.iscaspeech.org/archive/sp2004
19. Dohen, M., & Loevenbruck, H. (2005). Audiovisual production and perception of contrastive focus in French: A multispeaker study. In Interspeech 2005 (pp. 2413-2416). Retrieved February 2009, from http://www.isca-speech.org/ archive/interspeech-2005 (Pubitemid 43908582)
20. Dohen, M., Loevenbruck, H., & Hill, H. (2005). A multi-measurement approach to the identification of the audiovisual facial correlates of contrastive focus in French. In E. Vatikiotis-Bateson (Ed.), Auditory-visual processing workshop (pp. 115-116). Retrieved February 2009, from http://www.isca-speech.org/archive/avsp05
21. Dohen, M., Loevenbruck, H., & Hill, H. (2006). Visual correlates of prosodic contrastive focus in French: Description and inter-speaker variability. In R. Hoffmann & H. Mixdorff (Eds.), Speech prosody 2006 (pp. 221-224). Retrieved February 11, 2008, from http://www.isca-speech.org/archive/sp2006
22. Ekman, P. (1979). About brows: Emotional and conversational signals. In M. von Cranach, K. Foppa, W. Lepenies, & D. Ploog (Eds.), Human ethology: Claims and limits of a new discipline (pp. 169-202). Cambridge University Press.
23. Erickson, D. (2002). Articulation of extreme formant patterns for emphasized vowels. Phonetica, 59, 134-149.
24. Erickson, D., Fujimura, O., & Pardo, B. (1998). Articulatory correlates of prosodic control: emotion and emphasis. Language and Speech, 41, 395-413.
25. Fant, G., Kruckenberg, A., & Liljencrants, J. (2000). Acoustic-phonetic analysis of prominence in Swedish. In A. Botinis (Ed.), Intonation (pp. 55-86). Kluwer Academic Publishers.
26. Fry, D.B. (1958). Experiments in the perception of stress. Language and Speech, 1, 126-152.
27. Granström, B., House, D., & Lundeberg, M. (1999). Prosodic cues to multimodal speech perception. In Proceedings of the 14th International Congress on Phonetic Sciences (pp.655-658). San Francisco.
28. Grant, K.W., Ardell, L.A., Kuhl, P.K., & Sparks, D.W. (1986). The transmission of prosodic information via an electrotactile speech reading aid. Ear and Hearing, 7, 328-335. (Pubitemid 17168736)
29. Hadar, U., Steiner, T.J., Grant, E.C., & Clifford Rose, F. (1983). Head movement correlates of juncture and stress at sentence level. Language and Speech, 26, 117-129.
30. Harrington, J., Fletcher, J., & Beckman, M. (2000). Manner and place conflicts in the articulation of accent in Australian English. In M. B. Broe & J. Pierrehumbert (Eds.), Papers in laboratory phonology V (pp. 40-51). Cambridge: Cambridge University Press.
31. Hayes, B. (1995). Metrical stress theory: Principles and case studies. Chicago: University of Chicago Press.
32. Heldner, M. (2003). On the reliability of overall intensity and spectral emphasis as acoustic correlates of focal accents in Swedish. Journal of Phonetics, 31, 39-62. (Pubitemid 36338644)
33. Herment-Dujardin, S., & Hirst, D. (2002). Emphasis in English: A perceptual study based on modified synthetic speech. In B. Bel & I. Marlien (Eds.), Speech prosody 2004 (pp. 379-382). Retrieved February 2009, from http://www.isca-speech.org/archive/sp2004
34. Hirst, D., & Di Cristo, A. (1998). A survey of intonation systems. In D. Hirst & A. Di Cristo (Eds.), Intonation systems: A survey of twenty languages (pp. 1-44). Cambridge: Cambridge University Press.
35. House, D., Beskow, J., & Granström, B. (2001). Timing and interaction of visual cues for prominence in audiovisual speech perception. In by P. Dalsgaard, B. Lindberg, H. Benner, & Z. Tan (Eds.), Proceedings of Eurospeech 2001 (pp. 387-390). Retrieved February, 2009, from http://www.isca-speech.org/archive/eurospeech-2001
36. Jensen, C. (2003). Perception of prominence in Standard British English. In Proceedings of the 15th International Congress on Phonetic Sciences (pp. 1815-1819). Barcelona.
37. Jiang, J., Alwan, A., Keating, P.A., Auer, E.T., & Bernstein, L.E. (2002). On the relationship between face movements, tongue movements and speech acoustics. EURASIP Journal on Applied Signal Processing, 11, 1174-1188.
38. Jiang, J., Auer, E.T., Alwan, A., Keating, P.A., & Bernstein, L.E. (2007). Similarity structure in visual speech perception and optical phonetic signals. Perception and Psychophysics, 69, 1070-1083.
39. Keating, P.A., Cho, T., Baroni, M., Mattys, S., Bernstein, L.E., Chaney, B., ET AL. (2000). Articulation of word and sentence stress [abstract]. Journal of the Acoustical Society of America, 108, 2466.
40. Kendon, A. (1978). Differential perception and attentional frame: Two problems for investigation. Semiotica, 24, 305-315.
41. Kochanski, G., Grabe, E., Coleman, J., & Rosner, B. (2005). Loudness predicts prominence: Fundamental frequency lends little. Journal of the Acoustical Society of America, 118, 1038-1054. (Pubitemid 41129237)
42. Krahmer, E., Ruttkay, Z., Swerts, M., & Wesselink, W. (2002). Pitch, eyebrows and the perception of focus. In B. Bel & I. Marlien (Eds.), Speech prosody 2004 (pp. 443-446). Retrieved February 2009, from http://www.isca- speech.org/archive/sp2004
43. Krahmer, E., & Swerts, M. (2006). Hearing and seeing beats. In R. Hoffmann & H. Mixdorff (Eds.), Speech prosody 2006. Retrieved February 2009, from http://www.iscaspeech.org/archive/sp2006
44. Ladd, D.R. (1996). Intonational phonology. Cambridge: Cambridge University Press.
45. Lansing, C.R., & McConkie, G.W. (1999). Attention to facial regions in segmental and prosodic visual speech perception tasks. Journal of Speech, Language, and Hearing Research, 42, 526-539. (Pubitemid 29287933)
46. Lehiste, I. (1970). Suprasegmentals. Cambridge, MA: The MIT Press.
47. Massaro, D.W., & Beskow, J. (2002). Multimodal speech perception: A paradigm for speech science. In B. Granström, D. House, & I. Karlsson (Eds.), Multimodality in language and speech systems (pp. 45-71). Dordrecht: Kluwer Academic Publishers.
48. Munhall, K.G., Jones, J.A., Callan, D.E., Kuratate, T., & Vatikiotis-Bateson, E. (2004). Visual prosody and speech intelligibility: Head movement improves auditory speech perception. Psychological Science, 15, 133-138.
49. Pentland, A.P., & Darell, T. (1994). Visual perception of human bodies and faces for multi-modal interfaces. ICSLP 94, Yokohama, 543-546.
50. Rietveld, T., & Gussenhoven, C. (1985). On the relation between pitch excursion size and prominence. Journal of Phonetics, 13, 299-308.
51. Risberg, A., & Lubker, J. (1978). Prosody and speechreading. Speech Transmission Laboratory-Quarterly Progress Report, Status Report, 4, 1-16.
52. Silverman, K., Beckman, M., Pitrelli, J., Ostendorf, M., Wightman, C., Price, P., ET AL. (1992). ToBI: A standard for labeling English prosody. Proceedings of the 1992 International Conference on Spoken Language Processing, 2: 867-870.
53. Sluijter, A.M.C., van Heuven, V.J., & Pacilly, J.J. (1997). Spectral balance as a cue in the perception of linguistic stress. Journal of the Acoustical Society of America, 101, 503-513. (Pubitemid 27026662)
54. Srinivasan, R.J., & Massaro, D.W. (2003). Perceiving prosody from the face and voice: Distinguishing statements from echoic questions in English. Language and Speech, 46, 1-22. (Pubitemid 37244581)
55. Swerts, M., & Krahmer, E. (2008). Facial expression and prosodic prominence: Effects of modality and facial area. Journal of Phonetics, 36, 219-238.
56. Thompson, D.M. (1934). On the detection of emphasis in spoken sentences by means of visual, tactual, and visual-tactual cues. Journal of General Psychology, 11, 160-172.
57. van Kuijik, D., & Boves, L. (1999). Acoustic characteristics of lexical stress in continuous telephone speech. Speech Communication, 27, 95-111.
58. Vatikiotis-Bateson, E., Eigsti, I.-M., Yano, S., & Munhall, K.G. (1998). Eye movement of perceivers during audiovisual speech perception. Perception and Psychophysics, 60, 926-940. (Pubitemid 128462142)
59. Yehia, H.C., Kuratate, T., & Vatikiotis-Bateson, E. (2002). Linking facial animation, head motion and speech acoustics. Journal of Phonetics, 30, 555-568.