Are gesture and prosodic prominences always coordinated? Evidence from perception and production

Proceedings of the International Conference on Speech Prosody

Volumn , Issue , 2014, Pages 222-226

  Esteve Gibert, Núria ^a   Pons, Ferran ^b   Bosch, Laura ^b   Prieto, Pilar ^a,c  

^a UNIVERSITAT POMPEU FABRA   (Spain)

^b UNIVERSITY OF BARCELONA   (Spain)

^c INSTITUCIÓ CATALANA DE RECERCA I ESTUDIS AVANÇATS ICREA   (Spain)

Author keywords

Audiovisual prosody; Gesture speech synchronization; Multimodal prominence

Indexed keywords

ALIGNMENT; SPEECH;

AUDIOVISUAL PROSODY; GESTURE-SPEECH; MULTI-MODAL; POINTING GESTURES; SEMANTIC CONTENT; SEMANTIC INFORMATION; SPEECH MODALITY; TEMPORAL COORDINATION;

SEMANTICS;

EID: 84904629885     PISSN: 23332042     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.21437/speechprosody.2014-32     Document Type: Conference Paper

References (13)

1. Birdwhistell, R. L. "Introduction to kinesics: An annotated system for analysis of body motion and gesture". Washington, DC: Department of State, Foreign Service Institute, 1952.
2. Kendon, A. "Gesticulation and speech: Two aspects of the process of utterance". In M. R. Key (Ed.), The relationship of verbal and nonverbal communication (pp. 207-227). The Hague, the Netherlands: Mouton, 1980.
3. McNeill, D. "Hand and Mind: What Gestures Reveal About Thought". The Chicago University Press, Chicago, 1992.
4. Wagner, P., Malisz, Z., and Kopp, S. "Gesture and speech in interaction: An overview". Sp. Comm 57:209-232, 2014.
5. Loehr, D. "Aspects of rhythm in gesture and speech". Gesture 7: 179-214, 2007.
6. Rusiewicz, H., Shaiman, S., Iverson, J., Szuminsky, N., Smith, A., and van Lieshout, P. "Effects of Prosody and Position on the Timing of Deictic Gestures". J. Speech Lang. Hear. Res. 56(2):458-470, 2013.
7. De Ruiter, J. P. "Gesture and speech production". Doctoral dissertation. Katholieke Universiteit, Nijmegen, 1998.
8. Esteve-Gibert, N. and Prieto, P. "Prosodic structure shapes the temporal realization of intonation and manual gesture movements". J. Speech Lang. Hear. Res. 56(3): 850-864, 2013.
9. Bergmann, K., Aksu, V., and Kopp, S. "The Relation of Speech and Gestures: Temporal Synchrony Follows Semantic Synchrony" in Proceedings of the 2nd Workshop on Gesture and Speech in Interaction. Bielefeld, Germany, 2011.
10. Vatakis, A., and Spence, C. "Audiovisual temporal integration for complex speech, object-action, animal call, and musical stimuli". In M. J. Naumer & J. Kaiser (Eds.), Multisensory Object Perception in the Primate Brain. New York, Springer, 2010.
11. Jesse, A., and Mitterer, H. "Pointing gestures do not influence the perception of lexical stress" in Proceedings of the 12th Annual Conference of the International Speech Communication Association (Interspeech 2011): 2445-2448, 2011.
12. Treffner, P., Peter, M., and Kleidon, M. "Gestures and phases: the dynamics of speech-hand communication". Ecol. Psychol. 20:32-64, 2008.
13. Leonard, T. and Cummins, F. "The temporal relation between beat gestures and speech". Lang. Cognitive Proc. 20(1):32-64, 2008.