Gestures and phases: The dynamics of speech-hand communication

Ecological Psychology

Volumn 20, Issue 1, 2008, Pages 32-64

  Treffner, Paul ^a   Peter, Mira ^a   Kleidon, Mark ^b  

^a Metaffordance Consultants ^*

^b UNIVERSITY OF SOUTHERN QUEENSLAND   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 42549144027     PISSN: 10407413     EISSN: None     Source Type: Journal    
DOI: 10.1080/10407410701766643     Document Type: Article

References (79)

1. Abry, C., Lallouache, M.-T., & Cathiard, M.-A. (1996). How can coarticulation models account for speech sensitivity to audio-visual desynchronization? In D. Stork & M. Hennecke (Eds.), Speechrvading by humans and machines: Vol. 150 (pp. 247-256). Berlin: Springer-Verlag.
2. Alibali, M. W., & DiRusso, A. A. (1999). The function of gesture in learning to count: More than keeping track. Cognitive Development, 14, 37-56.
3. Arbib, M. A. (2005). From monkey-like action recognition to human language: An evolutionary framework, for neurolinguistics. Behavioral and Brain Sciences, 28, 105-124.
4. Blake, J., Olshansky, E., Vitale, G., & Macdonald, S. (1997). Are communicative gestures the substrate of language? Evolution of Communication, 1, 261-282.
5. Borghi, A. M. (2004). Object concepts and action: Extracting affordances from objects parts. Acta Psychologica, 115, 69-96.
6. Browman, C.P., & Goldstein, L. (1992). Articulatory phonology: An. overview. Phonetica, 49, 155-180.
7. Buccino, G., Binkfski, F., & Riggio, L. (2004). The mirror neuron system and action recognition. Brain and Language, 89, 370-376.
8. Corballis, M. (2002). From hand to mouth: The origins of language. Princeton, NJ: Princeton University Press.
9. Corballis, M. (2003). Laterality and human speciation. In T. J. Crow (Ed.), Speciation of modern Homo sapiens. Proceedings of the British Academy, 106, 137-152.
10. Fadiga, L., Craighero, L., Buccino, G., & Rizzolatti, G. (2002). Speech listening specifically modulates the excitability of tongue muscles: a TMS study. European Journal of Neumscience, 15, 399-402.
11. Ferrari, P. F., Gallese, V., Rizzolatti, G., & Fogassi, L. (2003). Mirror neurons responding to the observation of ingestive and communicative mouth actions. European Journal of Neumscience, 17, 1703-1714.
12. Floel, A., Ellger, T., Breitenstein C., & Knecht, S. (2003). Language perception activates the hand motor cortex: Implications for motor theories of speech perception. European Journal of Neuroscience, 18, 704-708.
13. Fowler, C. A. (1986). An event approach to the study of speech perception from a direct-realist perspective. Journal of Phonetics, 14, 3-28.
14. Furayama. N. (2002). Prolegomena of a theory of between-person coordination of speech and gesture. International Journal of Human-Computer Studies, 57, 347-374.
15. Gibson, J. J. (1975). Events are perceivable but time is not. In J. T. Fraser & N. Lawrence (Eds.), The study of time II (pp. 295-301). Berlin: Springer-Verlag.
16. Gibson, J. J. (1986). The ecological approach to visual perception. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. (Original work published 1979)
17. Goldin-Meadow, S. (1999). The role of gesture in communication and thinking. Trends in Cognitive Sciences, 3, 419-429.
18. Gullberg, M., & Holmquist, K. (1999). Keeping an eye on gestures: Visual perception of gestures in face-to-face communication. Pragmatics and Cognition, 7, 35-63.
19. Gullberg, M., & Holmquist, K. (2002). Visual attention towards gestures in face-to-face interaction vs. on screen. In I. Wachsmuth & T. Sowa (Eds.), Gesture and sign language based human-computer interaction (pp. 206-214). Berlin: Springer.
20. Haueisen, J., & Knosche, T. R. (2001). Involuntary motor activity in pianists evoked by music perception. Journal of Cognitive Neuroscience, 13, 786-792.
21. Hodges, B. H. (2007). Values define fields: The intentional dynamics of driving, carrying, leading, negotiating, and conversing. Ecological Psychology, 19, 153-178.
22. Iverson, J. M. & Goldin-Meadow, S. (1997). What's communication got to do with it? Gesture in congenitally blind children. Developmental Psychology, 33, 453-467.
23. Kelly, S. D., Kravitz, C., & Hopkins, M. (2004). Neural correlates of bimodal speech and gesture comprehension. Brain and Language, 89, 253-260.
24. Kelso, J. A. S. (1995). Dynamic patterns: The self-organization of brain and behaviour. Boston: MIT Press.
25. Kendon, A. (1972). Some relationships between body motion and speech. In A. Siegman & B. Pope (Eds.), Studies in dyadic communication. New York: Pergamon Press.
26. Kendon, A. (1974). Movement coordination in social interaction: Some examples described. In S. Weitz (Ed.), Nonverbal communication. New York: Oxford University Press.
27. Kendon, A. (2004). Review of Susan Goldin-Meadow (2003), Hearing gesture: How our hands help us think. Gesture, 4, 91-107.
28. Kimura, D. (1973). Manual activity during speaking-I. Right handers. Neuropsychologia, 11, 45-50.
29. Köhler, E., Keysers, C., Umilta, M. A., Fogassi, L., Gállese, V., & Rizzolatti, G. (2002). Hearing sounds, understanding actions: Action representation in mirror neurons. Science, 297, 846-848.
30. Krauss, R. M. (1998). Why do we gesture when we speak? Current Directions in Psychological Science, 7, 54-59.
31. Lausberg, H., & Kita, S. (2003). The content of the message influences the hand choice in co-speech gestures and in gesturing without speaking. Brain and Language, 86, 57-69.
32. Marsh, K. L., Richardson, M. J., Barron, R. M., & Schmidt, R. C. (2006). Contrasting approaches to perceiving and acting with others. Ecological Psychology, 18, 1-38.
33. Mayberry, R., & Jaques, J. (2000). Gesture production during stuttered speech: Insights into the nature of gesture-speech integration. In D. McNeill (Ed.), Gesture and language (pp. 199-213). Cambridge, UK: Cambridge University Press.
34. Mayberry, R. I., & Nicholadis, E. (2000). Gesture reflects language development: Evidence from bilingual children. Current Directions in Psychological Science, 9, 192-196.
35. Mayberry, R. I., & Shenker, R. C. (1997). Gesture mirrors speech motor control in stutterers. In W. Hulstijn, H. Peters, & P. van Lieshout (Eds.), Speech motor production and fluency disorders (pp. 183-190). Amsterdam: Elsevier Science.
36. McClave E. (1994). Gestural beats: The rhythm hypothesis. Journal of Psycholinguistic. Research, 23, 45-66.
37. McClave E. (1997). Pitch and manual gestures. Journal of Psycholinguistic Research, 27, 69-89.
38. McNeill, D. (1985). So you think gestures are nonverbal? Psychological Review, 92, 350-371.
39. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.
40. McNeill, D. (Ed.). (2000). Language and gesture. Cambridge, UK: Cambridge University Press.
41. McNeill, D. (2005). Gesture-first, but no gestures? Behavioral and Brain Sciences, 28, 138-139.
42. Meister, I. G., Boroojerdi, B., Foltys, H., Sparing, R., Huber, W., & Topper, R. (2003). Motor cortext hand area and speech: Implications for the development of language. Neuropsychologia, 41, 401-406.
43. Millikan, R. G. (1984). Language, thought and other biological categories. Cambridge, MA: MIT Press.
44. Morrel-Samuels, P., & Krauss, R. M. (1992). Word familiarity predicts the temporal asynchrony of hand gestures and speech. Journal of Experimental Psychology: Learning, Memory and Cognition., 18, 615-623.
45. Munhall, K. G., Gribble, P., Sacco, L., & Ward, M. (1996). Temporal constraints on the McGurk effect. Perception & Psychophysics, 58, 351-362.
46. Munhall, K. G., Jones, J. A., Callan, D. E., Kuratate, T., & Vatikiotis-Bateson, E. (2004). Visual prosody and speech intelligibility: Head movement improves auditory speech perception. Psychological Science, 15, 133-137.
47. Munhall, K. G., &Tohkura, Y. (1998). Audiovisual, gating and the time course of speech, perception. Journal of the Acoustical Society of America, 104, 530-539.
48. Nobe, S. (2000). Where do most spontaneous representational gestures actually occur with respect to speech? In D. McNeill (Ed.), Language and gesture (pp. 186-198). Cambridge, UK: Cambridge University Press.
49. Pagano, C. C., & Turvey, M. T. (1995). The inertia tensor as a basis for the perception of limb orientation. Journal of Experimental Psychology: Human Perception and Performance, 21, 1070-1087.
50. Pettito, L. A., Holowka, S., Sergio, L. E., Levy, B., & Ostry, D. J. (2004). Baby hands that move to the rhythm of language: Hearing babies acquiring sign languages babble silently on the hands. Cognition, 93, 43-73.
51. Reed, E. S. (1996). Encountering the world: Towards an ecological psychology. New York: Oxford University Press.
52. Rogers, W. (1978). The contribution of kinesic illustrators towards the comprehension of verbal behaviour within utterances. Human Communication Research, 5, 54-62.
53. Saltzman, E. L., & Byrd, D. (2000). Task-dynamics of gestural timing: Phase windows and multifrequency rhythms. Human Movement Science, 19, 499-526.
54. Santi, A., Servos, P., Vatikiotis-Bateson, E., Kuratate, T., & Munhall, K. (2003). Perceiving biological motion: Dissociating visible speech from walking. Journal of Cognitive Neuroscience, 15, 800-809.
55. Schmidt, R. C. (2007). Scaffolds for social meaning. Ecological Psychology, 19, 137-151.
56. Schmidt, R. C., & Turvey, M. T. (1994). Phase-entrainment dynamics of visually coupled rhythmic movements. Biological Cybernetics, 70, 369-376.
57. Shaw, R. E. (2001). Processes, acts, and experiences: Three stances on the problem of intentionality. Ecological Psychology, 13, 275-314.
58. Shaw, R. E. (2003). The agent-environment interface: Simon's indirect or Gibson's direct coupling? Ecological Psychology, 15, 37-106.
59. Shaw, R. E., Turvey, M. T., & Mace, W. M. (1981). Ecological psychology: The consequences of a commitment to realism. In W Weimer & D. Palermo (Eds.), Cognition and the symbolic processes: Vol. II (pp. 159-226). Hillsdale, NJ: Lawrence Erlbaum. Associates, Inc.
60. Singer, M. A., & Goldin-Meadow, S. (2005). Children, learn when their teacher's gestures and speech differ. Psychological Science, 16, 85-89.
61. Sousa-Poza, J. F., Rohrberg, R., & Mercure, A. (1979). Effects of type of information (abstract-concrete) and field dependence on asymmetry of hand movements during speech. Perceptual and Motor Skills, 48, 1323-1330.
62. Stanney, K. (2002). Handbook of virtual environments. Hillsdale, NJ: Lawrence Erlbaum. Associates, Inc.
63. Taira, M., Mine S., Georgopoulos, A. P., Murata, A., & Sakata, H. (1990). Parietal cortex neurons of the monkey related to the visual guidance of hand movement. Experimental Brain Research, 83, 29-36.
64. Treffner, P. J. (1999a). Resonance constraints on between-person polyrhythms. In M. A. Grealy & J. A. Thomson (Eds.), Studies inperception andaction V (pp. 165-169). Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
65. Treffner, P. J. (1999b). The common structure of concepts is the affordance in the ecology. Commentary on Ruth Millikan's "A common structure of individuals, stuffs, and real kinds." Behavioral and Bmin Sciences, 22, 729-733.
66. Treffner, P. J., & Barrett, R. (2004). Hands-free mobile phone speech while driving degrades coordination and control. Transportation Research, Part F: Traffic Psychology and Behaviour, 7, 229-246.
67. Treffner, P. J., Barrett, R., & Petersen, A. J. (2002). Stability and skill in. driving. Human Movement Science, 21, 749-784.
68. Treffner, P. J., & Kelso, J. A. S. (1999). Dynamic encounters: Long-memory during functional stabilization. Ecological Psychology, 11, 103-137.
69. Treffner, P. J., & Peter, M. (2002). Intentional and attentional. dynamics of speech-hand coordination. Human Movement Science, 21, 641-697.
70. Treffner, P. J., & Turvey, M. T. (1993). Resonance constraints on rhythmic movement. Journal of Experimental Psychology: Human Perception and Performance, 19, 1221-1237.
71. Treffner, P. J., & Turvey, M. T. (1995). Handedness and the asymmetric dynamics of bimanual rhythmic coordination. Journal of Experimental Psychology: Human Perception, and Performance, 21, 318-333.
72. Treffner, P. J., & Turvey, M. T. (1996). Symmetry, broken symmetry, and the dynamics of bimanual coordination. Experimental Brain Research, 107, 463-478.
73. Turvey, M. T., Shaw, R. E., Reed, E., & Mace, W. (1981). Ecological, laws of perceiving and acting: In reply to Fodor and Pylyshyn (1981). Cognition, 9, 237-304.
74. van Lieshout, P. H. H. M. (2004). Dynamical systems theory and its application in speech. In B. Maassen, R. Kent, H. Peters, P. van Lieshout, & W Hulstijn (Eds.), Speech motor control in normal and disordered speech (pp. 313-356). Oxford, UK: Oxford University Press.
75. van Lieshout, P. H. H. M.., Hulstijn, W., & Peters, R M. (1996). From, planning to articulation in speech production: What differentiates a person who stutters from, a person who does not stutter? Journal of Speech and Hearing Research, 39, 546-564.
76. Watkins, K. E., Strafella, A. P., & Paus, T. (2003). Seeing and hearing speech excites the motor system, involved in speech production. Neumpsychologia, 41, 989-994.
77. Wilson, A. D., Bingham, G. P., & Craig, J. C. (2003). Proprioceptive perception of phase variability. Journal of Experimental Psychology: Human Perception and Performance, 29, 1179-1190.
78. Yehia, H. C., Kuratate, T., & Vatikiotis-Bateson, E. (2002). Linking facial animation, head motion, and speech acoustics. Journal of Phonetics, 30, 555-568.
79. Yehia, H., Rubin, P., & Vatikiotis-Bateson, E. (1998). Quantitative association of vocal-tract and facial behaviour. Speech Communication, 26, 23-43.