Feature integration across multimodal perception and action: A review

Multisensory Research

Volumn 26, Issue 1-2, 2013, Pages 143-157

  Hommel, Bernhard ^a   Zmigrod, Sharon ^b  

^a LEIDEN UNIVERSITY MEDICAL CENTER   (Netherlands)

^b LEIDEN UNIVERSITY   (Netherlands)

Author keywords

event files; feature integration; intermodal integration; Multimodal perception; object files; perception and action; the binding problem

Indexed keywords

ANIMAL; BRAIN; HUMAN; MOVEMENT (PHYSIOLOGY); PERCEPTION; PHYSIOLOGY; PSYCHOMOTOR PERFORMANCE; REVIEW; TOUCH; VISION;

ANIMALS; BRAIN; HUMANS; MOVEMENT; PERCEPTION; PSYCHOMOTOR PERFORMANCE; TOUCH PERCEPTION; VISUAL PERCEPTION;

MLCS; MLOWN;

EID: 84879134809     PISSN: 22134794     EISSN: 22134808     Source Type: Journal    
DOI: 10.1163/22134808-00002390     Document Type: Article

References (74)

1. Barlow, H. B. (1972). Single units and sensation: a neuron doctrine for perceptual psychology? Perception 1, 371-394.
2. Bertelson, P., Vroomen, J., de Gelde, B. and Driver, J. (2000). The ventriloquist effect does not depend on the direction of deliberate visual attention, Percept. Psychophys. 62, 321-332.
3. Cinel, C., Humphreys, G. W. and Poli, R. (2002). Cross-modal illusory conjunctions between vision and touch, J. Exper. Psychol.: Human Percept. Perform. 28, 1243-1266.
4. Colzato, L. S. and Hommel, B. (2008). Cannabis, cocaine, and visuomotor integration: evidence for a role of dopamine D1 receptors in binding perception and action, Neuropsychologia 46, 1570-1575.
5. Colzato, L. S., Raffone, A. and Hommel, B. (2006a). What do we learn from binding features? Evidence for multilevel feature integration, J. Exper. Psychol.: Human Percept. Perform. 32, 705-716.
6. Colzato, L. S., van Wouwe, N. C., Lavender, T. J. and Hommel, B. (2006b). Intelligence and cognitive flexibility: fluid intelligence correlates with feature 'unbinding' across perception and action, Psychonom. Bull. Rev. 13, 1043-1048.
7. Colzato, L. S., vanWouwe, N. C. and Hommel, B. (2007). Feature binding and affect: emotional modulation of visuo-motor integration, Neuropsychologia 45, 440-446.
8. Corbett, B. A., Constantine, L. J., Hendren, R., Rocke, D. and Ozonoff, S. (2009). Examining executive functioning in children with autism spectrum disorder, attention deficit hyperactivity disorder and typical development, Psychiatry Res. 166, 210-222.
9. deCharms, R. C. and Merzenich, M. M. (1996). Primary cortical representation of sounds by the coordination of action-potential timing, Nature 381, 610-613.
10. Duncan, J., Seitz, R. J., Kolodny, J., Bor, D., Herzog, H., Ahmed, A., Newell, F. N. and Emslie, H. (2000). A neural basis for general intelligence, Science 289, 457-460.
11. Dutzi, I. B. and Hommel, B. (2009). The microgenesis of action-effect binding, Psycholog. Res. 73, 425-435.
12. Engel, A. K. and Singer, W. (2001). Temporal binding and the neural correlates of sensory awareness, Trends Cognit. Sci. 5, 16-25.
13. Engel, A. K., Konig, P. and Singer,W. (1991). Direct physiological evidence for scene segmentation by temporal coding, Proc. Nat. Acad. Sci. USA 88, 9136-9140.
14. Ernst, M. O. and Bülthoff, H. H. (2004). Merging the senses into a robust percept, Trends Cognit. Sci. 8, 162-169.
15. Evans, K. K. and Treisman, A. (2010). Natural cross-modal mappings between visual and auditory features, J. Vision 10, 1-12.
16. Found, A. and Müller, H. J. (1996). Searching for unknown feature targets on more than one dimension: investigating a 'dimension weighting' account, Percept. Psychophys. 58, 88-101.
17. Gao, T. and Scholl, B. J. (2010). Are objects required for object files? Roles of segmentation and spatiotemporal continuity in computing object persistence, Visual Cognition 18, 82-109.
18. Hall, M. D., Pastore, R. E., Acker, B. E. and Huang, W. (2000). Evidence for auditory feature integration with spatially distributed items, Percept. Psychophys. 6, 1243-1257.
19. Hill, E. L. (2004). Evaluating the theory of executive dysfunction in autism, Developmental Review 24, 189-233.
20. Hommel, B. (1998). Event files: evidences for automatic integration of stimulus-response episodes, Visual Cognition 5, 183-216.
21. Hommel, B. (2004). Event files: feature binding in and across perception and action, Trends Cognit. Sci. 8, 494-500.
22. Hommel, B. (2005). How much attention does an event file need? J. Exper. Psychol.: Human Percept. Perform. 31, 1067-1082.
23. Hommel, B. (2007). Feature integration across perception and action: event file affect response choice, Psycholog. Res. 7, 42-63.
24. Hommel, B. (2009). Action control according to TEC (theory of event coding), Psycholog. Res. 73, 512-526.
25. Hommel, B. and Colzato, L. S. (2004). Visual attention and the temporal dynamics of feature integration, Visual Cognition 11, 483-521.
26. Hommel, B. and Colzato, L. S. (2009). When an object is more than a binding of its features: evidence for two mechanisms of visual feature integration, Visual Cognition 17, 120-140.
27. Hommel, B., Müsseler, J., Aschersleben, G. and Prinz,W. (2001). The Theory of Event Coding (TEC): a framework for perception and action planning, Behav. Brain Sci. 24, 849-937.
28. Hommel, B., Kray, J. and Lindenberger, U. (2011). Feature integration across the lifespan: stickier stimulus-response bindings in children and older adults, Frontiers Psychol. 2, 268.
29. Hötting, K. and Röder, B. (2004). Hearing cheats touch, but less in congenitally blind than in sighted individuals, Psycholog. Sci. 15, 60-64.
30. Jeannerod, M. (1997). The Cognitive Neuroscience of Action. Blackwell Publishers, Oxford, UK.
31. Joliot, M., Ribary, U. and Llinás, R. (1994). Human oscillatory brain activity near 40 Hz coexists with cognitive, Proc. Nat. Acad. Sci. USA 91, 11748-11751.
32. Jordan, K. E., Clark, K. and Mitroff, S. R. (2010). See an object, hear an object file: object correspondence transcends sensory modality, J. Vision 18, 492-503.
33. Kahneman, D., Treisman, A. and Gibbs, B. J. (1992). The reviewing of object files: objectspecific integration of information, Cognit. Psychol. 24, 175-219.
34. Keele, S.W., Cohen, A. and Ivry, R. (1990). Motor programs: concepts and issues, in: Attention and Performance: Motor Representation and Control, M. Jeannerod (Ed.), Vol. 13, pp. 77-110. Erlbaum, Hillsdale, NJ, USA.
35. Keizer, A. W., Nieuwenhuis, S., Colzato, L. S., Theeuwisse, W., Rombouts, S. A. R. B. and Hommel, B. (2008). When moving faces activate the house area: an fMRI study of object file retrieval, Behav. Brain Functions 4, 50.
36. Keizer, A. W., Verment, R. and Hommel, B. (2010). Enhancing cognitive control through neurofeedback: a role of gamma-band activity in managing episodic retrieval, Neuroimage 49, 3404-3413.
37. Kühn, S., Keizer, A., Colzato, L. S., Rombouts, S. A. R. B. and Hommel, B. (2011). The neural underpinnings of event-file management: evidence for stimulus-induced activation of, and competition among stimulus-response bindings, J. Cognit. Neurosci. 23, 896-904.
38. Kunde, W. and Kiesel, A. (2006). See what you've done! Active touch affects the number of perceived visual objects, Psychonom. Bull. Rev. 13, 304-309.
39. Lee, C. C. and Winer, J. A. (2005). Principles governing auditory cortex connections, Cerebral Cortex 15, 1804-1814.
40. Lewald, J., Ehrenstein, W. H. and Guski, R. (2001). Spatio-temporal constraints for auditory-visual integration, Behav. Brain Res. 121, 69-79.
41. Lewkowicz, D. J. (1996). Perception of auditory-visual temporal synchrony in human infants, J. Exper. Psychol.: Human Percept. Perform. 22, 1094-1106.
42. McGinnis, E. M. and Keil, A. (2011). Selective processing of multiple features in the human brain: effects of feature type and salience, PLoS ONE 6, e16824. DOI:10.1371/journal.pone.0016824.
43. McGurk, H. and MacDonald, J. (1976). Hearing lips and seeing voices, Nature 264, 746-748.
44. Mitroff, S. R. and Alvarez, G. A. (2007). Space and time, not surface features, guide object persistence, Psychonom. Bull. Rev. 14, 1199-1204.
45. Memelink, J. and Hommel, B. (in press). Intentional weighting: a basic principle in cognitive control, Psycholog. Res.; published online in 2012: DOI: 10.1007/s00426-012-0435-y.
46. Mondor, T. A., Hurlburt, J. and Thorne, L. (2003). Categorizing sounds by pitch: effects of stimulus similarity and response repetition, Percept. Psychophys. 65, 107-114.
47. Murthy, V. N. and Fetz, E. E. (1992). Coherent 25-to 35-Hz oscillations in the sensorimotor cortex of awake behaving monkeys, Proc. Nat. Acad. Sci. USA 89, 5670-5674.
48. Murthy, V. N. and Fetz, E. E. (1996). Oscillatory activity in sensorimotor cortex of awake monkeys: synchronization of local field potentials and relation to behavior, J. Neurophysiol. 76, 3949-3967.
49. Nicolelis, M. A., Baccala, L. A., Lin, R. C. and Chapin, J. K. (1995). Sensorimotor encoding by synchronous neural ensemble activity at multiple levels of the somatosensory system, Science 268, 1353-1358.
50. Noë, A. (2004). Action in Perception. MIT Press, Cambridge, MA, USA.
51. Noles, N. S., Scholl, B. J. and Mitroff, S. R. (2005). The persistence of object file representations, Percept. Psychophys. 67, 324-334.
52. Raffone, A. and Wolters, G. (2001). A cortical mechanism for binding in visual working memory, J. Cognit. Neurosci. 13, 766-785.
53. Roelfsema, P. R., Engel, A. K., Konig, P. and Singer, W. (1997). Visuomotor integration is associated with zero time-lag synchronization among cortical areas, Nature 385, 157-161.
54. Shams, L., Kamitani, Y. and Shimojo, S. (2000). What you see is what you hear, Nature 408, 788.
55. Singer, W. (1994). A new job for the thalamus, Nature 369, 444-445.
56. Spapé, M. and Hommel, B. (2010). Actions travel with their objects: evidence for dynamic event files, Psycholog. Res. 74, 50-58.
57. Takegata, R., Brattico, E., Tervaniemi, M., Varyagina, O., Näätänen, R. and Winkler, I. (2005). Preattentive representation of feature conjunctions for concurrent spatially distributed audition objects, Cognit. Brain Res. 25, 169-179.
58. Tallon-Baudry, C. and Bertrand, O. (1999). Oscillatory gamma activity in humans and its role in object representation, Trends Cognit. Sci. 3, 151-162.
59. Treisman, A. (1996). The binding problem, Curr. Opinion Neurobiol. 6, 171-178.
60. Treisman, A. M. and Gelade, G. (1980). A feature-integration theory of attention, Cognit. Psychol. 12, 97-136.
61. van Dam, W. O. and Hommel, B. (2010). How object-specific are object files? Evidence for integration by location, J. Exper. Psychol.: Human Percept. Perform. 36, 1184-1192.
62. VanRullen, R. (2009). Binding hardwired versus on-demand feature conjunction, Visual Cognition 17, 103-119.
63. van Steenbergen, H., Band, G. P. H. and Hommel, B. (2009). Reward counteracts conflict adaptation: evidence for a role of affect in executive control, Psycholog. Sci. 20, 1473-1477.
64. van Wassenhove, V., Grant, K. W. and Poeppel, D. (2007). Temporal window of integration in bimodal speech, Neuropsychologia 45, 598-607.
65. Violentyev, A., Shimojo, S. and Shams, L. (2005). Touch-induced visual illusion, NeuroReport 16, 1107-1110.
66. von der Malsburg, C. (1999). The what and why of binding: the modeler's perspective, Neuron 24, 95-104.
67. von Stein, A., Rappelsberger, P., Sarnthein, J. and Petsche, H. (1999). Synchronization between temporal and parietal cortex during multimodal object processing in man, Cerbral Cortex 9, 137-150.
68. Vroomen, J., Bertelson, P. and de Gelder, B. (2001). The ventriloquist effect does not depend on the direction of automatic visual attention, Percept. Psychophys. 63, 651-659.
69. Zeki, S. (1993). A Vision of the Brain. Blackwell Scientific, Oxford, UK.
70. Zmigrod, S. and Hommel, B. (2009). Auditory event files: integrating auditory perception and action planning, Attention Percept. Psychophys. 71, 352-362.
71. Zmigrod, S. and Hommel, B. (2010). Temporal dynamics of unimodal and multimodal feature binding, Attention Percept. Psychophys. 72, 142-152.
72. Zmigrod, S. and Hommel, B. (2011). The relationship between feature binding and consciousness: evidence from asynchronous multi-modal stimuli, Conscious. Cognit. 20, 586-593.
73. Zmigrod, S., Spapé, M. and Hommel, B. (2009). Intermodal event files: integrating features across vision, audition, taction, and action, Psycholog. Res. 73, 674-684.
74. Zmigrod, S., de Sonneville, L. M., Colzato, L. S., Swaab, H. and Hommel, B. (in press). Cognitive control of feature bindings: evidence from children with autistic spectrum disorder, Psycholog. Res.; published online in 2011: DOI: 10.1007/s00426-011-0399-3.