Spatial orienting in complex audiovisual environments

Human Brain Mapping

Volumn 35, Issue 4, 2014, Pages 1597-1614

  Nardo, Davide ^a   Santangelo, Valerio ^a,b   Macaluso, Emiliano ^a  

^a IRCCS FONDAZIONE SANTA LUCIA   (Italy)

^b UNIVERSITY OF PERUGIA   (Italy)

Author keywords

Attention; Auditory; Ecological; Eye movements; Multisensory; Orienting; Posterior parietal cortex; Saliency; Space; Visual

Indexed keywords

ADULT; ARTICLE; AUDIOVISUAL EQUIPMENT; AUDITORY SYSTEM; BRAIN MAPPING; CONTROLLED STUDY; EYE MOVEMENT; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; FUNCTIONAL NEUROIMAGING; GAZE; HUMAN; HUMAN EXPERIMENT; LIFE EVENT; MALE; NORMAL HUMAN; POSTERIOR PARIETAL CORTEX; PRIORITY JOURNAL; SENSORY SYSTEM; SOUND; SPATIAL ORIENTATION; STIMULUS; TEMPORAL CORTEX; VIDEORECORDING; VISION; VISUAL CORTEX; VISUAL STIMULATION; YOUNG ADULT; AUDITORY STIMULATION; BIOLOGICAL MODEL; BRAIN; COMPUTER SIMULATION; DEPTH PERCEPTION; EVOKED RESPONSE; HEARING; NEUROPSYCHOLOGICAL TEST; NUCLEAR MAGNETIC RESONANCE IMAGING; OCULOGRAPHY; ORIENTATION; PHOTOSTIMULATION; PHYSIOLOGY;

ACOUSTIC STIMULATION; ADULT; AUDITORY PERCEPTION; BRAIN; BRAIN MAPPING; COMPUTER SIMULATION; EVOKED POTENTIALS; EYE MOVEMENT MEASUREMENTS; EYE MOVEMENTS; FEMALE; HUMANS; MAGNETIC RESONANCE IMAGING; MALE; MODELS, NEUROLOGICAL; NEUROPSYCHOLOGICAL TESTS; ORIENTATION; PHOTIC STIMULATION; SPACE PERCEPTION; VIDEO RECORDING; VISUAL PERCEPTION; YOUNG ADULT;

EID: 84896393752     PISSN: 10659471     EISSN: 10970193     Source Type: Journal    
DOI: 10.1002/hbm.22276     Document Type: Article

References (90)

1. Ahveninen J, Hämäläinen M, Jääskeläinen IP, Ahlfors SP, Huang S, Lin FH, Raij T, Sams M, Vasios CE, Belliveau JW (2011): Attention-driven auditory cortex short-term plasticity helps segregate relevant sounds from noise. Proc Natl Acad Sci USA 108:4182-4187.
2. Amlôt R, Walker R, Driver J, Spence C (2003): Multimodal visual-somatosensory integration in saccade generation. Neuropsychologia 41:1-15.
3. Andersen RA, Essick GK, andSiegel RM (1985): Encoding of spatial location by posterior parietal neurons. Science 230:456-458.
4. Arndt PA, Colonius H (2003): Two stages in crossmodal saccadic integration: Evidence from a visual-auditory focused attention task. Exp Brain Res 150:417-426.
5. Avillac M, Ben Hamed S, Duhamel JR (2007): Multisensory integration in the ventral intraparietal area of the macaque monkey. J Neurosci 27:1922-1932.
6. Balan PF, Gottlieb J (2006): Integration of exogenous input into a dynamic saliency map revealed by perturbing attention. J Neurosci 26:9239-9249.
7. Beck DM, Kastner S (2007): Stimulus similarity modulates competitive interactions in human visual cortex. J Vis 7:19.1-12.
8. Bisley JW, Goldberg ME (2003): Neuronal activity in the lateral intraparietal area and spatial attention. Science 299:81-86.
9. Bogler C, Bode S, Haynes JD (2011): Decoding successive computational stages of salience processing. Curr Biol 21:1667-1671.
10. Bordier C, Puja F, Macaluso E (2013): Sensory processing during viewing of cinematographic material: Computational modeling and functional neuroimaging. Neuroimage 67:213-226.
11. Bremmer F, Ilg UJ, Thiele A, Distler C, Hoffmann KP (1997): Eye position effects in monkey cortex. I. Visual and pursuit-related activity in extrastriate areas MT and MST. J Neurophysiol 77:944-961.
12. Bremmer F, Schlack A, Shah NJ, Zafiris O, Kubischik M, Hoffmann K, Zilles K, Fink GR (2001): Polymodal motion processing in posterior parietal and premotor cortex: A human fMRI study strongly implies equivalencies between humans and monkeys. Neuron 29:287-296.
13. Brotchie PR, Andersen RA, Snyder LH, Goodman SJ (1995): Head position signals used by parietal neurons to encode locations of visual stimuli. Nature 375:232-235.
14. Buschman TJ, Miller EK (2007): Top-down versus bottom-up control of attention in the prefrontal and posterior parietal cortices. Science 315:1860-1862.
15. Carmi R, Itti L (2006): Visual causes versus correlates of attentional selection in dynamic scenes. Vision Res 46:4333-4345.
16. Chait M, de Cheveigné A, Poeppel D, Simon JZ (2010): Neural dynamics of attending and ignoring in human auditory cortex. Neuropsychologia 48:3262-3271.
17. Colby CL, Goldberg ME (1999): Space and attention in parietal cortex. Annu Rev Neurosci 22:319-349.
18. Constantinidis C (2006): Posterior parietal mechanisms of visual attention. Rev Neurosci 17:415-427.
19. Constantinidis C, Steinmetz MA. (2005): Posterior parietal cortex automatically encodes the location of salient stimuli. J Neurosci 25:233-238.
20. Corbetta M, Miezin FM, Dobmeyer S, Shulman GL, Petersen SE (1990): Attentional modulation of neural processing of shape, color, and velocity in humans. Science 248:1556-1559.
21. Corbetta M, Akbudak E, Conturo TE, Snyder AZ, Ollinger JM, Drury HA, Linenweber MR, Petersen SE, Raichle ME, Van Essen DC, Shulman GL (1998): A common network of functional areas for attention and eye movements. Neuron 21:761-773.
22. Corneil BD, Munoz DP (1996): The influence of auditory and visual distractors on human orienting gaze shifts. J Neurosci 16:8193-8207.
23. Corneil BD, Van Wanrooij M, Munoz DP, Van Opstal AJ (2002): Auditory-visual interactions subserving goal-directed saccades in a complex scene. J Neurophysiol 88:438-454.
24. Desimone R, Duncan J (1995): Neural mechanisms of selective visual attention. Annu Rev Neurosci 18:193-222.
25. Duhamel JR, Bremmer F, Ben Hamed S, Graf W (1997): Spatial invariance of visual receptive fields in parietal cortex neurons. Nature 389:845-848.
26. Duvernoy HM.1991. The Human Brain: Surface, Three-Dimensional Sectional Anatomy and MRI. Wien:Springer.
27. Einhäuser W, Rutishauser U, Koch C (2008): Task-demands can immediately reverse the effects of sensory-driven salience in complex visual stimuli. J Vis 8:2.1-19.
28. Elazary L, Itti L (2008): Interesting objects are visually salient. J. Vis 3:1-15.
29. Ernst MO, Banks MS (2002): Humans integrate visual and haptic information in a statistically optimal fashion. Nature 415:429-433.
30. Farah MJ, Wong AB, Monheit MA, Morrow LA (1989): Parietal lobe mechanisms of spatial attention: Modality-specific or supramodal?Neuropsychologia 27:461-470.
31. Frens MA, Van Opstal AJ, Van der Willigen RF (1995): Spatial and temporal factors determine auditory-visual interactions in human saccadic eye movements. Percept Psychophys 57:802-816.
32. Friston KJ, Glaser DE, Henson RN, Kiebel S, Phillips C, Ashburner J (2002): Classical and Bayesian inference in neuroimaging: Applications. Neuroimage 16:484-512.
33. Giard MH, Peronnet F (1999): Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. J Cogn Neurosci 11:473-490.
34. Grady CL, Van Meter JW, Maisog JM, Pietrini P, Krasuski J, Rauschecker JP (1997): Attention-related modulation of activity in primary and secondary auditory cortex. Neuroreport 8:2511-2516.
35. Grefkes C, Fink GR (2005): The functional organization of the intraparietal sulcus in humans and monkeys. J Anat 207:3-17.
36. Gross CG, Graziano MS (1995): Multiple representations of space in the brain. Neuroscientist 1:43-50.
37. Grunewald A, Linden JF, Andersen RA (1999): Responses to auditory stimuli in macaque lateral intraparietal area. I. Effects of training. J Neurophysiol 82:330-342.
38. Helbig HB, Ernst MO, Ricciardi E, Pietrini P, Thielscher A, Mayer KM, Schultz J, Noppeney U (2012): The neural mechanisms of reliability weighted integration of shape information from vision and touch. Neuroimage 60:1063-1072.
39. Itti L, Koch C (2001): Computational modelling of visual attention. Nat Rev Neurosci 2:194-203.
40. Itti L, Koch C, Niebur E (1998): A model of salience-based visual attention for rapid scene analysis. IEEE Trans. Pattern Anal Mach Intell 20:1254-1259.
41. Jack AI, Patel GH, Astafiev SV, Snyder AZ, Akbudak E, Shulman GL, Corbetta M (2007): Changing human visual field organization from early visual to extra-occipital cortex. PLoS One 2:e452.
42. Kalinli O, Narayanan S (2009): Prominence detection using auditory attention cues and task-dependent high level information. IEEE Trans Audio Speech Lang Process 17:1009-1024.
43. Kastner S, Pinsk MA, De Weerd P, Desimone R, Ungerleider LG (1999): Increased activity in human visual cortex during directed attention in the absence of visual stimulation. Neuron 22:751-761.
44. Kayser C, Petkov CI, Lippert M, Logothetis NK (2005): Mechanisms for allocating auditory attention: An auditory saliency map. Curr Biol 15:1943-1947.
45. Koelewijn T, Bronkhorst A, Theeuwes J (2010): Attention and the multiple stages of multisensory integration: A review of audiovisual studies. Acta Psychol 134:372-384.
46. Koyama M, Hasegawa I, Osada T, Adachi Y, Nakahara K, Miyashita Y (2004): Functional magnetic resonance imaging of macaque monkeys performing visually guided saccade tasks: Comparison of cortical eye fields with humans. Neuron 41:795-807.
47. Leaver AM, Rauschecker JP (2010): Cortical representation of natural complex sounds: Effects of acoustic features and auditory object category. J Neurosci 30:7604-7612.
48. Leopold DA, Logothetis NK (1998): Microsaccades differentially modulate neural activity in the striate and extrastriate visual cortex. Exp Brain Res 123:341-345.
49. Lewis R, Noppeney U (2010): Audiovisual synchrony improves motion discrimination via enhanced connectivity between early visual and auditory areas. J Neurosci 30:12329-12339.
50. Lewis JW, Talkington WJ, Tallaksen KC, Frum CA (2012): Auditory object salience: Human cortical processing of non-biological action sounds and their acoustic signal attributes. Front Syst Neurosci 6:27.
51. Linden JF, Grunewald A, Andersen RA (1999): Responses to auditory stimuli in macaque lateral intraparietal area. II. Behavioral modulation. J Neurophysiol 82:343-358.
52. Macaluso E (2010): Orienting of spatial attention and the interplay between the senses. Cortex 46:282-297.
53. Macaluso E, Frith CD, Driver J (2000): Modulation of human visual cortex by crossmodal spatial attention. Science 289:1206-1208.
54. Macaluso E, Driver J, Frith CD (2003): Multimodal spatial representations engaged in human parietal cortex during both saccadic and manual spatial orienting. Curr Biol 13:990-999.
55. Mazzoni P, Bracewell RM, Barash S, Andersen RA (1996): Spatially tuned auditory responses in area LIP of macaques performing delayed memory saccades to acoustic targets. J Neurophysiol 75:1233-1241.
56. McDonald JJ, Ward LM (2000): Involuntary listening aids seeing: Evidence from human electrophysiology. Psychol Sci 11:167-171.
57. McDonald JJ, Teder-Sälejärvi WA, Hillyard SA (2000): Involuntary orienting to sound improves visual perception. Nature 407:906-908.
58. McDonald JJ, Teder-Sälejärvi WA, Di Russo F, Hillyard SA (2003): Neural substrates of perceptual enhancement by cross-modal spatial attention. J Cogn Neurosci 15:10-19.
59. McGurk H, MacDonald J (1976): Hearing lips and seeing voices. Nature 265:746-748.
60. McMains SA, Kastner S (2010): Defining the units of competition: Influences of perceptual organization on competitive interactions in human visual cortex. J Cogn Neurosci 22:2417-2426.
61. Meienbrock A, Naumer MJ, Doehrmann O, Singer W, Muckli L (2007): Retinotopic effects during spatial audio-visual integration. Neuropsychologia 45:531-539.
62. Nardo D, Santangelo V, Macaluso E (2011): Stimulus-driven orienting of visuo-spatial attention in complex dynamic environments. Neuron 69:1015-1028.
63. Nobre AC, Gitelman DR, Dias EC, Mesulam MM (2000): Covert visual spatial orienting and saccades: Overlapping neural systems. Neuroimage 11:210-216.
64. Noesselt T, Shah NJ, Jäncke L (2003): Top-down and bottom-up modulation of language related areas-An fMRI study. Biomed Chromatogr Neurosci 4:13.
65. Nuthmann A, Henderson JM (2010): Object-based attentional selection in scene viewing. J Vis 10:20.
66. Okamoto H, Stracke H, Bermudez P, Pantev C (2011): Sound processing hierarchy within human auditory cortex. J Cogn NeuroscI 23:1855-1863.
67. Onat S, Libertus K, König P (2007): Integrating audiovisual information for the control of overt attention. J Vis 7:11.1-11.16.
68. Penny W, Holmes A (2004): Random effects analysis. In: Frackowiak RS, Friston KJ, Frith CD, Dolan RJ, Price CJ, Zeki S, Ashburner J, Penny W, editors. Human Brain Function II, 2nd ed.San Diego, CA:Elsevier. pp843-851.
69. Petkov CI, Kang X, Alho K, Bertrand O, Yund EW, Woods DL. (2004): Attentional modulation of human auditory cortex. Nat Neurosci 7:658-663.
70. Ptak R (2012): The frontoparietal attention network of the human brain: Action, salience, and a priority map of the environment. Neuroscientist 18:502-515.
71. Recanzone GH, Wurtz RH (2000): Effects of attention on MT and MST neuronal activity during pursuit initiation. J Neurophysiol 83:777-790.
72. Reddy L, Kanwisher NG, VanRullen R (2009): Attention and biased competition in multi-voxel object representations. Proc Natl Acad Sci USA 106:21447-21452.
73. Rizzolatti G, Riggio L, Dascola I, Umiltá C (1987): Reorienting attention across the horizontal and vertical meridians: Evidence in favor of a premotor theory of attention. Neuropsychologia 25:31-40.
74. Romei V, Murray MM, Cappe C, Thut G (2009): Preperceptual and stimulus-selective enhancement of low-level human visual cortex excitability by sounds. Curr Biol 19:1799-1805.
75. Santangelo V, Olivetti Belardinelli M, Spence C, Macaluso E (2009): Interactions between voluntary and stimulus-driven spatial attention mechanisms across sensory modalities. J Cogn Neurosci 21:2384-2397.
76. Schiller PH (1993): The effects of V4 and middle temporal (MT) area lesions on visual performance in the rhesus monkey. Vis Neurosci 10:717-746.
77. Schlack A, Sterbing-D'Angelo SJ, Hartung K, Hoffmann KP, Bremmer F (2005): Multisensory space representations in the macaque ventral intraparietal area. J Neurosci 25:4616-4625.
78. Schönwiesner M, Novitski N, Pakarinen S, Carlson S, Tervaniemi M, Näätänen R (2007): Heschl's gyrus, posterior superior temporal gyrus, and mid-ventrolateral prefrontal cortex have different roles in the detection of acoustic changes. J Neurophysiol 97:2075-2082.
79. Snyder LH, Grieve KL, Brotchie P, Andersen RA (1998): Separate body- and world-referenced representations of visual space in parietal cortex. Nature 394:887-891.
80. Spence C, Driver J (1997): Audiovisual links in exogenous covert spatial orienting. Percept Psychophys 59:1-22.
81. Spence C, Nicholls ME, Gillespie N, Driver J (1998): Cross-modal links in exogenous covert spatial orienting between touch, audition and vision. Percept Psychophys 60:544-557.
82. Stein BE, Meredith MA (1993):The Merging of the Senses. Cambridge:MIT Press.
83. Stein BE, Meredith MA, Huneycutt WS, McDade L (1989): Behavioral indices of multisensory integration: Orientation to visual cues is affected by auditory stimuli. J Cogn Neurosci 1:12-24.
84. Störmer VS, McDonald JJ, Hillyard SA (2009): Cross-modal cueing of attention alters appearance and early cortical processing of visual stimuli. Proc Natl Acad Sci USA 106:22456-22461.
85. Sussman E, Winkler I, Huotilainen M, Ritter W, Näätänen R (2002): Top-down effects can modify the initially stimulus-driven auditory organization. Brain Res Cogn Brain Res 13:393-405.
86. Talsma D, Senkowski D, Soto-Faraco S, Woldorff MG (2010): The multifaceted interplay between attention and multisensory integration. Trends Cogn Sci 14:400-410.
87. Teki S, Chait M, Kumar S, von Kriegstein K, Griffiths TD (2011): Brain bases for auditory stimulus-driven figure-ground segregation. J Neurosci 31:164-171.
88. Tseng PH, Carmi R, Cameron IG, Munoz DP, Itti L (2009): Quantifying center bias of observers in free viewing of dynamic natural scenes. J Vis 9:4-16.
89. Woods DL, Stecker GC, Rinne T, Herron TJ, Cate AD, Yund EW, Liao I, Kang X (2009): Functional maps of human auditory cortex: Effects of acoustic features and attention. PLoS One 4:e5183.
90. Worsley KJ, Marrett S, Neelin P, Vandal AC, Friston KJ, Evans AC (1996): A unified statistical approach for determining significant signals in images of cerebral activation. Hum Brain Mapp 4:58-73.