Perceptual decisions formed by accumulation of audiovisual evidence in prefrontal cortex

Journal of Neuroscience

Volumn 30, Issue 21, 2010, Pages 7434-7446

  Noppeney, Uta ^a   Ostwald, Dirk ^a   Werner, Sebastian ^a  

^a MAX PLANCK INSTITUTE FOR BIOLOGICAL CYBERNETICS   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ADULT; ARTICLE; AUDITORY STIMULATION; BAYES THEOREM; BEHAVIOR; DECISION MAKING; FEMALE; FUNCTIONAL MAGNETIC RESONANCE IMAGING; HUMAN; HUMAN EXPERIMENT; MALE; NORMAL HUMAN; PERCEPTIVE DISCRIMINATION; PREFRONTAL CORTEX; PRIORITY JOURNAL; PSYCHOPHYSICS; REACTION TIME; RELIABILITY; SELECTIVE ATTENTION; SEMANTICS; SENSORY EVALUATION; SUBICULUM; VIDEORECORDING; VISUAL STIMULATION;

ACOUSTIC STIMULATION; ADULT; ANALYSIS OF VARIANCE; ATTENTION; AUDITORY PERCEPTION; BRAIN MAPPING; DECISION MAKING; FEMALE; FUNCTIONAL LATERALITY; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; MAGNETIC RESONANCE IMAGING; MALE; MENTAL PROCESSES; MODELS, NEUROLOGICAL; NONLINEAR DYNAMICS; OXYGEN; PHOTIC STIMULATION; PREFRONTAL CORTEX; PSYCHOPHYSICS; REACTION TIME; VISUAL PERCEPTION; YOUNG ADULT;

EID: 77953078834     PISSN: 02706474     EISSN: 15292401     Source Type: Journal    
DOI: 10.1523/JNEUROSCI.0455-10.2010     Document Type: Article

References (91)

1. Amedi A, von Kriegstein K, van Atteveldt NM, Beauchamp MS, Naumer MJ (2005) Functional imaging of human crossmodal identification and object recognition. Exp Brain Res 166:559-571. (Pubitemid 41502099)
2. Beauchamp MS, Lee KE, Argall BD, MartinA (2004) Integration of auditory and visual information about objects in superior temporal sulcus. Neuron 41:809-823.
3. Ben-Artzi E, Marks LE (1995) Visual-auditory interaction in speeded classification: role of stimulus difference. Percept Psychophys 57:1151-1162.
4. Binder JR, Liebenthal E, Possing ET, Medler DA, Ward BD (2004) Neural correlates of sensory and decision processes in auditory object identification. Nat Neurosci 7:295-301.
5. Botvinick M, Nystrom LE, Fissell K, Carter CS, Cohen JD (1999) Conflict monitoring versus selection-for-action in anterior cingulate cortex. Nature 402:179-181.
6. Botvinick MM, Braver TS, Barch DM, Carter CS, Cohen JD (2001) Conflict monitoring and cognitive control. Psychol Rev 108:624-652.
7. Botvinick MM, Cohen JD, Carter CS (2004) Conflict monitoring and anterior cingulate cortex: an update. Trends Cogn Sci 8:539-546. (Pubitemid 39536251)
8. Brown JW, Braver TS (2005) Learned predictions of error likelihood in the anterior cingulate cortex. Science 307:1118-1121.
9. Calvert GA, Lewis JW (2004) Hemodynamic studies of audio-visual interactions. In: The handbook of multi-sensory processes (Calvert GA, Spence C, Stein BE, eds), pp 483-502. Cambridge: MIT Press.
10. Calvert GA, Campbell R, Brammer MJ (2000) Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex. Curr Biol 10:649-657. (Pubitemid 30354700)
11. Calvert GA, Hansen PC, Iversen SD, Brammer MJ (2001) Detection of audio-visual integration sites in humans by application of electrophysiological criteria to the BOLD effect. Neuroimage 14:427-438.
12. Carter CS, van Veen V (2007) Anterior cingulate cortex and conflict detection: an update of theory and data. Cogn Affect Behav Neurosci 7:367-379.
13. Chao LL, Haxby JV, Martin A (1999) Attribute-based neural substrates in temporal cortex for perceiving and knowing about objects. Nat Neurosci 2:913-919.
14. Dakin SC, Hess RF, Ledgeway T, Achtman RL (2002) What causes non-monotonic tuning of fMRI response to noisy images? Curr Biol 12:R476-R477.
15. de Lange FP, Jensen O, Dehaene S (2010) Accumulation of evidence during sequential decision making: the importance of top-down factors. J Neurosci 30:731-738.
16. Doehrmann O, Naumer MJ (2008) Semantics and the multisensory brain: how meaning modulates processes of audio-visual integration. Brain Res 1242:136-150.
17. Driver J, Noesselt T (2008) Multisensory interplay reveals crossmodal influences on 'sensory-specific' brain regions, neural responses, and judgments. Neuron 57:11-23.
18. Duncan J, Owen AM (2000) Common regions of the human frontal lobe recruited by diverse cognitive demands. Trends Neurosci 23:475-483.
19. Egner T, Hirsch J (2005) Cognitive control mechanisms resolve conflict through cortical amplification of task-relevant information. Nat Neurosci 8:1784-1790. (Pubitemid 41683207)
20. Evans AC, Collins DL, Milner B (1992) An MRI-based stereotactic atlas from 250 young normal subjects. Soc Neurosci Abstr 18:408.
21. Forster B, Pavone EF (2008) Electrophysiological correlates of crossmodal visual distractor congruency effects: evidence for response conflict. Cogn Affect Behav Neurosci 8:65-73.
22. Foxe JJ, Morocz IA, Murray MM, Higgins BA, Javitt DC, Schroeder CE (2000) Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Brain Res Cogn Brain Res 10:77-83.
23. Friston K (2009) The free-energy principle: a rough guide to the brain? Trends Cogn Sci 13:293-301.
24. Friston K, Kiebel S (2009) Predictive coding under the free-energy principle. Philos Trans R Soc Lond B Biol Sci 364:1211-1221.
25. Friston KJ, Holmes A, Worsley KJ, Poline JB, Frith CD, Frackowiak R (1995) Statistical parametric mapping: a general linear approach. Hum Brain Mapp 2:189-210.
26. Friston KJ, Harrison L, PennyW (2003) Dynamic causal modelling. Neuro-image 19:1273-1302.
27. Ghazanfar AA, Schroeder CE (2006) Is neocortex essentially multisensory? Trends Cogn Sci 10:278-285.
28. Gold JI, Shadlen MN (2002) Banburismus and the brain: decoding the relationship between sensory stimuli, decisions, and reward. Neuron 36:299-308.
29. Gold JI, ShadlenMN (2007) The neural basis of decision making. Annu Rev Neurosci 30:535-574.
30. Grinband J, Hirsch J, Ferrera VP (2006) A neural representation of categorization uncertainty in the human brain. Neuron 49:757-763. (Pubitemid 43290134)
31. Heekeren HR, Marrett S, Bandettini PA, Ungerleider LG (2004) A general mechanism for perceptual decision-making in the human brain. Nature 431:859-862. (Pubitemid 39434083)
32. Hein G, Doehrmann O, Müller NG, Kaiser J, Muckli L, Naumer MJ (2007) Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J Neurosci 27:7881-7887. (Pubitemid 47173312)
33. Ho TC, Brown S, Serences JT (2009) Domain general mechanisms of perceptual decision making in human cortex. J Neurosci 29:8675-8687.
34. Kaiser J, Lennert T, LutzenbergerW (2007) Dynamics of oscillatory activity during auditory decision making. Cereb Cortex 17:2258-2267. (Pubitemid 47517461)
35. Kass RE, Raftery AE (1995) Bayes factors. J Am Stat Assoc 90:773-795.
36. Kayser C, Petkov CI, Augath M, Logothetis NK (2005) Integration of touch and sound in auditory cortex. Neuron 48:373-384.
37. Kerns JG, Cohen JD, MacDonald AW 3rd, Cho RY, Stenger VA, Carter CS (2004) Anterior cingulate conflict monitoring and adjustments in control. Science 303:1023-1026.
38. Kiani R, Hanks TD, Shadlen MN (2008) Bounded integration in parietal cortex underlies decisions even when viewing duration is dictated by the environment. J Neurosci 28:3017-3029.
39. Kim JN, ShadlenMN (1999) Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque. Nat Neurosci 2:176-185.
40. Kleiner M, Wallraven C, Bulthoff HH (2004) The MPI VideoLab - a system for high quality synchronous recording of video and audio from multiple viewpoints. Tübingen, Germany: MPI 123.
41. Lakatos P, Chen CM, O'Connell MN, Mills A, Schroeder CE (2007) Neuronal oscillations and multisensory interaction in primary auditory cortex. Neuron 53:279-292.
42. Laurienti PJ, Wallace MT, Maldjian JA, Susi CM, Stein BE, Burdette JH (2003) Cross-modal sensory processing in the anterior cingulate and medial prefrontal cortices. Hum Brain Mapp 19:213-223.
43. Laurienti PJ, Kraft RA, Maldjian JA, Burdette JH, Wallace MT (2004) Semantic congruence is a critical factor in multisensory behavioral performance. Exp Brain Res 158:405-414.
44. Lewis JW, Wightman FL, Brefczynski JA, Phinney RE, Binder JR, DeYoe EA (2004) Human brain regions involved in recognizing environmental sounds. Cereb Cortex 14:1008-1021.
45. Lewis JW, Brefczynski JA, Phinney RE, Janik JJ, DeYoe EA (2005) Distinct cortical pathways for processing tool versus animal sounds. J Neurosci 25:5148-5158.
46. Liu YS, Yu A, Holmes P (2009) Dynamical analysis of Bayesian inference models for the Eriksen task. Neural Comput 21:1520-1553.
47. Lo CC, Wang XJ (2006) Cortico-basal ganglia circuit mechanism for a decision threshold in reaction time tasks. Nat Neurosci 9:956-963.
48. Macaluso E, George N, Dolan R, Spence C, Driver J (2004) Spatial and temporal factors during processing of audiovisual speech: a PET study. Neuroimage 21:725-732. (Pubitemid 38229935)
49. Mazurek ME, Roitman JD, Ditterich J, ShadlenMN (2003) A role for neural integrators in perceptual decision making. Cereb Cortex 13:1257-1269. (Pubitemid 37323002)
50. Miller BT, D'Esposito M (2005) Searching for "the top" in top-down control. Neuron 48:535-538.
51. Miller EK, Cohen JD (2001) An integrative theory of prefrontal cortex function. Annu Rev Neurosci 24:167-202. (Pubitemid 32695227)
52. Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ (2002) Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. Brain Res Cogn Brain Res 14:115-128.
53. Naumer MJ, Doehrmann O, Müller NG, Muckli L, Kaiser J, Hein G (2009) Cortical plasticity of audio-visual object representations. Cereb Cortex 19:1641-1653.
54. Noppeney U (2010) Characterization of multisensory integration with fMRI - experimental design, statistical analysis and interpretation. In: Frontiers in the neural bases of multisensory processes (Murray MM, Wallace MT, eds). London: Taylor and Francis.
55. Noppeney U, Price CJ (2002) A PET study of stimulus- and task-induced semantic processing. Neuroimage 15:927-935. (Pubitemid 34852506)
56. Noppeney U, Price CJ, Penny WD, Friston KJ (2006) Two distinct neural mechanisms for category-selective responses. Cereb Cortex 16:437-445. (Pubitemid 43209218)
57. Noppeney U, Josephs O, Hocking J, Price CJ, Friston KJ (2008) The effect of prior visual information on recognition of speech and sounds. Cereb Cortex 18:598-609. (Pubitemid 351264229)
58. Orr JM, Weissman DH (2009) Anterior cingulate cortex makes 2 contributions to minimizing distraction. Cereb Cortex 19:703-711.
59. Paus T (2001) Primate anterior cingulate cortex: where motor control, drive and cognition interface. Nat Rev Neurosci 2:417-424.
60. Penny WD, Stephan KE, Mechelli A, Friston KJ (2004) Comparing dynamic causal models. Neuroimage 22:1157-1172.
61. Philiastides MG, Sajda P (2006) Temporal characterization of the neural correlates of perceptual decision making in the human brain. Cereb Cortex 16:509-518.
62. Pleger B, Ruff CC, Blankenburg F, Bestmann S, Wiech K, Stephan KE, Capilla A, Friston KJ, Dolan RJ (2006) Neural coding of tactile decisions in the human prefrontal cortex. J Neurosci 26:12596-12601. (Pubitemid 44864524)
63. Ploran EJ, Nelson SM, Velanova K, Donaldson DI, Petersen SE, Wheeler ME (2007) Evidence accumulation and the moment of recognition: dissociating perceptual recognition processes using fMRI. J Neurosci 27:11912-11924. (Pubitemid 350072112)
64. Rao RP, Ballard DH (1999) Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nat Neurosci 2:79-87.
65. Ratcliff R, Smith PL (2004) Acomparison of sequential sampling models for two-choice reaction time. Psychol Rev 111:333-367. (Pubitemid 38516421)
66. Romo R, Hernández A, Zainos A (2004) Neuronal correlates of a perceptual decision in ventral premotor cortex. Neuron 41:165-173. (Pubitemid 38068442)
67. Rushworth MF, Walton ME, Kennerley SW, BannermanDM (2004) Action sets and decisions in the medial frontal cortex. Trends Cogn Sci 8:410-417. (Pubitemid 39179904)
68. Sadaghiani S, Maier JX, Noppeney U (2009) Natural, metaphoric, and linguistic auditory direction signals have distinct influences on visual motion processing. J Neurosci 29:6490-6499.
69. Schall JD (2003) Neural correlates of decision processes: neural and mental chronometry. Curr Opin Neurobiol 13:182-186. (Pubitemid 36556275)
70. Schneider TR, Debener S, Oostenveld R, Engel AK (2008) Enhanced EEG gamma-band activity reflects multisensory semantic matching in visualto-auditory object priming. Neuroimage 42:1244-1254.
71. Schroeder CE, Foxe J (2005) Multisensory contributions to low-level, 'unisensory' processing. Curr Opin Neurobiol 15:454-458.
72. Schroeder CE, Foxe JJ (2002) The timing and laminar profile of converging inputs to multisensory areas of the macaque neocortex. Brain Res Cogn Brain Res 14:187-198.
73. Servan-Schreiber D, Bruno RM, Carter CS, Cohen JD (1998a) Dopamine and the mechanisms of cognition: Part I. A neural network model predicting dopamine effects on selective attention. Biol Psychiatry 43:713-722.
74. Servan-Schreiber D, Carter CS, Bruno RM, Cohen JD (1998b) Dopamine and the mechanisms of cognition: Part II. D-amphetamine effects in human subjects performing a selective attention task. Biol Psychiatry 43:723-729.
75. Shadlen MN, Newsome WT (2001) Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. J Neurophysiol 86:1916-1936.
76. Smith PL, Ratcliff R (2004) Psychology and neurobiology of simple decisions. Trends Neurosci 27:161-168.
77. Stephan KE, PennyWD,Daunizeau J, Moran RJ, Friston KJ (2009) Bayesian model selection for group studies. Neuroimage 46:1004-1017.
78. Stevenson RA, James TW (2009) Audiovisual integration in human superior temporal sulcus: inverse effectiveness and the neural processing of speech and object recognition. Neuroimage 44:1210-1223.
79. Sugihara T, Diltz MD, Averbeck BB, Romanski LM (2006) Integration of auditory and visual communication information in the primate ventrolateral prefrontal cortex. J Neurosci 26:11138-11147.
80. Sugrue LP, Corrado GS, Newsome WT (2004) Matching behavior and the representation of value in the parietal cortex. Science 304:1782-1787.
81. Summerfield C, Koechlin E (2008) A neural representation of prior information during perceptual inference. Neuron 59:336-347.
82. Talairach J, Tournoux P (1988) Co-planar stereotaxic atlas of the human brain. Stuttgart: Thieme.
83. Thielscher A, Pessoa L (2007) Neural correlates of perceptual choice and decision making during fear-disgust discrimination. J Neurosci 27:2908-2917.
84. van Atteveldt N, Formisano E, Goebel R, Blomert L (2004) Integration of letters and speech sounds in the human brain. Neuron 43:271-282.
85. van Atteveldt NM, Formisano E, Goebel R, Blomert L (2007) Top-down task effects overrule automatic multisensory responses to letter-sound pairs in auditory association cortex. Neuroimage 36:1345-1360.
86. Weissman DH, Warner LM, Woldorff MG (2004) The neural mechanisms for minimizing cross-modal distraction. J Neurosci 24:10941-10949.
87. Werner S, Noppeney U (2009) Superadditive responses in superior temporal sulcus predict audiovisual benefits in object categorization. Cereb Cortex. Advance online publication. Retrieved May 17, 2010. doi:10.1093/ cercor/bhp248.
88. Werner S, NoppeneyU (2010) Distinct functional contributions of primary sensory and association areas to audiovisual integration in object categorization. J Neurosci 30:2662-2675.
89. Yu AJ, Dayan P, Cohen JD (2009) Dynamics of attentional selection under conflict: toward a rational Bayesian account. J Exp Psychol Hum Percept Perform 35:700-717.
90. Yuval-Greenberg S, Deouell LY (2007) What you see is not (always) what you hear: induced gamma band responses reflect cross-modal interactions in familiar object recognition. J Neurosci 27:1090-1096.
91. Yuval-Greenberg S, Deouell LY (2009) The dog's meow: asymmetrical interaction in cross-modal object recognition. Exp Brain Res 193:603-614.