Simultaneous modeling of visual saliency and value computation improves predictions of economic choice

Proceedings of the National Academy of Sciences of the United States of America

Volumn 110, Issue 40, 2013, Pages

  Towal, R Blythe ^a   Mormann, Milica ^a,b   Koch, Christof ^a,c  

^a CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^b UNIVERSITY OF MIAMI   (United States)

^c ALLEN INSTITUTE FOR BRAIN SCIENCE   (United States)

Author keywords

Attention; Eye tracking; Multiple targets; Preference; Search

Indexed keywords

ACCURACY; ADULT; ARTICLE; ATTENTION; DECISION MAKING; DIFFUSION; DRIFT DIFFUSION MODEL; ECONOMIC ASPECT; EYE MOVEMENT; EYE TRACKING; FEMALE; FORCED CHOICE METHOD; GAZE; HUMAN; HUMAN EXPERIMENT; MALE; MATHEMATICAL MODEL; NORMAL HUMAN; PERCEPTION; PREDICTION; PRIORITY JOURNAL; PSYCHOPHYSIOLOGY; QUANTITATIVE ANALYSIS; TASK PERFORMANCE; VISION; VISUAL MEMORY; ANALYSIS OF VARIANCE; COMPARATIVE STUDY; ECONOMICS; MULTIPLE TARGETS; PHYSIOLOGY; PREFERENCE; PSYCHOLOGICAL MODEL; SEARCH;

ATTENTION; EYE TRACKING; MULTIPLE TARGETS; PREFERENCE; SEARCH;

ADULT; ANALYSIS OF VARIANCE; CHOICE BEHAVIOR; ECONOMICS; EYE MOVEMENTS; FEMALE; HUMANS; MALE; MODELS, PSYCHOLOGICAL; VISUAL PERCEPTION;

EID: 84885066831     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1304429110     Document Type: Article

References (88)

1. Summerfield C, Tsetsos K (2012) Building bridges between perceptual and economic decision-making: Neural and computational mechanisms. Front Neurosci 6:70.
2. Milosavljevic M, Navalpakkam V, Koch C, Rangel A (2012) Relative visual saliency differences induce sizable bias in consumer choice. J Consum Psychol 22(1):67-74.
3. Smith PL, Ratcliff R (2004) Psychology and neurobiology of simple decisions. Trends Neurosci 27(3):161-168.
4. Vickers D (1979) Decision Processes in Visual Perception (Academic Press, New York), pp 374-396.
5. Usher M, McClelland JL (2001) The time course of perceptual choice: The leaky, competing accumulator model. Psychol Rev 108(3):550-592.
6. Krajbich I, Armel C, Rangel A (2010) Visual fixations and the computation and comparison of value in simple choice. Nat Neurosci 13(10):1292-1298.
7. Milosavljevic M, Malmaud J, Huth A, Koch C, Rangel A (2010) The drift diffusion model can account for the accuracy and reaction time of value-based choices under high and low time pressure. Judgm Decis Mak 5(6):437-449.
8. Krajbich I, Rangel A (2011) Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions. Proc Natl Acad Sci USA 108(33):13852-13857.
9. Krajbich I, Lu D, Camerer C, Rangel A (2012) The attentional drift-diffusion model extends to simple purchasing decisions. Front Psychol 3:193.
10. Gold JI, Shadlen MN (2007) The neural basis of decision making. Annu Rev Neurosci 30:535-574.
11. Busemeyer JR, Townsend JT (1993) Decision field theory: A dynamic-cognitive approach to decision making in an uncertain environment. Psychol Rev 100(3):432-459.
12. Usher M, McClelland JL (2004) Loss aversion and inhibition in dynamical models of multialternative choice. Psychol Rev 111(3):757-769.
13. Kustov AA, Robinson DL (1996) Shared neural control of attentional shifts and eye movements. Nature 384(6604):74-77.
14. Mohler CW, Wurtz RH (1976) Organization of monkey superior colliculus: Intermediate layer cells discharging before eye movements. J Neurophysiol 39(4):722-744.
15. Walther D, Koch C (2006) Modeling attention to salient proto-objects. Neural Netw 19(9):1395-1407.
16. Reichle ED, Rayner K, Pollatsek A (2003) The E-Z reader model of eye-movement control in reading: Comparisons to other models. Behav Brain Sci 26(4):445-476, discussion 477-526.
17. Wolfe JM (2007) Guided Search 4.0: Current Progress with a Model of Visual Search in Integrated Models of Cognitive Systems, ed. Gray WD (Oxford Univ Press, New York), pp 99-119.
18. Treisman A, Gormican S (1988) Feature analysis in early vision: Evidence from search asymmetries. Psychol Rev 95(1):15-48.
19. Navalpakkam V, Itti L (2007) Search goal tunes visual features optimally. Neuron 53(4):605-617.
20. Bundesen C, Habekost T, Kyllingsbaek S (2005) A neural theory of visual attention: Bridging cognition and neurophysiology. Psychol Rev 112(2):291-328.
21. Itti L, Koch C (2001) Computational modelling of visual attention. Nat Rev Neurosci 2(3):194-203.
22. Hou X, Zhang L (2008) Dynamic visual attention: Searching for coding length increments. Advances in Neural Information Processing Systems, eds Koller D, Schuurmans D, Bengio Y, Bottou L, Culotta A (MIT Press, Cambridge, MA), pp 651-688.
23. Zhao Q, Koch C (2011) Learning a saliency map using fixated locations in natural scenes. J Vis 11(3):1-15.
24. Purcell BA, Schall JD, Logan GD, Palmeri TJ (2012) From salience to saccades: Multiplealternative gated stochastic accumulator model of visual search. J Neurosci 32(10): 3433-3446.
25. Schall JD, Purcell BA, Heitz RP, Logan GD, Palmeri TJ (2011) Neural mechanisms of saccade target selection: Gated accumulator model of the visual-motor cascade. Eur J Neurosci 33(11):1991-2002.
26. Schall JD, Hanes DP (1993) Neural basis of saccade target selection in frontal eye field during visual-search. Nature 366(6454):467-469.
27. Gottlieb JP, Kusunoki M, Goldberg ME (1998) The representation of visual salience in monkey parietal cortex. Nature 391(6666):481-484.
28. Bichot NP, Schall JD (1999) Effects of similarity and history on neural mechanisms of visual selection. Nat Neurosci 2(6):549-554.
29. Constantinidis C, Steinmetz MA (2005) Posterior parietal cortex automatically encodes the location of salient stimuli. J Neurosci 25(1):233-238.
30. Ipata AE, Gee AL, Gottlieb J, Bisley JW, Goldberg ME (2006) LIP responses to a popout stimulus are reduced if it is overtly ignored. Nat Neurosci 9(8):1071-1076.
31. Thomas NWD, Paré M (2007) Temporal processing of saccade targets in parietal cortex area LIP during visual search. J Neurophysiol 97(1):942-947.
32. Hanes DP, Schall JD (1996) Neural control of voluntary movement initiation. Science 274(5286):427-430.
33. Hanes DP, Patterson WF, 2nd, Schall JD (1998) Role of frontal eye fields in countermanding saccades: Visual, movement, and fixation activity. J Neurophysiol 79(2):817-834.
34. Paré M, Hanes DP (2003) Controlled movement processing: Superior colliculus activity associated with countermanded saccades. J Neurosci 23(16):6480-6489.
35. Boucher L, Palmeri TJ, Logan GD, Schall JD (2007) Inhibitory control in mind and brain: An interactive race model of countermanding saccades. Psychol Rev 114(2):376-397.
36. Ratcliff R, Hasegawa YT, Hasegawa RP, Smith PL, Segraves MA (2007) Dual diffusion model for single-cell recording data from the superior colliculus in a brightnessdiscrimination task. J Neurophysiol 97(2):1756-1774.
37. Shimojo S, Simion C, Shimojo E, Scheier C (2003) Gaze bias both reflects and influences preference. Nat Neurosci 6(12):1317-1322.
38. Summerfield C, Koechlin E (2010) Economic value biases uncertain perceptual choices in the parietal and prefrontal cortices. Front Hum Neurosci 4:208.
39. McMillen T, Holmes P (2006) The dynamics of choice among multiple alternatives. J Math Psychol 50(1):30-57.
40. Busemeyer JR, Johnson JG (2004) Computational models of decision making. Blackwell Handbook of Judgment and Decision Making, eds Koehler DJ, Harvey N (Blackwell Publishing, Oxford), pp 133-154.
41. Bogacz R (2007) Optimal decision-making theories: Linking neurobiology with behaviour. Trends Cogn Sci 11(3):118-125.
42. Schrödinger E (1915) The theory of drop and rise tests on Brownian motion particles. Phys Z 16:289-295.
43. Smoluchowski M (1915) Notes on the calculation of the Brownian motion of molecules in Millikan-honourable test arrangements. Phys Z 16:318-321.
44. Schmolesky MT, et al. (1998) Signal timing across the macaque visual system. J Neurophysiol 79(6):3272-3278.
45. Wurtz RH, Goldberg ME (1972) Activity of superior colliculus in behaving monkey. 3. Cells discharging before eye-movements. J Neurophysiol 35(4):575-586.
46. Bogacz R, Brown E, Moehlis J, Holmes P, Cohen JD (2006) The physics of optimal decision making: A formal analysis of models of performance in two-alternative forced-choice tasks. Psychol Rev 113(4):700-765.
47. White CT, Eason RG, Bartlett NR (1962) Latency and duration of eye movements in the horizontal plane. J Opt Soc Am 52(2):210-213.
48. Van Der Lans R, Pieters R, Wedel M (2008) Competitive brand salience. Mark Sci 27(5): 922-931.
49. Navalpakkam V, Koch C, Rangel A, Perona P (2010) Optimal reward harvesting in complex perceptual environments. Proc Natl Acad Sci USA 107(11):5232-5237.
50. Yuille A, Bulthoff HH (1996) Bayesian Decision Theory and Psychophysics in Perception as Bayesian Inference (Cambridge Univ Press, Cambridge, UK), pp 123-162.
51. Markowitz DA, Shewcraft RA, Wong YT, Pesaran B (2011) Competition for visual selection in the oculomotor system. J Neurosci 31(25):9298-9306.
52. Theeuwes J (2010) Top-down and bottom-up control of visual selection. Acta Psychol 135(2):77-99.
53. Mackay M, Cerf M, Koch C (2012) Evidence for two distinct mechanisms directing gaze in natural scenes. J Vis 12(4):9.
54. Mannan SK, Kennard C, Husain M (2009) The role of visual salience in directing eye movements in visual object agnosia. Curr Biol 19(6):R247-R248.
55. Reddi BAJ, Carpenter RHS (2000) The influence of urgency on decision time. Nat Neurosci 3(8):827-830.
56. Wald A (1947) Foundations of a general theory of sequential decision functions. Econometrica 15(4):279-313.
57. Koch C, Idan S (1998) Methods in Neuronal Modeling (MIT Press, Cambridge, MA).
58. Ditterich J, Mazurek ME, Shadlen MN (2003) Microstimulation of visual cortex affects the speed of perceptual decisions. Nat Neurosci 6(8):891-898.
59. Posner MI, Cohen Y (1984) Components of visual orienting. Attention and Performance X: Control of Language Processes, 32, eds Bouma H, Bouwhuis DG (Erlbaum, Hilsdale, NJ) pp 531-556.
60. Klein RM (2000) Inhibition of return. Trends Cogn Sci 4(4):138-147.
61. Shadlen MN, Kiani R, Hanks TD, Churchland AK (2008) Neurobiology of decisionmaking: An intentional framework. Better Than Conscious?, eds. Engel C, Singer W (MIT Press, Cambridge, MA), pp 71-101.
62. Rangel A, Camerer CF, Montague PR (2008) A framework for studying the neurobiology of value-based decision making. Nat Rev Neurosci 9(7):545-556.
63. Kable JW, Glimcher PW(2009) The neurobiology of decision: Consensus and controversy. Neuron 63(6):733-745.
64. Ungerleider LG, Mishkin M (1982) Two cortical visual systems. Analysis of Visual Behavior, eds Ingle DJ, Goodale MA, Mansfield RJW (MIT Press, Cambridge, MA), pp 549-586.
65. Thompson KG, Bichot NP (2005) A visual salience map in the primate frontal eye field. Prog Brain Res 147:251-262.
66. Fecteau JH, Munoz DP (2006) Salience, relevance, and firing: A priority map for target selection. Trends Cogn Sci 10(8):382-390.
67. Bisley JW, Goldberg ME (2010) Attention, intention, and priority in the parietal lobe. Annu Rev Neurosci 33:1-21.
68. Montague PR, Berns GS (2002) Neural economics and the biological substrates of valuation. Neuron 36(2):265-284.
69. Platt ML, Glimcher PW (1998) Response fields of intraparietal neurons quantified with multiple saccadic targets. Exp Brain Res 121(1):65-75.
70. Musallam S, Corneil BD, Greger B, Scherberger H, Andersen RA (2004) Cognitive control signals for neural prosthetics. Science 305(5681):258-262.
71. Schultz J, Imamizu H, Kawato M, Frith CD (2004) Activation of the human superior temporal gyrus during observation of goal attribution by intentional objects. J Cogn Neurosci 16(10):1695-1705.
72. Padoa-Schioppa C, Assad JA (2006) Neurons in the orbitofrontal cortex encode economic value. Nature 441(7090):223-226.
73. Deaner RO, Khera AV, Platt ML (2005) Monkeys pay per view: Adaptive valuation of social images by rhesus macaques. Curr Biol 15(6):543-548.
74. Plassmann H, O'Doherty JP, Rangel A (2007) Orbitofrontal cortex encodes willingness to pay in everyday economic transactions. J Neurosci 27(37):9984-9988.
75. Hare TA, Camerer CF, Knoepfle DT, Rangel A (2010) Value computations in ventral medial prefrontal cortex during charitable decision making incorporate input from regions involved in social cognition. J Neurosci 30(2):583-590.
76. Lim SL, O'Doherty JP, Rangel A (2011) The decision value computations in the vmPFC and striatum use a relative value code that is guided by visual attention. J Neurosci 31(37):13214-13223.
77. Jenison RL, Rangel A, Oya H, Kawasaki H, HowardMA (2011) Value encoding in single neurons in the human amygdala during decision making. J Neurosci 31(1):331-338.
78. Itti L, Koch C, Niebur E (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Trans Patt Anal Machine Intell 20(11):1254-1259.
79. Gao D, Mahadevan V, Vasconcelos N (2007) The discriminant center-surround hypothesis for bottom-up saliency. Advances in Neural Information Processing Systems 20, eds Platt JC, Koller D, Singer Y, Roweis S (MIT Press, Cambridge, MA), pp 497-504.
80. Bruce ND, Tsotsos JK (2009) Saliency, attention, and visual search: An information theoretic approach. J Vis 9(3):1-24.
81. Hare TA, Camerer CF, Rangel A (2009) Self-control in decision-making involves modulation of the vmPFC valuation system. Science 324(5927):646-648.
82. Hare TA, O'Doherty JP, Camerer CF, Schultz W, Rangel A (2008) Dissociating the role of the orbitofrontal cortex and the striatum in the computation of goal values and prediction errors. J Neurosci 28(22):5623-5630.
83. Tatler BW (2007) The central fixation bias in scene viewing: Selecting an optimal viewing position independently of motor biases and image feature distributions. J Vis 7(14):4.1-17.
84. Tatler BW, Baddeley RJ, Gilchrist ID (2005) Visual correlates of fixation selection: Effects of scale and time. Vision Res 45(5):643-659.
85. Milosavljevic M, Koch C, Rangel A (2011) Consumers can make choices in as little as a third of a second. Judgm Decis Mak 6(6):520-530.
86. Reutskaya E, Nagel R, Camerer C, Rangel A (2011) Search dynamics in consumer choice under time pressure: An eye-tracking study. Am Econ Rev 101(2):900-926.
87. Van Zandt T (2000) How to fit a response time distribution. Psychon Bull Rev 7(3): 424-465.
88. Ratcliff R, Tuerlinckx F (2002) Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability. Psychon Bull Rev 9(3):438-481.