The role of uncertainty and reward on eye movements in a virtual driving task

Journal of Vision

Volumn 12, Issue 13, 2012, Pages

  Sullivan, Brian T ^a,b   Johnson, Leif ^c   Rothkopf, Constantin A ^d,e   Ballard, Dana ^c   Hayhoe, Mary ^b  

^a SMITH KETTLEWELL EYE RESEARCH INSTITUTE   (United States)

^b UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^c University of Texas at Austin   (United States)

^d FRANKFURT INSTITUTE FOR ADVANCED STUDIES   (Germany)

^e UNIVERSITY OF OSNABRÜCK   (Germany)

Author keywords

Cognition; Driving; Eye movements; Natural tasks; Reinforcement learning

Indexed keywords

ADULT; ARTICLE; CAR DRIVING; COMPARATIVE STUDY; COMPUTER SIMULATION; EYE FIXATION; EYE MOVEMENT; FEMALE; HUMAN; MALE; PHYSIOLOGY; PSYCHOLOGICAL ASPECT; PSYCHOMOTOR PERFORMANCE; UNCERTAINTY;

ADULT; AUTOMOBILE DRIVING; COMPUTER SIMULATION; EYE MOVEMENTS; FEMALE; FIXATION, OCULAR; HUMANS; MALE; PSYCHOMOTOR PERFORMANCE; UNCERTAINTY;

EID: 84878210547     PISSN: None     EISSN: 15347362     Source Type: Journal    
DOI: 10.1167/12.13.19     Document Type: Article

References (79)

1. Aggarwal, M., Hyland, B. I., & Wickens, J. R. (2012). Neural control of dopamine neurotransmission: Implications for reinforcement learning. European Journal of Neuroscience, 35(7), 1115-1123.
2. Andersen, G. J., & Sauer, C. W. (2007). Optical information for car following: The driving by visual angle (DVA) model. Human factors: The journal of the human factors and ergonomics society, 49(5), 878-896.
3. Anderson, J. R. (1996). ACT: A simple theory of complex cognition. American Psychologist, 51(4), 355.
4. Anderson, J. R. (2007). How can the human mind occur in the physical universe? (Vol. 3). New York: Oxford University Press.
5. Atkins, J. E., Jacobs, R. A., & Knill, D. C. (2003). Experience-dependent visual cue recalibration based on discrepancies between visual and haptic percepts. Vision Research, 43(25), 2603-2613.
6. Bromberg-Martin, E. S., & Hikosaka, O. (2009). Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron, 63(1), 119.
7. Bruce, N. D., & Tsotsos, J. K. (2009). Saliency, attention, and visual search: An information theoretic approach. Journal of Vision, 9(3), 5, 1- 24, http://www.journalofvision.org/content/9/3/5, doi:10.1167/9.3.5. [PubMed] [Article]
8. Deaner, R. O., Khera, A. V., & Platt, M. L. (2005). Monkeys pay per view: Adaptive valuation of social images by rhesus macaques. Current Biology, 15(6), 543-548.
9. Dorris, M. C., & Glimcher, P. W. (2004). Activity in posterior parietal cortex is correlated with the relative subjective desirability of action. Neuron, 44(2), 365-378.
10. Droll, J. A., Hayhoe, M. M., Triesch, J., & Sullivan, B. T. (2005). Task demands control acquisition and storage of visual information. Journal of Experimental Psychology: Human Perception and Performance, 31(6), 1416.
11. Fecteau, J. H. (2007). Priming of pop-out depends upon the current goals of observers. Journal of Vision, 7(6):1, 1-11, http://www.journalofvision.org/content/7/6/1, doi:10.1167/7.6.1. [PubMed] [Article]
12. Folk, C. L., Remington, R. W., & Wright, J. H. (1994). The structure of attentional control: Contingent attentional capture by apparent motion, abrupt onset, and color. Journal of Experimental Psychology: Human Perception and Performance, 20(2), 317.
13. Folk, C. L., & Remington, R. (1998). Selectivity in distraction by irrelevant featural singletons: Evidence for two forms of attentional capture. Journal of Experimental Psychology: Human Perception and Performance, 24(3), 847.
14. Forster, S., & Lavie, N. (2008). Failures to ignore entirely irrelevent distractors: The role of load. Journal of Experimental Psychology: Applied, 14(1), 73. Glimcher, P. W. (2003). The neurobiology of visualsaccadic decision making. Annual Review of Neuroscience, 26(1), 133-179.
15. Glimcher, P. W. (2003). The neurobiology of visualsaccadic decision making. Annual Review of Neuroscience, 26(1), 133-179.
16. Gottlieb, J., & Balan, P. (2010). Attention as a decision in information space. Trends in Cognitive Sciences, 14(6), 240.
17. Graf, E. W., Warren, P. A., & Maloney, L. T. (2005). Explicit estimation of visual uncertainty in human motion processing. Vision Research, 45(24), 3050- 3059.
18. Hayhoe, M., & Ballard, D. (2005). Eye movements in natural behavior. Trends in Cognitive Sciences, 9(4): 188-194.
19. Hayhoe, M. M., Bensinger, D. G., & Ballard, D. H. (1998). Task constraints in visual working memory. Vision Research, 38(1), 125-137.
20. Hikosaka, O., Nakamura, K., & Nakahara, H. (2006). Basal ganglia orient eyes to reward. Journal of Neurophysiology, 95(2), 567-584.
21. Itti, L., & Baldi, P. (2006). Bayesian surprise attracts human attention. Advances in neural information processing systems, 18, 547.
22. Itti, L., & Koch, C. (2001). Computational modeling of visual attention. Nature Reviews Neuroscience, 2(3), 194-203.
23. Janssen, C. P., Brumby, D. P., & Garnett, R. (2012). Natural break points: The influence of priorities and cognitive and motor cues on dual-task interleaving. Journal of Cognitive Engineering and Decision Making, 6(1), 5-29.
24. Jovancevic, J., Sullivan, B., & Hayhoe, M. (2006). Control of attention and gaze in complex environments. Journal of Vision, 6(12):9, 1431-1450, http://www.journalofvision.org/content/6/12/9, doi:10. 1167/6.12.9. [PubMed] [Article]
25. Jovancevic-Misic, J., & Hayhoe, M. (2009). Adaptive gaze control in natural environments. The Journal of Neuroscience, 29(19), 6234-6238.
26. Kanan, C., Tong, M. H., Zhang, L., & Cottrell, G. W. (2009). SUN: Top-down saliency using natural statistics. Visual Cognition, 17(6-7), 979-1003.
27. Knudsen, E. I. (2007). Fundamental components of attention. Annual Reivew of Neuroscience, 30, 57- 78.
28. Land, M. F., & Hayhoe, M. (2001). In what ways do eye movements contribute to everyday activities?. Vision research, 41(25), 3559-3565.
29. Land, M. F., & Horwood, J. (1995). Which parts of the road guide steering?. Nature, 377, 339-340.
30. Land, M. F., & Lee, D. N. (1994). Where we look when we steer. Nature, 369(6483), 742-744.
31. Land, M., Mennie, N., & Rusted, J. (1999). The roles of vision and eye movements in the control of activities of daily living. Perception, 28(11), 1311- 1328.
32. Lauwereyns, J., Takikawa, Y., Kawagoe, R., Kobayashi, S., Koizumi, M., & Coe, B., et al. (2002). Feature-based anticipation of cues that predict reward in monkey caudate nucleus. Neuron, 33(3), 463-473.
33. Leber, A. B., & Egeth, H. E. (2006). It's under control: Top-down search strategies can override attentional capture. Pyschonomic Bulletin & Review, 13(1), 132-138.
34. Lee, D., Seo, H., & Jung, M. W. (2012). Neural basis of reinforcement learning and decision making. Annual Review of Neuroscience, 35, 287-308.
35. Lu, S., & Han, S. (2009). Attentional capture is contingent on the interaction between task demand and stimulus salience. Attention, Perception, & Psychophysics, 71(5), 1015-1026.
36. Maunsell, J. H. R. (2004). Neuronal representations of cognitive state: Reward or attention?. Trends in Cognitive Science, 8, 261-265.
37. Morvan, C., & Maloney, L. T. (2012). Human visual search does not maximize the post-saccadic probability of identifying targets. PLoS Computational Biology, 8(2), e1002342.
38. Najemnik, J., & Geisler, W. S. (2005). Optimal eye movement strategies in visual search. Nature, 434(7031), 387-391.
39. Navalpakkam, V., Koch, C., & Perona, P. (2009). Homo economicus in visual search. Journal of Vision, 9(1):31, 1-16, http://www.journalofvision.org/content/9/1/31, doi:10.1167/9.1.31. [PubMed] [Article]
40. Navalpakkam, V., Koch, C., Rangel, A., & Perona, P. (2010). Optimal reward harvesting in complex perceptual environments. Proceedings of the National Academy of Sciences, 107(11), 5232-5237.
41. Ng, A. Y., & Russell, S. (2000). Algorithms for inverse reinforcement learning. In Proceedings of the Seventeenth International Conference on Machine Learning, (pp. 663-670).
42. Nunez-Varela, J., Ravindran, B., & Wyatt, J. L. (2012, May). Where do I look now? Gaze allocation during visually guided manipulation. In Robotics and Automation (ICRA), 2012, IEEE International Conference, pp. 4444-4449. IEEE.
43. Peck, C. J., Jangraw, D. C., Suzuki, M., Efem, R., & Gottlieb, J. (2009). Reward modulates attention independently of action value in posterior parietal cortex. The Journal of Neuroscience, 29(36), 11 182-11 191.
44. Pelz, J., Hayhoe, M., & Loeber, R. (2001). The coordination of eye, head, and hand movements in a natural task. Experimental Brain Research, 139(3), 266-277.
45. Platt, M. L., & Glimcher, P. W. (1999). Neural correlates of decision variables in parietal cortex. Nature, 400(6741), 233-238.
46. Renninger, L. W., Verghese, P., & Coughlan, J. (2007). Where to look next? Eye movements reduce local uncertainty. Journal of Vision, 7(3):6, 1-17, http://www.journalofvision.org/content/7/3/6, doi:10. 1167/7.3.6. [PubMed] [Article]
47. Rothkopf, C. A., & Ballard, D. H. (Submitted). Modular inverse reinforcement learning for visuomotor behavior.
48. Rothkopf, C., & Dimitrakakis, C. (2011). Preference elicitation and inverse reinforcement learning. Machine learning and knowledge discovery in databases, Lecture Notes in Computer Science, Vol. 6913, pp. 34-48. European Conference, ECML PKDD 2011, Athens, Greece, September 5-9, 2011 Proceedings, Part III. Berlin: Springer.
49. Rothkopf, C. A., & Ballard, D. H. (2010). Credit assignment in multiple goal embodied visuomotor behavior. Frontiers in Psychology, 1(173), 1-13, doi: 10.3389/fpsyg.2010.00173.
50. Rothkopf, C. A., Ballard, D. H., & Hayhoe, M. H. (2007). Task and scene context determines where you look. Journal of Vision, 7(14):16, 1-20, http://www.journalofvision.org/content/7/14/16, doi:10. 1167/7.14.16. [PubMed] [Article]
51. Rothkopf, C. A., & Pelz, J. B. (2004). Head movement estimation for wearable eye tracker. In A T. Duchowski & R. Vertegaal (eds.), Proceedings of the 2004 Symposium on Eye Tracking Research & Applications. (pp. 123-130). ACM.
52. Salvucci, D. D., & Gray, R. (2004). A two-point visual control model of steering. Perception-London, 33(10), 1233-1248.
53. Salvucci, D. D., & Taatgen, N. A. (2008). Threaded cognition: An integrated theory of concurrent multitasking. Psychological Review, 115(1), 101.
54. Salvucci, D. D., & Taatgen, N. A. (2011). Toward a unified view of cognitive control. Topics in Cognitive Science, 3(2), 227-230.
55. Schlicht, E. J., & Schrater, P. R. (2007). Effects of visual uncertainty on grasping movements. Experimental Brain Research, 182(1), 47-57.
56. Schultz, W. (1998). Predictive reward signal of dopamine neurons. Journal of Neurophysiology, 80(1), 1-27.
57. Schutz, A. Trommershauser, J., & Gegenfurtner, K. (2012). Dynamic integration of information about salience and value for saccadic eye movements. Proceedings of the National Academy of Sciences, 109, 7547-7552.
58. Senders, J. W. (1980). Visual scanning processes. Soest, the Netherlands: Drukkerij Neo Print.
59. Seo, H., & Less, D. (2007). Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. Journal of Neuroscience, 27, 8366-8377.
60. Sohn, J.-W., & Lee, D. (2006). Effects of reward expectancy on sequential eye movements in monkeys. Neural Networks, 19, 1181-1191.
61. Sprague, N., & Ballard, D. (2003). Eye movements for reward maximization. Advances in Neural Information Processing Systems, 16, 1467.
62. Sprague, N., Ballard, D., & Robinson, A. (2007). Modeling embodied visual behaviors. ACM Transactions on Applied Perception (TAP), 4(2), 11.
63. Stritzke, M., & Trommershäuser, J. (2007). Eye movements during rapid pointing under risk. Vision research, 47(15), 2000-2009.
64. Stritzke, M., Trommershäuser, J., & Gegenfurtner, K.R. (2009). Effects of salience and reward information during saccadic decisions under risk.Journal of the Optical Society of America, 26, B1- B13.
65. Stuphorn, V., Taylor, T. L., & Schall, J. D. (2000). Performance monitoring by the supplementary eye field. Nature, 408(6814), 857-860.
66. Stuphorn, V. & Schall, J. D. (2006). Executive control of contermanding saccades by the supplementary eye field. Nature Neuroscience, 9(7), 925-931.
67. Sugrue, L. P., Corrado, G. S., & Newsome, W. T. (2004). Matching behavior and the representation of value in the parietal cortex. Science, 304(5678), 1782-1787.
68. Sullivan, B. T., Johnson, L. M., Ballard, D. H., & Hayhoe, M. M. (2011). A modular reinforcement learning model for human visumoto behavior in a driving task. Proceedings of the AISB 2011 Symposium, 33-40.
69. Sutton R. S. (1988). Learning to predict by the methods of temporal differences. Machine Learning, 3(1), 9-44.
70. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction (Vol. 1, No. 1). Cambridge, MA: MIT Press.
71. Tatler, B. W., Hayhoe, M. M., Land, M. F., & Ballard, D. H. (2011). Eye guidance in natural vision: Reinterpreting salience. Journal of Vision, 11(5):5, 1-23, http://www.journalofvision.org/content/11/5/5, doi:10.1167/11.5.5. [PubMed] [Article]
72. Theeuwes, J. (2004). Top-down search strategies cannot override attentional capture. Psychonomic Bulletin & Review, 11(1), 65-70.
73. Treisman, A. M., & Gelade, G. (1980). A featureintegration theory of attention. Cognitive Psychology, 12(1), 97-136.
74. Warren, P. A., Graf, E. W., Champion, R. A., & Maloney, L. T. (2012). Visual extrapolation under risk: human observers estimate and compensate for exogenous uncertainty. Proceedings of the Royal Society B: Biological Sciences, 279(1736), 2171- 2179.
75. Watanabe, K. Lauwereyns, J., & Hikosaka, O. (2003). Neural correlates of rewarded and unrewarded eye movements in the primate caudate nucleus. The Journal of Neuroscience, 23(31), 10052-10057.
76. Wolfe, J. M., Butcher, S. J., Lee, C., & Hyle, M. (2003). Changing your mind: On the contributions of topdown and bottom-up guidance in visual search for feature singletons. Journal of Experimental Psychology: Human Perception and Performance, 29(2), 483.
77. Wolfe, J. M. , & Horowitz, T. S. (2004). What attributes guide the deployment of visual attention and how do they do it?. Nature Reviews Neuroscience, 5(6), 495-501
78. Yantis, S, & Egeth, H. E. (1999). On the distinction between visual salience and stimulus-driven attentional capture. Journal of Experimental Psychology: Human Perception and Performance, 25(3), 661.
79. Zhang, H., Morvan, C., & Maloney, L. T. (2010). Gambling in the visual periphery: A conjointmeasurement analysis of human ability to judge visual uncertainty. PLoS Computational Biology, 6(12), e1001023.