Learning where to look for a hidden target

Proceedings of the National Academy of Sciences of the United States of America

Volumn 110, Issue SUPPL2, 2013, Pages 10438-10445

  Chukoskie, Leanne ^a   Snider, Joseph ^a   Mozer, Michael C ^a,b,c   Krauzlis, Richard J ^d   Sejnowski, Terrence J ^a,e,f,g  

^a UNIVERSITY OF CALIFORNIA SAN DIEGO   (United States)

^b UNIVERSITY OF CALIFORNIA SAN DIEGO   (United States)

^c UCB 450   (United States)

^d UNIVERSITY OF COLORADO   (United States)

^e NATIONAL EYE INSTITUTE   (United States)

^f HOWARD HUGHES MEDICAL INSTITUTE   (United States)

^g SALK INSTITUTE FOR BIOLOGICAL STUDIES   (United States)

Author keywords

Ideal observer; Oculomotor; Reinforcement learning; Saccades

Indexed keywords

ARTICLE; FORAGE; FORAGING; FORAGING BEHAVIOR; GAZE; HUMAN; INFORMATION RETRIEVAL; LEARNING; OBSERVER VARIATION; PRIORITY JOURNAL; REINFORCEMENT; REINFORCEMENT LEARNING; SACCADIC EYE MOVEMENT; TASK PERFORMANCE;

IDEAL OBSERVER; OCULOMOTOR; REINFORCEMENT LEARNING; SACCADES;

ANIMALS; FEMALE; HUMANS; MALE; MODELS, BIOLOGICAL; PROBLEM SOLVING; PROBLEM-BASED LEARNING; VISUAL PERCEPTION;

ANIMALIA;

EID: 84879297139     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1301216110     Document Type: Article

References (66)

1. Potter MC (1975) Meaning in visual search. Science 187 4180:965-966.
2. Itti L, Koch C (2000) A saliency-based search mechanism for overt and covert shifts of visual attention. Vision Res 40(10-12):1489-1506
3. Oliva A, Torralba A (2006) Building the gist of a scene: The role of global image features in recognition. Prog Brain Res 155:23-36.
4. Torralba A, Oliva A, Castelhano MS, Henderson JM (2006) Contextual guidance of eye movements and attention in real-world scenes: The role of global features in object search. Psychol Rev 113 4:766-786.
5. Neider MB, Zelinsky GJ (2006) Scene context guides eye movements during visual search. Vision Res 46 5:614-621.
6. Rayner K, Castelhano MS, Yang J (2009) Eye movements when looking at unusual/weird scenes: Are there cultural differences? J Exp Psychol Learn Mem Cogn 35(1):254-259.
7. Vo ML, Henderson JM (2010) The time course of initial scene processing for eye movement guidance in natural scene search. J Vis 10(3):11-13.
8. Castelhano MS, Heaven C (2010) The relative contribution of scene context and target features to visual search in scenes. Atten Percept Psychophys 72(5):1283-1297.
9. Castelhano MS, Heaven C (2011) Scene context influences without scene gist: Eye movements guided by spatial associations in visual search. Psychon Bull Rev 18(5):890-896.
10. Hayhoe M, Ballard D (2005) Eye movements in natural behavior. Trends Cogn Sci 9(4):188-194.
11. Parkhurst DJ, Niebur E (2003) Scene content selected by active vision. Spat Vis 16(2):125-154.
12. Tatler BW, Vincent BT (2009) The prominence of behavioural biases in eye guidance. Vis Cogn 17(6-7):1029-1054
13. Chun MM, Jiang Y (1998) Contextual cueing: Implicit learning and memory of visual context guides spatial attention. Cognit Psychol 36(1):28-71.
14. Milstein DM, Dorris MC (2007) The influence of expected value on saccadic preparation. J Neurosci 27(18):4810-4818.
15. Xu-Wilson M, Zee DS, Shadmehr R (2009) The intrinsic value of visual information affects saccade velocities. Exp Brain Res 196(4):475-481.
16. Shadmehr R (2010) Control of movements and temporal discounting of reward. Curr Opin Neurobiol 20(6):726-730.
17. Shadmehr R, Orban de Xivry JJ, Xu-Wilson M, Shih TY (2010) Temporal discounting of reward and the cost of time in motor control. J Neurosci 30(31):10507-10516.
18. Reinagel P, Zador AM (1999) Natural scene statistics at the centre of gaze. Network 10(4):341-350.
19. Tatler BW, Baddeley RJ, Vincent BT (2006) The long and the short of it: Spatial statistics at fixation vary with saccade amplitude and task. Vision Res 46(12):1857-1862.
20. Açik A, Sarwary A, Schultze-Kraft R, Onat S, König P (2010) Developmental changes in natural viewing behavior: Bottom-up and top-down differences between children, young adults and older adults. Front Psychol 1:207.
21. Buswell GT (1935) How People Look at Pictures: A Study of the Psychology of Perception in Art (Univ of Chicago Press, Chicago).
22. Yarbus AL (1967) Eye Movements and Vision (Plenum, New York).
23. Castelhano MS, Mack ML, Henderson JM (2009) Viewing task influences eye movement control during active scene perception. J Vis 9(3):1-15.
24. Greene MR, Liu T, Wolfe JM (2012) Reconsidering Yarbus: A failure to predict observers' task from eye movement patterns. Vision Res 62:1-8.
25. Schütz AC, Trommershäuser J, Gegenfurtner KR (2012) Dynamic integration of information about salience and value for saccadic eye movements. Proc Natl Acad Sci USA 109 19:7547-7552.
26. Stritzke M, Trommershäuser J (2007) Eye movements during rapid pointing under risk. Vision Res 47 15:2000-2009.
27. Geng JJ, Behrmann M (2005) Spatial probability as an attentional cue in visual search. Percept Psychophys 67 7:1252-1268.
28. Wolfe JM, Vo ML, Evans KK, Greene MR (2011) Visual search in scenes involves selective and nonselective pathways. Trends Cogn Sci 15 2:77-84.
29. Charnov EL (1976) Optimal foraging, the marginal value theorem. Theor Popul Biol 9 2:129-136.
30. Bénichou O, Coppey M, Moreau M, Suet P-H, Voituriez R (2005) Optimal search strategies for hidden targets. Phys Rev Lett 94(19):198101- 198104.
31. Gilchrist ID, North A, Hood B (2001) Is visual search really like foraging? Perception 30 12:1459-1464.
32. Smith AD, Gilchrist ID, Hood BM (2005) Children's search behaviour in large-scale space: Developmental components of exploration. Perception 34(10):1221-1229.
33. Sutton RS (1988) Learning to predict by the method of temporal differences. Mach Learn 3 1:9-44.
34. Montague PR, Sejnowski TJ (1994) The predictive brain: Temporal coincidence and temporal order in synaptic learning mechanisms. Learn Mem 1(1):1-33.
35. Lee D, Seo H, Jung MW (2012) Neural basis of reinforcement learning and decision making. Annu Rev Neurosci 35:287-308.
36. Niv Y, Joel D, Meilijson I, Ruppin E (2002) Evolution of reinforcement learning in uncertain environments: A simple explanation for complex foraging beaviors. Adapt Behav 10 1:5-24.
37. Snider J (2011) Optimal random search for a single hidden target. Phys Rev E Stat Nonlin Soft Matter Phys 83(1 Pt 1):011105.
38. Yu AJ, Cohen JD (2008) Sequential effects: Superstition or rational behavior?, eds Koller D, Schuurmans D, Bengio Y, Bottou L Advances in Neural Information Processing Systems (MIT Press, Cambridge, MA), Vol. 21, pp 1873-1880.
39. Sutton RS, Barto AG (1998) Reinforcement Learning: An Introduction (MIT Press, Cambridge, MA), p xviii.
40. Rayner K (1998) Eye movements in reading and information processing: 20 years of research. Psychol Bull 124 3:372-422.
41. Araujo C, Kowler E, Pavel M (2001) Eye movements during visual search: The costs of choosing the optimal path. Vision Res 41(25-26):3613-3625
42. Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275 5306:1593-1599.
43. Humphries NE, et al. (2010) Environmental context explains Lévy and Brownian movement patterns of marine predators. Nature 465 7301:1066-1069.
44. James A, Plank MJ, Edwards AM (2011) Assessing Lévy walks as models of animal foraging. J R Soc Interface 8 62:1233-1247.
45. Viswanathan GM, et al. (1999) Optimizing the success of random searches. Nature 401(6756):911-914.
46. Adams GK, Watson KK, Pearson J, Platt ML (2012) Neuroethology of decision-making. Curr Opin Neurobiol 22 6:982-989.
47. Fecteau JH, Munoz DP (2003) Exploring the consequences of the previous trial. Nat Rev Neurosci 4 6:435-443.
48. Wilder MH, Mozer MC, Wickens CD (2011) An integrative, experience-based theory of attentional control. J Vis 11(2), 10.1167/11.2.8.
49. Herrnstein RJ (1961) Relative and absolute strength of response as a function of frequency of reinforcement. J Exp Anal Behav 4:267-272.
50. Greggers U, Mauelshagen J (1997) Matching behavior of honeybees in a multiplechoice situation: The differential effect of environmental stimuli on the choice process. Anim Learn Behav 25 4:458-472.
51. Lau B, Glimcher PW (2005) Dynamic response-by-response models of matching behavior in rhesus monkeys. J Exp Anal Behav 84 3:555-579.
52. Bradshaw CM, Szabadi E, Bevan P (1976) Behavior of humans in variable-interval schedules of reinforcement. J Exp Anal Behav 26 2:135-141.
53. Gallistel CR, et al. (2007) Is matching innate? J Exp Anal Behav 87 2:161-199.
54. Mark TA, Gallistel CR (1994) Kinetics of matching. J Exp Psychol Anim Behav Process 20 1:79-95.
55. Baum WM (1982) Choice, changeover, and travel. J Exp Anal Behav 38 1:35-49.
56. Bisley JW, Goldberg ME (2010) Attention, intention, and priority in the parietal lobe. Annu Rev Neurosci 33:1-21.
57. Fecteau JH, Munoz DP (2006) Salience, relevance, and firing: A priority map for target selection. Trends Cogn Sci 10 8:382-390.
58. Mirpour K, Arcizet F, Ong WS, Bisley JW (2009) Been there, seen that: A neural mechanism for performing efficient visual search. J Neurophysiol 102 6:3481-3491.
59. Klein JT, Deaner RO, Platt ML (2008) Neural correlates of social target value in macaque parietal cortex. Curr Biol 18 6:419-424.
60. Platt ML, Glimcher PW (1999) Neural correlates of decision variables in parietal cortex. Nature 400 6741:233-238.
61. Hayden BY, Pearson JM, Platt ML (2011) Neuronal basis of sequential foraging decisions in a patchy environment. Nat Neurosci 14 7:933-939.
62. Mirpour K, Ong WS, Bisley JW (2010) Microstimulation of posterior parietal cortex biases the selection of eye movement goals during search. J Neurophysiol 104(6):3021-3028.
63. Carello CD, Krauzlis RJ (2004) Manipulating intent: Evidence for a causal role of the superior colliculus in target selection. Neuron 43 4:575-583.
64. Nakahara H, Hikosaka O (2012) Learning to represent reward structure: A key to adapting to complex environments. Neurosci Res 74(3-4):177-183
65. Yasuda M, Yamamoto S, Hikosaka O (2012) Robust representation of stable object values in the oculomotor Basal Ganglia. J Neurosci 32 47:16917-16932.
66. Brainard DH (1997) The Psychophysics Toolbox. Spat Vis 10 4:433-436.