Information-Seeking, Learning and the Marginal Value Theorem: A Normative Approach to Adaptive Exploration

Proceedings of the 38th Annual Meeting of the Cognitive Science Society, CogSci 2016

Volumn , Issue , 2016, Pages 1793-1798

  Geana, Andra ^a   Wilson, Robert C ^b   Daw, Nathaniel ^a   Cohen, Jonathan D ^a  

^a PRINCETON UNIVERSITY   (United States)

^b University of Arizona   (United States)

Author keywords

decision making; exploration; explore exploit tradeoff; information seeking

Indexed keywords

INFORMATION USE;

ADAPTIVE EXPLORATIONS; AFFECTIVE STATE; DAILY LIVES; DECISIONS MAKINGS; EXPLORE-EXPLOIT TRADEOFF; FIXED NUMBERS; INFORMATION SEEKING; LEARNING ABOUT THE ENVIRONMENTS; MARGINAL VALUES; PERKINS;

DECISION MAKING;

EID: 85027061421     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (21)

1. Behrens. T.E.J., Woolrich, M.W., Walton, M.E. * Rushworth, M.F.S. (2007). Learning the value of information in an uncertain world. Nature Neuroscience 10: 1214 - 1221
2. Charnov, E. L. (1976). Optimal foraging, the marginal value theorem.Theoretical population biology, 9(2), 129-136.
3. Cohen, J. D., McClure, S. M. & Yu, A. J. (2007).Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Philosophical Transactions of the Royal Society: Biological Sciences. 362, 933-942
4. Daw, N. D., O'Doherty, J. P., Dayan, P., Seymour, B., & Dolan, R. J. (2006). Cortical substrates for exploratory decisions in humans. Nature, 441, 876 - 879
5. Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination in human behavior.
6. Diuk, C., Tsai, K., Wallis, J., Botvinick, M., & Niv, Y. (2013). Hierarchical learning induces two simultaneous, but separable, prediction errors in human basal ganglia. The Journal of Neuroscience, 33(13), 5797-5805.
7. Eastwood, J. D., Frischen, A., Fenske, M. J., & Smilek, D. (2012). The Unengaged mind defining boredom in terms of attention. Perspectives on Psychological Science, 7(5), 482-495.
8. Fowler, H. (1967). Satiation and Curiosity: Constructs for a Drive and Incentive-Motivational Theory of Exploration. Psychology of learning and motivation, 1, 157-227.
9. Gallistel, C.R. (1990). The Organization of Learning. Cambridge, MA: Bradford Books/MIT Press.
10. Goldstone, R. L., & Ashpole, B. C. (2004). Human foraging behavior in a virtual environment. Psychonomic bulletin & review, 11(3), 508-514.
11. Hill, A. B., & Perkins, R. E. (1985). Towards a model of boredom. British Journal of Psychology, 76(2), 235-240.
12. Kaelbling, L., Littman, M., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237-285.
13. Krebs, J. R., Kacelnik, A. & Taylor, P. (1978) Tests of optimal sampling by foraging great tits. Nature 275, 27-31Lai & Robbins 1985
14. Kurzban, R., Duckworth, A., Kable, J. W., & Myers, J. (2013). An opportunity cost model of subjective effort and task performance. Behavioral and Brain Sciences, 36(06), 661-679.
15. Nassar, M. R., Wilson, R. C., Heasly, B., & Gold, J. I. (2010). An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. The Journal of Neuroscience, 30(37), 12366-12378.
16. Olsson, O., & S Brown, J. (2006). The foraging benefits of information and the penalty of ignorance. Oikos, 112(2), 260-273.
17. Payzan-LeNestour, E., & Bossaerts, P. (2012). Do not bet on the unknown versus try to find out more: Estimation uncertainty and “unexpected uncertainty” both modulate exploration. Frontiers in Neuroscience
18. Pearson, J. M., Hayden, B. Y., Raghavachari, S., & Platt, M. L. (2009). Neurons in posterior cingulate cortex signal exploratory decisions in a dynamic multioption choice task. Current biology, 19(18), 1532-1537.
19. Schmidhuber, J. (1991). A possibility for implementing curiosity and boredom in model-building neural controllers. In From animals to animats: proceedings of the first international conference on simulation of adaptive behavior (SAB90).
20. Şimşek, O., & Barto, A. (2006). An intrinsic reward mechanism for efficient exploration. Proceedings of the 23rd international conference, 833-840.
21. Wilson, R. C., Geana, A., White, J. M., Ludvig, E. A., & Cohen, J. D. (2014). Humans use directed and random exploration to solve the explore-exploit dilemma. Journal of Experimental Psychology: General, 143(6), 2074.