The cognitive mechanisms of optimal sampling

Behavioural Processes

Volumn 89, Issue 2, 2012, Pages 77-85

  Lea, Stephen E G ^a   McLaren, Ian P L ^a   Dow, Susan M ^b   Graft, Donald A ^c  

^a UNIVERSITY OF EXETER   (United Kingdom)

^b Veterinary Department   (United Kingdom)

^c STMICROELECTRONICS   (Italy)

Author keywords

Foraging; Matching law; Neural networks; Optimal sampling; Reinforcement learning; Rescorla Wagner model; Two armed bandit

Indexed keywords

ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; COGNITION; LEARNING; NUMERICAL MODEL; OPTIMIZATION; SAMPLING; TRADE-OFF;

ARTICLE; ARTIFICIAL INTELLIGENCE; BIRD; COGNITION; DYNAMICS; LEARNING; MEMORY; MOTIVATION; NERVE CELL NETWORK; NONHUMAN; PERFORMANCE;

ANIMALS; APPETITIVE BEHAVIOR; CHOICE BEHAVIOR; COGNITION; CONDITIONING, OPERANT; MODELS, PSYCHOLOGICAL; REWARD;

ANIMALIA; AVES;

EID: 84856383116     PISSN: 03766357     EISSN: 18728308     Source Type: Journal    
DOI: 10.1016/j.beproc.2011.10.004     Document Type: Article

References (61)

1. Abarca N., Fantino E. Choice and foraging. J. Exper. Anal. Behav. 1982, 38:117-123.
2. Aparicio C.F., Baum W.M. Dynamics of choice: relative rate and amount affect local preference at three different time scales. J. Exp. Anal. Behav. 2009, 91:293-317.
3. Bellman R. A problem in the sequential design of experiments. Sankhya 1956, 16:221-229.
4. Bitterman M.E., Wodinsky J., Candland D.K. Some comparative psychology. Am. J. Psychol. 1958, 71:94-110.
5. Bradt R.N., Karlin S. On the design and comparison of certain dichotomous experiments. Ann. Math. Stat. 1956, 27:390-409.
6. Bush R.R., Mosteller F. A mathematical model for simple learning. Psychol. Rev. 1951, 58:313-323.
7. Charnov E.L., Orians G.H. Optimal Foraging: Some Theoretical Explorations 1973, Author, Vancouver, BC.
8. Clayton N.S., Dickinson A. Episodic-like memory during cache recovery by scrub jays. Nature 1998, 395:272-274.
9. Cohen J.D., McClure S.M., Yu A.J. Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration. Proc. Royal Soc. B 2007, 362:933-942.
10. Collier G.H., Rovee-Collier C.K. A comparative analysis of optimal foraging behavior: laboratory simulations. Foraging Behavior 1981, 39-76. Garland, New York. A.C. Kamil, T.D. Sargent (Eds.).
11. Colton T. A model for selecting one of two medical treatments. J. Am. Stat. Assoc. 1963, 58:388-400.
12. Dall S.R.X., Giraldeau L.A., Olsson O., McNamara J.M., Stephens D.W. Information and its use by animals in evolutionary ecology. Trends Ecol. Evol. 2005, 20:187-193.
13. Dawson M.R.W., Dupuis B., Spetch M.L., Kelly D.M. Simple artificial neural networks that match probability and exploit and explore when confronting a multiarmed bandit. IEEE Trans. Neural Networks 2009, 20:1368-1371.
14. Dow S.M., Lea S.E.G. Foraging in a changing environment: simulations in the operant laboratory. Quantitative Analyses of Behavior, vol. 6: Foraging 1987, 89-113. Erlbaum, Hillsdale, NJ. M.L. Commons, A. Kacelnik, S.J. Shettleworth (Eds.).
15. Dow S.M., Lea S.E.G. Sampling of schedule parameters by pigeons: tests of optimizing theory. Anim. Behav. 1987, 35:102-114.
16. Erev I., Barron G. On adaptation, maximization, and reinforcement learning among cognitive strategies. Psychol. Rev. 2005, 112:912-931.
17. Findley J.D. Preference and switching under concurrent scheduling. J. Exp. Anal. Behav. 1958, 1:123-144.
18. Fisher J., Hinde R.A. The opening of milk bottles by birds. Br. Birds 1949, 42:347-357.
19. Fox B.L. Finite horizon behavior of policies for two-arm bandits. J. Am. Stat. Assoc. 1974, 69:963-965.
20. Gigerenzer G., Goldstein D.G. Reasoning the fast and frugal way: models of bounded rationality. Psychol. Rev. 1996, 103:650-669.
21. Gross R., Houston A.I., Collins E.J., McNamara J.M., Dechaume-Moncharmont F.X., Franks N.R. Simple learning rules to cope with changing environments. J. Royal Soc. Interface 2008, 5:1193-1202.
22. Herrnstein R.J. Relative and absolute strengths of response as a function of frequency of reinforcement. J. Exp. Anal. Behav. 1961, 4:267-272.
23. Herrnstein R.J. On the law of effect. J. Exp. Anal. Behav. 1970, 13:243-266.
24. Herrnstein R.J., Loveland D.H. Maximizing and matching on concurrent ratio schedules. J. Exp. Anal. Behav. 1975, 24:107-116.
25. Herrnstein R.J., Loveland D.H., Cable C. Natural concepts in pigeons. J. Exp. Psychol. Anim. Behav. Proc. 1976, 2:285-302.
26. Hills T.T., Hertwig R. Information search in decisions from experience: do our patterns of sampling foreshadow our decisions?. Psychol. Sci. 2010, 21:1787-1792.
27. Houston A.I., Kacelnik A., McNamara J. Some learning rules for acquiring information. Functional Ontogeny 1982, 140-191. Pitman, Boston. D.J. McFarland (Ed.).
28. Kacelnik, A., 1979. Studies of foraging behaviour and time budgeting in great tits (Parus major). Unpublished DPhil thesis, University of Oxford.
29. Kamil A.C., Balda R.P. Cache recovery and spatial memory in Clark's nutcrackers (Nucifraga columbiana). J. Exp. Psychol. Anim. Behav. Proc. 1985, 11:95-111.
30. Keasar T., Rashkovich E., Cohen D., Shmida A. Bees in two-armed bandit situations: foraging choices and possible decision mechanisms. Behav. Ecol. 2002, 13:757-765.
31. Kersten M., Visser W. The rate of food processing in the oystercatcher: food intake and energy expenditure constrained by a digestive bottleneck. Funct. Ecol. 1996, 10:440-448.
32. Killeen P.R. Incentive theory: III. Adaptive clocks. Ann. N. Y. Acad. Sci. 1984, 423:515-527.
33. Krebs J.R., Kacelnik A., Taylor P. Test of optimal sampling by foraging great tits. Nature 1978, 275:27-31.
34. Lea S.E.G. Foraging and reinforcement schedules in the pigeon: optimal and non-optimal aspects of choice. Anim. Behav. 1979, 27:875-886.
35. Lea S.E.G. Operant psychology and ethology: failures and successes in interdisciplinary interaction. Behaviour Analysis and Contemporary Psychology 1985, 43-51. Erlbaum, London. C.F. Lowe, M. Richelle, D.E. Blackman, C.M. Bradshaw (Eds.).
36. Lea S.E.G., Dow S.M. The integration of reinforcements over time. Ann. N. Y. Acad. Sci. 1984, 423:269-277.
37. MacArthur R.H., Pianka E.R. On the optimal use of a patchy environment. Am. Nat. 1966, 100:603-609.
38. Mackintosh N.J., Wilson B., Boakes R.A. Differences in mechanisms of intelligence among vertebrates. Phil. Trans. Royal Soc. B: Biol. Sci. 1985, 308:53-65.
39. Mazur J.E. Choice behavior in transition: development of preference with ratio and interval schedules. J. Exp. Psychol. Anim. Behav. Proc. 1992, 18:364-378.
40. McNamara J. Optimal patch use in a stochastic environment. Theor. Popul. Biol. 1982, 21:269-288.
41. McNamara J.M. Risk-prone behavior under rules which have evolved in a changing environment. Am. Zool. 1996, 36:484-495.
42. McNamara J.M., Houston A.I. Optimal foraging and learning. J. Theor. Biol. 1985, 117:231-249.
43. Myerson J., Miezin F.M. The kinetics of choice: an operant systems analysis. Psychol. Rev. 1980, 87:160-174.
44. Oaten A. Optimal foraging in patches: case for stochasticity. Theor. Popul. Biol. 1977, 12:263-285.
45. Plowright C.M.S., Plowright R.C. Oversampling by great tits: a critique of Krebs, Kacelnik, and Taylor's (1978) test of optimal sampling by great tits. Can. J. Zool. 1987, 65:1282-1283.
46. Plowright C.M.S., Shettleworth S.J. Time horizon and choice by pigeons in a prey-selection task. Anim. Learn. Behav. 1991, 19:103-112.
47. Rayburn-Reeves R.M., Molet M., Zentall T.R. Simultaneous discrimination reversal learning in pigeons and humans: anticipatory and perseverative errors. Learn. Behav. 2011, 39:125-137.
48. Rayburn-Reeves, R.M., Stagner, J.P., Kirk, C.R., Zentall, T.R., in press. Mid-session simultaneous discrimination reversal differences between rats and pigeons. J. Comp. Psychol.
49. Rescorla R.A., Wagner A.R. A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. Classical Conditioning II: Current Research and Theory 1972, 64-99. Appleton-Century-Crofts, New York. A.H. Black, W.F. Prokasy (Eds.).
50. Shettleworth S.J., Plowright C.M.S. Time horizons of pigeons on a two-armed bandit. Anim. Behav. 1989, 37:610-623.
51. Simon H.A. Rational choice and the structure of the environment. Psychol. Rev. 1956, 63:129-138.
52. Stephens D.W. Models of information use. Foraging 2007, 31-58. Chicago University Press, Chicago. D.W. Stephens, J.S. Brown, R.C. Ydenberg (Eds.).
53. Stephens D.W., Krebs J.R. Foraging Theory 1986, Princeton University Press, Princeton, NJ.
54. Foraging 2007, Chicago University Press, Chicago. D.W. Stephens, J.S. Brown, R.C. Ydenberg (Eds.).
55. Sutton R.S., Barto A.B. Reinforcement Learning: An Introduction 1998, MIT Press, Cambridge, MA.
56. Thomas G., Kacelnik A., Van Der Meulen J. The three-spined stickleback and the two-armed bandit. Behaviour 1985, 93:227-240.
57. Thompson W.R. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 1933, 25:285-294.
58. Tolman E.C. Principles of performance. Psychol. Rev. 1955, 62:315-326.
59. Ward D. The role of satisficing in foraging theory. Oikos 1992, 63:312-317.
60. Wills A.J., McLaren I.P.L. A connectionist account of differences in gradient after discriminative and non-discriminative training. Q. J. Exp. Psychol. 1997, 50:607-630.
61. Zwarts L., Dirksen S. Digestive bottleneck limits the increase in food intake of whimbrels preparing to migrate from the Banc d'Arguin, Mauritania. Ardea 1990, 78:257-278.