The effects of the previous outcome on probabilistic choice in rats

Journal of Experimental Psychology: Animal Behavior Processes

Volumn 39, Issue 1, 2013, Pages 24-38

  Marshall, Andrew T ^a   Kirkpatrick, Kimberly ^a  

^a KANSAS STATE UNIVERSITY   (United States)

Author keywords

Probabilistic choice; Rats; Reward value; Risky choice

Indexed keywords

ANALYSIS OF VARIANCE; ANIMAL; ARTICLE; AUDITORY STIMULATION; DECISION MAKING; INSTRUMENTAL CONDITIONING; LEARNING; MALE; PHYSIOLOGY; RAT; REINFORCEMENT; SPRAGUE DAWLEY RAT; UNCERTAINTY;

ACOUSTIC STIMULATION; ANALYSIS OF VARIANCE; ANIMALS; CHOICE BEHAVIOR; CONDITIONING, OPERANT; MALE; PROBABILITY LEARNING; RATS; RATS, SPRAGUE-DAWLEY; REINFORCEMENT (PSYCHOLOGY); REINFORCEMENT SCHEDULE; UNCERTAINTY;

MLCS; MLOWN;

EID: 84879284737     PISSN: 00977403     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0030765     Document Type: Article

References (61)

1. Bacotti, A. V. (1976). Home cage feeding time controls responding under multiple schedules. Animal Learning & Behavior, 4, 41-44. doi: 10.3758/BF03211983
2. Bateson, M., & Kacelnik, A. (1995). Preferences for fixed and variable food sources: Variability in amount and delay. Journal of the Experimental Analysis of Behavior, 63, 313-329. doi:10.1901/jeab.1995.63-313
3. Battalio, R. C., Kagel, J. H., & MacDonald, D. N. (1985). Animals' choices over uncertain outcomes: Some initial experimental results. The American Economic Review, 75, 597-613.
4. Bayer, H. M., & Glimcher, P. W. (2005). Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron, 47, 129-141. doi:10.1016/j.neuron.2005.05.020
5. Bush, R. R., & Mosteller, F. (1951). A mathematical model for simple learning. Psychological Review, 58, 313-323. doi:10.1037/h0054388
6. Cardinal, R. N., Daw, N., Robbins, T. W., & Everitt, B. J. (2002). Local analysis of behaviour in the adjusting-delay task for assessing choice of delayed reinforcement. Neural Networks, 15, 617-634. doi:10.1016/S0893-6080(02)00053-9
7. Cardinal, R. N., & Howes, N. J. (2005). Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards. BMC Neuroscience, 6, 37. doi:10.1186/1471-2202-6-37
8. Crossman, E. (1983). Las Vegas knows better. The Behavior Analyst, 6, 109-110.
9. Demaree, H. A., Burns, K. J., DeDonno, M. A., Agarwala, E. K., & Everhart, D. E. (2012). Risk dishabituation: In repeated gambling, risk is reduced following low-probability "surprising" events (wins or losses). Emotion, 12, 495-502. doi:10.1037/a0025780
10. Devenport, J. A., Patterson, M. R., & Devenport, L. D. (2005). Dynamic averaging and foraging decisions in horses (Equus callabus). Journal of Comparative Psychology, 119, 352-358. doi:10.1037/0735-7036.119.3 .352
11. Devenport, L. D., & Devenport, J. A. (1994). Time-dependent averaging of foraging information in least chipmunks and golden-mantled ground squirrels. Animal Behaviour, 47, 787-802. doi:10.1006/anbe.1994.1111
12. Devenport, L., Hill, T., Wilson, M., & Ogden, E. (1997). Tracking and averaging in variable environments: A transition rule. Journal of Experimental Psychology: Animal Behavior Processes, 23, 450-460. doi: 10.1037/0097-7403.23.4.450
13. Dixon, M. R., Hayes, L. J., Rehfeldt, R. A., & Ebbs, R. E. (1998). A possible adjusting procedure for studying outcomes of risk-taking. Psychological Reports, 82, 1047-1050. doi:10.2466/pr0.1998.82.3.1047
14. Dunlap, A. S., & Stephens, D. W. (2012). Tracking a changing environment: Optimal sampling, adaptive memory and overnight effects. Behavioural Processes, 89, 86-94. doi:10.1016/j.beproc.2011.10.005
15. Glimcher, P. W. (2011). Understanding dopamine and reinforcement learning: The dopamine reward prediction error hypothesis. PNAS Proceedings of the National Academy of Sciences of the United States of America, 108, 15647-15654. doi:10.1073/pnas.1014269108
16. Green, L., Myerson, J., & Calvert, A. L. (2010). Pigeons' discounting of probabilistic and delayed reinforcers. Journal of the Experimental Analysis of Behavior, 94, 113-123. doi:10.1901/jeab.2010.94-113
17. Greggers, U., & Menzel, R. (1993). Memory dynamics and foraging strategies of honeybees. Behavioral Ecology and Sociobiology, 32, 17-29. doi:10.1007/BF00172219
18. Guilhardi, P., Yi, L., & Church, R. M. (2007). A modular theory of learning and performance. Psychonomic Bulletin & Review, 14, 543-559. doi:10.3758/BF03196805
19. Hayden, B. Y., & Platt, M. L. (2009). Gambling for Gatorade: Risk-sensitive decision making for fluid rewards in humans. Animal Cognition, 12, 201-207. doi:10.1007/s10071-008-0186-8
20. Herrnstein, R. J. (1961). Relative and absolute strength of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4, 267-272. doi:10.1901/jeab.1961.4-267
21. Hollenbeck, J. R., Ilgen, D. R., Phillips, J. M., & Hedlund, J. (1994). Decision risk in dynamic two-stage contexts: Beyond the status quo. Journal of Applied Psychology, 79, 592-598. doi:10.1037/0021-9010.79 .4.592
22. Kacelnik, A., Vasconcelos, M., Monteiro, T., & Aw, J. (2011). Darwin's "tug-of-war" vs. starlings' "horse-racing": How adaptations for sequential encounters drive simultaneous choice. Behavioral Ecology and So-ciobiology, 65, 547-558. doi:10.1007/s00265-010-1101-2
23. Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47, 263-291. doi:10.2307/1914185
24. Kennerley, S. W., Walton, M. E., Behrens, T. E. J., Buckley, M. J., & Rushworth, M. F. S. (2006). Optimal decision making and the anterior cingulate cortex. Nature Neuroscience, 9, 940-947. doi:10.1038/nn1724
25. Keren, G., & Wagenaar, W. A. (1987). Violation of utility theory in unique and repeated gambles. Journal of Experimental Psychology: Learning, Memory, and Cognition, 13, 387-391. doi:10.1037/0278-7393.13.3.387
26. Kirkpatrick, K. (2002). Packet theory of conditioning and timing. Behavioural Processes, 57, 89-106. doi:10.1016/S0376-6357(02)00007-4
27. Lau, B., & Glimcher, P. W. (2005). Dynamic response-by-response models of matching behavior in rhesus monkeys. Journal of the Experimental Analysis of Behavior, 84, 555-579. doi:10.1901/jeab.2005.110-04
28. Lea, S. E. G., & Dow, S. M. (1984). The integration of reinforcements over time. Annals of the New York Academy of Sciences, 423, 269-277. doi:10.1111/j.1749-6632.1984.tb23436.x
29. Leopard, A. (1978). Risk preference in consecutive gambling. Journal of Experimental Psychology: Human Perception and Performance, 4, 521-528. doi:10.1037/0096-1523.4.3.521
30. Madden, G. J., Ewan, E. E., & Lagorio, C. H. (2007). Toward an animal model of gambling: Delay discounting and the allure of unpredictable outcomes. Journal of Gambling Studies, 23, 63-83. doi:10.1007/s10899-006-9041-5
31. March, J. G. (1996). Learning to be risk averse. Psychological Review, 103, 309-319. doi:10.1037/0033-295X.103.2.309
32. Marsh, B., & Kacelnik, A. (2002). Framing effects and risky decisions in starlings. Proceedings of the National Academy of Sciences, 99, 3352-3355. doi:10.1073/pnas.042491999
33. Mazur, J. E. (1987). An adjusting procedure for studying delayed reinforcement. In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative Analyses of Behavior: Vol. 5, The Effect of Delay and of Intervening Events on Reinforcement Value (pp. 55-73). Hills-dale, NJ: Erlbaum.
34. Mazur, J. E. (1988). Choice between small certain and large uncertain reinforcers. Animal Learning & Behavior, 16, 199-205. doi:10.3758/BF03209066
35. Mazur, J. E. (1989). Theories of probabilistic reinforcement. Journal of the Experimental Analysis of Behavior, 51, 87-99. doi:10.1901/jeab.1989 .51-87
36. Mazur, J. E. (1995). Development of preference and spontaneous recovery in choice behavior with concurrent variable-interval schedules. Animal Learning & Behavior, 23, 93-103. doi:10.3758/BF03198020
37. Mazur, J. E. (1996). Past experience, recency, and spontaneous recovery in choice behavior. Animal Learning & Behavior, 24, 1-10. doi:10.3758/BF03198948
38. McCoy, A. N., & Platt, M. L. (2005). Risk-sensitive neurons in macaque posterior cingulate cortex. Nature Neuroscience, 8, 1220-1227. doi: 10.1038/nn1523
39. McGlothlin, W. H. (1956). Stability of choices among uncertain alternatives. The American Journal of Psychology, 69, 604-615. doi:10.2307/1419083
40. Mobini, S., Body, S., Ho, M.-Y., Bradshaw, C. M., Szabadi, E., Deakin, J. F. W., & Anderson, I. M. (2002). Effects of lesions of the orbitofrontal cortex on sensitivity to delayed and probabilistic reinforcement. Psycho-pharmacology, 160, 290-298. doi:10.1007/s00213-001-0983-0
41. Montague, P. R., Dayan, P., Person, C., & Sejnowski, T. J. (1995). Bee foraging in uncertain environments using predictive hebbian learning. Nature, 377, 725-728. doi:10.1038/377725a0
42. Myers, J. L., & Fort, J. G. (1963). A sequential analysis of gambling behavior. Journal of General Psychology, 69, 299-309. doi:10.1080/00221309.1963.9920562
43. Myerson, J., & Green, L. (1995). Discounting of delayed rewards: Models of individual choice. Journal of the Experimental Analysis of Behavior, 64, 263-276. doi:10.1901/jeab.1995.64-263
44. Myerson, J., Green, L., Hanson, J. S., Holt, D. D., & Estle, S. J. (2003). Discounting delayed and probabilistic rewards: Processes and traits. Journal of Economic Psychology, 24, 619-635. doi:10.1016/S0167-4870(03)00005-9
45. Peters, J., & Büchel, C. (2010). Neural representations of subjective reward value. Behavioural Brain Research, 213, 135-141. doi:10.1016/j.bbr .2010.04.031
46. Rachlin, H. (1990). Why do people gamble and keep gambling despite heavy losses? Psychological Science, 1, 294-297. doi:10.1111/j.1467-9280.1990.tb00220.x
47. Rachlin, H., & Frankel, M. (1969). Choice, rate of response, and rate of gambling. Journal of Experimental Psychology, 80, 444-449. doi: 10.1037/h0027488
48. Rachlin, H., Logue, A. W., Gibbon, J., & Frankel, M. (1986). Cognition and behavior in studies of choice. Psychological Review, 93, 33-45. doi:10.1037/0033-295X.93.1.33
49. Rachlin, H., Raineri, A., & Cross, D. (1991). Subjective probability and delay. Journal of the Experimental Analysis of Behavior, 55, 233-244. doi:10.1901/jeab.1991.55-233
50. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical Conditioning II: Current Research and Theory (pp. 64-99). New York, NY: Appleton-Century-Crofts.
51. Searcy, G. D., & Pietras, C. J. (2011). Optimal risky choice in humans: Effects of amount of variability. Behavioural Processes, 87, 88-99. doi:10.1016/j.beproc.2011.01.008
52. Shapiro, M. S., Siller, S., & Kacelnik, A. (2008). Simultaneous and sequential choice as a function of reward delay and magnitude: Normative, descriptive and process-based models tested in the European starling (Sturnus vulgaris). Journal of Experimental Psychology: Animal Behavior Processes, 34, 75-93. doi:10.1037/0097-7403.34.1.75
53. Slattery, J. P., & Ganster, D. C. (2002). Determinants of risk taking in a dynamic uncertain context. Journal of Management, 28, 89-106. doi: 10.1177/014920630202800106
54. Smethells, J. R., Fox, A. T., Andrews, J. J., & Reilly, M. P. (2012). Immediate postsession feeding reduces operant responding in rats. Journal of the Experimental Analysis of Behavior, 97, 203-214. doi:10.1901/jeab.2012.97-203
55. Stopper, C. M., & Floresco, S. B. (2011). Contributions of the nucleus accumbens and its subregions to different aspects of risk-based decision making. Cognitive, Affective, & Behavioral Neuroscience, 11, 97-112. doi:10.3758/s13415-010-0015-9
56. Tatham, T. A., & Zurn, K. R. (1989). The MED-PC experimental apparatus programming system. Behavior Research Methods, Instruments & Computers, 21, 294-302. doi:10.3758/BF03205598
57. Thaler, R. H., & Johnson, E. J. (1990). Gambling with the house money and trying to break even: The effects of prior outcomes on risky choice. Management Science, 36, 643-660. doi:10.1287/mnsc.36.6.643
58. Vasconcelos, M., Monteiro, T., Aw, J., & Kacelnik, A. (2010). Choice in multi-alternative environments: A trial-by-trial implementation of the Sequential Choice Model. Behavioural Processes, 84, 435-439. doi: 10.1016/j.beproc.2009.11.010
59. Weatherly, J. N., & Derenne, A. (2007). Rats playing a slot machine: A preliminary attempt at an animal gambling model. Analysis of Gambling Behavior, 1, 79-89.
60. Winstanley, C. A. (2011). Gambling rats: Insight into impulsive and addictive behavior. Neuropsychopharmacology, 36, 359. doi:10.1038/npp.2010.136
61. Winterrowd, M. F., & Devenport, L. D. (2004). Balancing variable patch quality with predation risk. Behavioural Processes, 67, 39-46. doi: 10.1016/j.beproc.2004.02.003