When good news leads to bad choices

Journal of the Experimental Analysis of Behavior

Volumn 105, Issue 1, 2016, Pages 23-40

  McDevitt, Margaret A ^a   Dunn, Roger M ^b   Spetch, Marcia L ^c   Ludvig, Elliot A ^d  

^a MCDANIEL COLLEGE   (United States)

^b SAN DIEGO STATE UNIVERSITY   (United States)

^c UNIVERSITY OF ALBERTA   (Canada)

^d UNIVERSITY OF WARWICK   (United Kingdom)

Author keywords

conditioned reinforcement; context; pigeons; preference; probabilistic; signals; suboptimal choice

Indexed keywords

ANIMAL; DECISION MAKING; DELAY DISCOUNTING; PIGEONS AND DOVES; PSYCHOTRAUMA; REINFORCEMENT; REWARD; TIME FACTOR;

ANIMALS; CHOICE BEHAVIOR; COLUMBIDAE; DELAY DISCOUNTING; PSYCHOLOGICAL TRAUMA; REINFORCEMENT (PSYCHOLOGY); REWARD; TIME FACTORS;

EID: 84968786793     PISSN: 00225002     EISSN: 19383711     Source Type: Journal    
DOI: 10.1002/jeab.192     Document Type: Review

References (80)

1. Alsop, B., & Davison, M. (1986). Preference for multiple versus mixed schedules of reinforcement. Journal of the Experimental Analysis of Behavior, 45(1), 33–45. doi:10.1901/jeab.1986.45-33
2. Anselme, P., Robinson, M. F., & Berridge, K. C. (2013). Reward uncertainty enhances incentive salience attribution as sign-tracking. Behavioural Brain Research, 238, 53-61. doi:10.1016/j.bbr.2012.10.006
3. Ariely, D. (2008). Predictably irrational: the hidden forces that shape our decisions. New York, NY: HarperCollins.
4. Beckmann, J. S., & Chow, J. J. (2015). Isolating the incentive salience of reward-associated stimuli: Value, choice, and persistence. Learning & Memory, 22(2), 116–127. doi:10.1101/lm.037382.114
5. Beierholm, U. R., & Dayan, P. (2010). Pavlovian-instrumental interaction in ‘observing behavior’. PLoS Comput Biol, 6(9), e1000903.
6. Belke, T. W., & Spetch, M. L. (1994). Choice between reliable and unreliable reinforcement alternatives revisited: Preference for unreliable reinforcement. Journal of the Experimental Analysis of Behavior, 62(3), 353. doi:10.1901/jeab.1994.62-353
7. Bell, M. C., & McDevitt, M. A. (2014). Conditioned reinforcement. In F. K. McSweeney and E. S. Murphy (Eds.), The Wiley-Blackwell handbook of operant and classical conditioning. Oxford, UK : John Wiley & Sons, Ltd. doi: 10.1002/9781118468135.ch10
8. Bell, M. C., & Williams, B. A. (2013). Conditioned reinforcement in chain schedules when time to reinforcement is held constant. Journal of the Experimental Analysis of Behavior, 99(2), 179–188. doi:10.1002/jeab.10
9. Blanchard, T. C., Hayden, B. Y., & Bromberg-Martin, E. S. (2015). Orbitofrontal cortex uses distinct codes for different choice attributes in decisions motivated by curiosity. Neuron, 85(3), 602–614.
10. Bradshaw, C. M., & Szabadi, E. (1992). Choice between delayed reinforcers in a discrete-trials schedule: The effect of deprivation level. The Quarterly Journal of Experimental Psychology B: Comparative and Physiological Psychology, 44B(1), 1–16.
11. Bromberg-Martin, E. S., & Hikosaka, O. (2009). Midbrain dopamine neurons signal preference for advance information about upcoming rewards. Neuron, 63(1), 119–126. doi:10.1016/j.neuron.2009.06.009
12. Davison, M., & Baum, W. M. (2006). Do conditional reinforcers count? Journal of the Experimental Analysis of Behavior, 86, 269–283. doi:10.1901/jeab.2006.56-05
13. Dinsmoor, J. A. (1983). Observing and conditioned reinforcement. Behavioral and Brain Sciences, 6, 693–728. doi:10.1017/S0140525X00017969
14. Dinsmoor, J. A., Browne, M. P., & Lawrence, C. E. (1972). A test of the negative discriminative stimulus as a reinforcer of observing. Journal of the Experimental Analysis of Behavior, 18(1), 79–85. doi:10.1901/jeab.1972.18-79
15. Dunn, R., & Spetch, M. L. (1990). Choice with uncertain outcomes: Conditioned reinforcement effects. Journal of the Experimental Analysis of Behavior, 53(2), 201–218. doi:10.1901/jeab.1990.53-201
16. Eisenberger, R., Masterson, F. A., & Lowman, K. (1982). Effects of previous delay of reward, generalized effort, and deprivation on impulsiveness. Learning and Motivation, 13(3), 378–389.
17. Fantino, E. (1969). Choice and rate of reinforcement. Journal of the Experimental Analysis of Behavior, 12(5), 723–730. doi:10.1901/jeab.1969.12-723
18. Fantino, E. (2012). Optimal and non-optimal behavior across species. Comparative Cognition & Behavior Reviews, 7, 44–54. Retrieved from http://comparative-cognition-and-behavior-reviews.org/doi:10.3819/ccbr.2012.70003
19. Fantino, E., & Moore, J. (1980). Uncertainty reduction, conditioned reinforcement, and observing. Journal of the Experimental Analysis of Behavior, 33(1), 3–13. doi:10.1901/jeab.1980.33-3
20. Fantino, E., Preston, R. A., & Dunn, R. (1993). Delay reduction: Current status. Journal of the Experimental Analysis of Behavior, 60(1), 159–169. doi:10.1901/jeab.1993.60-159
21. Gipson, C. D., Alessandri, J. J. D., Miller, H. C., & Zentall, T. R. (2009). Preference for 50% reinforcement over 75% reinforcement by pigeons. Learning & Behavior, 37(4), 289–298. doi:10.3758/LB.37.4.289
22. Green, L., Myerson, J., Lichtman, D., Rosen, S., & Fry, A. (1996). Temporal discounting in choice between delayed rewards: The role of age and income. Psychology and Aging, 11(1), 79–84. doi:10.1037/0882-7974.11.1.79
23. Green, L., & Rachlin, H. (1977). Pigeons’ preferences for stimulus information: Effects of amount of information. Journal of the Experimental Analysis of Behavior, 27(2), 255–263. doi:10.1901/jeab.1977.27-255
24. Hursh, S. R., & Fantino, E. (1974). An appraisal of preference for multiple versus mixed schedules. Journal of the Experimental Analysis of Behavior, 22(1), 31. doi:10.1901/jeab.1974.22-31
25. Kahneman, D. (2011). Thinking, fast and slow. New York, NY: Farrar, Straus, & Giroux.
26. Kendall, S. B. (1974). Preference for intermittent reinforcement. Journal of the Experimental Analysis of Behavior, 21(3), 463–473. doi:10.1901/jeab.1974.21-463
27. Kendall, S. B. (1985). A further study of choice and percentage reinforcement. Behavioural Processes, 10, 399–413. doi:10.1016/0376-6357(85)90040-3
28. Killeen, P., Wald, B., & Cheney, C. D. (1980). Observing behavior and information. The Psychological Record, 30(2), 181–190.
29. Lalli, J. S., & Mauro, B. C. (1995). The paradox of preference for unreliable reinforcement: The role of context and conditioned reinforcement. Journal of Applied Behavior Analysis, 28(3), 389–394. doi:10.1901/jaba.1995.28-389
30. Lalli, J. S., Mauro, B. C., & Mace, F. C. (2000). Preference for unreliable reinforcement in children with mental retardation: The role of conditioned reinforcement. Journal of Applied Behavior Analysis, 33(4), 533–544. doi:10.1901/jaba.2000.33-533
31. Laude, J. R., Beckmann, J. S., Daniels, C. W., & Zentall, T. R. (2014). Impulsivity affects suboptimal gambling-like choice by pigeons. Journal of Experimental Psychology: Animal Behavior Processes, 40(1), 2–11. doi:10.1037/xan0000001
32. Laude, J. R., Pattison, K. F., & Zentall, T. R. (2012). Hungry pigeons make suboptimal choices, less hungry pigeons do not. Psychonomic Bulletin & Review, 19, 884–891. doi: 10.3758/s13423-012-0282-2
33. Laude, J. R., Stagner, J. P., & Zentall, T. R. (2014). Suboptimal choice by pigeons may result from the diminishing effect of nonreinforcement. Journal of Experimental Psychology: Animal Learning and Cognition, 40(1), 12–21. doi:10.1037/xan0000010
34. Ludvig, E. A., Spetch, M. L., Dunn, R. M., & McDevitt, M. A. (June, 2015). When good news leads to bad choices: A reinforcement-learning model of information-driven suboptimal choice. Paper presented at RLDM: The Multi-disciplinary Conference on Reinforcement Learning and Decision Making. Edmonton, Canada
35. Mazur, J. E. (1984). Tests of an equivalence rule for fixed and variable reinforcer delays. Journal of Experimental Psychology: Animal Behavior Processes, 10(4), 426–436. doi:10.1037/0097-7403.10.4.426
36. Mazur, J. E. (1989). Theories of probabilistic reinforcement. Journal of The Experimental Analysis of Behavior, 51(1), 87–99. doi:10.1901/jeab.1989.51-87
37. Mazur, J. E., (1991), Choice with probabilistic reinforcement: Effects of delay and conditioned reinforcers. Journal of the Experimental Analysis of Behavior, 55(1), 63–77.
38. Mazur, J. E. (1995). Conditioned reinforcement and choice with delayed and uncertain primary reinforcers. Journal of the Experimental Analysis of Behavior, 63(2), 139–150. doi: 10.1901/jeab.1995.63-139
39. Mazur, J. E. (2001). Hyperbolic value addition and general models of animal choice. Psychological Review, 108(1), 96–112. doi:10.1037/0033-295X.108.1.96
40. Mazur, J. E. (2005). Effects of reinforcer probability, delay, and response requirements on the choices of rats and pigeons: Possible species differences. Journal of the Experimental Analysis of Behavior, 83(3), 263–279. doi:10.1901/jeab.2005.69-04
41. Mazur, J. E. (2007). Species differences between rats and pigeons in choices with probabilistic and delayed reinforcers. Behavioural Processes, 75(2), 220–224. doi:10.1016/j.beproc.2007.02.004
42. Mazur, J. E., & Biondi, D. R. (2009). Delay-amount tradeoffs in choices by pigeons and rats: Hyperbolic versus exponential discounting. Journal of the Experimental Analysis of Behavior, 91(2), 197–211. doi:10.1901/jeab.2009.91-197
43. McDevitt, M. A., Spetch, M. L., & Dunn, R. (1997). Contiguity and conditioned reinforcement in probabilistic choice. Journal of the Experimental Analysis of Behavior, 68(3), 317–327. doi:10.1901/jeab.1997.68-317
44. McDevitt, M. A., & Williams, B. A. (2001). Effects of signalled versus unsignalled delay of reinforcement on choice. Journal of the Experimental Analysis of Behavior, 75(2), 165–182. doi:10.1901/jeab.2001.75-165
45. McDevitt, M. A., & Williams, B. A. (2010). Dual effects on choice of conditioned reinforcement frequency and conditioned reinforcement value. Journal of the Experimental Analysis of Behavior, 93(2), 147–155. doi:10.1901/jeab.2010.93-147
46. McMillan, J. C. (1974). Average uncertainty as a determinant of observing behavior. Journal of the Experimental Analysis of Behavior, 22(2), 401–408. doi: 10.1901/jeab.1974.22-401
47. Molet, M., Miller, H. C., Laude, J. R., Kirk, C., Manning, B., & Zentall, T. R. (2012). Decision making by humans in a behavioral task: Do humans, like pigeons, show suboptimal choice? Learning & Behavior, 40(4), 439–447. doi:10.3758/s13420-012-0065-7
48. Oliveira, L., Calvert, A. L., Green, L., & Myerson, J. (2013). Level of deprivation does not affect degree of discounting in pigeons. Learning & Behavior, 41(2), 148–158. doi:10.3758/s13420-012-0092-4
49. Pattison, K. F., Laude, J. R., & Zentall, T. R. (2013). Environmental enrichment affects suboptimal, risky, gambling-like choice by pigeons. Animal Cognition, 16(3), 429–434. doi: 10.1007/s10071-012-0583-x
50. Petry, N. M. (2001). Pathological gamblers, with and without substance abuse disorders, discount delayed rewards at high rates. Journal of Abnormal Psychology, 110(3), 482–487. doi:10.1037/0021-843X.110.3.482
51. Pisklak, J. M., McDevitt, M. A., Dunn, R. M., & Spetch M. L. (in press). When good pigeons make bad decisions: Choice with probabilistic delays and outcome. Journal of the Experimental Analysis of Behavior.
52. Pyke, G.H., Pulliam, H. R., & Charnov, E. L. (1977). Optimal foraging: A selective review of theory and tests. The Quarterly Review of Biology, 52(2) 137–154. http://www.jstor.org/stable/2824020
53. Rachlin, H., & Green, L. (1972). Commitment, choice and self-control. Journal of the Experimental Analysis of Behavior, 17(1), 15–22. doi: 10.1901/jeab.1972.17-15
54. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II (pp. 64–99). New York: Appleton-Century-Crofts.
55. Roper, K. L., & Zentall, T. R. (1999). Observing behavior in pigeons: The effect of reinforcement probability and response cost using a symmetrical choice procedure. Learning and Motivation, 30, 201–220. doi:10.1006/lmot.1999.1030
56. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593–1599.
57. Shahan, T. A. (2010). Conditioned reinforcement and response strength. Journal of the Experimental Analysis of Behavior, 93(2), 269–289. doi:10.1901/jeab.2010.93-269
58. Shahan, T. A. (2013). Attention and conditioned reinforcement. In G. J. Madden, W. V. Dube, T. D. Hackenberg, G. P. Hanley, & K. A. Lattal (Eds.), APA handbook of behavior analysis, vol. 1: methods and principles (pp. 387–410). Washington, DC: American Psychological Association. doi:10.1037/13937-017
59. Shahan, T. A., & Cunningham, P. (2015). Conditioned reinforcement and information theory reconsidered. Journal of the Experimental Analysis of Behavior, 103, 405–418. doi:10.1002/jeab.142
60. Shahan, T. A., Podlesnik, C. A., & Jimenez-Gomez, C. (2006). Matching and conditioned reinforcement rate. Journal of the Experimental Analysis of Behavior, 85, 167–180. doi:10.1901/jeab.2006.34-05
61. Silberberg, A., & Fantino, E. (2010). Observing responses: Maintained by good news only? Behavioural Processes, 85(1), 80–82. doi:10.1016/j.beproc.2010.06.002
62. Singer, R. A., Berry, L. M., & Zentall, T. R. (2007). Preference for a stimulus that follows a relatively aversive event: Contrast or delay reduction? Journal of the Experimental Analysis of Behavior, 87(2), 275–285. doi:10.1901/jeab.2007.39-06
63. Singer, R. A., & Zentall, T. R. (2011). Preference for the outcome that follows a relatively aversive event: Contrast or delay reduction? Learning and Motivation, 42(3), 255–271. doi:10.1016/j.lmot.2011.06.001
64. Spetch, M., Belke, T., Barnet, R., Dunn, R., & Pierce, W. (1990). Suboptimal choice in a percentage-reinforcement procedure: Effects of signal condition and terminal-link length. Journal of the Experimental Analysis of Behavior, 53(2), 219–234. doi: 10.1901/jeab.1990.53-219.
65. Spetch, M. L., & Dunn, R. (1987). Choice between reliable and unreliable outcomes: Mixed percentage-reinforcement in concurrent chains. Journal of the Experimental Analysis of Behavior, 47(1), 57–72. doi:10.1901/jeab.1987.47-57
66. Spetch, M., Mondloch, M., Belke, T., & Dunn, R. (1994). Determinants of pigeons’ choice between certain and probabilistic outcomes. Animal Learning & Behavior, 22(3), 239–251. doi: 10.3758/BF03209832.
67. Stagner, J. P., Laude, J. R., & Zentall, T. R. (2011). Sub-optimal choice in pigeons does not depend on avoidance of the stimulus associated with the absence of reinforcement. Learning and Motivation, 42(4), 282–287. doi:10.1016/j.lmot.2011.09.001
68. Stagner, J. P., Laude, J. R., & Zentall, T. R. (2012). Pigeons prefer discriminative stimuli independently of the overall probability of reinforcement and of the number of presentations of the conditioned reinforcer. Journal of Experimental Psychology. Animal Behavior Processes, 38(4), 446–452. doi:10.1037/a0030321
69. Stagner, J. P., & Zentall, T. R. (2010). Suboptimal choice behavior by pigeons. Psychonomic Bulletin & Review, 17(3), 412–416. doi:10.3758/PBR.17.3.412
70. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
71. Vasconcelos, M., Monteiro, T., & Kacelnik, A. (2015). Irrational choice and the value of information. Scientific Reports, 5, 13874.
72. Williams, B. A. (1994). Conditioned reinforcement: Neglected or outmoded explanatory construct? Psychonomic Bulletin & Review, 1(4), 457–475. doi:10.3758/BF03210950
73. Williams, B. A. & Dunn, R. (1991). Preference for conditioned reinforcement. Journal of the Experimental Analysis of Behavior, 55(1), 37–46. doi:10.1901/jeab.1991.55-37
74. Wyckoff, L. B., Jr. (1952). The role of observing responses in discrimination learning. Part I. Psychological Review, 59(6), 431–442. doi:10.1037/h0053932
75. Zentall, T. R. (2015). When animals misbehave: analogs of human biases and suboptimal choice. Behavioural Processes, 112, 3–13. doi:10.1016/j.beproc.2014.08.001
76. Zentall, T. R., & Laude, J. R. (2013). Do pigeons gamble? I wouldn't bet against it. Current Directions in Psychological Science, 22(4), 271–277. doi: 10.1177/0963721413480173
77. Zentall, T. R., Laude, J. R, Stagner, J. P. & Smith, A. P. (2015) Suboptimal choice by pigeons: Evidence that the value of the conditioned reinforcer rather than its frequency determines choice. Psychological Record, 65, 223–229. doi 10.1007/s40732-015-0119-2
78. Zentall, T. R., & Stagner, J. (2011a). Maladaptive choice behavior by pigeons: An animal analogue and possible mechanism for gambling (sub-optimal human decision-making behavior). Proceedings of the Royal Society B: Biological Sciences, 278, 1203–1208. doi:10.1098/rspb.2010.1607
79. Zentall, T. R., & Stagner, J. (2011b). Sub-optimal choice by pigeons: Failure to support the Allais paradox. Learning and Motivation, 42, 245–254. doi: 10.1016/j.lmot.2011.03.002
80. Zentall, T. R., & Stagner, J. P. (2012). Do pigeons prefer information in the absence of differential reinforcement? Learning & Behavior, 40(4), 465–475. doi:10.3758/s13420-012-0067-5