The rat approximates an ideal detector of changes in rates of reward: Implications for the law of effect

Journal of Experimental Psychology: Animal Behavior Processes

Volumn 27, Issue 4, 2001, Pages 354-372

  Gallistel, C R ^a,d   Mark, Terence A ^a   King, Adam Philip ^b   Latham, P E ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b FAIRFIELD UNIVERSITY   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

^d RUTGERS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ANIMAL; ANIMAL BEHAVIOR; ARTICLE; BRAIN; CONDITIONING; DISCRIMINATION LEARNING; MALE; PHYSIOLOGY; POISSON DISTRIBUTION; RAT; REWARD; SPRAGUE DAWLEY RAT; TIME;

ANIMALS; BEHAVIOR, ANIMAL; BRAIN; CONDITIONING (PSYCHOLOGY); DISCRIMINATION LEARNING; MALE; POISSON DISTRIBUTION; RATS; RATS, SPRAGUE-DAWLEY; REWARD; TIME FACTORS;

EID: 0035490184     PISSN: 00977403     EISSN: None     Source Type: Journal    
DOI: 10.1037/0097-7403.27.4.354     Document Type: Article

References (40)

1. Baum, W. M. (1981). Optimization and the matching law as accounts of instrumental behavior. Journal of the Experimental Analysis of Behavior, 36, 387-403.
2. Bouton, M. E. (1991). Context and retrieval in extinction and in other examples of interference in simple associative learning. In L. Dachowski & C. R. Flaherty (Eds.), Current topics in animal learning (pp. 25-53). Hillsdale, NJ: Erlbaum.
3. Bouton, M. E. (1993a). Context, ambiguity, and classical conditioning. Current Directions in Psychological Science, 3, 49-53.
4. Bouton, M. E. (1993b). Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychological Bulletin, 114, 80-99.
5. Brown, P. L., & Jenkins, H. M. (1968). Autoshaping of the pigeon's key-peck. Journal of the Experimental Analysis of Behavior, 11, 1-8.
6. Buytendijk, F. J. J. (1930). Über das Umlernen [On reversal learning]. Archiv der neerländiscen Physiologie, 15, 283-310.
7. Davison, M., & McCarthy, D. (1988). The matching law: A research review. Hillsdale, NJ: Erlbaum.
8. Devenport, L. D. (1998). Spontaneous recovery without interference: Why remembering is adaptive. Animal Learning and Behavior, 26, 172-181.
9. Devenport, L., Hill, T., Wilson, M., & Ogden, E. (1997). Tracking and averaging in variable environments: A transition rule. Journal of Experimental Psychology: Animal Behavior Processes, 23, 450-460.
10. DeWeese, M., & Zador, A. (1998). Asymmetric dynamics in optimal variance adaptation. Neural Computation, 10, 1179-1202.
11. Dreyfus, L. R. (1991). Local shifts in relative reinforcement rate and time allocation on concurrent schedules. Journal of Experimental Psychology: Animal Behavior Processes, 17, 486-502.
12. Dufort, R. H., Gutman, N., & Kimble, G. A. (1956). One-trial discrimination reversal in the white rat. Journal of Comparative and Physiological Psychology, 47, 248-249.
13. Gallistel, C. R., & Gibbon, J. (2000). Time, rate and conditioning. Psychological Review, 107, 289-344.
14. Gamzu, E., & Williams, D. R. (1971). Classical conditioning of a complex skeletal response. Science, 171, 923-925.
15. Gibbon, J. (1995). Dynamics of time matching: Arousal makes better seem worse. Psychonomic Bulletin and Review, 2, 208-215.
16. Harper, D. G. C. (1982). Competitive foraging in mallards: Ideal free ducks. Animal Behavior, 30, 575-584.
17. Herrnstein, R. J. (1961). Relative and absolute strength of response as a function of frequency of reinforcement. Journal of the Experimental Analysis of Behavior, 4, 267-272.
18. Herrnstein, R. J. (1991). Experiments on stable sub-optimality in individual behavior. American Economic Review, 81, 360-364.
19. Herrnstein, R. J., & Prelec, D. (1991). Melioration: A theory of distributed choice. Journal of Economic Perspectives, 5, 137-156.
20. Herrnstein, R. J., & Vaughan, W. J. (1980). Melioration and behavioral allocation. In J. E. R. Staddon (Ed.), Limits to action: The allocation of individual behavior (pp. 143-176). New York: Academic.
21. Heyman, G. M. (1979). A Markov model description of changeover probabilities on concurrent variable-interval schedules. Journal of the Experimental Analysis of Behavior, 31, 41-51.
22. Heyman, G. M. (1982). Is time allocation unconditioned behavior? In M. Commons, R. Herrnstein, & H. Rachlin (Eds.), Quantitative analyses of behavior: Matching and maximizing accounts (Vol. 2, pp. 459-490). Cambridge, MA: Ballinger Press.
23. Heyman, G. M., & Luce, R. D. (1979). Operant matching is not a logical consequence of maximizing reinforcement rate. Animal Learning and Behavior, 7, 133-140.
24. Krechevsky, I. (1932). Antagonistic visual discrimination habits in the white rat. Journal of Comparative Psychology, 14, 263-277.
25. Lea, S. E. G., & Dow, S. M. (1984). The integration of reinforcements over time. In J. Gibbon & L. Allan (Eds.), Timing and time perception (Vol. 423, pp. 269-277). New York: Annals of the New York Academy of Sciences.
26. Leon, M. I., & Gallistel, C. R. (1998). Self-stimulating rats combine subjective reward magnitude and subjective reward rate multiplicatively. Journal of Experimental Psychology: Animal Behavior Processes, 24, 265-277.
27. Mahadevan, S., & Connell, J. (1992). Automatic programming of behavior-based robots using reinforcement learning. Artificial Intelligence, 55, 311-365.
28. Mark, T. A., & Gallistel, C. R. (1994). The kinetics of matching. Journal of Experimental Psychology: Animal Behavior Processes, 20, 79-95.
29. Mazur, J. E. (1995). Development of preference and spontaneous recovery in choice behavior with concurrent variable-interval schedules. Animal Learning & Behavior, 23, 93-103.
30. Mazur, J. E. (1996). Past experience, recency, and spontaneous recovery in choice behavior. Animal Learning & Behavior, 24, 1-10.
31. Montague, P. R., Dayan, P., & Sejnowski, T. J. (1996). A framework for mesencephalic dopamine systems based on predictive Hebbian learning. Journal of Neuroscience, 16(5), 1936-1947.
32. Myerson, J., & Miezin, F. M. (1980). The kinetics of choice: An operant systems analysis. Psychological Review, 87, 160-174.
33. North, A. J. (1950). Improvement in successive discrimination reversals. Journal of Comparative and Physiological Psychology, 43, 442-460.
34. Pliskoff, S. S. (1971). Effects of symmetrical and asymmetrical changeover delays on concurrent performances. Journal of the Experimental Analysis of Behavior, 16, 249-256.
35. Schmajuk, N. A. (1997). Animal learning and cognition. New York: Cambridge University Press.
36. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593-1599.
37. Staddon, J. E. R. (1977). On Herrnstein's equations and related forms. Journal of the Experimental Analysis of Behavior, 28, 163-170.
38. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning. Cambridge, MA: MIT Press.
39. Williams, B. A. (1988). Reinforcement, choice, and response strength. In R. C. Atkison, R. J. Herrnstein, G. Lindzey, & R. D. Luce (Eds.), Steven's handbook of experimental psychology, 2nd ed. (Vol. 2, pp. 167-244). New York: Wiley.
40. Williams, D. R., & Williams, H. (1969). Automaintenance in the pigeon: Sustained pecking despite contingent non-reinforcement. Journal of the Experimental Analysis of Behavior, 12, 511-520.