A dynamic theory of acquisition and extinction in operant learning

Neural Networks

Volumn 10, Issue 2, 1997, Pages 201-229

  Dragoi, Valentin ^a  

^a DUKE UNIVERSITY   (United States)

Author keywords

assignment of credit; contingency; expectancy; long term memory; operant conditioning; recurrent choice; reinforcement learning; short term memory

Indexed keywords

ASSOCIATIVE STORAGE; LEARNING SYSTEMS; MATHEMATICAL MODELS; PROBABILITY;

ASSIGNMENT OF CREDIT; CONTINGENCY; EXPECTANCY; LONG TERM MEMORY; OPERANT CONDITIONING; OPERANT LEARNING; RECURRENT CHOICE; REINFORCEMENT LEARNING; SHORT TERM MEMORY;

NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; AVOIDANCE BEHAVIOR; CONDITIONING; LONG TERM MEMORY; PRIORITY JOURNAL; REINFORCEMENT;

EID: 0031104847     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(96)00067-6     Document Type: Article

References (78)

1. Adams, C. D. (1982). Variations in the sensitivity of instrumental responding to reinforcer devaluation. Quarterly Journal of Experimental Psychology, 34B, 77-98.
2. Amsel, A. (1962). Frustrative nonreward in partial reinforcement and discrimination learning. Psychological Review, 69, 306-328.
3. Bacon, W. E. (1962). Partial-reinforcement extinction effect following different amounts of training. Journal of Comparative and Physiological Psychology, 55, 998-1003.
4. Bailey, J. T., & Mazur, J. E. (1990). Choice behavior in transition: development of preference for the higher probability of reinforcement. Journal of the Experimental Analysis of Behavior, 53, 409-422.
5. Benefield, R., Oscos, A., & Ehrefreund, D. (1974). Role of frustration in successive positive contrast effect. Journal of Comparative and Physiological Psychology, 86, 648-651.
6. Bernheim, J. W., & Williams, D. R. (1967). Time-dependent contrast effects in a multiple schedule of food reinforcement. Journal of the Experimental Analysis of Behavior, 10, 243-249.
7. Black, R. W. (1968). Shifts in magnitude of reward and contrast effects in instrumental conditioning. Psychological Review, 75, 114-126.
8. Bower, G. H. (1961). A contrast effect in differential conditioning. Journal of Experimental Psychology, 62, 196-199.
9. Burstein, K. R. (1967). Spontaneous recovery: A (Hullian) noninhibition interpretation. Psychonomic Science, 7, 389-390.
10. Capaldi, E. J. (1967). Sequential versus nonsequential variables in partial delay of reward. Journal of Experimental Psychology, 74, 161-166.
11. Capaldi, E. J. (1971). Memory and learning: a sequential viewpoint. In W. K. Honig & P.H.R. James (Eds.). Animal memory (pp. 111-154). New York: Academic Press.
12. Capaldi, E. J., & Minkoff, R. (1967). Reward schedule effects at a relatively long intertrial interval. Psychonomic Science, 9, 169-170.
13. Chung, S. H., & Herrnstein, R. J. (1967). Choice and delay of reinforcement. Journal of the Experimental Analysis of Behavior, 10, 67-74.
14. Cohen, S. L., Riley, D. S., & Weigle, P. A. (1993). Tests of behavior momentum in simple and multiple schedules with rats and pigeons. Journal of the Experimental Analysis of Behavior, 60, 255-291.
15. Collwill, R. M., & Rescorla, R. A. (1985). Instrumental responding remains sensitive to reinforcer devaluation after extensive training. Journal of Experimental Psychology: Animal Behavior Processes, 11, 520-536.
16. Cox, W. M. (1975). A review of recent incentive contrast studies involving discrete trial procedures. The Psychological Record, 25, 373-393.
17. Crespi, L. P. (1952). Quantitative variation of incentive and performance in the white rat. American Journal of Psychology, 55, 467-517.
18. Daly, H. B., & Daly, J. T. (1982). A mathematical model of reward and aversive nonreward: its application in over 30 appetitive learning situations. Journal of Experimental Psychology: General, 111, 441-480.
19. Davis, D. G. S., Staddon, J. E. R., Machado, A., & Palmer, R. (1993). The process of recurrent choice. Psychological Review, 100, 320-341.
20. Di Lollo, V., & Beez, V. (1966). Negative contrast effect as a function of magnitude of reward decrement. Psychonomic Science, 5, 99-100.
21. Dragoi, V., & Staddon, J. E. R. (1993). A competitive neural network model for the process of recurrent choice. In M. C. Mozer, P. Smolensky, D. S. Touretzky, J. L. Elman, & A. S. Weigend (Eds.), Proceedings of the 1993 Connectionist Models Summer School (pp. 65-73). Hillsdale, N.J.: Lawrence Erlbaum Associates.
22. Estes, W. K. (1955). Statistical theory of spontaneous recovery and regression. Psychological Review, 62, 145-154.
23. Franchina, J. J., & Brown, T. S. (1971). Reward magnitude shifts effects in rats with hippocampal lesions. Journal of Comparative Physiological Psychology, 76, 365-370.
24. Gray, J. A. (1971). The psychology of fear and stress. London: Weidenfeld and Nicholson.
25. Gray, J. A. (1982). The neuropsychology of anxiety: an enquiry into the functions of the septo-hippocampal system. New York: Oxford University Press.
26. Grossberg, S. (1972). A neural theory of punishment and avoidance, II: Quantitative theory. Mathematical Biosciences, 15, 39-67.
27. Grossberg, S. (1975). A neural model of attention, reinforcement, and discrimination learning. International Review of Neurobiology, 18, 263-325.
28. Grossberg, S. (1981). Psychophysiological substrates of schedule interactions and behavioral contrast. SIAM-AMS Proceedings, 13, 157-186.
29. Grossberg, S. (1982). Processing of expected and unexpected events during conditioning and attention: a psychophysiological theory. Psychological Review, 89, 529-572.
30. Gutman, A. (1977). Positive contrast, negative induction, and inhibitory stimulus control in rat. Journal of the Experimental Analysis of Behavior, 27, 219-233.
31. Harlow, H. F. (1949). The formation of learning sets. Psychological Review, 56, 51-65.
32. Herrnstein, R. J. (1961). Relative and absolute strength of response as a function of frequency of reinforcement. Journal of Experimental Analysis and Behavior, 4, 267-272.
33. Herrnstein, R. J. (1970). On the law of effect. Journal of the Experimental Analysis of Behavior, 13, 243-266.
34. Hull, C. L. (1943). Principles of behavior. New York: Appleton-Century-Crofts.
35. Kacelnik, A., Krebs, J. R. & Ens, B. (1987). Foraging in a changing environment: an experiment with starlings (sturnus vulgaris). In M. L. Commons, A. Kacelnik & S. J. Shettleworth (Eds.), Quantitative analyses of behavior VI: foraging (pp. 63-87). Hillsdale, NJ: Laurence Erlbaum.
36. Killeen, P. (1968). on the measurement of reinforcement frequency in the study of preference. Journal of the Experimental Analysis of Behavior, 11, 263-269.
37. Killeen, P. (1970). Preference for fixed-interval schedules of reinforcement. Journal of the Experimental Analysis of Behavior, 14, 127-131.
38. Klopf, A. H. (1988). A neuronal model of classical conditioning. Psychobiology, 16, 85-125.
39. Luce, R. D. (1959). Individual choice behavior: a theoretical analysis. New York: John Wiley.
40. Mackintosh, N. J. (1974). The psychology of animal learning. New York: Academic Press.
41. Maxwell, F. R., Calef, R. S., Murray, D. W., Shephard, D. C. & Norville, R. A. (1976). Positive and negative successive contrast effects following multiple shifts in reward magnitude under high drive and immediate reinforcement. Animal Learning and Behavior, 4, 480-484.
42. Mazur, J. E. (1992). Choice behavior in transition: development of preference with ratio and interval schedules. Journal of Experimental Psychology: Animal Behavior Processes, 18, 364-378.
43. Mazur, J. E. (1995). Development of preference and spontaneous recovery in choice behavior with concurrent variable-interval schedules. Animal Learning and Behavior, 23, 93-103.
44. McEwen, D. (1972). The effects of terminal-link fixed-interval and variable-interval schedules on responding under concurrent chained schedules. Journal of the Experimental Analysis of Behavior, 18, 253-261.
45. McSweeney, F. K., Roll, J. M. & Weatherly, J. N. (1994). Within-session changes in responding during several simple schedules. Journal of the Experimental Analysis of Behavior, 62, 109-132.
46. Meyer, D. R. (1951). The effects of differential rewards on discrimination reversal learning by monkeys. Journal of Experimental Psychology, 41, 268-274.
47. Mikulka, P. J., Lehr, R., and Pavlik, V. B. (1967). Effect of reinforcement schedules on reward shifts. Journal of Experimental Psychology, 74, 57-61.
48. Minsky, M. L. (1963). Steps toward artificial intelligence. Proceedings of the Institute of Radio Engineers, 49, 8-30, 1961. Reprinted in E. A. Feigenbaum & J. Feldman (Eds.), Computers and thought (pp. 406-450). New York: MacGraw- Hill.
49. Minsky, M. L. (1963). Steps toward artificial intelligence. Proceedings of the Institute of Radio Engineers, 49, 8-30, 1961. Reprinted in E. A. Feigenbaum & J. Feldman (Eds.), Computers and thought (pp. 406-450). New York: MacGraw-Hill.
50. Mowrer, O. H. (1960). Learning theory and behavior. New York: Wiley (Krieger Edition, 1973).
51. Nevin, J. A. (1988). Behavioral momentum and the partial reinforcement effect. Psychological Bulletin, 103, 44-56.
52. Nevin, J. A., & Shettleworth, S. J. (1966). An analysis of contrast effects in multiple schedules. Journal of the Experimental Analysis of Behaviour, 9, 305-315.
53. Pavlov, I. P. (1927). Conditioned reflexes. Oxford: Oxford University Press.
54. Pearce, J. M., & Hall, G. (1980). A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. Psychological Review, 87, 532-552.
55. Rescorla, R. A., & Wanger, A. R. (1972). A theory of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: current research and theory. New York: Appleton-Century-Crofts.
56. Reynolds, B. (1961). Behavioral contrast. Journal of Experimental Analysis of Behavior, 4, 57-71.
57. Robbins, S. J. (1990). Mechanisms underlying spontaneous recovery in autoshaping, Journal of Experimental Psychology: Animal Behavior Processes, 16, 235-249.
58. Rudy, J. W. (1971). Sequential variables as determiners of the rat's discrimination of reinforcement events: effects of extinction performance. Journal of Comparative Physiological Psychology, 77, 476-481.
59. Rudy, J. W., & Sutherland, R. J. (1989). The hippocampal formation is necessary for rats to learn and remember configural discriminations. Behavioral Brain Research, 34, 97-109.
60. Samuel, A. L. (1963). Some studies in machine learning using the game of checkers. In E. A. Feigenbaum & J. Feldman (Eds.), Computers and thought. New York: McGraw-Hill. (Reprinted from IBM Journal on Research & Development, 1959, 3, 210- 229.)
61. Samuel, A. L. (1963). Some studies in machine learning using the game of checkers. In E. A. Feigenbaum & J. Feldman (Eds.), Computers and thought. New York: McGraw-Hill. (Reprinted from IBM Journal on Research & Development, 1959, 3, 210-229.)
62. Schmajuk, N. A. (1995). Animal learning and cognition: A neural network approach. Cambridge: Cambridge University Press (in press).
63. Schmajuk, N. A., & DiCarlo, J. J. (1992). Stimulus configuration, classical conditioning, and hippocampal function. Psychological Review, 99, 268-305.
64. Schwartz, B., & Gamzu, E. (1977). Pavlovian control of operant behavior: an analysis of autoshaping and its implication for operant conditioning. In W. K. Honig & J. E. R. Staddon (Eds.), Handbook of operant behavior. Englewood Cliffs, NJ: Prentice Hall.
65. Skinner, B. F. (1938). The behavior of organisms. New York: Appleton-Century-Crofts.
66. Skinner, B. F. (1950). Are theories of learning necessary? Psychological Review, 57, 193-216.
67. Sokolov, E. N. (1960). Neuronal models and the orienting reflex. In M. A. B. Brazier (Ed.), The central nervous system and behavior, 3rd Conference (pp. 187-276). New York: Josiah Macy Jr Foundation.
68. Spence, K. W. (1956). Behavior theory and conditioning. New Haven, CN: Yale University Press.
69. Staddon, J. E. R. (1983). Adaptive behavior and learning. Cambridge: Cambridge University Press.
70. Staddon, J. E. R., & Hinson, J. M. (1978). Behavioral competition: a mechanism for schedule interactions. Science, 202, 432-434.
71. Staddon, J. E. R., & Zhang, Y. (1991). On the assignment-of-credit problem in operant learning. In M. L. Commons, S. Grossberg, & J. E. R. Staddon (Eds.), Neural network models of conditioning and action (pp. 279-393). Hillsdale, N.J.: Lawrence Erlbaum.
72. Sutton, R. S., & Barto, A. G. (1981). Toward a modern theory of adaptive networks: expectation and prediction. Psychological Review, 88, 135-170.
73. Sutton, R. S., & Barto, A. G. (1990). Time-derivative models of Pavlovian reinforcement. In M. Gabriel & J. W. Moore, (Eds.), Learning and computational neuroscience: Foundations of adaptive networks. Cambridge, MA: MIT Press.
74. Swanson, L. W. (1978). The anatomical organization of septohippocampal projections. In K. Elliot & J. Whelan (Eds.), Functions of the septo-hippocampal system (pp. 25-43). Ciba Foundation Symposium 58 (New Series).
75. Thorndike, E. L. (1911). Animal intelligence. New York: MacMillan.
76. Tolman, E. C. (1932). Purposive behavior in animals and men. New York: Appleton-Century-Crofts.
77. Vinogradova, O. S. (1975). Functional organization of the limbic system in the process of registration of information: facts and hypotheses. In R. L. Isaacson & K. H. Pribram (Eds.), The hippocampus, v. 2, Neurophysiology and behavior (pp. 1-70). New York: Plenum Press.
78. Weinstock, S. (1958). Acquisition and extinction of a partially reinforced running response at a 24-hour intertrial interval. Journal of Experimental Psychology, 47, 151-158.