The Dynamics of Conditioning and Extinction

Journal of Experimental Psychology: Animal Behavior Processes

Volumn 35, Issue 4, 2009, Pages 447-472

  Killeen, Peter R ^a   Sanabria, Federico ^a   Dolgov, Igor ^a  

^a ARIZONA STATE UNIVERSITY   (United States)

Author keywords

autoshaping; behavioral momentum; classical conditioning; dynamic analyses; instrumental conditioning

Indexed keywords

ANIMAL; ANIMAL BEHAVIOR; ARTICLE; CONDITIONED REFLEX; FEEDING BEHAVIOR; INSTRUMENTAL CONDITIONING; NONLINEAR SYSTEM; PHYSIOLOGY; PIGEONS AND DOVES; PSYCHOLOGICAL MODEL; REACTION TIME; REINFORCEMENT;

ANIMALS; BEHAVIOR, ANIMAL; COLUMBIDAE; CONDITIONING, CLASSICAL; CONDITIONING, OPERANT; EXTINCTION, PSYCHOLOGICAL; FEEDING BEHAVIOR; MODELS, PSYCHOLOGICAL; NONLINEAR DYNAMICS; REACTION TIME; REINFORCEMENT (PSYCHOLOGY); REINFORCEMENT SCHEDULE;

COLUMBA;

EID: 70350434161     PISSN: 00977403     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0015626     Document Type: Article

References (123)

1. rep statistic as a measure of confidence in model fitting. Psychonomic Bulletin & Review 15 (2008) 16-27
2. Atkinson R.C., and Estes W.K. Stimulus sampling theory (1962), Institute for Mathematical Studies in the Social Science, Applied Mathematics and Statistics Laboratories, Stanford University, Palo Alto, CA
3. Baker A.G., Murphy R.A., and Mehta R. Learned irrelevance and retrospective correlation learning. Quarterly Journal of Experimental Psychology: Journal of Comparative and Physiological Psychology 56 B (2003) 90-101
4. Balsam P.D., and Schwartz A.L. Rapid contextual conditioning in autoshaping. Journal of Experimental Psychology: Animal Behavior Processes 7 (2004) 382-393
5. Benedict J.O., and Ayres J.J. Factors affecting conditioning in the truly random control procedure in the rat. Journal of Comparative and Physiological Psychology 78 (1972) 323-330
6. Bindra D. How adaptive behavior is produced: A perceptual-motivational alternative to response-reinforcement. Behavioral Brain Sciences 1 (1978) 41-91
7. Bitterman M.E. Classical conditioning since Pavlov. Review of General Psychology 10 (2006) 365-376
8. Bouton M.E. Context, time, and memory retrieval in the interference paradigms of Pavlovian learning. Psychological Bulletin 114 (1993) 80-99
9. Bower G.H. A turning point in mathematical learning theory. Psychological Review 101 (1994) 290-300
10. Brandon S.E., Vogel E.H., and Wagner A.R. Stimulus representation in SOP: I. Theoretical rationalization and some implications. Behavioral Processes 62 (2003) 5-25
11. Burgos J.E. Evolving artificial networks in Pavlovian environments. In: Donahoe J.W., and Packard-Dorsel V. (Eds). Neural-network models of cognition: Biobehavioral foundations (1997), Elsevier, New York 58-79
12. Burke C.J., and Estes W.K. A component model for stimulus variables in discrimination learning. Psychometrika 22 (1956) 133-145
13. Burnham K.P., and Anderson D.R. Model selection and multimodel inference: A practical information-theoretic approach. 2nd ed. (2002), Springer-Verlag, New York
14. Bush R.R., and Mosteller F. A mathematical model for simple learning. Psychological Review 58 (1951) 313-323
15. Colwill R.M. The effect of noncontingent outcomes on extinction of the response-outcome association. Animal Learning & Behavior 29 (2001) 153-164
16. Colwill R.M., and Rescorla R.A. Associative structures in instrumental learning. In: Bower G.H. (Ed). The psychology of learning and motivation 20 (1986), Academic Press, New York 55-103
17. Colwill R.M., and Rescorla R.A. The role of response-reinforcer associations increases throughout extended instrumental training. Animal Learning & Behavior 16 (1988) 105-111
18. Colwill R.M., and Triola S.M. Instrumental responding remains under the control of the consequent outcome after extended training. Behavioural Processes 57 (2002) 51-64
19. Corrado G.S., Sugrue L.P., Seung H.S., and Newsome W.T. Linear-nonlinear-Poisson models of primate choice dynamics. Journal of the Experimental Analysis of Behavior 84 (2005) 581-617
20. Couvillon P.A., and Bitterman M.E. Analysis of choice in honeybees. Animal Learning & Behavior 13 (1985) 246-252
21. Cox D.R., and Hinkley D.V. Theoretical statistics (1974), Chapman & Hall, London
22. Danks D. Equilibria of the Rescorla-Wagner model. Journal of Mathematical Psychology 47 (2003) 109-121
23. Davison M., and Baum W.M. Choice in a variable environment: Every reinforcer counts. Journal of the Experimental Analysis of Behavior 74 (2000) 1-24
24. Davison M., and Baum W.M. Do conditional reinforcers count?. Journal of the Experimental Analysis of Behavior 86 (2006) 269-283
25. Davol G., Steinhauer G., and Lee A. The role of preliminary magazine training in acquisition of the autoshaped key peck. Journal of the Experimental Analysis of Behavior 1977 (2002) 99-106
26. Dayan P., Niv Y., Seymour B., and Daw N.D. The misbehavior of value and the discipline of the will. Neural Networks 19 (2006) 1153-1160
27. de la Piedad X., Field D., and Rachlin H. The influence of prior choices on current choice. Journal of the Experimental Analysis of Behavior 85 (2006) 3-21
28. Donahoe J.W., Palmer D.C., and Burgos J.E. The S-R issue: Its status in behavior analysis and in Donahoe and Palmer's learning and complex behavior. Journal of the Experimental Analysis of Behavior 67 (1997) 193-211
29. Donahoe J.W., Palmer D.C., and Burgos J.E. The unit of selection: What do reinforcers reinforce?. Journal of the Experimental Analysis of Behavior 67 (1997) 259-273
30. Donahoe J.W., Palmer D.C., and Dorsel V.P. Learning and complex behavior (1994), Allyn & Bacon, Boston
31. Downing K., and Neuringer A. Autoshaping as a function of prior food presentations. Journal of the Experimental Analysis of Behavior 26 (2003) 463-469
32. Estes W.K. Toward a statistical theory of learning. Psychological Review 57 (1950) 94-107
33. Estes W.K. The problem of inference from curves based on group data. Psychological Bulletin 53 (1956) 134-140
34. Estes W.K. Learning theory. Annual Review of Psychology 13 (1962) 107-144
35. Frey P.W., and Sears R.J. Model of conditioning incorporating the Rescorla-Wagner associative axiom, a dynamic attention process, and a catastrophe rule. Psychological Review 85 (1978) 321-340
36. Gallistel C.R., and Gibbon J. Time, rate, and conditioning. Psychological Review 107 (2000) 289-344
37. Gibbon J., Baldock M.D., Locurto C.M., Gold L., and Terrace H.S. Trial and intertrial durations in autoshaping. Journal of Experimental Psychology: Animal Behavior Processes 3 (1977) 264-284
38. Gibbon J., Farrell L., Locurto C.M., Duncan H.J., and Terrace H.S. Partial reinforcement in autoshaping with pigeons. Animal Learning & Behavior 8 (1980) 45-59
39. Grace R.C. Acquisition of preference in concurrent chains: Comparing linear-operator and memory-representational models. Journal of Experimental Psychology: Animal Behavior Processes 28 (2002) 257-276
40. Guilhardi P., Yi L., and Church R.M. A modular theory of learning and performance. Psychonomic Bulletin & Review 14 (2007) 543-559
41. Hall G. Associative structures in Pavlovian and instrumental conditioning. In: Gallistel R. (Ed). Steven's handbook of experimental psychology (2002), Wiley, New York 1-46
42. Hanson, S. J. (1977). The Rescorla-Wagner model and the temporal control of behavior. Unpublished master's thesis, Arizona State University, Tempe.
43. Healy A.F., Kosslyn S.M., and Shiffrin R.M. From learning theory to connectionist theory: Essays in honor of William K. Estes (1992), Erlbaum, Hillsdale, NJ
44. Hearst E. The classical-instrumental distinction: Reflexes, voluntary behavior, and categories of associative learning. In: Estes W.K. (Ed). Handbook of learning and cognitive processes 2 (1975), Erlbaum, Mahwah, NJ 181-223
45. Hearst E., and Jenkins H.M. Sign-tracking: The stimulus-reinforcer relation and directed action (1974), Psychonomic Society, Austin, TX
46. Holland P.C. Differential effects of omission contingencies on various components of Pavlovian appetitive conditioned responding in rats. Journal of Experimental Psychology: Animal Behavior Processes 5 (1979) 178-193
47. Humphreys L.G. The effect of random alternation of reinforcement on the acquisition and extinction of conditioned eyelid reactions. Journal of Experimental Psychology 25 (1939) 141-158
48. James W. Classics in the history of psychology: The principles of psychology (1890). http://psychclassics.yorku.ca/James/Principles Retrieved March 25, 2009, from http://psychclassics.yorku.ca/James/Principles
49. James W. Principles of psychology (1890), Holt, New York
50. Janssen M., Farley J., and Hearst E. Temporal location of unsignaled food deliveries: Effects on conditioned withdrawal (inhibition) in pigeon signtracking. Journal of Experimental Psychology: Animal Behavior Processes 21 (1995) 116-128
51. Jenkins H.M. Sensitivity of different response systems to stimulus-reinforcer and response-reinforcer relations. In: Davis H., and Hurwitz H. (Eds). Operant-Pavlovian interactions (1977), Erlbaum, Mahwah, NJ 47-66
52. Kakade S., and Dayan P. Acquisition and extinction in autoshaping. Psychological Review 109 (2002) 533-544
53. Killeen P.R. Reinforcement frequency and contingency as factors in fixed-ratio behavior. Journal of the Experimental Analysis of Behavior 12 (1969) 391-395
54. Killeen P.R. Averaging theory. In: Bradshaw C.M., Szabadi E., and Lowe C.F. (Eds). Quantification of steady-state operant behaviour (1981), Elsevier, Amsterdam 21-34
55. Killeen P.R. Reflections on a cumulative record. Behavior Analyst 8 (1985) 177-183
56. Killeen P.R. Mechanics of the animate. Journal of the Experimental Analysis of Behavior 57 (1992) 429-463
57. Killeen P.R. Writing and overwriting short-term memory. Psychonomic Bulletin & Review 8 (2001) 18-43
58. Killeen P.R. Complex dynamic processes in sign-tracking with an omission contingency (negative automaintenance). Journal of Experimental Psychology: Animal Behavior Processes 29 (2003) 49-61
59. Killeen P.R., Hall S.S., Reilly M.P., and Kettle L.C. Molecular analyses of the principal components of response strength. Journal of the Experimental Analysis of Behavior 78 (2002) 127-160
60. Killeen P.R., Hanson S.J., and Osborne S.R. Arousal: Its genesis and manifestation as response rate. Psychological Review 85 (1978) 571-581
61. Levine G., and Burke C.J. Mathematical model techniques for learning theories (1972), Academic Press, New York
62. Locurto C.M., Duncan H., Terrace H.S., and Gibbon J. Autoshaping in the rat: Interposing delays between responses and food. Animal Learning & Behavior 8 (1980) 37-44
63. Locurto C.M., Terrace H.S., and Gibbon J. Autoshaping and conditioning theory (1981), Academic Press, New York
64. Loftus G.R., and Masson M.E.J. Using confidence intervals in within-subject designs. Psychonomic Bulletin & Review 1 (1994) 476-490
65. Mackintosh N.J. The psychology of animal learning (1974), Academic Press, New York
66. Mackintosh N.J. A theory of attention: Variations in the associability of stimuli with reinforcement. Psychological Review 82 (1975) 276-298
67. Mazur J.E. Mathematical models and the experimental analysis of behavior. Journal of the Experimental Analysis of Behavior 85 (2006) 275
68. Miller R.R., and Barnet R.C. The role of time in elementary associations. Current Directions in Psychological Science 2 (1993) 106-111
69. Miller R.R., Barnet R.C., and Grahame N.J. Assessment of the Rescorla-Wagner model. Psychological Bulletin 117 (1995) 363-386
70. Myerson J. Leverpecking elicited by signaled presentation of grain. Bulletin of the Psychonomic Society 4 (1974) 499-500
71. Myung I.J. Tutorial on maximum likelihood estimation. Journal of Mathematical Psychology 47 (2003) 90-100
72. Myung I.J., and Pitt M.A. Applying Occam's razor in modeling cognition: A Bayesian approach. Psychonomic Bulletin & Review 4 (1997) 79-95
73. Nadel L., and Willner J. Context and conditioning: A place for space. Physiological Psychology 8 (1980) 218-228
74. Nevin J.A. Behavioral momentum and the partial reinforcement effect. Psychological Bulletin 103 (1988) 44-56
75. Nevin J.A. The momentum of compliance. Journal of Applied Behavior Analysis 29 (1996) 535-547
76. Nevin J.A., and Grace R.C. Behavioral momentum and the law of effect. Behavioral and Brain Sciences 23 (2001) 73-90
77. Nevin J.A., Mandell C., and Atak J.R. The analysis of behavioral momentum. Journal of the Experimental Analysis of Behavior 39 (1983) 49-59
78. Nevin J.A., Tota M., Torquato R.D., and Shull R.L. Alternative reinforcement increases resistance to change: Pavlovian or operant contingencies?. Journal of the Experimental Analysis of Behavior 53 (1990) 359-379
79. Osborne S., and Killeen P.R. Temporal properties of responding during stimuli that precede response-independent food. Learning and Motivation 8 (1977) 533-550
80. Perkins C.C., Beavers W.O., Hancock R.A., Hemmendinger D., and Ricci J.A. Some variables affecting rate of key pecking during response-independent procedures (autoshaping). Journal of the Experimental Analysis of Behavior 24 (1975) 59-72
81. Plaud J.J., and Gaither G.A. Human behavioral momentum: Implications for applied behavior analysis and therapy. Journal of Behavior Therapy and Experimental Psychiatry 27 (1996) 139-148
82. Poincaré H. Science and hypothesis (1952), Dover, New York (Original work published 1905)
83. Popper K.R. The logic of scientific discovery (2002), Routledge, London
84. Reboreda J.C., and Kacelnik A. The role of autoshaping in cooperative two-player games between starlings. Journal of the Experimental Analysis of Behavior 60 (1993) 67-83
85. Rescorla R.A. Response-independent outcome presentation can leave instrumental R-O associations intact. Animal Learning & Behavior (1992) 104-111
86. Rescorla R.A. Within-subject partial reinforcement extinction effect in autoshaping. Quarterly Journal of Experimental Psychology: Journal of Comparative and Physiological Psychology 52 B (1999) 75-87
87. Rescorla R.A. Associative changes in excitors and inhibitors differ when they are conditioned in compound. Journal of Experimental Psychology: Animal Behavior Processes 26 (2000) 428-438
88. Rescorla R.A. Extinction can be enhanced by a concurrent excitor. Journal of Experimental Psychology: Animal Behavior Processes 26 (2000) 251-260
89. Rescorla R.A. Are associative changes in acquisition and extinction negatively accelerated?. Journal of Experimental Psychology: Animal Behavior Processes 27 (2001) 307-315
90. Rescorla R.A. Comparison of the rates of associative change during acquisition and extinction. Journal of Experimental Psychology: Animal Behavior Processes 28 (2002) 406-415
91. Rescorla R.A. Savings tests: Separating differences in rate of learning from differences in initial levels. Journal of Experimental Psychology: Animal Behavior Processes 28 (2002) 369-377
92. Rescorla R.A., and Wagner A.R. A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In: Black A.H., and Prokasy W.F. (Eds). Classical conditioning II: Current research and theory (1972), Appleton-Century-Crofts, New York 64-99
93. Roberts S., and Pashler H. How persuasive is a good fit? A comment on theory testing. Psychological Review 107 (2000) 358-367
94. Roe R.M., Busemeyer J.R., and Townsend J.T. Multialternative decision field theory: A dynamic connectionist model of decision making. Psychological Review 108 (2001) 370-392
95. Sanabria F., Sitomer M.T., and Killeen P.R. Negative automaintenance omission training is effective. Journal of the Experimental Analysis of Behavior 86 (2006) 1-10
96. Schachtman T. Pavlovian conditioning: Basic associative processes 17 (2004), International Society for Comparative Psychology 3-5
97. Schmajuk N.A., Lamoureux J.A., and Holland P.C. Occasion setting: A neural network approach. Psychological Review 105 (1998) 3-32
98. Schwarz G. Estimating the dimension of a model. Annals of Statistics 6 (1978) 461-464
99. Seligman M.E.P. On the generality of the laws of learning. Psychological Review 77 (1970) 406-418
100. Shull R.L. Mathematical description of operant behavior: An introduction. In: Iversen I.H., and Lattal K.A. (Eds). Experimental analysis of behavior 2 (1991), Elsevier, New York 243-282
101. Shull R.L., Gaynor S.T., and Grimes J.A. Response rate viewed as engagement bouts: Effects of relative reinforcement and schedule type. Journal of the Experimental Analysis of Behavior 75 (2001) 247-274
102. Shull R.L., and Grimes J.A. Resistance to extinction following variable-interval reinforcement: Reinforcer rate and amount. Journal of the Experimental Analysis of Behavior 85 (2006) 23-39
103. Siegel S., and Allan L.G. The widespread influence of the Rescorla-Wagner model. Psychonomic Bulletin & Review 3 (1996) 314-321
104. Skinner B.F. The behavior of organisms (1938), Appleton-Century-Crofts, New York
105. Skinner B.F. Farewell, my lovely. Journal of the Experimental Analysis of Behavior 25 (1976) 218
106. Smith A.C., Frank L.M., Wirth S., Yanike M., Hu D., Kubota Y., et al. Dynamic analysis of learning in behavioral experiments. Journal of Neuroscience 24 (2004) 447
107. Sperling S.E., Perkins M.E., and Duncan H.J. Stimulus generalization from feeder to response key in the acquisition of autoshaped pecking. Journal of the Experimental Analysis of Behavior 27 (1977) 469-478
108. Steinhauer G.D. Acquisition and maintenance of autoshaped key pecking as a function of food stimulus and key stimulus similarity. Journal of the Experimental Analysis of Behavior 38 (1982) 281-289
109. Stout S.C., and Miller R.R. Sometimes-competing retrieval (SOCR): A formalization of the comparator hypothesis. Psychological Review 114 (2007) 759-783
110. Terrace H.S., Gibbon J., Farrell L., and Baldock M.D. Temporal factors influencing the acquisition and maintenance of an autoshaped keypeck. Animal Learning & Behavior 3 (1975) 53-62
111. Timberlake W. Behavior systems, associationism, and Pavlovian conditioning. Psychonomic Bulletin & Review 1 (1994) 405-420
112. Timberlake W. Biological behaviorism. In: O'Donahue W., and Kitchener R. (Eds). Handbook of behaviorism (1999), Academic Press, New York 243-284
113. Timberlake W. Motivational modes in behavior systems. In: Mowrer R.R., and Klein S.B. (Eds). Handbook of contemporary learning theories (2000), Erlbaum, Mahwah, NJ 155-209
114. Timberlake W. Is the operant contingency enough for a science of purposive behavior?. Behavior and Philosophy 32 (2003) 197-229
115. Tonneau F. Windows. Behavioural Processes 69 (2005) 237-247
116. Tonneau F., Ríos A., and Cabrera F. Measuring resistance to change at the within-session level. Journal of the Experimental Analysis of Behavior 86 (2006) 109-121
117. Vogel E.H., Castro M.E., and Saavedra M.A. Quantitative models of Pavlovian conditioning. Brain Research Bulletin 63 (2006) 173-202
118. Wagner A.R. SOP: A model of automatic memory processing in animal behavior. In: Spear N.E., and Miller R.R. (Eds). Information processing in animals: Memory mechanisms (1981), Erlbaum, Hillsdale, NJ 5-47
119. Wasserman E.A., Hunter N., Gutowski K., and Bader S. Autoshaping chicks with heat reinforcement: The role of stimulus-reinforcer and response-reinforcer relations. Journal of Experimental Psychology: Animal Behavior Processes 104 (1975) 158-169
120. Wasserman E.A., and Miller R.R. What's elementary about associative learning?. Annual Review of Psychology 48 (1997) 573-607
121. Williams D.R., and Williams H. Auto-maintenance in the pigeon: Sustained pecking despite contingent non-reinforcement. Journal of the Experimental Analysis of Behavior 12 (1969) 511-520
122. Woodruff G., Conner N., Gamzu E., and Williams D.R. Associative interaction: Joint control of key pecking by stimulus-reinforcer and response-reinforcer relationships. Journal of the Experimental Analysis of Behavior 28 (1977) 133
123. Yamaguchi M. Complete solution of the Rescorla-Wagner model for relative validity. Behavioural Processes 71 (2006) 70-73