Extinction under a behavioral microscope: Isolating the sources of decline in operant response rate

Behavioural Processes

Volumn 90, Issue 1, 2012, Pages 111-123

  Cheung, Timothy H C ^a,b   Neisewander, Janet L ^a,b   Sanabria, Federico ^a  

^a ARIZONA STATE UNIVERSITY   (United States)

^b ARIZONA STATE UNIVERSITY   (United States)

Author keywords

ADHD; Bout; Drug seeking; Extinction; Hierarchical model; Microstructure of behavior; Motivation

Indexed keywords

EXTINCTION; HIERARCHICAL SYSTEM; MEMORY; MICROSTRUCTURE; PARADIGM SHIFT; PSYCHOLOGY;

ARTICLE; BAYES THEOREM; BEHAVIORAL SCIENCE; BIEXPONENTIAL REFRACTORY MODEL; INSTRUMENTAL CONDITIONING; MAXIMUM LIKELIHOOD METHOD; NONHUMAN; PSYCHOLOGICAL MODEL; REINFORCEMENT; RESPONSE TIME;

ANIMALS; BAYES THEOREM; COCAINE; CONDITIONING, OPERANT; EXTINCTION, PSYCHOLOGICAL; MODELS, PSYCHOLOGICAL; PSYCHOLOGICAL THEORY; RATS; RATS, INBRED SHR; RATS, INBRED WKY; REINFORCEMENT (PSYCHOLOGY); REINFORCEMENT SCHEDULE; SELF ADMINISTRATION;

EID: 84860126859     PISSN: 03766357     EISSN: 18728308     Source Type: Journal    
DOI: 10.1016/j.beproc.2012.02.012     Document Type: Article

References (85)

1. Anderson D.R. Model Based Inference in the Life Sciences: A Primer on Evidence 2008, Springer, New York, NY.
2. Babyak M.A. What you see may not be what you get: a brief, nontechnical introduction to overfitting in regression-type models. Psychosom. Med. 2004, 66:411-421.
3. Bennett J.A., Hughes C.E., Pitts R.C. Effects of methamphetamine on response rate: a microstructural analysis. Behav. Processes 2007, 75:199-205.
4. Bindra D. How adaptive behavior is produced: a perceptual-motivational alternative to response-reinforcement. Behav. Brain Sci. 1978, 1:41-52.
5. Bouton M. Context and behavioral processes in extinction. Learn. Mem. 2004, 11:485-494.
6. Bowers M.T., Hill J., Palya W.L. Interresponse time structures in variable-ratio and variable-interval schedules. J. Exp. Anal. Behav. 2008, 90:345-362.
7. Brackney, R.J., Cheung, T.H.C., Herbst, K., Hill, J.C., Sanabria, F. Extinction learning deficit in a rodent model of attention-deficit hyperactivity disorder, submitted for publication.
8. Brackney R.J., Cheung T.H.C., Neisewander J.L., Sanabria F. The isolation of motivational, motoric, and schedule effects on operant performance: a modeling approach. J. Exp. Anal. Behav. 2011, 96:17-38.
9. Burnham K.P., Anderson D.R. Model Selection and Multimodel Inference: A Practical Information-theoretic Approach 2002, Springer, Secaucus, NJ. 2nd edn.
10. Clark F.C. Some quantitative properties of operant extinction data. Psychol. Rep. 1959, 5:131-139.
11. Conover K., Fulton S., Shizgal P. Operant tempo varies with reinforcement rate: implications for measurement of reward efficacy. Behav. Processes 2001, 56:85-101.
12. Cowles M.K., Carlin B.P. Markov chain Monte Carlo convergence diagnostics: a comparative review. J. Am. Stat. Assoc. 1996, 91:883-904.
13. Dickinson A., Balleine B. Motivational control of goal-directed action. Anim. Learn. Behav. 1994, 22:1-18.
14. Dickinson A., Balleine B., Watt A., Gonzalez F., Boakes R.A. Motivational control after extended instrumental training. Anim. Learn. Behav. 1995, 23:197-206.
15. Dulaney A.E., Bell M.C. Resistance to extinction, generalization decrement, and conditioned reinforcement. Behav. Processes 2008, 78:253-258.
16. Epstein D.H., Preston K.L., Stewart J., Shaham Y. Toward a model of drug relapse: an assessment of the validity of the reinstatement procedure. Psychopharmacology (Berl.) 2006, 189:1-16.
17. Flegal J.M., Jones G.L. Batch means and spectral variance estimators in Markov chain Monte Carlo. Ann. Stat. 2010, 38:1034-1070.
18. Freedman D.A. A note on screening regression equations. Am. Stat. 1983, 37:152-155.
19. Fuchs R.A., Tran-Nguyen L.T.L., Specio S.E., Groff R.S., Neisewander J.L. Predictive validity of the extinction/reinstatement model of drug craving. Psychopharmacology (Berl.) 1998, 135:151-160.
20. Gelman A. Prior distributions for variance parameters in hierarchical models (comment on an article by Browne and Draper). Bayesian Anal. 2006, 1:515-533.
21. Gelman A., Carlin J.B., Stern H.S., Rubin D.B. Bayesian Data Analysis 2004, Chapman and Hall/CRC, Boca Raton, FL. 2nd edn.
22. Gelman A., van Dyk D.A., Huang Z.Y., Boscardin W.J. Using redundant parameterizations to fit hierarchical models. J. Comput. Graph. Stat. 2008, 17:95-122.
23. Geweke J. Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments. Bayesian Statistics 1992, vol. 4:169-193. Oxford University Press, Oxford, UK.
24. Gilbert T.F. Fundamental dimensional properties of the operant. Psychol. Rev. 1958, 65:272-282.
25. Griffiths T.L., Kemp C., Tenenbaum J.B. Bayesian models of cognition. The Cambridge Handbook of Computational Psychology 2008, 9-100. Cambridge University Press. R. Sun (Ed.).
26. Hawkins D.M. The problem of overfitting. J. Chem. Inf. Comput. Sci. 2004, 44:1-12.
27. Herrnstein R.J. On the law of effect. J. Exp. Anal. Behav. 1970, 13:243-266.
28. Hill J.C., Herbst K., Sanabria F. Characterizing operant hyperactivity in the Spontaneously Hypertensive Rat. Behav. Brain Funct. 2012, 8:15.
29. Ho M.Y., Mobini S., Chiang T.J., Bradshaw C.M., Szabadi E. Theory and method in the quantitative analysis of impulsive choice behaviour: implications for psychopharmacology. Psychopharmacology (Berl.) 1999, 146:362-372.
30. Hodos W. Progressive ratio as a measure of reward strength. Science 1961, 134:943-944.
31. Hursh S.R., Silberberg A. Economic demand and essential value. Psychol. Rev. 2008, 115:186-198.
32. Johnson J.E., Bailey J.M., Newland M.C. Using pentobarbital to assess the sensitivity and independence of response-bout parameters in two mouse strains. Pharmacol. Biochem. Behav. 2011, 97:470-478.
33. Johnson J.E., Pesek E.F., Newland M.C. High-rate operant behavior in two mouse strains: a response-bout analysis. Behav. Processes 2009, 81:309-315.
34. Kessel R., Lucke R.L. An analytic form for the interresponse time analysis of Shull, Gaynor, and Grimes with applications and extensions. J. Exp. Anal. Behav. 2008, 90:363-386.
35. Killeen P., Hanson S., Osborne S. Arousal: its genesis and manifestation as response rate. Psychol. Rev. 1978, 85:571-581.
36. Killeen P.R. Mathematical principles of reinforcement. Behav. Brain Sci. 1994, 17:105-135.
37. Killeen P.R. The first principle of reinforcement. Models of Action: Mechanisms for Adaptive Behavior 1998, 127-156. Lawrence Erlbaum Associates, Mahwah, NJ. C.D.L. Wynne, J.E.R. Staddon (Eds.).
38. Killeen P.R. An alternative to null-hypothesis significance tests. Psychol. Sci. 2005, 16:345-353.
39. Killeen P.R., Hall S.S., Reilly M.P., Kettle L.C. Molecular analyses of the principal components of response strength. J. Exp. Anal. Behav. 2002, 78:127-160.
40. Killeen P.R., Sanabria F., Dolgov I. The dynamics of conditioning and extinction. J. Exp. Psychol. Anim. Behav. Process. 2009, 35:447-472.
41. Killeen P.R., Sitomer M.T. MPR. Behav. Processes 2003, 62:49-64.
42. Langton S.D., Collett D., Sibly R.M. Splitting behaviour into bouts; a maximum likelihood approach. Behaviour 1995, 132:781-799.
43. Leslie J.C., Shaw D., McCabe C., Reynolds D.S., Dawson G.R. Effects of drugs that potentiate GABA on extinction of positively-reinforced operant behaviour. Neurosci. Biobehav. Rev. 2004, 28:229-238.
44. Limpert E., Stahel W.A., Abbt M. Log-normal distributions across the sciences: keys and clues. Bioscience 2001, 51:341-352.
45. Machlis L. An analysis of the temporal patterning of pecking in chicks. Behaviour 1977, 63:1-70.
46. MacKay D.J.C. Information Theory, Inference and Learning Algorithms 2003, Cambridge University Press, Cambridge.
47. Margulies S. Response duration in operant level, regular reinforcement, and extinction. J. Exp. Anal. Behav. 1961, 4:317-321.
48. Mellgren R.L., Elsmore T.F. Extinction of operant behavior: an analysis based on foraging considerations. Anim. Learn. Behav. 1991, 19:317-325.
49. Myung I.J. Tutorial on maximum likelihood estimation. J. Math. Psychol. 2003, 47:90-100.
50. Nakagawa S., Cuthill I.C. Effect size, confidence interval and statistical significance: a practical guide for biologists. Biol. Rev. 2007, 82:591-605.
51. Neal R.M. Slice sampling. Ann. Stat. 2003, 31:705-741.
52. Nevin J.A. Behavioral momentum and the partial reinforcement effect. Psychol. Bull. 1988, 103:44-56.
53. Nevin J.A., Grace R.C. Behavioral momentum and the law of effect. Behav. Brain Sci. 2000, 23:73-90.
54. Nevin J.A., McLean A.P., Grace R.C. Resistance to extinction: contingency termination and generalization decrement. Anim. Learn. Behav. 2001, 29:176-191.
55. Ntzoufras I. Bayesian Modeling using WinBUGS 2009, Wiley, New York.
56. Olmstead M.C., Lafond M.V., Everitt B.J., Dickinson A. Cocaine seeking by rats is a goal-directed action. Behav. Neurosci. 2001, 115:394-402.
57. Olton D.S. The radial arm maze as a tool in behavioral pharmacology. Physiol. Behav. 1987, 40:793-797.
58. Pentkowski N.S., Duke F.D., Weber S.M., Pockros L.A., Teer A.P., Hamilton E.C., Thiel K.J., Neisewander J.L. Stimulation of medial prefrontal cortex serotonin 2C (5-HT(2C)) receptors attenuates cocaine-seeking behavior. Neuropsychopharmacology 2010, 35:2037-2048.
59. Pockros L.A., Pentkowski N.S., Swinford S.E., Neisewander J.L. Blockade of 5-HT2A receptors in the medial prefrontal cortex attenuates reinstatement of cue-elicited cocaine-seeking behavior in rats. Psychopharmacology (Berl.) 2011, 213:307-320.
60. Podlesnik C.A., Jimenez-Gomez C., Ward R.D., Shahan T.A. Resistance to change of responding maintained by unsignaled delays to reinforcement: a response-bout analysis. J. Exp. Anal. Behav. 2006, 85:329-347.
61. Podlesnik C.A., Sanabria F. Repeated extinction and reversal learning of an approach response supports an arousal-mediated learning model. Behav. Processes 2010, 87:125-134.
62. Podlesnik C.A., Shahan T.A. Extinction, relapse, and behavioral momentum. Behav. Processes 2010, 84:400-411.
63. Quirk G.J., Mueller D. Neural mechanisms of extinction learning and retrieval. Neuropsychopharmacology 2008, 33:56-72.
64. Reed P. An experimental analysis of steady-state response rate components on variable ratio and variable interval schedules of reinforcement. J. Exp. Psychol. Anim. Behav. Process. 2011, 37:1-9.
65. Rescorla R. Experimental extinction. Handbook of Contemporary Learning Theories 2001, 119-154. Lawrence Erlbaum Associates, Mahwah, NJ. R.R. Mowrer, S.B. Klein (Eds.).
66. Robbins T.W. The 5-choice serial reaction time task: behavioural pharmacology and functional neurochemistry. Psychopharmacology (Berl.) 2002, 163:362-380.
67. Robert C.P., Casella G. Monte Carlo Statistical Methods 1999, Springer-Verlag, New York.
68. Rouder J.N., Lu J. An introduction to Bayesian hierarchical models with an application in the theory of signal detection. Psychon. Bull. Rev. 2005, 12:573-604.
69. Sagvolden T. Behavioral validation of the spontaneously hypertensive rat (SHR) as an animal model of attention-deficit/hyperactivity disorder (AD/HD). Neurosci. Biobehav. Rev. 2000, 24:31-39.
70. Salamone J.D., Correa M., Farrar A.M., Nunes E.J., Pardo M. Dopamine, behavioral economics, and effort. Front. Behav. Neurosci. 2009, 3:12.
71. Sanabria F., Killeen P.R. Better statistics for better decisions: rejecting null hypotheses statistical tests in favor of replication statistics. Psychol. Schools 2007, 44:471-481.
72. Sanabria F., Killeen P.R. Evidence for impulsivity in the spontaneously hypertensive rat drawn from complementary response-withholding tasks. Behav. Brain Funct. 2008, 4:17.
73. See R.E. Neural substrates of cocaine-cue associations that trigger relapse. Eur. J. Pharmacol. 2005, 526:140-146.
74. Shiffrin R., Lee M., Kim W., Wagenmakers E.J. A survey of model evaluation approaches with a tutorial on hierarchical Bayesian methods. Cogn. Sci. 2008, 32:1248-1284.
75. Shull R.L. Bouts of responding on variable-interval schedules: effects of deprivation level. J. Exp. Anal. Behav. 2004, 81:155-167.
76. Shull R.L., Gaynor S.T., Grimes J.A. Response rate viewed as engagement bouts: effects of relative reinforcement and schedule type. J. Exp. Anal. Behav. 2001, 75:247-274.
77. Shull R.L., Gaynor S.T., Grimes J.A. Response rate viewed as engagement bouts: resistance to extinction. J. Exp. Anal. Behav. 2002, 77:211-231.
78. Shull R.L., Grimes J.A. Bouts of responding from variable-interval reinforcement of lever pressing by rats. J. Exp. Anal. Behav. 2003, 80:159-171.
79. Shull R.L., Grimes J.A. Resistance to extinction following variable-interval reinforcement: reinforcer rate and amount. J. Exp. Anal. Behav. 2006, 85:23-39.
80. Shull R.L., Grimes J.A., Bennett J.A. Bouts of responding: the relation between bout rate and the rate of variable-interval reinforcement. J. Exp. Anal. Behav. 2004, 81:65-83.
81. Skinner B.F. The generic nature of the concepts of stimulus and response. J. Gen. Psychol. 1935, 12:40-65.
82. Stewart J., de Wit H. Reinstatement of drug-taking behavior as a method of assessing incentive motivational properties of drugs. Method of Assessing the Reinforcing Properties of Abused Drugs 1987, 221-227. Springer, New York, NY. M.A. Bozarth (Ed.).
83. Tonneau F., Ortiz G., Cabrera F. Early-session increases in responding during extinction. Behav. Processes 2000, 49:121-129.
84. Weiner H. Response cost effects during extinction following fixed-interval reinforcement in humans. J. Exp. Anal. Behav. 1964, 7:333-335.
85. Wetzels R., Lee M.D., Wagenmakers E.J. Bayesian inference using WBDev: a tutorial for social scientists. Behav. Res. Methods 2010, 42:884-897.