Absence of "warm-up" during active avoidance learning in a rat model of anxiety vulnerability: Insights from computational modeling

Frontiers in Behavioral Neuroscience

Volumn 8, Issue AUG, 2014, Pages

  Myers, Catherine E ^a,b   Smith, Ian M ^a   Servatius, Richard J ^a,b   Beck, Kevin D ^a,b  

^a VETERANS AFFAIRS MEDICAL CENTER   (United States)

^b RUTGERS NEW JERSEY MEDICAL SCHOOL   (United States)

Author keywords

Acquisition; Anxiety vulnerability; Extinction; Learning and memory; Reinforcement learning model

Indexed keywords

ANIMAL EXPERIMENT; ANXIETY; ARTICLE; AVOIDANCE BEHAVIOR; BEHAVIORAL SCIENCE; CONTROLLED STUDY; EXPLORATORY BEHAVIOR; MALE; MATHEMATICAL MODEL; NONHUMAN; RAT; RAT MODEL; REINFORCEMENT; SIMULATION; SPRAGUE DAWLEY RAT; STRAIN DIFFERENCE; WISTAR KYOTO RAT;

EID: 84922465782     PISSN: 16625153     EISSN: None     Source Type: Journal    
DOI: 10.3389/fnbeh.2014.00283     Document Type: Article

References (65)

1. Barad, M. (2005). Fear extinction in rodents: basic insight to clinical promise. Curr. Opin. Neurobiol. 15, 710-715. doi:10.1016/j.conb.2005.10.005
2. Barto, A. G., Sutton, R. S., and Anderson, C. W. (1983). Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Trans. Syst. Man Cybern. 13, 834-846. doi:10.1109/TSMC.1983.6313077
3. Beck, K. D., Jiao, X., Pang, K. C., and Servatius, R. J. (2010). Vulnerability factors in anxiety determined through differences in active-avoidance behavior. Prog. Neuropsychopharmacol. Bol. Psychiatry 34, 852-860. doi:10.1016/j.pnpbp.2010. 03.036
4. Beck, K. D., Jiao, X., Ricart, T. M., Myers, C. E., Minor, T. R., Pang, K. C. H., etal. (2011). Vulnerabillity factors in anxiety: strain and sex differences in the use of signals associated with non-threat during the acquisition and extinction of active-avoidance behavior. Prog. Neuro-Psychopharmacol. Biol. Psychiatry 35, 1659-1670. doi:10.1016/j.pnpbp.2011.05.002
5. Berger, D. F., and Brush, F. R. (1975). Rapid acquisition of discrete-trial lever-press avoidance: effects of signal-shock interval. J. Exp. Anal. Behav. 24, 227-239. doi:10.1901/jeab.1975.24-227
6. Berger, D. F., and Starzec, J. J. (1988). Contrasting lever-pressavoidance behaviors of spontaneouslyhypertensive and normotensive rats (Rattus norvegicus). J. Comp. Psychol. 102,279-286. doi:10.1037/0735-7036.102.3.279
7. Biederman, J., Rosenbaum, J. F., Bolduc-Murphy, E. A., Faraone, S. V., Chaloff, J., Hirshfeld, D. R., et al. (1993). A 3-year follow-up of children with and without behavioral inhibition. J. Am. Acad. Child Adolesc. Psychiatry 32, 814-821. doi:10.1097/00004583-199307000-00016
8. Brennan, F. X. (2004). Genetic differences in leverpress escape/avoidance conditioning in seven mouse strains. Genes Brain Behav. 3, 110-114. doi:10.1111/j.1601-183X.2003.0057.x
9. Brennan, F. X., Beck, K. D., Ross, R. J., and Servatius, R. J. (2005). Stress-induced increases in avoidance responding: an animal model of post-traumatic stress disorder behavior? Neuropsychiatr. Dis. Treat. 1, 69-72. doi:10.2147/nedt.1.1.69. 52292
10. Brennan, F. X., Beck, K. D., and Servatius, R. J. (2006). Predator odor facilitates acquisition ofa leverpress avoidance response in rats. Neuropsychiatr. Dis. Treat. 2, 65-69.
11. Bullock, D. H. (1960). Repeated conditioning: extinction sessions as a function of the reinforcement schedule. J. Exp. Anal. Behav. 3, 241-243. doi:10.1901/jeab. 1960.3-241
12. Bussey, T. J., Wise, S. P., and Murray, E. A. (2001). The role of ventral and orbitofrontal prefrontal cortex in conditional visuomotor learning and strategy use in rhesus monkeys (Macaca mulatta). Behav. Neurosci. 115, 971-982. doi:10.1037/0735-7044.115.5.971
13. Cole, M., and Fantino, E. (1966). Temporal variables and trial discreteness in leverpress avoidance. Psychon. Sci. 6, 217-218. doi:10.3758/BF03328035
14. Daw, N. D., Niv, Y., and Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nat. Neurosci. 8,1704-1711. doi:10.1038/nn1560
15. Dayan, P., and Balleine, B. (2002). Reward, motivation, and reinforcement learning. Neuron 36,285-298. doi:10.1016/S0896-6273(02)00963-7
16. De La Garza, R. II, and Mahoney, J. J. III (2004). A distinct neurochemical profile in WKY rats at baseline and in response to acute stress: implications for animal models of anxiety and depression. Brain Res. 1021, 209-218. doi:10.1016/j.brainres.2004.06.052
17. Drolet, G., Proulx, K., Pearson, D., Rochford, J., and Deschepper, C. F. (2002). Comparisons of behavioral and neurochemical characteristics between WKY, WKHA, and Wistar rat strains. Neuropsychopharmacology 27, 400-409. doi:10.1016/S0893-133X(02)00303-2
18. Fox, N. A., Henderson, H. A., Marshall, P. J., Nichols, K. E., and Ghera, M. M. (2005). Behavioral inhibition: linking biology and behavior within a develop-mentalframework. Annu. Rev. Psychol. 56,235-262. doi:10.1146/annurev.psych. 55.090902.141532
19. Goldman-Rakic, P. (1992). Working memoryand the mind. Sci. Am. 267,110-117. doi:10.1038/scientificamerican0992-110
20. Gould, T. D., and Gottesman, I.I. (2006). Psychiatric endophenotypes andthe development of valid animal models. Genes Brain Behav. 5, 113-119. doi:10.1111/j. 1601-183X.2005.00186.x
21. Hineline, P. N. (1978). Warmup in avoidance as a function of time since prior training. J. Exp. Anal. Behav. 29, 87-103. doi:10.1901/jeab.1978.29-87
22. Hirshfeld, D. R., Rosenbaum, J. F., Biederman, J., Bolduc, E. A., Faraone, S. V., Snidman, N., et al. (1992). Stable behavioral inhibition and its association with anxiety disorder. J. Am. Acad. Child Adolesc. Psychiatry 31, 103-111. doi:10.1097/00004583-199201000-00016
23. Hirshfeld-Becker, D. R., Biederman, J., Henin, A., Faraone, S. V., Davis, S., Harrington, K., et al. (2007). Behavioral inhibition in preschool children at risk is a specific predictor of middle childhood social anxiety: a five-year follow-up. J. Dev. Behav. Pediatr. 28, 225-233. doi:10.1097/01.DBP.0000268559.34463.d0
24. Hoffman, H. S., and Fleshler, M. (1962). The course of emotionalityin the development of avoidance. J. Exp. Psychol. 64, 288-294. doi:10.1037/h0041933
25. Hoffman, H. S., Fleshler, M., and Chorny, H. (1961). Discriminated bar-press avoidance. J. Exp. Anal. Behav. 4, 309-316. doi:10.1901/jeab.1961.4-309
26. Hollerman, J., and Schultz, W. (1998). Dopamine neurons report an error in the temporal prediction of reward during learning. Nat. Neurosci. 1, 304-309. doi:10.1038/1124
27. Houk, J., and Wise, S. (1995). Distributed modular architectures linking basal ganglia, cerebellum, and cerebral cortex: their role in planning and controlling action. Cereb. Cortex 5, 95-110.
28. Jiao, X., Beck, K. D., Pang, K. C. H., and Servatius, R. J. (2011a). Animal models of anxiety vulnerability-the Wistar Kyoto Rat, in Different Views of Anxiety Disorders, ed. D. S. Selek (Rijeka, Croatia: InTech), 95-120.
29. Jiao, X., Pang, K. C., Beck, K. D., Minor, T. R., and Servatius, R. J. (2011b). Avoidance perseveration during extinction training in Wistar-Kyoto rats: an interaction of innate vulnerability and stressor intensity. Behav. Brain Res. 221, 98-107. doi:10.1016/j.bbr.2011.02.029
30. Jiao, X., Pare, W. P., and Tejani-Butt, S. (2003). Strain differences in the distribution of dopamine transporter sites in rat brain. Prog. Neuro-Psychopharmacol. Biol. Psychiatry 27, 913-919. doi:10.1016/S0278-5846(03)00150-7
31. Johnson, J. D., Li, W., Li, J., and Klopf, A. H. (2002). A computational model of learned avoidance behavior in a one-way avoidance experiment. Adapt. Behav. 9,91-104. doi:10.1177/105971230200900205
32. Kagan, J., Reznick, J. S., and Snidman, N. (1987). The physiology and psychology of behavioral inhibition in children. Child Dev. 58, 1459-1473. doi:10.2307/ 1130685
33. Kamin, L. J. (1963). Retention of an incompletely learned avoidance response: some further analyses. J. Comp. Physiol. Psychol. 56, 713-718. doi:10.1037/h0043941
34. Karamustafalioglu, O. K., Zohar, J., Güveli, M., Gal, G., Bakim, B., Fostick, L., et al. (2006). Natural course of posttraumatic stress disorder: a 20-month prospective study of Turkish earthquake survivors. J. Clin. Psychiatry 67, 882-889. doi:10.4088/JCP.v67n0604
35. Kashdan, T. B., Morina, N., and Priebe, S. (2009). Post-traumatic stress disorder, social anxiety disorder, and depression in survivors of the Kosovo War: experiential avoidance as a contributor to distress and quality of life. J. Anxiety Disord. 23,185-196. doi:10.1016/j.janxdis.2008.06.006
36. Kessler, R. C., Berglund, P., Demler, O., Jin, R., Merikangas, K. R., and Walters, E. E. (2005). Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch. Gen. Psychiatry 62, 593-602. doi:10.1001/archpsyc.62.6.617
37. LeDoux, J. E., Sakaguchi, A., and Reis, D. J. (1983). Strain differences in fear between spontaneously hypertensive and normotensive rats. Brain Res. 277, 137-143. doi:10.1016/0006-8993(83)90915-0
38. Lemos, J. C., Zhang, G., Walsh, T., Kirby, L. G., Akanwa, A., Brooks-Kayal, A., et al. (2011). Stress-hyperresponsive WKY rats demonstrate depressed dorsal raphe neuronal excitability and dysregulated CRF-mediated responses. Neuropsychopharmacology 36, 721-734. doi:10.1038/npp.2010.200
39. Maia, T. V. (2010). Two-factor theory, the actor-critic model, and conditioned avoidance. Learn. Behav. 38, 50-67. doi:10.3758/LB.38.1.50
40. McAuley, J. D., Stewart, A. L., Webber, E. S., Cromwell, H. C., Servatius, R. J., and Pang, K. C. H. (2009). Wistar-Kyoto rats as an animal model of anxiety vulnerability: Support for a hypervigilance hypothesis. Behav. Brain Res. 204, 162-168. doi:10.1016/j.bbr.2009.05.036
41. McSweeney, F. K., and Roll, J. M. (1993). Responding changes systematically within sessions during conditioning procedures. J. Exp. Anal. Behav. 60, 621-640. doi:10.1901/jeab.1993.60-621
42. Moustafa, A. A., Keri, S., Herzallah, M. M., Myers, C. E., and Gluck, M. A. (2010). A neural model of hippocampal-striatal interactions in associative learning and transfer generalization in various neurological and psychiatric patients. Brain Cogn. 74, 132-144. doi:10.1016/j.bandc.2010.07.013
43. Moustafa, A. A., Myers, C. E., and Gluck, M. A. (2009). A neurocomputational model of classical conditioning phenomena: a putative role for the hippocampal region in associative learning. Brain Res. 1276,180-195. doi:10.1016/j.brainres.2009.04. 020
44. Moutoussis, M., Bentall, R. P., Williams, J., and Dayan, P. (2008). A temporal difference account of avoidance learning. Network 19, 137-160. doi:10.1080/ 09548980802192784
45. Mowrer, O. H. (1951). Two-factor learning theory: summary and comment. Psychol. Rev. 58, 350-354. doi:10.1037/h0058956
46. Myers, K. M., and Davis, M. (2002). Behavioral and neural analysis of extinction. Neuron 36, 567-584. doi:10.1016/S0896-6273(02)01064-4
47. Nakamura, C. Y., and Anderson, N. H. (1962). Avoidance behavior differences within and between strains of rats. J. Comp. Physiol. Psychol. 55, 740-747. doi:10.1037/h0044433
48. North, C. S., Pfefferbaum, B., Tivis, L., Kawasaki, A., Reddy, C., and Spitznagel, E. L. (2004). The course of posttraumatic stress disorder in a follow-up study of survivors of the Oklahoma City bombing. Ann. Clin. Psychiatry 16, 209-215. doi:10.1080/10401230490522034
49. O'Donnell, M. L., Elliott, P., Lau, W., and Creamer, M. (2007). PTSD symptom trajectories: from early to chronic response. Behav. Res. Ther. 45, 601-606. doi:10.1016/j.brat.2006.03.015
50. Pare, W. P. (1992). The performance of WKY rats on three tests of emotional behavior. Physiol. Behav. 51, 1051-1056. doi:10.1016/0031-9384(92)90091-F
51. Pare, W. P. (1993). Passive-avoidance behavior in Wistar-Kyoto (WKY), Wistar, and Fischer-344 rats. Physiol. Behav. 54, 845-852. doi:10.1016/0031-9384(93) 90291-M
52. Perrotti, L. I., Dennis, T. S., Jiao, X., Servatius, R. J., Pang, K. C. H., and Beck, K. D. (2013). Activation of extracellular signal-related kinase (ERK) and AFosB in emotion-associated neural circuitry after asymptotic levels of active avoidance behavior are attained. Brain Res. Bull. 98, 102-110. doi:10.1016/j.brainresbull. 2013.07.004
53. Powell, R. W. (1972). Analysis of warm-up effects during avoidance in wild and domesticated rodents. J. Comp. Physiol. Psychol. 78, 311-316. doi:10.1037/ h0032191
54. Rosenbaum, J. F., Biederman, J., Bolduc-Murphy, E. A., Faraone, S. V., Chaloff, J., Hirshfeld, D. R., et al. (1993). Behavioral inhibition in childhood: a risk factor for anxiety disorders. Harv. Rev. Psychiatry 1, 2-16. doi:10.3109/ 10673229309017052
55. Rosenbaum, J. F., Biederman, J., Hirshfeld, D. R., Bolduc, E. A., Faraone, S. V., Kagan, J., et al. (1991). Further evidence of an association between behavioral inhibition and anxiety disorders: results from a family study of children from a non-clinical sample. J. Psychiatr. Res. 25, 49-65. doi:10.1016/0022-3956(91) 90015-3
56. Schultz, W. (1998). Predictive reward signal of dopamine neurons. J. Neurophysiol. 80, 1-27.
57. Schultz, W., and Dickinson, A. (2000). Neuronal coding of prediction errors. Annu. Rev. Neurosci. 23, 473-500. doi:10.1146/annurev.neuro.23.1.473
58. Seligman, M., and Johnston, J. (1973). "A cognitive theory of avoidance learning," in ContemporaryApproaches to ConditioningandLearning, eds F. McGuigan and D. Lumsden (Washington, DC: Winston-Wiley), 69-110.
59. Servatius, R. J., Jiao, X., Beck, K. D., Pang, K. C., and Minor, T. R. (2008). Rapid avoidance acquisition in Wistar-Kyoto rats. Behav. Brain Res. 192, 191-197. doi:10.1016/j.bbr.2008.04.006
60. Smith, A., Li, M., Becker, S., and Kapur, S. (2004). A model of antipsychotic action in conditioned avoidance: a computational approach. Neuropsychopharmacology 29,1040-1049. doi:10.1038/sj.npp.1300414
61. Solberg, L. C., Olson, S. L., Turek, F. W., and Redei, E. (2001). Altered hormone levels and circadian rhythm of activity in the WKY rat, a putative animal model of depression. Am. J. Physiol. Regul. Integr. Comp. Physiol. 281, R786-R794.
62. Somers, J. M., Goldner, E. M., Waraich, P., and Hsu, L. (2006). Prevalence and incidence studies of anxiety disorders: a systematic review of the literature. Can. J. Psychiatry 51, 100-113.
63. Spear, N. E., Gordon, W. C., and Martin, P. A. (1973). Warm-up decrement as failure in memory retrieval in the rat. J. Comp. Physiol. Psychol. 85, 601-614. doi:10.1037/h0035314
64. Sutton, R. (1988). Learning to predict bythe methods of temporal differences. Mach Learn 3, 9-44. doi:10.1007/BF00115009
65. Tolman, E. (1932). Purposive behavior in animals and men. New York: Appleton-Century-Croft.