Protocol for a statewide randomized controlled trial to compare three training models for implementing an evidence-based treatment

Implementation Science

Volumn 10, Issue 1, 2015, Pages

  Herschell, Amy D ^a   Kolko, David J ^a   Scudder, Ashley T ^a   Taber Thomas, Sarah ^a   Schaffner, Kristen F ^b   Hiegel, Shelley A ^b   Iyengar, Satish ^c   Chaffin, Mark ^d   Mrozowski, Stanley ^e  

^a UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

^b UNIVERSITY OF PITTSBURGH MEDICAL CENTER   (United States)

^c UNIVERSITY OF PITTSBURGH   (United States)

^d GEORGIA STATE UNIVERSITY   (United States)

^e Office of Mental Health and Substance Abuse Services   (United States)

Author keywords

Cascading model; Distance education; Evidence based treatment; Implementation; Learning collaborative; Parent Child Interaction Therapy; Therapist training; Train the trainer

Indexed keywords

ATTENTION DEFICIT AND DISRUPTIVE BEHAVIOR DISORDERS; CHILD; CHILD PARENT RELATION; CLINICAL COMPETENCE; CONTROLLED STUDY; COOPERATION; COST; EDUCATION; EVIDENCE BASED MEDICINE; FAMILY THERAPY; FEMALE; HUMAN; IN SERVICE TRAINING; MALE; METHODOLOGY; ORGANIZATION AND MANAGEMENT; PENNSYLVANIA; PRESCHOOL CHILD; PROGRAM EVALUATION; RANDOMIZED CONTROLLED TRIAL;

ATTENTION DEFICIT AND DISRUPTIVE BEHAVIOR DISORDERS; CHILD; CHILD, PRESCHOOL; CLINICAL COMPETENCE; COOPERATIVE BEHAVIOR; COSTS AND COST ANALYSIS; EDUCATION, DISTANCE; EVIDENCE-BASED MEDICINE; FAMILY THERAPY; FEMALE; HUMANS; INSERVICE TRAINING; MALE; PARENT-CHILD RELATIONS; PENNSYLVANIA; PROGRAM EVALUATION; RESEARCH DESIGN;

EID: 84942525961     PISSN: None     EISSN: 17485908     Source Type: Journal    
DOI: 10.1186/s13012-015-0324-z     Document Type: Article

References (121)

1. Lavigne JV, Gibbons RD, Christoffel KK, Rosenbaum D, Binns H, Dawson N, et al. Prevalence rates and correlates of psychiatric disorders among preschool children. J Am Acad Child Adolesc Psychiatry. 1996;35(2):204-14.
2. Tremblay RE, Pihl RO, Vitaro F, Dobkin PL. Predicting early onset of male antisocial behavior from preschool behavior. Arch Gen Psychiatry. 1994;51(9):732-6.
3. Block J, Block JH, Keyes S. Longitudinally foretelling drug usage in adolescence: early childhood personality and environmental precursors. Child Dev. 1988;59(2):336-55.
4. Caspi A, Moffitt TE, Newman DL, Silva PA. Behavioral observations at age 3 years predict adult psychiatric disorders. Arch Gen Psychiatry. 1996;53:1033-9.
5. Egger H. Toddler with temper tantrums: A careful assessment of a dysregulated preschool child. In: Galanter CA, Jensen P, editors. DSM-IV-TR casebook and treatment guide for child mental health. Arlington, VA, US: Am Psychiatr US; 2009. p. 365-84.
6. Campbell SB. Behavior problems in preschool children: a review of recent research. J Child Psychol Psychiatry. 1995;36:113-49.
7. Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walers RR. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62:593-603.
8. Kazdin AE. Conduct disorders in childhood and adolescence. 2nd ed. Thousand Oaks: Sage; 1995.
9. Schumann E, Durning P, Eyberg SM, Boggs SR. Screening for conduct problem behavior in pediatric settings using the Eyberg Child Behavior Inventory. Ambul Child Health. 1996;2:35-41.
10. Brestan EV, Eyberg SM. Effective pscyhosocial treatments of conduct-disordered children and adolescents: 29 years, 82 studies, and 5,272 kids. J Clin Psychol. 1998;27(2):180-9.
11. Eyberg SM. Members of the Child Study Laboratory. Parent-child Interaction Therapy: integrity checklists and session materials. 1999.
12. Herschell AD, Calzada EJ, Eyberg SM, McNeil CB. Parent-child interaction therapy: new directions in research. Cogn Behav Pract. 2002;9:9-16.
13. Gallagher N. Effects of Parent-child Interaction Therapy on young children with disruptive behavior disorders. Bridges Practice-Based Res Syntheses. 2003;1:1-17.
14. Thomas R, Simmer-Gembeck MJ. Behavioral outcomes of Parent-child Interaction Therapy and Triple P-Positive Parenting Program: a review and meta-analysis. J Abnorm Child Psychol. 2007;35:475-95.
15. Eisenstadt TH, Eyberg SM, McNeil CB, Newcomb K, Funderburk B. Parent-child interaction therapy with behavior problem children: relative effectiveness of two stages and overall treatment outcome. J Clin Child Psychol. 1993;22:42-51.
16. McNeil CB, Clemens-Mowrer L, Gurwitch RH, Funderbunk BW. Assessment of a new procedure to prevent timeout escape in preschoolers. Child Fam Behav Ther. 1994;16(3):27-35.
17. Schuhmann E, Foote R, Eyberg SM, Boggs SR, Algina J. Parent-child interaction therapy: interim report of a randomized trial with short-term maintenance. J Clin Child Psychol. 1998;27:34-45.
18. Boggs SR, Eyberg SM, Edwards DL, Rayfield A, Jacobs J, Bagner D, et al. Outcomes of Parent-child Interaction Therapy: a comparison of treatment completers and study dropouts one to three years later. Child Fam Behav Ther. 2004;26:1-22.
19. Eyberg SM, Funderbunk BW, Hembree-Kigin T, McNeil CB, Querido JG, Hood KK. Parent-child interaction therapy with behavior problem children: one and two year maintenance of treatment effects in the family. Child Fam Behav Ther. 2001;23:1-20.
20. Funderburk B, Eyberg SM, Newcomb K, McNeil CB, Hembree-Kigin T, Capage L. Parent-child Interaction Therapy with behavior problem children: maintenance of treatment effects in the school setting. Child Fam Behav Ther. 1998;20:17-38.
21. Hood KK, Eyberg SM. Outcomes of Parent-child Interaction Therapy: mothers' reports of maintenance three to six years after treatment. J Clin Child Adolesc Psychol. 2003;32(3):419-29.
22. Chaffin M, Silovsky JF, Funderburk B, Valle LA, Brestan EV, Balachova T, et al. Parent-child interaction therapy with physically abusive parents: efficacy for reducing future abuse reports. J Consult Clin Psychol. 2004;72:500-10.
23. Lanier P, Kohl PL, Benz J, Swinger D, Mousette P, Drake B. Parent-child interaction therapy in a community setting: examining outcomes, attrition, and treatment setting. Res Soc Work Pract. 2011;24(1):689-98.
24. Lyon AR, Budd KS. A community mental health implementation of Parent-child interaction therapy. J Child Fam Stud. 2010;19:654-68.
25. U.S. Public Health Service US Department of Health and Human Services, Office of the Surgeon General. Mental health. A report of the Surgeon General. Rockville, MD: Author; 2009.
26. President's New Freedom Commission on Mental Health. Report of the President's New Freedom Commission on Mental Health. 2004.
27. McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments: a review of the current efforts. Am Psychol. 2010;65(2):73-84.
28. Herschell AD, Kolko DJ, Baumann BL, Davis AC. The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations. Clin Psychol Rev. 2010;30:448-66.
29. Institute for Healthcare Improvement. The breakthrough series: IHI's collaborative model for achieveing breakthrough improvement in innovation series. Boston: Institute for Healthcare Improvement; 2003.
30. Kilo CM. A framework for collaborative improvement: lessons from the Institute for Healthcare Improvement's breakthrough series. Qual Manag Health Care. 1998;6:1-13.
31. Markiewicz J, Ebert L, Ling D, Amaya-Jackson L, Kisiel C. Learning collaborative toolkit. National Center for Child Traumatic Stress: Los Angeles, CA and Durham, NC; 2006.
32. Flamm BL, Berwick DM, Kabcenall AI. Reducing cesarean section rates safely: lessons learned from a "breakthrough series" collaborative. Birth. 1998;25:117-24.
33. Lendon BE, Wilson IB, McInnes K, Landrum B, Hirschhon L, Marsden PV, et al. Effects of a quality improvement collaborative on the outcome care of patients with HIV infection: the EQHIV study. Ann Intern Med. 2004;140:887-96.
34. Glasgow RE, Funnell MM, Bonomi AE, Davis C, Beckham V, Wagner EH. Self-management aspects of the improving chronic illness care breakthrough series: Implementation with diabetes and heart failure. Ann Behav Med. 2002;24:80-7.
35. Cretin S, Shortell SM, Keller EB. An evaluation of collaborative interventions to improve chronic illness care: framework and study design. Eval Rev. 2004;28(1):28-51.
36. Young PC, Glade GB, Stoddard GJ, Norlin C. Evaluation of a learning collaborative to improve the delivery of preventive services by pediatric practices. Pediatrics. 2006;117(5):1469-79.
37. Ebert L, Amaya-Jackson L, Markiewicz J, Kisiel C, Fairbank J. Use of the breakthrough series collaborative to support broad and sustained use of evidence-based trauma treatment for children in community practice settings. Adm Policy Ment Health. 2011:1-13. doi: 10.1007/s10488-011-0347-y.
38. Cavaleri MA, Gopalan G, McKay M, Appel A, Bannon Jr WM, Bigley MF, et al. Impact of a learning collaborative to improve child mental health service use among low-income urban youth and families. Best Pract Ment Health. 2006;2(2):67-79.
39. Cavaleri MA, Franco LM, McKay M, Appel A, Bannon Jr WM, Bigley MF, et al. The sustainability of a learning collaborative to improve mental health service use among low-income urban youth and families. Best Pract Ment Health. 2007;3(2):52-61.
40. Cavaleri MA, Perez M, Burton G, Penn M, Beharie N, Hoagwood KE. Developing the Support, Teambuilding, and Referral (STAR) Intervention: a research/community partnership. Child Adolesc Ment Health. 2010;15(1):56-9.
41. Rutkowski BA, Gallon S, Rawson RA, Freese TE, Bruehl A, Crevecoeur-MacPhail D, et al. Improving client engagement and retention in treatment: The Los Angeles County experience. J Subst Abuse Treat. 2010;39:78-86.
42. Solberg LI, Kottke TE, Brekke ML, Magnan S, Davidson G, Calomeni CA, et al. Failure of a continuous quality improvement intervention to increase the delivery of preventative services: a randomized trial. Eff Clin Pract. 2000;3(3):105-15.
43. Rogers ES, Cohen BF, Danley KS, Hutschinson D, Anthony WA. Training mental health workers in psychiatric rehabilitation. Schizophr Bull. 1986;12(4):709-19.
44. Chamberlain P, Price J, Reid J, Landsverk J. Cascading implementation of a foster and kinship parent intervention. Child Welfare. 2008;87(5):27.
45. Hein D, Litt LC, Cohen L, Miele GM, Campbell A. Preparing, training, and integrating staff in providing integrated treatment. In: Trauma services for women in substance abuse: an integrated approach. Washington D.C: American Psychological Association; 2009. p. 143-61.
46. Martino S, Brigham GS, Higgins C, Gallon S, Freese TE, Albright LM, et al. Partnerships and pathways of dissemination: the NIDA-SAMHSA Blending Initiative in the Clinical Trials Network. J Subst Abuse Treat. 2010;38 Suppl 1:S31-43.
47. Coogle CL. The families who care project: meeting educational needs of African American and rural family caregivers dealing with dementia. Educ Gerontol. 2002;28(1):59-71.
48. Gabel LL, Pearsol JA. The twin epidemics of substance use and HIV: a state-level response using a train-the-trainer model. Fam Pract. 1993;10(4):400-5.
49. Jason LA, Lapp C, Kenney KK, Lupton T. An innovative approach in training health care workers to diagnose and manage patients with CFS. In: Alexandra AM, editor. Advances in pyschology research. Hauppauge, NY: Nova Science Publishers; 2008. p. 273-81.
50. Nyamathi A, Vatsa M, Khakha DC, McNeese-Smith D, Leake B, Fahey JL. HIV knowledge improvement among nurses in India: using a train-the-trainer program. JANAC. 2008;19(6):443-9.
51. Pullum JD, Sanddal ND, Obbink K. Training for rural prehospital providers: a retrospective analysis from Montana. Prehosp Emerg Care. 1999;3(3):231-8.
52. Tibbles LR, Smith AE, Manzi SC. Train-the-trainer for hospital-wide safety training. J Nurs Staff Dev. 1993;9(6):266-9.
53. Virani R, Malloy P, Ferrell BR, Kelly K. Statewide efforts in promoting palliative care. J Palliat Med. 2008;11(7):991-6.
54. Booth-Kewley S, Gilman PA, Shaffer RA, Brodine SK. Evaluation of sexually transmitted disease/human immunodeficiency virus prevention train-the-trainer program. Mil Med. 2001;166(4):304-10.
55. Coogle CL, Osgood NJ, Parham IA. A statewide model detection and prevention program for geriatric alcoholism and alcohol abuse: increased knowledge among service providers. Community Ment Health J. 2000;36(2):137-48.
56. Gadomski AM, Wolff D, Tripp M, Lewis C, Short LM. Changes in health care providers' knowledge, attitudes, beliefs, and behaviors regarding domestic violence, following a multifaceted intervention. Acad Med. 2001;76(10):1045-52.
57. Moon RY, Calabrese T, Aird L. Reducing the risk of sudden infanct death syndrome in child care and changing provider practices: lessons learned from a demonstration project. Pediatrics. 2008;122(4):788-98.
58. Tziraki C, Graubard BI, Manley M, Kosary C, Moler JE, Edwards BK. Effect of training on adoption of cancer prevention nutrition-related activities by primary care practices: results of a randomized, controlled study. JGIM. 2000;15(3):155-62.
59. Well M, Stokols D, McMahan S, Clitheroe C. Evaluation of a worksite injury and illness prevention program: do the effects of the REACH OUT training program reach the employees? J Occup Health Psychol. 1997;2(1):25-34.
60. Zapata LB, Coreil J, Entrekin N. Evaluation of triple touch: an assessment of program delivery. Cancer Pract. 2001;9 Suppl 1:S23-30.
61. Demchak M, Browder DM. An evaluation of the pyramid model of staff training in group homes for adults with severe handicaps. Educ Train Ment Retard. 1990;25(2):150-63.
62. Shore BA, Iwata BA, Vollmer TR, Lerman DC, Zarcone JR. Pyramidal staff training in the extension of treatment for severe behavior disorders. J Appl Behav Anal. 1995;28(3):323-30.
63. Anderson SE, Youngson SC. Introduction of a child sexual abuse policy in a health district. Child Soc. 1990;4:401-19.
64. Miller WR, Sorenson JL, Selzer JA, Brigham GS. Disseminating evidence-based practices in substance abuse treatment: a review with suggestions. J Subst Abuse Treat. 2006;31(1):25-39.
65. Worrall JM, Fruzzetti A. Improving peer supervision ratings of therapist performance in dialectical behavior therapy: an internet-based training system. Psychother Theory Res Pract Train. 2009;46(4):476-9.
66. Sholomskas DE, Syracuse-Siewert G, Rounsaville BJ, Ball SA, Nuro KF. We don't train in vain: a dissemination trial of three strategies of training clinicians in cognitive behavioral therapy. J Consult Clin Psychol. 2005;73(1):106-15.
67. National Crime Victims Research & Treatment Center. TF-CBT web: first year report. Charleston, SC: Medical University of South Carolina; 2007.
68. Suda KT, Miltenberger RG. Evaluation of staff management strategies to increase positive interactions in a vocational setting. Behav Res Treat. 1993;8(2):69-88.
69. Miller WR, Yahne CE, Moyers TB, Martinez J, Pirritano M. A randomized trial of methods to help clinicians learn motivational learning. J Consult Clin Psychol. 2004;72(6):1050-62.
70. Sanders MR, Turner KMT, Markie-Dadds C. The development and dissemination of the Triple P-Positive Parenting Program: a multilevel, evidence-based system of parenting and family support. Prev Sci. 2002;3:173-89.
71. Ford JK, Weissbein DA. Training of transfer: an updated review. Perform Improv Q. 1997;10:22-41.
72. Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009;36:24-34.
73. Baldwin T, Ford JK. Transfer of training: a review and directions for future research. Pers Psychol. 1988;41:63-105.
74. Kraiger K, Ford JD, Salas E. Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. J Appl Psychol. 1993;78:311-28.
75. Proctor EK, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2010. doi: 10.1007/s10488-010-0319-7.
76. Herschell AD, Kogan JN, Celedonia KL, Lindhiem O, Stein BD. Evaluation of an implementation initiative for embedding dialectical behavior therapy in community settings. Eval Program Plann. 2014;43:55-63.
77. Kolko DJ, Baumann BL, Herschell AD, Hart JA, Holden EA, Wisniewski SR. Implementation of AF-CBT with community practitioners serving child welfare and mental health: a randomized trial. Child Maltreat. in press.
78. Rollins AL, Salyers MP, Tsai J, Lydick JM. Staff turnover in a statewide implementation of ACT: Relationship with ACT fidelity and other team characteristics. Admin Policy Mental Health Mental Health Serv Res. 2010;37:417-26.
79. Woltmann EM, Whitley R, McHugo GJ, Brunette M, Torrey WC, Coots L, et al. The role of staff turnover in the implementation of evidence-based practices in mental health care. Psychiatr Serv. 2008;59(7):732-7.
80. Southam-Gerow MA, Weisz JR, Chu BC, McLeod BD, Gordis EB, Connor-Smith JK. Does cognitive behavioral therapy for youth anxiety outperform usual care in community clinics? An initial effectiveness test. J Am Acad Child Adolesc Psychiatry. 2010;49(10):1043-52.
81. Weisz JR, Gordis EB, Chu BC, McLeod BD, Updegraff A, Southam-Gerow MA, et al. Cognitive-behavioral therapy versus usual clinical care for youth depression: an Initial test of transportability to community clinics and clinicians. J Consult Clin Psychol. 2009;77(3):383-96.
82. Chamberlain P, Brown CH, Saldana L, Reid J, Wand W, Marsenich L, et al. Engaging and recruiting counties in an experiment on implementing evidence-based practice in California. Adm Policy Ment Health. 2008;35:250-60.
83. Aarons GA, Wells RS, Zagursky K, Fettes DL, Palinkas LA. Implementing evidence-based practice in community mental health agencies: a multiple stakeholder analysis. Am J Public Health. 2009;99(11):2087-95. doi: 10.2105/ajph.2009.161711.
84. Wang W, Saldana L, Brown CH, Chamberlain P. Factors that influenced county system leaders to implement an evidence-based program: a baseline survey within a randomized controlled trial. Implement Sci. 2010;5:72.
85. Eyberg SM, Funderburk B, McNeil CB, Niec L, Urquiza AJ, Zebbell NM. Training guidelines for parent-child interaction therapy. 2009.
86. TF-CBT web: a web-based learning course for trauma-focused cognitive-behavioral therapy. http://tfcbt.musc.edu (2005) Assessed 30 March 2015.
87. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Admin Policy Mental Health Mental Health Serv Res. 2011;38(2):65-76.
88. Baron RM, Kenny DM. The moderator-mediator variable distinction in social psychological research: conceptual, strategic and statistical considerations. J Pers Soc Psychol. 1986;51:1173-82.
89. Kraemer HC et al. Mediators and moderators of treatment effects in randomized clinical trials. Arch Gen Psychiatry. 2002;59:877-83.
90. Little R, Rubin D. Statistical analysis with missing data. John Wiley & Sons, Inc. Hoboken,NJ: 2002.
91. Cohen J. Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates, Inc; 1998. p. 2.
92. Henggeler SW, Schoenwald SK, Liao JG, Letourneau EJ, Edwards DL. Transporting efficacious treatments to field settings: the link between supervisory practices and therapist fidelity in MST programs. J Clin Child Adolesc Psychol. 2002;31(2):155-67.
93. Beidas RS, Kendall PC. Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clin Psychol Sci Pract. 2010;17(1):1-30.
94. Martino S, Ball S, Nich C, Canning-Ball M, Rounsaville B, Carroll K. Teaching community program clinicians motivational interviewing using expert and train-the-trainer strategies. Addiction. 2010;106(2):428-41.
95. Scudder AB, Herschell AD. Building an evidence-base for the training of evidence-based treatments: use of an expert informed approach. Child Youth Serv Rev. 2015;55:84-92.
96. Funderburk B, Nelson M. PCIT Coaches Quiz. 2014.
97. Funderburk B, Gurwitch R, Shanley J, Chase R, Nelson M, Bard E, et al. FIRST Coach Coding System for CDI Coaching Manual. 2014.
98. Kolko DJ. Treatment of child physical abuse: an effectiveness trial. 2012.
99. Herschell AD, McNeil CB, Urquiza AJ, McGrath JM, Zebbell NM, Timmer SG, et al. Evaluation of a treatment manual and workshops for disseminating, Parent-child Interaction Therapy. Admin Policy Mental Health Mental Health Serv Res. 2009;36:63-81.
100. Chafouleas SM, Briesch AM, Riley-Tillman TC, McCoach DB. Moving beyond assessment of treatment acceptability: an examination of the factor structure of the Usage Rating Profile-Intervention (URP-I). Sch Psychol Q. 2009;24:36-47.
101. Addis ME, Krasnow AD. A national survey of practicing psychologists' attitudes toward psychotherapy treatment manuals. J Consulting Clin Psychol. 2000;68(2):331-9.
102. Schoenwald SK et al. A survey of the infrastructure for children's mental health services: implications for the implementation of empirically supported treatments. Adm Policy Ment Health. 2008;35:84-97.
103. Hoagwood K, Schoenwald SK, Chapman JE. Dimensions of organizational readiness-revised (DOOR-R). Unpublished Instrument. 2003
104. Lehman WE, Greener JM, Simpson DD. Assessing organizational readiness for change. J Subst Abuse Treat. 2002;22(4):197-209.
105. Kolko DJ, Herschell AD, Baumann BL. Treatment Implementation Feedback Form. Unpublished Manuscript. Pittsburgh PA: University of Pittsburgh School of Medicine. 2006.
106. Kolko DJ, Dorn LD, Burkstein OG, Pardini D, Holden EA, Hart JD. Community vs. clinic-based modular treatment of children with early-onset ODD or CD: a clinical trial with three-year follow-up. J Abnorm Child Psychol. 2009;37:591-609.
107. Eyberg S, Funderburk B. Parent-Child Interaction Therapy protocol. PCIT International, Inc. Gainsville, FL: 2011.
108. Herschell AD. Translating an evidence-based treatment into community practice: implementing parent-child interaction therapy in an urban domestic violence shelter. 2011.
109. Olmstead T, Carroll K, Canning-Ball M, Martino S. Cost and cost-effectiveness of three strategies for training clinicians in motivational interviewing. in press.
110. Aarons GA. Evidence-based practice implementation: the impact of public versus private sector organization type on organizational support, provider attitudes, and adoption of evidence-based practice. Impl Sci. 2009;4:83.
111. Eyberg SM, Pincus D. Eyberg Child Behavior Inventory and Sutter-Eyberg Behavior Inventory: revised professional manual. Odessa, FL: Psychological Assessment Resources, Inc.; 2000.
112. Wolraich ML, Lambert W, Doffing MA, Bickman L, Simmons T, Worley K. Psychometric properties of the Vanderbilt ADHD diagnostic parent rating scale in a referred population. J Pediatr Psychol. 2003;28(8):559-68.
113. Frick PJ, Hare RD. Antisocial process screening device: APSD. Toronto: Multi-Health Systems; 2001.
114. Kroenke K, Spitzer RL, Williams JB. The Phq-9. JGIM. 2001;16(9):606-13.
115. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092-7.
116. Elgar FJ, Waschbusch DA, Dadds MR, Sigvaldason N. Development and validation of a short form of the Alabama Parenting Questionnaire. J Child Fam Stud. 2007;16(2):243-59.
117. Kazdin AE, Holland L, Crowley M, Breton S. Barriers to treatment participation scale: evaluation and validation in the context of child outpatient treatment. J Child Psychol Psychiatry. 1997;38(8):1051-62.
118. Brestan EV, Jacobs JR, Rayfield AD, Eyberg SM. A consumer satisfaction measure for parent-child treatments and its relation to measures of child behavior change. Behav Ther. 2000;30(1):17-30.
119. Gyamfi P, Walrath C, Burns BJ, Stephens RL, Geng Y, Stambaugh L. Family education and support services in systems of care. J Emotion Behav Disord. 2009;18(1):14-26.
120. Ben-Dror R. Employee turnover in community mental health organizations: A developmental stages study. Community Ment Health J. 1994;30:14.
121. Sheidow AJ, Schoenwald SK, Wagner HR, Allred CA, & Burns BJ. Predictors of workforce turnover in a tranposted treatment program. Adm Policy Ment Health and Ment Health Serv Res. 2007;34(1):45-56.