Validating the Implementation Climate Scale (ICS) in child welfare organizations

Child Abuse and Neglect

Volumn 53, Issue , 2016, Pages 17-26

  Ehrhart, Mark G ^a   Torres, Elisa M ^b,c   Wright, Lisa A ^a   Martinez, Sandra Y ^a   Aarons, Gregory A ^b,c  

^a SAN DIEGO STATE UNIVERSITY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c CHILD AND ADOLESCENT SERVICES RESEARCH CENTER   (United States)

Author keywords

Evidence based practice; Implementation climate; Measurement; Organizational climate; Organizational context

Indexed keywords

CHILD; CHILD WELFARE; CLIMATE; FACTORIAL ANALYSIS; HUMAN; MAJOR CLINICAL STUDY; MENTAL HEALTH; MODEL; ORGANIZATION; RELIABILITY; VALIDITY; ADULT; AGED; CALIFORNIA; CHILD PROTECTION; CLINICAL TRIAL; EVIDENCE BASED PRACTICE; FEMALE; HEALTH CARE PERSONNEL; HEALTH CARE PLANNING; ILLINOIS; MALE; MIDDLE AGED; MULTICENTER STUDY; OKLAHOMA; ORGANIZATION AND MANAGEMENT; ORGANIZATIONAL POLICY; REPRODUCIBILITY; STANDARDS;

ADULT; AGED; CALIFORNIA; CHILD; CHILD PROTECTIVE SERVICES; CHILD WELFARE; EVIDENCE-BASED PRACTICE; FEMALE; HEALTH PERSONNEL; HEALTH PLAN IMPLEMENTATION; HUMANS; ILLINOIS; MALE; MIDDLE AGED; OKLAHOMA; ORGANIZATIONAL INNOVATION; ORGANIZATIONAL POLICY; REPRODUCIBILITY OF RESULTS;

EID: 84962079673     PISSN: 01452134     EISSN: 18737757     Source Type: Journal    
DOI: 10.1016/j.chiabu.2015.10.017     Document Type: Article

References (66)

1. Aarons G.A., Ehrhart M.G., Farahnak L.R. The Implementation Leadership Scale (ILS): Development of a brief measure of unit level implementation leadership. Implementation Science 2014, 9:45.
2. Aarons G.A., Ehrhart M.G., Farahnak L.R., Hurlburt M.S. Leadership and organizational change for implementation (LOCI): A randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science 2015, 10:11.
3. Aarons G.A., Ehrhart M.G., Farahnak L.R., Sklar M. Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health 2014, 35:255-274.
4. Aarons G.A., Fettes D.L., Hurlburt M.S., Palinkas L.A., Gunderson L., Willging C.E., Chaffin M.J. Collaboration, negotiation, and coalescence for interagency-collaborative teams to scale-up evidence-based practice. Journal of Clinical Child & Adolescent Psychology 2014, 43(6):915-928.
5. Aarons G.A., Glisson C., Green P.D., Hoagwood K., Kelleher K.J., Landsverk J.A., Research Network on Youth Mental Health The organizational social context of mental health services and clinician attitudes toward evidence-based practice: A United States national study. Implementation Science 2012, 7:56.
6. Aarons G.A., Green A.E., Palinkas L.A., Self-Brown S., Whitaker D.J., Lutzker J.R., Chaffin M.J. Dynamic adaptation process to implement an evidence-based child maltreatment intervention. Implementation Science 2012, 7:32.
7. Aarons G.A., Hurlburt M., Horwitz S.M. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research 2011, 38(1):4-23.
8. Aarons G.A., Palinkas L.A. Implementation of evidence-based practice in child welfare: Service provider perspectives. Administration and Policy in Mental Health and Mental Health Services Research 2007, 34:411-419.
9. Barbee A.P., Christensen D., Antle B., Wandersman A., Cahn K. Successful adoption and implementation of a comprehensive casework practice model in a public child welfare agency: Application of the Getting to Outcomes (GTO) model. Children and Youth Services Review 2011, 33(5):622-633.
10. wg indices. Organizational Research Methods 2005, 8(2):165-184.
11. Byrne B.M., Stewart S.M. The MACS approach to testing for multigroup invariance of a second-order structure: A walk through the process. Structural Equation Modeling 2006, 13:287-321.
12. Chen F.F. Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling 2007, 14:464-504.
13. Cheung G.W., Rensvold R.B. Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling 2002, 9:233-255.
14. Dogherty E.J., Harrison M.B., Graham I.D., Vandyk A.D., Keeping-Burke L. Turning knowledge into action at the point-of-care: The collective experience of nurses facilitating the implementation of evidence-based practice. Worldviews on Evidence-Based Nursing 2013, 10(3):129-139.
15. Dolan M., Smith K., Casanueva C., Ringeisen H. NSCAW II Baseline Report: Caseworker Characteristics, Child Welfare Services, and Experiences of Children Placed in Out-of-Home Care 2011, OPRE Report #2011-27e, Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services, Washington, DC.
16. Ehrhart K.H., Witt L.A., Schneider B., Perry S.J. Service employees give as they get: Internal service as a moderator of the service climate-service outcomes link. Journal of Applied Psychology 2011, 96(2):423-431.
17. Ehrhart M.G., Aarons G.A., Farahnak L.R. Assessing the organizational context for EBP implementation: The development and validity testing of the Implementation Climate Scale (ICS). Implementation Science 2014, 9:157.
18. Ehrhart M.G., Schneider B., Macey W.H. Organizational climate and culture: An introduction to theory, research, and practice 2014, Routledge, New York.
19. Fogarty International Center Frequently Asked Questions about Implementation Science 2013, May, http://www.fic.nih.gov/News/Events/implementation-science/Pages/faqs.aspx.
20. Garland A.F., Hurlburt M.S., Hawley K.M. Examining psychotherapy processes in a services research context. Clinical Psychology: Science and Practice 2006, 13(1):30-46.
21. Guo Q., Li F., Chen X., Wang W., Meng Q. Performance of fit indices in different conditions and selection of cut-off values. Acta Psychologica Sinica 2008, 40(1):109-118.
22. Glisson C., Green P. Organizational climate, services, and outcomes in child welfare systems. Child Abuse & Neglect 2011, 35(8):582-591.
23. Glisson C., Hemmelgarn A. The effects of organizational climate and interorganizational coordination on the quality and outcomes of children's service systems. Child Abuse & Neglect 1998, 22(5):401-421.
24. Glisson C., Green P., Williams N.J. Assessing the Organizational Social Context (OSC) of child welfare systems: Implications for research and practice. Child Abuse & Neglect 2012, 36(9):621-632.
25. Glisson C., James L.R. The cross-level effects of culture and climate in human service teams. Journal of Organizational Behavior 2002, 23(6):767-794.
26. Glisson C., Landsverk J., Schoenwald S., Kelleher K., Hoagwood K.E., Mayberg S., Green P. Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Administration and Policy in Mental Health and Mental Health Services Research 2008, 35(1-2):98-113.
27. Helfrich C., Li Y., Sharp N., Sales A. Organizational readiness to change assessment (ORCA): Development of an instrument based on the promoting action on research in health services (PARIHS) framework. Implementation Science 2009, 4:38.
28. Holmes L., Landsverk J., Ward H., Rolls-Reutz J., Saldana L., Wulczyn F., Chamberlain P. Cost calculator methods for estimating casework time in child welfare services: A promising approach for use in implementation of evidence-based practices and other service innovations. Children and Youth Services Review 2014, 39:169-176.
29. Horwitz S.M., Chamberlain P., Landsverk J., Mullican C. Improving the mental health of children in child welfare through the implementation of evidence-based parenting interventions. Administration and Policy in Mental Health and Mental Health Services Research 2010, 37(1-2):27-39.
30. Hu L.T., Bentler P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal 1999, 6(1):1-55.
31. Jacobs S.R., Weiner B.J., Bunger A.C. Context matters: Measuring implementation climate among individuals and groups. Implementation Science 2014, 9:46.
32. Jacobs S.R., Weiner B.J., Reeve B.B., Hofmann D.A., Christian M., Weinberger M. Determining the predictors of innovation implementation in healthcare: A quantitative analysis of implementation effectiveness. BMC Health Services Research 2015, 15(1):6.
33. Jung D., Wu A., Chow C.W. Towards understanding the direct and indirect effects of CEOs' transformational leadership on firm innovation. The Leadership Quarterly 2008, 19(5):582-594.
34. Kaye S., DePanfilis D., Bright C.L., Fisher C. Applying implementation drivers to child welfare systems change: Examples from the field. Journal of Public Child Welfare 2012, 6(4):512-530.
35. Kessler M., Gira E., Poertner J. Moving best practice to evidence-based practice in child welfare. Families in Society: The Journal of Contemporary Social Services 2005, 86(2):244-250.
36. Klein K.J., Conn A.B., Sorra J.S. Implementing computerized technology: An organizational analysis. Journal of Applied Psychology 2001, 86:811-824.
37. Kolko D.J., Baumann B.L., Herschell A.D., Hart J.A., Holden E.A., Wisniewski S.R. Implementation of AF-CBT by community practitioners serving child welfare and mental health: A randomized trial. Child Maltreatment 2012, 10.1177/1077559511427346.
38. Kuenzi M., Schminke M. Assembling fragments into a lens: A review, critique, and proposed research agenda for the organizational work climate literature. Journal of Management 2009, 35:634-717.
39. Lewin K., Lippitt R., White R.K. Patterns of aggressive behavior in experimentally created 'social climates'. Journal of Social Psychology 1939, 10:271-299.
40. Lobb R., Colditz G. Implementation science and its application to population health. Annual Review of Public Health 2013, 34(1):235-251.
41. MacCallum R.C., Browne M.W., Cai L. Tested differences between nested covariance structure models: Power analyses and null hypotheses. Psychological Methods 2005, 11:19-35.
42. Mayer D., Nishii L., Schneider B., Goldstein H. The precursors and products of justice climates: Group leader antecedents and employee attitudinal consequences. Personnel Psychology 2007, 60(4):929-963.
43. McCormack B., Kitson A., Harvey G., Rycroft-Malone J., Titchen A., Seers K. Getting evidence into practice: The meaning of "context". Journal of Advanced Nursing 2002, 38:94-104.
44. Meredith W. Measurement invariance, factor analysis, and factorial invariance. Psychometrika 1993, 58:525-543.
45. Mildon R., Shlonsky A. Bridge over troubled water: Using implementation science to facilitate effective services in child welfare. Child Abuse & Neglect 2011, 35(9):753-756.
46. Mitchell P. Evidence-based practice in real-world services for young people with complex needs: New opportunities suggested by recent implementation science. Children and Youth Services Review 2011, 33(2):207-216.
47. Muthén L.K., Muthén B.O. Mplus user's guide 1998-2012, Muthén & Muthén, Los Angeles, CA. 7th ed.
48. Ostroff C., Kinicki A.J., Muhammad R.S. Organizational culture and climate. Handbook of psychology, Vol. 12: Industrial and organizational psychology 2012, 643-676. Wiley, Hoboken, NJ. 2nd ed.
49. Pipkin S., Sterrett E.M., Antle B., Christensen D.N. Washington State's adoption of a child welfare practice model: An illustration of the getting to outcomes implementation framework. Children and Youth Services Review 2013, 35(12):1923-1932.
50. Pecora P.J., Whittaker J.K., Maluccio A.N., Barth R.P. The child welfare challenge: Policy, practice, and research 2012, AldineTransaction.
51. Ployhart R.E., Oswald F.L. Applications of mean and covariance structure analysis: Integrating correlational and experimental approaches. Organizational Research Methods 2004, 7:27-65.
52. Proctor L.K., Landsverk J., Aarons G.A., Chambers D., Glisson C., Mittman B. Implementation research in mental health services: An emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research 2009, 36:24-34.
53. Proctor E.K., Powell B.J., Feely M. Measurement in dissemination and implementation science. Dissemination and implementation of evidence-based practices in child and adolescent mental health 2014, 22-43. Oxford University Press, New York. R.S. Beidas, P.C. Kendall (Eds.).
54. Rabin B.A., Purcell P., Naveed S., Moser R.P., Henton M.D., Proctor E.K., Brownson R.C., Glasgow R.E. Advancing the application, quality and harmonization of implementation science measures. Implementation Science 2012, 7:119.
55. Roesch S.C., Norman G.J., Merz E.L., Sallis J.F., Patrick K. Longitudinal measurement invariance of psychosocial measures in physical activity research: An application to adolescent data. Journal of Applied Social Psychology 2013, 43(4):721-729.
56. Salanova M., Agut S., Peiró J.M. Linking organizational resources and work engagement to employee performance and customer loyalty: The mediation of service climate. Journal of Applied Psychology 2005, 90:1217-1227.
57. Schein E.H. Organizational culture and leadership 2010, Jossey-Bass, San Francisco. 4th ed.
58. Schneider B., Ehrhart M.G., Macey W.H. Organizational climate research: Achievements and the road ahead. The handbook of organizational culture and climate 2011, 29-49. Sage, Thousand Oaks, CA. 2nd ed. N.M. Ashkanasy, C.P.M. Wilderom, M.F. Peterson (Eds.).
59. Schneider B., Ehrhart M.G., Mayer D.M., Saltz J.L., Niles-Jolly K. Understanding organization-customer links in service settings. Academy of Management Journal 2005, 48(6):1017-1032.
60. Schneider B., White S.S., Paul M.C. Linking service climate and customer perceptions of service quality: Test of a causal model. Journal of Applied Psychology 1998, 83:150-163.
61. Shea C.M., Jacobs S.R., Esserman D.A., Bruce K., Weiner B.J. Organizational readiness for implementing change: A psychometric assessment of a new measure. Implementation Science 2014, 9:7.
62. Stirman S., Crits-Christoph P., DeRubeis R. Achieving successful dissemination of empirically supported psychotherapies: A synthesis of dissemination theory. Clinical Psychology: Science and Practice 2004, 11(4):343-359.
63. Wallace J.C., Popp E., Mondore S. Safety climate as a mediator between foundation climates and occupational accidents: A group-level investigation. Journal of Applied Psychology 2006, 91:681-688.
64. Weiner B., Belden C., Bergmire D., Johnston M. The meaning and measurement of implementation climate. Implementation Science 2011, 6:78.
65. Williams N.J., Glisson C. Testing a theory of organizational culture, climate and youth outcomes in child welfare systems: A United States national study. Child Abuse & Neglect 2014, 38(4):757-767.
66. Zohar D., Luria G. A multilevel model of safety climate: Cross-level relationships between organization and group-level climates. Journal of Applied Psychology 2005, 90(4):616-628.