Studying implementation quality of a school-based prevention curriculum in frontier Alaska: Application of video-recorded observations and expert panel judgment

Prevention Science

Volumn 11, Issue 3, 2010, Pages 275-286

  Johnson, Knowlton W ^a   Ogilvie, Kristen A ^a   Collins, David A ^a   Shamblen, Stephen R ^a   Dirks, Lisa G ^a   Ringwalt, Christopher L ^a   Norland, Jennifer J ^a  

^a PACIFIC INSTITUTE FOR RESEARCH AND EVALUATION   (United States)

Author keywords

Expert panel assessment; Implementation quality; School based prevention; Video recorded observations; Youths' harmful legal product use

Indexed keywords

ADDICTION; ARTICLE; CHILD; CURRICULUM; HEALTH CARE QUALITY; HUMAN; ORGANIZATION AND MANAGEMENT; PREVENTIVE HEALTH SERVICE; RURAL POPULATION; SCHOOL; STANDARD; UNITED STATES; PROGRAM EVALUATION; STANDARDS; SUBSTANCE-RELATED DISORDERS;

ALASKA; CHILD; CURRICULUM; HUMANS; PREVENTIVE HEALTH SERVICES; PROGRAM EVALUATION; RURAL POPULATION; SCHOOLS; SUBSTANCE-RELATED DISORDERS;

EID: 77954622232     PISSN: 13894986     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11121-010-0174-5     Document Type: Article

References (50)

1. Backer, T. E. (2001). Finding the balance: Program fidelity and adaptation in substance abuse prevention. Rockville, MD: Center for Substance Abuse Prevention.
2. Berman, P., Greenwood, P. W., McLaughlin, M. W., & Pincus, J. (1975). Federal programs supporting educational change. Santa Monica, CA: RAND.
3. Berman, P., & McLaughlin, M. W. (1976). Implementation of educational innovation. The Educational Forum, 40, 345-370.
4. Bierman, K. L., Cole, J. D., Dodge, K. A., Greenberg, M. T., Lochman, J. E., & McMahon, R. J. (1997). Implementing a comprehensive program for the prevention of conduct problems in rural communities: The Fast Track experience. American Journal of Community Psychology, 25, 493-514.
5. Bosworth, K. (2000). Protective schools: Linking drug abuse prevention with student success. A guide for educators, policy makers, and families. Tucson: University of Arizona.
6. Botvin, G. J., Griffin, K. W., Diaz, T., Scheier, L. M., Williams, C., & Epstein, J. A. (2000). Preventing illicit drug use in adolescents: Long-term follow-up data from a randomized control trial of a school population. Addictive Behaviors, 25, 769-774.
7. Brandon, P. R., Taum, A. K. H., Young, D. B., Pottenger, F. M., III, & Speitel, T. W. (2008). The complexity of measuring the quality of program implementation with observations. American Journal of Evaluation, 29, 235-250.
8. Centers for Disease Control and Prevention. (1994). Guidelines for school health programs to prevent tobacco use and addiction. MMWR, 1994, 43.
9. Cohen, J. (1960). A coefficient for agreement for nominal scales. Education and Psychological Measurement, 20, 37-46.
10. CSAP (2001). Model programs: Bringing effective prevention to every community. Retrieved May 30, 2007, from http://modelprograms. samhsa. gov/model. htm.
11. Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23-45.
12. Domitrovich, C. E., & Greenberg, M. T. (2000). The study of implementation: Current findings from effective programs that prevent mental disorders in school-aged children. Journal of Educational and Psychological Consultation, 11, 193-221.
13. Drug Strategies (1999). Making the grade: A guide to drug prevention programs (Revised Edition). Washington, DC: Drug Strategies.
14. Dusenbury, L., Brannigan, R., Falco, M., & Hansen, W. B. (2003). A review of research on fidelity of implementation: Implications for drug abuse prevention in school settings. Health Education Research, 18, 237-256.
15. Dusenbury, L., Brannigan, R., Hansen, W. B., Walsh, J., & Falco, M. (2005). Quality of implementation: Developing measures crucial to understanding the diffusion of preventive interventions. Health Education Research, 20, 308-313.
16. Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological Bulletin, 76, 378-382.
17. Fullan, M., & Pomfret, A. (1977). Research on curriculum and instruction implementation. Review of Educational Research, 52, 7-30.
18. Giles, S. M., Jackson-Newsom, J., Pankratz, M. M., Hansen, W. B., Ringwalt, C. L., & Dusenbury, L. (2008). Measuring quality of delivery in a substance use prevention program. Journal of Primary Prevention, 29, 489-501.
19. Gingiss, P. M., Robert-Graw, C., & Boer, M. (2006). Bridge-it: A system for predicting implementation fidelity for school-based tobacco prevention programs. Prevention Science, 7, 197-207.
20. Hansen, W. B., & McNeal, R. B. (1999). Drug education practice: Results of an observational study. Health Education Research, 14, 85-97.
21. Johnson, K. W. (1990). Impact of a university-government partnership in criminal justice: A linkage model for the 1990s. The Journal of Criminal Justice Education, 1, 167-194.
22. Johnson, K. W., Holder, H., Ogilvie, K. A., Collins, D. A., Courser, M., Miller, B., et al. (2007). A community prevention intervention to combat inhalants and other harmful legal products among pre-adolescents. Journal of Drug Education, 37, 227-247.
23. Johnson, K. W., Shamblen, S. R., Ogilvie, K. A., Collins, D. A., & Saylor, B. L. (2009). Preventing youth's use of inhalants and other harmful legal products in frontier Alaskan communities: A randomized trial. Prevention Science, 10, 298-312.
24. Lillehoj, C. J., Griffin, K. W., & Spoth, R. (2004). Program provider and observer ratings of school-based preventive intervention implementation: Agreement and relation to youth outcomes. Health Education and Behavior, 31, 242-257.
25. McCormick, L., Steckler, A., & McLeroy, K. (1995). Diffusion of innovations in schools: A study of adoption and implementation of school-based tobacco prevention curricula. American Journal of Health Promotion, 9, 210-219.
26. McGrew, J. H., Bond, G. R., Dietzen, L., & Salyers, M. (1994). Measuring the fidelity of implementation of a mental health program model. Journal of Consulting and Clinical Psychology, 62, 670-678.
27. Melde, C., Esbensen, F. A., & Tusinski, K. (2006). Addressing program fidelity using onsite observations and program provider descriptions of program delivery. Evaluation Review, 30, 714-740.
28. Meyer, M. A., & Booker, J. M. (2001). Eliciting and analyzing expert judgment: A practical guide. Philadelphia: Society for Industrial and Applied Mathematics.
29. National Center for Frontier Communities (2007). List of frontier counties from 2000 U. S. Census. Retrieved May 25, 2007, from http://frontierus. org/2000census. htm.
30. NCLB (2004). Safe and drug-free schools and communities act, state grants, guidance for state and local implementation of programs. Retrieved May 1, 2009, from www. ed. gov/programs/dvpformula/guidance. doc.
31. Nevo, D. (1989). Expert opinion in program evaluation. New Directions for Program Evaluation, 42, 85-93.
32. NREPP (2007). SAMHSA'S National registry of evidence based programs and practices: Finding interventions. Retrieved May 30, 2007, from www. nrepp. samhsa. gov/find. asp.
33. Ogilvie, D. C., Coulehan, H. M., Ogilvie, K. A., & Johnson, K. W. (2006). Think Smart: Problem solving for Alaskan youth, A curriculum to prevent substance abuse including inhalants and other harmful everyday legal products. Anchorage, AK: Akeela.
34. OJJDP. (2002). Blueprints for violence prevention. Retrieved May 30, 2007, from www. colorado. edu/cspv/blueprints/model/overview. html.
35. Pankratz, M. M., Jackson-Newsom, J., Giles, S. M., Ringwalt, C. L., Bliss, K., & Bell, M. L. (2006). Implementation fidelity in a teacher-led alcohol use prevention curriculum. Journal of Drug Education, 36, 317-333.
36. Ridgely, M. S., & Jerrell, J. M. (1996). Analysis of three interventions for substance abuse treatment of severely mentally ill people. Community Mental Health Journal, 32, 561-572.
37. Ringwalt, C. L., & Bliss, K. (2006). The cultural tailoring of a substance use prevention curriculum for American Indian youth. Journal of Drug Education, 36, 159-177.
38. Ringwalt, C. L., Ennett, S., Rohrbach, L. A., Simons-Rudolph, A., Vincus, A., & Thorne, J. (2003). Factors associated with fidelity to substance use prevention curriculum guides on the nation's middle schools. Health Education & Behavior, 30, 375-391.
39. Ringwalt, C. L., Pankratz, M. M., Jackson-Newsom, J., Gottfredson, N. C., Hansen, W. B., Giles, S. M., et al. (2009). Three year trajectory of teachers' fidelity to a drug prevention curriculum. Prevention Science (in press).
40. Robertson, E. B., David, S. L., & Rao, S. A. (2003). Preventing drug use among children and adolescents: A research-based guide for parents, educators, and community leaders (2nd ed.). Bethesda, MD: National Institute on Drug Abuse.
41. Rohrbach, L. A., Dent, C. W., Skara, S., Sun, P., & Sussman, S. (2007). Fidelity of implementation in Project Towards No Drug Abuse (TND): A comparison of classroom teachers and program specialists. Prevention Science, 8, 125-132.
42. Schinke, S. P., Orlandi, M. A., Botvin, G. J., Gilchrist, L. D., Trimble, J. E., & Locklear, V. S. (1988). Preventing substance abuse among American-Indian adolescents: A bicultural competence skills approach. Journal of Counseling Psychology, 35, 87-90.
43. Schinke, S. P., Tepavac, L., & Cole, K. C. (2000). Preventing substance use among Native American youth: Three-year results. Addictive Behaviors, 25, 387-397.
44. Shrout, P., & Fleiss, J. L. (1979). Intraclass correlation: Uses in assessing rater reliability. Psychological Bulletin, 86, 420-428.
45. Sobol, D. F., Rohrbach, L. A., Dent, C. W., Gleason, L., Brannon, B. R., Johnson, C. A., et al. (1989). The integrity of smoking prevention curriculum delivery. Health Education Research, 4, 59-67.
46. Spoth, R., Redmond, C., & Shin, C. (1988). Direct and indirect latent-variable parenting outcomes of two universal family-focused preventative interventions: Extending a public health oriented research base. Journal of Consulting and Clinical Psychology, 66, 385-399.
47. Spoth, R., Guyll, M., Lillehoj, C. J., Redmond, C., & Greenberg, M. (2007). Prosper study of evidence-based intervention implementation quality by community-university partnerships. Journal of Community Psychology, 35, 981-999.
48. Tricker, R., & Davis, L. G. (1988). Implementing drug education in schools: An analysis of the costs and teacher perceptions. Journal of School Health, 58, 181-185.
49. U. S. DOE (2001). Safe, disciplined, and drug-free schools expert panel. Retrieved May 30, 2007, from www. ed. gov/admins/lead/safety/exemplary01/index. html.
50. Yeaton, W. H., & Sechrest, L. (1981). Critical dimensions in the choice and maintenance of successful treatments: Strength, integrity and effectiveness. Journal of Consulting and Clinical Psychology, 49, 156-167.