Examining variation in the impact of school-wide positive behavioral interventions and supports: Findings from a randomized controlled effectiveness trial

Journal of Educational Psychology

Volumn 107, Issue 2, 2015, Pages 546-557

  Bradshaw, Catherine P ^a   Waasdorp, Tracy E ^a   Leaf, Philip J ^a  

^a JOHNS HOPKINS UNIVERSITY   (United States)

Author keywords

Aggressive and disruptive behavior; Prevention; Randomized controlled trial; School wide positive behavioral interventions and supports (SWPBIS); Schools; Social emotional adjustment

Indexed keywords

EID: 84930341504     PISSN: 00220663     EISSN: 19392176     Source Type: Journal    
DOI: 10.1037/a0037630     Document Type: Article

References (66)

1. Arbuckle, J. L. (1996). Full information estimation in the presence of incomplete data. In G. A. Marcoulides & R. E. Schumacker (Eds.), Advanced structural equation modeling: Issues and techniques (pp. 243-277). Mahwah, NJ: Erlbaum.
2. Asparouhov, T., & Muthén, B. O. (2010). Multiple imputation with Mplus. Retrieved from http://www.statmodel.com/download/Imputations7.pdf
3. Barrett, S. B., Bradshaw, C. P., & Lewis-Palmer, T. (2008). Maryland statewide PBIS initiative. Journal of Positive Behavior Interventions, 10, 105-114. doi:10.1177/1098300707312541
4. Bierman, K. (2002). Evaluation of the first 3 years of the fast track prevention trial with children at high risk for adolescent conduct problems. Journal of Abnormal Child Psychology, 30, 19-35. doi:10.1023/A:1014274914287
5. Bradshaw, C. P., Koth, C. W., Thornton, L. A., & Leaf, P. J. (2009). Altering school climate through School-Wide Positive Behavioral Interventions and Supports: Findings from a group-randomized effectiveness trial. Prevention Science, 10, 100-115. doi:10.1007/s11121-008-0114-9
6. Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010). Examining the effects of School-Wide Positive Behavioral Interventions and Supports on student outcomes: Results from a randomized controlled effectiveness trial in elementary schools. Journal of Positive Behavior Interventions, 12, 133-148. doi:10.1177/1098300709334798
7. Bradshaw, C. P., O'Brennan, L. M., & McNeely, C. A. (2008). Core competencies and the prevention of school failure and early school leaving [Special issue]. New Directions for Child and Adolescent Development, 122, 19-32. doi:10.1002/cd.226
8. Bradshaw, C. P., & Pas, E. T. (2011). A state-wide scale-up of Positive Behavioral Interventions and Supports (PBIS): A description of the development of systems of support and analysis of adoption and implementation. School Psychology Review, 40, 530-548.
9. Bradshaw, C. P., Pas, E. T., Bloom, J., Barrett, S., Hershfeldt, P., Alexander, A., . . . Leaf, P. J. (2012). A state-wide partnership to promote safe and supportive schools: The PBIS Maryland Initiative. Administration and Policy in Mental Health and Mental Health Services Research, 39, 225-237. doi:10.1007/s10488-011-0384-6
10. Bradshaw, C. P., Pas, E. T., Goldweber, A., Rosenberg, M., & Leaf, P. (2012). Integrating school-wide Positive Behavioral Interventions and Supports with Tier 2 coaching to student support teams: The PBIS plus model. Advances in School Mental Health Promotion, 5, 177-193. doi:10.1080/1754730X.2012.707429
11. Bradshaw, C. P., Reinke, W. M., Brown, L. D., Bevans, K. B., & Leaf, P. J. (2008). Implementation of school-wide Positive Behavioral Interventions and Supports (PBIS) in elementary schools: Observations from a randomized trial. Education and Treatment of Children, 31, 1-26. doi: 10.1353/etc.0.0025
12. Bradshaw, C. P., Waasdorp, T. E., & Leaf, P. J. (2012). Effects of school-wide Positive Behavioral Interventions and Supports on child behavior problems. Pediatrics, 130, e1136-1145. doi:10.1542/peds.2012-0243
13. Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students' social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405-432. doi:10.1111/j.1467-8624.2010.01564.x
14. Farrell, A. D., Henry, D. B., & Bettencourt, A. (2013). Methodological challenges examining subgroup differences: Examples from universal school-based youth violence prevention trials. Prevention Science, 14, 121-133. doi:10.1007/s11121-011-0200-2
15. Flay, B. R., Biglan, A., Boruch, R. F., González Castro, F., Gottfredson, D., Kellam, S., . . . Ji, P. (2005). Standards of evidence: Criteria for efficacy, effectiveness and dissemination. Prevention Science, 6, 151-175. doi: 10.1007/s11121-005-5553-y
16. Hawken, L. S., Vincent, C. G., & Schumann, J. (2008). Response to intervention for social behavior: Challenges and opportunities. Journal of Emotional and Behavioral Disorders, 16, 213-225. doi:10.1177/1063426608316018
17. Horner, R. H., Sugai, G., & Anderson, C. M. (2010). Examining the evidence base for School-Wide Positive Behavior Support. Focus on Exceptional Children, 42, 1-14.
18. Horner, R. H., Sugai, G., Smolkowski, K., Eber, L., Nakasato, J., Todd, A. W., & Esperanza, J. (2009). A randomized, wait-list controlled effectiveness trial assessing School-Wide Positive Behavior Support in elementary schools. Journal of Positive Behavior Interventions, 11, 133-144. doi:10.1177/1098300709332067
19. Horner, R. H., Todd, A., Lewis-Palmer, T., Irvin, L., Sugai, G., & Boland, J. (2004). The school-wide evaluation tool (SET): A research instrument for assessing School-Wide Positive Behavior Support. Journal of Positive Behavior Interventions, 6, 3-12. doi:10.1177/10983007040060010201
20. Horwitz, S. M., Hoagwood, K., Stiffman, A. R., Summerfeld, T., Weisz, J. R., Costello, E. J., . . . Norquist, G. (2001). Reliability of the Services Assessment for Children and Adolescents (SACA). Psychiatric Services, 52, 1088-1094. doi:10.1176/appi.ps.52.8.1088
21. Ialongo, N. S., Werthamer, L., & Kellam, S. G. (1999). Proximal impact of two first-grade preventive interventions on the early risk behaviors for later substance abuse, depression, and antisocial behavior. American Journal of Community Psychology, 27, 599-641. doi:10.1023/A:1022137920532
22. Institute of Medicine and National Research Council. (2009). Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington, DC: National Academies Press.
23. Jelićić, H., Phelps, E., & Lerner, R. M. (2010). Why missing data matter in the longitudinal study of adolescent development: Using the 4-H study to understand the uses of different missing data methods. Journal of Youth and Adolescence, 39, 816-835. doi:10.1007/s10964-010-9542-5
24. Kellam, S. G., Ling, X., Merisca, R., Brown, C. H., & Ialongo, N. (1998). The effect of the level of aggression in the first grade classroom on the course and malleability of aggressive behavior into middle school. Development and Psychopathology, 10, 165-185. doi:10.1017/S0954579498001564
25. Kellam, S. G., Rebok, G. W., Ialongo, N., & Mayer, L. S. (1994). The course and malleability of aggressive behavior from early first-grade into middle school: Results of a developmental epidemiologically based preventive trial. Journal of Child Psychology and Psychiatry, 35, 259-281. doi:10.1111/j.1469-7610.1994.tb01161.x
26. Koth, C. W., Bradshaw, C. P., & Leaf, P. J. (2009). Teacher observation of classroom adaptation- checklist: Development and factor structure. Measurement and Evaluation in Counseling and Development, 42, 15-30. doi:10.1177/0748175609333560
27. Lanza, S. T., & Rhoades, B. T. (2013). Latent class analysis: An alternative perspective on subgroup analysis in prevention and treatment. Prevention Science, 14, 157-168. doi:10.1007/s11121-011-0201-1
28. Lewis, T. J., & Sugai, G. (1999). Effective behavior support: A systems approach to proactive schoolwide management. Focus on Exceptional Children, 31, 1-24.
29. Little, R. J. (1995). Modeling the dropout mechanism in repeated-measures studies. Journal of the American Statistical Association, 90, 1112-1121. doi:10.1080/01621459.1995.10476615
30. Loehlin, J. C. (1998). Latent variable models: An introduction to factor, path, and structural analysis (3rd ed.). Mahwah, NJ: Erlbaum.
31. Lubke, G. H., & Muthén, B. O. (2007). Performance of factor mixture models as a function of model size, covariate effects, and class-specific parameters. Structural Equation Modeling, 14, 26-47. doi:10.1207/s15328007sem1401_2
32. Mayer, G. R. (1995). Preventing antisocial behavior in the schools. Journal of Applied Behavior Analysis, 28, 467-478. doi:10.1901/jaba.1995.28-467
33. McCutcheon, A. L. (1987). Latent class analysis. Newbury Park, CA: Sage.
34. Moffitt, T. E. (2006). Life-course-persistent versus adolescence-limited antisocial behavior. In D. Cicchetti & D. J. Cohen (Eds.), Developmental Psychopathology (Vol. 3, pp. 570-598). Hoboken, NJ: Wiley.
35. Murray, D. M. (1998). Design and analysis of group-randomized trials. New York, NY: Oxford Press.
36. Muthén, B. O., Brown, C. H., Masyn, K., Jo, B., Khoo, S. T., Yang, C. C., . . . Liao, J. (2002). General growth mixture modeling for randomized preventive interventions. Biostatistics, 3, 459-475. doi:10.1093/biostatistics/3.4.459
37. Muthén, L. K., & Muthén, B. O. (1998-2012). Mplus user's guide (7th ed.). Los Angeles, CA: Muthén & Muthén.
38. Nylund, K., Asparouhov, T., & Muthén, B. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling, 14, 535-569. doi:10.1080/10705510701575396
39. Owens, P. L., Hoagwood, K., Horwitz, S. M., Leaf, P. J., Poduska, J. M., Kellam, S. G., & Ialongo, N. S. (2002). Barriers to children's mental health services. Journal of the American Academy of Child & Adolescent Psychiatry, 41, 731-738. doi:10.1097/00004583-200206000-00013
40. Pas, E. T., Bradshaw, C. P., & Mitchell, M. M. (2011). Examining the validity of office discipline referrals as an indicator of student behavior problems. Psychology in the Schools, 48, 541-555. doi:10.1002/pits.20577
41. Petras, H., Chilcoat, H. D., Leaf, P. J., Ialongo, N. S., & Kellam, S. G. (2004). Utility of TOCA-R scores during the elementary school years in identifying later violence among adolescent males. Journal of the American Academy of Child & Adolescent Psychiatry, 43, 88-96. doi: 10.1097/00004583-200401000-00018
42. Ramaswamy, V., Desarbo, W. S., Reibstein, D. J., & Robinson, W. T. (1993). An empirical pooling approach for estimating marketing mix elasticities with PIMS data. Marketing Science, 12, 103-124. doi: 10.1287/mksc.12.1.103
43. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Thousand Oaks, CA: Sage.
44. Rubin, D. B. (1987). Multiple imputation for nonresponse in surveys. New York, NY: Wiley. doi:10.1002/9780470316696
45. Schaeffer, C. M., Petras, H., Ialongo, N., Poduska, J., & Kellam, S. (2003). Modeling growth in boys' aggressive behavior across elementary school: Links to later criminal involvement, conduct disorder, and antisocial personality disorder. Developmental Psychology, 39, 1020-1035. doi:10.1037/0012-1649.39.6.1020
46. Schafer, J. L., & Graham, J. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7, 147-177. doi:10.1037/1082-989X.7.2.147
47. Schlomer, G. L., Bauman, S., & Card, N. A. (2010). Best practices for missing data management in counseling psychology. Journal of Counseling Psychology, 57, 1-10. doi:10.1037/a0018082
48. Schochet, P. Z., Puma, M., & Deke, J. (2014). Understanding variation in treatment effects in education impact evaluations: An overview of quantitative methods (NCEE Report 2014-4017). Washington, DC: U.S. Department of Education, Institute of Education Sciences.
49. Sclove, S. L. (1987). Application of model-selection criteria to some problems in multivariate analysis. Psychometrika, 52, 333-343. doi: 10.1007/BF02294360
50. Spilt, J. L., Koot, J. M., & van Lier, P. A. C. (2013). For whom does it work? Subgroup differences in the effects of a school-based universal prevention program. Prevention Science, 14, 479-488. doi:10.1007/s11121-012-0329-7
51. Stormshak, E. A., Bierman, K. L., Bruschi, C., Dodge, K. A., & Coie, J. D. (1999). The relation between behavior problems and peer preference in different classroom contexts. Child Development, 70, 169-182. doi: 10.1111/1467-8624.00013
52. Stuart, E. A., Cole, S. R., Bradshaw, C. P., & Leaf, P. J. (2011). The use of propensity scores to assess the generalizability of results from randomized trials. Journal of the Royal Statistical Society, Series A: Statistics in Society, 174, 369-386. doi:10.1111/j.1467-985X.2010.00673.x
53. Sugai, G., & Horner, R. H. (2002). The evolution of discipline practices: School-wide positive behavior supports. Child & Family Behavior Therapy, 24, 23-50. doi:10.1300/J019v24n01_03
54. Sugai, G., & Horner, R. R. (2006). A promising approach for expanding and sustaining School-Wide Positive Behavior Support. School Psychology Review, 35, 245-259.
55. Sugai, G., Horner, R. H., & Gresham, F. M. (2002). Behaviorally effective school environments. In M. R. Shinn, H. M. Walker, & G. Stoner (Eds.), Interventions for academic and behavior problems II: Preventive and remedial approaches (pp. 315-350). Washington, DC: National Association of School Psychologists.
56. Supplee, L. H., Kelly, B. C., MacKinnon, D. M., & Barofsky, M. Y. (2013). Introduction to the special issue: Subgroup analysis in prevention and intervention research. Prevention Science, 14, 107-110. doi: 10.1007/s11121-012-0335-9
57. Taylor-Greene, S. J., Brown, D., Nelson, L., Longton, J., Gassman, T., Cohen, J., . . . Hall, S. (1997). School-Wide Behavioral Support: Starting the year off right. Journal of Behavioral Education, 7, 99-112. doi: 10.1023/A:1022849722465
58. Taylor-Greene, S. J., & Kartub, D. T. (2000). Durable implementation of School-Wide Behavior Support: The High Five Program. Journal of Positive Behavior Interventions, 2, 233-235. doi:10.1177/109830070000200408
59. van Lier, P. A. C., Vuijk, P., & Crijnen, A. A. M. (2005). Understanding mechanisms of change in the development of antisocial behavior: The impact of a universal intervention. Journal of Abnormal Child Psychology, 33, 521-535. doi:10.1007/s10802-005-6735-7
60. Waasdorp, T. E., Bradshaw, C. P., & Leaf, P. J. (2012). The impact of Schoolwide Positive Behavioral Interventions and Supports on bullying and peer rejection: A randomized controlled effectiveness trial. Archives of Pediatrics & Adolescent Medicine, 166, 149-156. doi:10.1001/archpediatrics.2011.755
61. Walker, H. M., Horner, R. H., Sugai, G., Bullis, M., Sprague, J. R., Bricker, D., & Kaufman, M. J. (1996). Integrated approaches to preventing antisocial behavior patterns among school-age children and youth. Journal of Emotional and Behavioral Disorders, 4, 194-209. doi:10.1177/106342669600400401
62. Wang, R., & Ware, J. H. (2013). Detecting moderator effects using subgroup analyses. Prevention Science, 14, 111-120. doi:10.1007/s11121-011-0221-x
63. Werthamer-Larsson, L., Kellam, S., & Wheeler, L. (1991). Effect of first grade classroom environment on shy behavior, aggressive behavior, and concentration problems. American Journal of Community Psychology, 19, 585-602. doi:10.1007/BF00937993
64. Wilson, D. B. (2014). Practical meta-analysis effect size calculator. Retrieved March 25, 2014, from http://www.campbellcollaboration.org/escalc/html/EffectSizeCalculator-SMD10.php
65. Wothke, W. (2000). Longitudinal and multigroup modeling with missing data. In T. D. Little, K. U. Schnabel, & J. Baumert (Eds.), Modeling longitudinal and multilevel data: Practical issues, applied approaches, and specific examples (pp. 219-240, 269-281). Mahwah, NJ: Erlbaum.
66. Zima, B. T., Hurlburt, M., Knapp, P., Ladd, H., Tang, L., Duan, N., . . . Wells, K. (2005). Quality of publicly-funded outpatient specialty mental health care for common childhood psychiatric disorders in California. Journal of the American Academy of Child & Adolescent Psychiatry, 44, 130-144. doi:10.1097/00004583-200502000-00005