Methodology Considerations in School Mental Health Research

School Mental Health

Volumn 6, Issue 2, 2014, Pages 68-83

  Fabiano, Gregory A ^a   Chafouleas, Sandra M ^b   Weist, Mark D ^c   Carl Sumi, W ^d   Humphrey, Neil ^e  

^a UNIVERSITY AT BUFFALO   (United States)

^b UNIVERSITY OF CONNECTICUT   (United States)

^c UNIVERSITY OF SOUTH CAROLINA   (United States)

^d SRI INTERNATIONAL   (United States)

^e UNIVERSITY OF MANCHESTER   (United Kingdom)

Author keywords

Outcome measurement; Research design; Research methods; School mental health

Indexed keywords

EID: 85016260988     PISSN: 18662625     EISSN: 18662633     Source Type: Journal    
DOI: 10.1007/s12310-013-9117-1     Document Type: Article

References (104)

1. Achenbach, T. M., McConaughy, S. H., & Howell, C. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101, 213-232.
2. Agency for Healthcare Research. (2013). Methods guide for effectiveness and comparative effectiveness reviews. Rockville, MD: Agency for Healthcare Research.
3. Andis, P., Cashman, J., Praschil, R., Oglesby, D., Adelman, H., Taylor, L., et al. (2002). A strategic and shared agenda to advance mental health in schools through family and system partnerships. International Journal of Mental Health Promotion, 4, 28-35.
4. Atkins, M. S., Hoagwood, K. E., Kutash, K., & Seidman, E. (2010). Toward the integration of education and mental health services. Administration and Policy in Mental Health and Mental Health Services Research, 37, 40-47.
5. Berkowitz, H. (1968). A preliminary assessment of the extent of interaction between child psychiatric clinics and public schools. Psychology in the Schools, 5, 291-295.
6. Brandt, N. E., Glimpse, C., Fette, C., Lever, N. A., Cammack, N. L., & Cox, J. (2014). Advancing effective family-school-community partnerships. In M. Weist, N. Lever, C. Bradshaw, & J. Owens (Eds.), Handbook of school mental health: Research, training, practice, and policy (pp. 209-222). New York: Springer.
7. Briesch, A. M., Chafouleas, S. M., Neugebauer, S. R., & Riley-Tillman, T. C. (2013). Assessing influences on intervention use: Revision of the usage rating profile-intervention. Journal of School Psychology, 51, 81-96.
8. Campbell, D., & Stanley, J. (1963). Experimental and quasi-experimental designs for research. Chicago, IL: Rand-McNally.
9. Campbell, M. K., Thomson, S., Ramsay, C. R., MacLennan, G. S., & Grimshaw, J. M. (2004). Sample size calculator for cluster randomized trials. Computers in Biology and Medicine, 34, 113-125.
10. Chafouleas, S. M., Briesch, A. M., Riley-Tillman, T. C., Christ, T. C., Black, A. C., & Kilgus, S. P. (2010a). An investigation of the generalizability and dependability of Direct Behavior Rating Single Item Scales (DBR-SIS) to measure academic engagement and disruptive behavior of middle school students. Journal of School Psychology, 48, 219-246. doi: 10. 1016/j. jsp. 2010. 02. 001.
11. Chafouleas, S. M., Briesch, A. M., Riley-Tillman, T. C., Christ, T. J., Volpe, R. J., & Gresham, F. (2009, August). Review of methods for formative assessment of child social behavior. In Symposium presented at the meeting of the American Psychological Association, Toronto, Canada.
12. Chafouleas, S. M., Kilgus, S. P., Riley-Tillman, T. C., Jaffery, R., & Harrison, S. (2012). Preliminary evaluation of various training components on accuracy of direct behavior ratings. Journal of School Psychology. doi: 10. 1016/j. jsp. 2011. 11. 007.
13. Chafouleas, S. M., Volpe, R. J., Gresham, F. M., & Cook, C. (2010b). School-based behavioral assessment within problem-solving models: Current status and future directions. School Psychology Review, 34, 343-349.
14. Clauss-Ehlers, C., Serpell, Z., & Weist, M. D. (2013). Handbook of culturally responsive school mental health: Advancing research, training, practice, and policy. New York: Springer.
15. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.
16. Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston, MA: Houghton Mifflin.
17. Cordray, D. S., & Pion, G. M. (2006). Treatment strength and integrity: Models and methods. In R. R. Bootzin & P. E. McKnight (Eds.), Strengthening research methodology: Psychological measurement and evaluation (pp. 103-124). Washington, D. C.: American Psychological Association.
18. Craig, W. M., & Pepler, D. J. (1998). Observations of bullying and victimization in the school yard. Canadian Journal of School Psychology, 13, 41-59.
19. Cunningham, C. E., Vaillancourt, T., Rimas, H., Deal, K., Cunningham, L., Short, K., et al. (2009). Modeling the bullying prevention program preferences of educators: A discrete choice conjoint experiment. Journal of Abnormal Child Psychology, 37, 929-943.
20. Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and secondary prevention: Are implementation effects out of control? Clinical Psychology Review, 18, 23-45.
21. Donner, A., & Klar, N. (2004). Pitfalls of and controversies in cluster randomization trials. American Journal of Public Health, 94, 416-422.
22. Dumas, J. E., Lynch, A. M., Laughlin, J. E., Phillips Smith, E., & Prinz, R. J. (2001). Promoting intervention fidelity: Conceptual issues, methods, and preliminary results from the Early Alliance prevention trial. American Journal of Preventive Medicine, 20, 38-47.
23. Duncan, G. J., & Magnuson, K. (2007). Penny wise and effect size foolish. Child Development Perspectives, 1, 46-51.
24. DuPaul, G. J., Eckert, T. L., & Vilardo, B. (2012). The effects of school-based interventions for attention deficit hyperactivity disorder: A meta-analysis 1996-2010. School Psychology Review, 41, 387-412.
25. Durlak, J. A., & DuPre, E. P. (2008). Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology, 41, 327-350.
26. Evans, S. W., Allen, J., Moore, S., & Strauss, V. (2005). Measuring symptoms and functioning of youth with ADHD in middle schools. Journal of Abnormal Child Psychology, 33, 695-706.
27. Evans, S. W., Brady, C. E., Harrison, J. R., Bunford, N., Kern, L., State, T., et al. (2013). Measuring ADHD and ODD symptoms and impairment using high school teachers' ratings. Journal of Clinical Child and Adolescent Psychology, 42, 197-207.
28. Evans, C., Margison, F., & Barkham, M. (1998). The contribution of reliable and clinically significant change methods to evidence-based mental health. Evidence-Based Mental Health, 1, 70-73.
29. Fabiano, G. A., Pelham, W. E., Gnagy, E. M., Burrows-MacLean, L., Coles, E. K., Chacko, A., et al. (2007). The single and combined effects of multiple intensities of behavior modification and multiple intensities of methylphenidate in a classroom setting. School Psychology Review, 36, 195-216.
30. Fabiano, G. A., Pelham, W. E., Waschbusch, D., Gnagy, E. M., Lahey, B. B., Chronis, A. M., et al. (2006). A practical impairment measure: Psychometric properties of the impairment rating scale in samples of children with attention-deficit/hyperactivity disorder and two school-based samples. Journal of Clinical Child and Adolescent Psychology, 35, 369-385.
31. Fabiano, G. A., Vujnovic, R., Pelham, W. E., Waschbusch, D. A., Massetti, G. M., Yu, J., et al. (2010). Enhancing the effectiveness of special education programming for children with ADHD using a daily report card. School Psychology Review, 39, 219-239.
32. Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. (FMHI Publication #231). Tampa, FL: University of South Florida, National Implementation Research Network.
33. Fonagy, P., Twemlow, S. W., Vernberg, E. M., Nelson, J. M., Dill, E. J., Little, T. D., et al. (2009). A cluster randomized controlled trial of child-focused psychiatric consultation and a school systems-focused intervention to reduce aggression. Journal of Child Psychology and Psychiatry, 50, 607-616.
34. Frazier, S. L., Formoso, D., Birman, D., & Atkins, M. S. (2008). Closing the research to practice gap: Redefining feasibility. Clinical Psychology: Science and Practice, 15, 125-129.
35. Gottfredson, G. D., & Gottfredson, D. C. (2001). What schools do to prevent problem behavior and promote safe environments. Journal of Educational and Psychological Consultation, 12, 313-344.
36. Gresham, F. M. (1989). Assessment of treatment integrity in school consultation and prereferral intervention. School Psychology Review, 18, 37-50.
37. Hagermoser Sanetti, L. M., & Kratochwill, T. R. (2009). Toward developing a science of treatment integrity: Introduction to the special series. School Psychology Review, 38, 445-459.
38. Hill, C., Bloom, H., Black, A. R., & Lipsey, M. W. (2008). Empirical benchmarks for interpreting effect sizes in research. Child Development Perspectives, 2, 172-177.
39. Humphrey, N. (2013). Social and emotional learning: A critical appraisal. London: Sage.
40. Humphrey, N., Hanley, T., Lendrum, A., & Wigelsworth, M. (2012). Assessing therapeutic outcomes. In T. Hanley, N. Humphrey, & C. Lennie (Eds.), Adolescent counselling psychology: Theory, research and practice (pp. 157-170). London: Routledge.
41. Humphrey, N., Kalambouka, A., Wigelsworth, M., Lendrum, A., Lennie, C., & Farrell, P. (2010). New beginnings: Evaluation of a short social-emotional intervention for primary-aged children. Educational Psychology, 30, 513-532.
42. Humphrey, N., Lendrum, A., Barlow, A., Wigelsworth, M., & Squires, G. (2013). Achievement for All: Improving psychosocial outcomes for students with special educational needs and disabilities. Research in Developmental Disabilities, 34, 1210-1225.
43. Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-19.
44. Kataoka, S., Jaycox, L. H., Wong, M., Nadeem, E., Langley, A., Tang, L., et al. (2011). Effects on school outcomes in low-income minority youth: Preliminary findings from a community-partnered study of a school-based trauma intervention. Ethnicity and Disease, 21, S1-71-S1-77.
45. Kataoka, S., Stein, B. D., Jaycox, L. H., Wong, M., Escudero, P., Tu, W., et al. (2003). A school-based mental health program for traumatized Latino immigrant children. Journal of the American Academy of Child and Adolescent Psychiatry, 42(3), 311-318.
46. Kazdin, A. E. (1982). Single-case research designs: Methods for clinical and applied settings. New York, NY: Oxford University Press.
47. Kazdin, A. E. (2005). Evidence-based assessment for children and adolescents: Issues in measurement development and clinical application. Journal of Clinical Child and Adolescent Psychology, 34, 548-558.
48. Kendall, P. C., & Grove, W. M. (1988). Normative comparisons in therapy outcome. Psychological Assessment, 10, 147-158.
49. Kerry, S., & Bland, J. (1998). Statistics notes: Sample size in cluster randomisation. British Medical Journal, 316, 549.
50. Killip, S., Mahfoud, Z., & Pearce, K. (2004). What is an intracluster correlation coefficient? Crucial concepts for researchers. The Annals of Family Medicine, 3, 204-208.
51. Kolko, D. J., Bukstein, O. G., & Barron, J. (1999). Methylphenidate and behavior modification in children with ADHD and comorbid ODD and CD: Main and incremental effects across settings. Journal of the American Academy of Child and Adolescent Psychiatry, 38, 578-586.
52. Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., et al. (2010). Single-case designs technical documentation. Retrieved from: http://ies. ed. gov/ncee/wwc/pdf/wwc_scd. pdf.
53. Lane, K. L., Beebe-Frankenberger, M. E., Lambros, K. M., & Pierson, M. (2001). Designing effective interventions for children at risk for antisocial behavior: An integrated model of components necessary for making valid inferences. Psychology in the Schools, 38, 365-379.
54. Langberg, J. M., Vaughn, A. J., Williamson, P., Epstein, J. N., Girio-Herrera, E., & Becker, S. P. (2011). Refinement of an organizational skills intervention for adolescents with ADHD for implementation by school mental health providers. School Mental Health, 3, 143-155.
55. Lendrum, A., & Humphrey, N. (2012). The importance of studying the implementation of school-based interventions. Oxford Review of Education, 38, 635-652.
56. Lever, N. A., Adelsheim, S., Prodente, C., Christodulu, K. V., Ambrose, M. G., Schlitt, J., et al. (2003). System, agency and stakeholder collaboration to advance mental health programs in schools. In M. D. Weist, S. W. Evans, & N. A. Lever (Eds.), Handbook of school mental health: Advancing practice and research (pp. 149-162). New York, NY: Springer.
57. Lipsey, M. W., Puzio, K., Yun, C., Hebert, M. A., Steinka-Fry, K., Cole, M. W., et al. (2012). Translating the statistical representation of effects of education interventions into more readily interpretable forms (NCSER 2013-3000). Washington, D. C.: National Center for Special Education Research, Institute of Education Sciences, U. S. Department of Education.
58. Lonigan, C. J., Elbert, J. C., & Johnson, S. B. (1998). Empirically supported psychosocial interventions for children: An overview. Journal of Clinical Child Psychology, 27, 138-145.
59. Losen, D. J., & Gillespie, J. (2012, August). Opportunities suspended: The disparate impact of disciplinary exclusion from school. University of California, Los Angeles: The Civil Rights Project. Retrieved from http://civilrightsproject. ucla. edu/resources/projects/center-for-civil-rights-remedies/school-to-prison-folder/federal-reports/upcoming-ccrr-research.
60. Martens, B. K. (1993). Social labeling, precision of measurement, and problem-solving: Key issues in the assessment of children's emotional problems. School Psychology Review, 22, 308-312.
61. Mash, E. J., & Hunsley, J. (2005). Evidence-based assessment of child and adolescent disorders: Issues and challenges. Journal of Clinical Child and Adolescent Psychology, 34, 362-379.
62. Miller, F. G., Johnson, A. H., Welsh, M., Chafouleas, S. M., Riley-Tillman, T. C., & Fabiano, G. (2013, July). Evaluation of universal screening methods to identify behavioral risk. In Poster presented at the meeting of the American Psychological Association, Honolulu, Hawaii.
63. Miller, F. G., Neugebauer, S. R., Chafouleas, S. M., Briesch, A. M., & Riley-Tillman, T. C. (2013, July). Examining innovation usage: Construct validation of the Usage Rating Profile- Assessment. In Poster presented at the meeting of the American Psychological Association, Honolulu, Hawaii.
64. Moncher, F. J., & Prinz, R. J. (1991). Treatment fidelity in outcome studies. Clinical Psychology Review, 11, 247-266.
65. Muijs, D. (2010). Doing quantitative research in education with SPSS (2nd ed.). London: Sage.
66. Murphy, S. A. (2005). An experimental design for the development of adaptive treatment strategies. Statistics in Medicine, 24, 1455-1481.
67. Muthén, B., Brown, C. H., Masyn, K., Jo, B., Khoo, S.-T., Yang, C.-C., et al. (2002). General growth mixture modeling for randomized preventive interventions. Biostatistics, 3, 459-475.
68. Nahum-Shani, I., Qian, M., Almirall, D., Pelham, W. E., Gnagy, E. M., Fabiano, G. A., et al. (2012a). Q-learning: A data analysis method for constructing adaptive interventions. Psychological Methods, 17, 478-494.
69. Nahum-Shani, I., Qian, M., Almirall, D., Pelham, W. E., Gnagy, E. M., Fabiano, G. A., et al. (2012b). Experimental design and primary data analysis methods for comparing adaptive interventions. Psychological Methods, 17, 457-477.
70. Nelson, J. R., Benner, G. J., Reid, R. C., Epstein, M. H., & Currin, D. (2002). The convergent validity of office discipline referrals with the CBCL-TRF. Journal of Emotional and Behavioral Disorders, 10, 181-188.
71. O'Connell, M., Boat, T., & Warner, K. E. (2009). Preventing mental, emotional, and behavioral disorders among young people: Progress and possibilities. Washington, DC: The National Academies Press.
72. O'Leary, K. D., & O'Leary, S. G. (1982). Classroom management: The successful use of behavior modification. New York, NY: Pergamon Press Inc.
73. Paul, G. L. (1967). Strategy of outcome research in psychotherapy. Journal of Consulting Psychology, 31, 109-118.
74. Pelham, W. E., Carlson, C., Sams, S. E., Vallano, G., Dixon, J., & Hoza, B. (1993). Separate and combined effects of methylphenidate and behavior modification on boys with ADHD in the classroom. Journal of Consulting and Clinical Psychology, 61, 506-515.
75. Pelham, W. E., Fabiano, G. A., & Massetti, G. M. (2005). Evidence-based assessment of attention-deficit/hyperactivity disorder in children and adolescents. Journal of Clinical Child and Adolescent Psychology, 34, 449-476.
76. Pfiffner, L. J., Kaiser, N. M., Burner, C., Zalecki, C., Rooney, M., Setty, P., et al. (2011). From clinic to school: Translating a collaborative school-home behavioral intervention for ADHD. School Mental Health, 3, 127-142.
77. Pfiffner, L. J., & O'Leary, S. G. (1987). The efficacy of all-positive management as a function of the prior use of negative consequences. Journal of Applied Behavior Analysis, 20, 265-271.
78. Pfiffner, L. J., Rosen, L. A., & O'Leary, S. G. (1985). The efficacy of an all-positive approach to classroom management. Journal of Applied Behavior Analysis, 18, 257-261.
79. Power, T. J., Blom-Hoffman, J., Clarke, A. T., Riley-Tillman, T. C., Kelleher, C., & Manz, P. H. (2005). Reconceptualizing intervention integrity: A partnership-based framework for linking research with practice. Psychology in the Schools, 42, 495-507.
80. Power, T. J., Mautone, J. A., Soffer, S. L., Clarke, A. T., Marshall, S. A., Sharman, J., et al. (2012). A family-school intervention for children with ADHD: Results of a randomized clinical trial. Journal of Consulting and Clinical Psychology, 80, 611-623.
81. Puffer, S., Torgerson, D. J., & Watson, J. (2005). Cluster randomized controlled trials. Journal of Evaluation in Clinical Practice, 11, 479-483.
82. Raudenbusch, S. W., Spybrook, J., Congdon, R., Liu, X., Martinez, A., Bloom, H., et al. (2011). Optimal design plus empirical evidence (version 3. 0). Retrieved from: http://www. wtgrantfoundation. org/resources/consultation-service-and-optimal-design.
83. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models. Thousands Oaks, CA: Sage.
84. Riley-Tillman, T. R., & Burns, M. K. (2009). Evaluating educational interventions: Single-case designs for measuring response to intervention. New York, NY: The Guilford Press.
85. Robb, J. A., Sibley, M. H., Pelham, W. E., Foster, E. M., Molina, B. S. G., Gnagy, E. M., et al. (2011). The estimated annual cost of ADHD to the U. S. education system. School Mental Health, 3, 169-177.
86. Schnoes, C., Reid, R., Wagner, M., & Marder, C. (2006). ADHD among students receiving special education services: A national survey. Exceptional Children, 72, 483-496.
87. Schulte, S. C., Easton, J. E., & Parker, J. (2009). Advances in treatment integrity research: Multidisciplinary perspectives on the conceptualization, measurement, and enhancement of treatment integrity. School Psychology Review, 38, 460-475.
88. Sellström, E., & Bremberg, S. (2006). Is there a "school effect" on pupil outcomes? A review of multilevel studies. Journal of Epidemiology and Community Health, 60, 149-155.
89. Shernoff, E. S., Marinez-Lora, A. M., Frazier, S. L., Jakobsons, L. J., Atkins, M. S., & Bonner, D. (2011). Teachers supporting teachers in urban schools: What iterative research designs can teach us. School Psychology Review, 40, 465-485.
90. Skiba, R. J., Horner, R. H., Chung, C. G., Rausch, M. K., May, S. L., & Tobin, T. (2011). Race is not neutral: A national investigation of African American and Latino disproportionality in school discipline. School Psychology Review, 40, 85-107.
91. Social and Character Development Research Consortium. (2010). Efficacy of schoolwide programs to promote social and character development and reduce problem behavior in elementary school children (NCER 2011-2001). Washington, D. C.: National Center for Education Research, Institute of Education Sciences, U. S. Department of Education.
92. Spoth, R., & Redmond, C. (1993). Identifying program preferences through conjoint analysis: Illustrative results from a parent sample. American Journal of Health Promotion, 8, 124-133.
93. Stein, B. D., Jaycox, L. H., Kataoka, S. H., Wong, M., Tu, W., Elliott, M. N., et al. (2003). A mental health intervention for schoolchildren exposed to violence: A randomized controlled trial. Journal of the American Medical Association, 290(5), 603-611.
94. Sugai, G., Horner, R., & Walker, H. (2000). Preventing school violence: The use of office referral to assess and monitor school-wide discipline interventions. Journal of Emotional and Behavioral Disorders, 8, 94-101.
95. U. S. Department of Education, Institute of Education Sciences, & What Works Clearinghouse. (2012, June). WWC review of report: Enhancing the effectiveness of special education programming for children with attention-deficit/hyperactivity disorder using a daily report card. Retrieved from http://whatworks. ed. gov.
96. Volpe, R., & Fabiano, G. A. (2013). Daily behavior report cards: An evidence-based system of assessment and intervention. New York: The Guilford Press.
97. Wagner, M., Kutash, K., Duchnowski, A. J., Epstein, M. H., & Sumi, W. C. (2005). The children and youth we serve: A national picture of the characteristics of students with emotional disturbances receiving special education. Journal of Emotional and Behavioral Disorders, 13, 79-96.
98. Waltz, J., Addis, M. E., Koerner, K., & Jacobson, N. S. (1993). Testing the integrity of a psychotherapy protocol: Assessment of adherence and competence. Journal of Consulting and Clinical Psychology, 61, 620-630.
99. Weist, M. D., Lever, N., Bradshaw, C., & Owens, J. S. (2014). Further advancing the field of school mental health. In M. Weist, N. Lever, C. Bradshaw, & J. S. Owes (Eds.), Handbook of school mental health: Research, training, practice, and policy (pp. 1-16). New York: Springer.
100. Weist, M. D., Lever, N., Stephan, S., Youngstrom, E., Moore, E., Harrison, B., et al. (2009). Formative evaluation of a framework for high quality, evidence-based services in school mental health. School Mental Health., 1(3), 196-211.
101. Weisz, J. R., Thurber, C. A., Sweeney, L., Proffitt, V. D., & LeGagnoux, G. L. (1997). Brief treatment of mild-to-moderate child depression using primary and secondary control enhancement training. Journal of Consulting and Clinical Psychology, 65, 703-707.
102. What Works Clearinghouse. (2010). What Works Clearinghouse: Procedures and standards handbook (version 2. 1). Retrieved from: http://ies. ed. gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v2_1_standards_handbook. pdf.
103. What Works Clearinghouse. (2013). What Works Clearinghouse: Procedures and standards handbook (version 3. 0). Retrieved from: http://ies. ed. gov/ncee/wwc/pdf/reference_resources/wwc_procedures_v3_0_draft_standards_handbook. pdf.
104. Wigelsworth, M., Humphrey, N., Kalambouka, A., & Lendrum, A. (2010). A review of key issues in the measurement of children's social and emotional skills. Educational Psychology in Practice, 26, 173-186.