Improving the Power of an Efficacy Study of a Social and Emotional Learning Program: Application of Generalizability Theory to the Measurement of Classroom-Level Outcomes

Prevention Science

Volumn 15, Issue 2, 2014, Pages 146-155

  Mashburn, Andrew J ^a   Downer, Jason T ^b   Rivers, Susan E ^c   Brackett, Marc A ^c   Martinez, Andres ^d  

^a PORTLAND STATE UNIVERSITY   (United States)

^b UNIVERSITY OF VIRGINIA   (United States)

^c YALE UNIVERSITY   (United States)

^d UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Generalizability theory; Interventions; Setting level outcomes; Social and emotional learning

Indexed keywords

ADOLESCENT; CHILD; EMOTIONAL INTELLIGENCE; FEMALE; HUMAN; HUMAN RELATION; MALE; PEER GROUP; PROGRAM EVALUATION; PSYCHOLOGY; SCHOOL; SOCIAL BEHAVIOR; SOCIALIZATION; STATISTICS AND NUMERICAL DATA; STUDENT; THEORETICAL MODEL;

ADOLESCENT; CHILD; EMOTIONAL INTELLIGENCE; FEMALE; HUMANS; INTERPERSONAL RELATIONS; MALE; MODELS, THEORETICAL; PEER GROUP; PROGRAM EVALUATION; SCHOOLS; SOCIAL BEHAVIOR; SOCIALIZATION; STUDENTS;

EID: 84897024076     PISSN: 13894986     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11121-012-0357-3     Document Type: Article

References (25)

1. Brennan, R. L., & Kane, M. T. (1977). An index of dependability for mastery tests. Journal of Educational Measurement, 14, 277-289. doi: 10. 1111/j. 1745-3984. 1977. tb00045. x.
2. Brown, J. L., Jones, S. M., LaRusso, M., & Aber, J. L. (2010). Improving classroom quality: Teacher influences and experimental impacts of the 4Rs program. Journal of Educational Psychology, 102, 153-167. doi: 10. 1037/a0018160.
3. Bloom, H. S. (2005). Randomizing groups to evaluate place-based programs. In H. S. Bloom (Ed.), Learning more from social experiments (pp. 115-172). New York: Sage.
4. Cairns, R. B., & Green, J. A. (1979). How to assess personality and social patterns: Observations or ratings? In R. B. Cairns (Ed.), The analysis of social interactions: Methods, issues, and illustrations (pp. 209-226). Hillsdale: Lawrence Erlbaum.
5. Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: Wiley.
6. Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing students' social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82, 405-432. doi: 10. 1111/j. 1467-8624. 2010. 01564. x.
7. Erlich, O., & Borich, C. (1979). Occurrence and generalizability of scores on a classroom interaction instrument. Journal of Educational Measurement, 16, 11-18. doi: 10. 1111/j. 1745-3984. 1979. tb00081. x.
8. Greenberg, M. T., Weissberg, R. P., O'Brien, M. U., Zins, J. E., Fredericks, L., Resnik, H., & Elias, M. J. (2003). Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning. American Psychologist, 58, 466-474. doi: 10. 1037/0003-066X. 58. 6-7. 466.
9. Hintze, J. M. (2005). Psychometrics of direct observation. School Psychology Review, 34, 507-519.
10. Judd, C. M., & Kenny, D. A. (1981). Process analysis: Estimating mediation in treatment evaluations. Evaluation Review, 5, 602-619. doi: 10. 1177/0193841X8100500502.
11. Liu, X., Spybrook, J., Congdon, R., Martinez, A., & Raudenbush, S. W. (2009). Optimal design for longitudinal and multilevel research (version 2) [computer software]. Available at http://www. wtgrantfoundation. org.
12. MacKinnon, D. P. (1994). Analysis of mediating variables in prevention intervention studies. In A. Cazares & L. A. Beatty (Eds.), Scientific methods for prevention intervention research. NIDA research monograph 139, DHHS pub. 94-3631 (pp. 127-153). Washington, DC: US Department of Health and Human Services.
13. Marcoulides, G. (1989). The application of generalizability analysis to observational studies. Quality and Quantity, 23, 115-127. doi: 10. 1007/BF00151898.
14. Mashburn, A. J., Pianta, R. C., Hamre, B. K., Downer, J. T., Barbarin, O., et al. (2008). Measures of pre-k quality and children's development of academic, language and social skills. Child Development, 79, 732-749. doi: 10. 1111/j. 1467-8624. 2008. 01154. x.
15. Meyer, J. P., Henry, A. E., & Mashburn, A. J. (2011). Occasions and the reliability of classroom observations: Alternative conceptualizations and methods of analysis. Educational Assessment Journal, 16, 227-243. doi: 10. 1080/10627197. 2011. 638884.
16. Murray, D. M. (1998). Design and analysis of group-randomized trials. New York: Oxford University Press.
17. Pianta, R. C., Belsky, J., Vandergrift, N., Houts, R., & Morrison, F. J. (2008). Classroom effects on children's achievement trajectories in elementary school. American Educational Research Journal, 45, 365-397. doi: 10. 3102/0002831207308230.
18. Pianta, R. C., La Paro, K., & Hamre, B. K. (2008). Classroom assessment scoring system (CLASS). Baltimore: Paul H. Brookes.
19. Raudenbush, S. W., & Sadoff, S. (2008). Statistical inference when classroom quality is measured with error. Journal of Research on Educational Effectiveness, 1, 138-154. doi: 10. 1080/19345740801982104.
20. Rivers, S. E., & Brackett, M. A. (2011). Achieving standards in the English language arts (and more) using The RULER Approach to social and emotional learning. Reading & Writing Quarterly, 27, 75-100. doi: 10. 1080/10573569. 2011. 532715.
21. Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. Newbury Park: Sage.
22. Snyder, J., Reid, J., Stoolmiller, M., Howe, G., Brown, H., Dagne, G., & Cross, W. (2006). The role of behavior observation in measurement systems for randomized prevention trials. Prevention Science, 7, 43-56. doi: 10. 1007/s11121-005-0020-3.
23. Suen, H. K. (1990). Principles of test theories. Hillsdale: Lawrence Erlbaum.
24. Suen, H. K., & Ary, D. (1989). Analyzing quantitative behavioral observation data. Hillsdale: Lawrence Erlbaum.
25. Zins, J. E., Bloodworth, M. R., Weissberg, R. P., & Walberg, H. J. (2004). The scientific base linking social and emotional learning to school success. In J. E. Zins, R. P. Weissberg, M. C. Wang, & H. J. Walberg (Eds.), Building academic success on social and emotional learning: What does the research say? (pp. 3-22). New York: Teachers College Press.