Measuring Afterschool Program Quality Using Setting-Level Observational Approaches

Journal of Early Adolescence

Volumn 35, Issue 5-6, 2015, Pages 681-713

  Oh, Yoonkyung ^a   Osgood, D Wayne ^a   Smith, Emilie P ^a  

^a PENNSYLVANIA STATE UNIVERSITY   (United States)

Author keywords

afterschool program assessment; Caregiver Interaction Scale; generalizability theory; Promising Practices Rating Scale; reliability; validity

Indexed keywords

EID: 84937208934     PISSN: 02724316     EISSN: 15525449     Source Type: Journal    
DOI: 10.1177/0272431614561261     Document Type: Article

References (61)

1. J.L.Arbuckle (2011). IMB SPSS Amos 20 user’s guide. Armonk, NY: IBM.
2. J.Arnett (1989). Caregivers in day-care centers: Does training matter?Journal of Applied Developmental Psychology, 10, 541-552.
3. P.M.Bentler (1990). Comparative fit indices in structural models. Psychological Bulletin, 107, 238-246.
4. J.BlazevskiC.Smith (2007). After-school quality and school-day outcomes in Michigan’s 21st CCLC program. Ypsilanti, MI: High/Scope Educational Research Foundation.
5. U.Bronfenbrenner (1979). The ecology of human development: Experiments by nature and design. Cambridge, MA: Harvard University Press.
6. T.A.Brown (2006). Confirmatory factor analysis for applied research. New York, NY: Guilford Press.
7. M.W.BrowneR.Cudeck (1992). Alternative ways of assessing model fit. Sociological Methods & Research, 21, 230-258.
8. P.H.CampbellS.A.Milbourne (2005). Improving the quality of infant-toddler care through professional development. Topics in Early Childhood Special Education, 25, 3-14.
9. G.W.CheungR.B.Rensvold (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255.
10. H.A.Davis (2006). Exploring the contexts of relationship quality between middle school students and teachers. The Elementary School Journal, 106, 193-223.
11. R.E.L.de KruifR.A.McWilliamS.M.RidleyM.B.Wakely (2000). Classification of teachers’ interaction behaviors in early childhood classrooms. Early Childhood Research Quarterly, 15, 247-268. doi:10.1016/S0885-2006(00)00051-X
12. D.DuBoisB.HollowayJ.ValentineH.Cooper (2002). Effectiveness of mentoring programs for youth: A meta-analytic review. American Journal of Community Psychology, 30, 157-197. doi:10.1023/a:1014628810714
13. J.A.DurlakR.P.WeissbergM.Pachan (2010). A meta-analysis of after-school programs that seek to promote personal and social skills in children and adolescents. American Journal of Community Psychology, 45, 294-309. doi:10.1007/s10464-010-9300-6
14. J.S.EcclesJ.A.Gootman (2002). Community programs to promote youth development. Washington, DC: National Academies Press.
15. J.S.EcclesR.W.Roeser (2009). Schools, academic motivation, and stage-environment fit. In R.M.LernerL.Steinberg (Eds.), Handbook of adolescent psychology (3rd ed., pp. 404-434). Hoboken, NJ: John Wiley.
16. J.L.Fleiss (1981). The measurement of interrater agreement. In J.L.Fleiss (Ed.), Statistical methods for rates and proportions (pp. 212-236). New York, NY: John Wiley.
17. N.FrancoM.J.Levitt (1998). The social ecology of middle childhood: Family support, friendship quality, and self-esteem. Family Relations, 47, 315-321.
18. B.J.Fraser (1998). Classroom environment instruments: Development, validity and applications. Learning Environments Research, 1, 7-33.
19. S.A.GerstenblithD.A.SouleD.C.GottfredsonS.LuM.A.KelltromS.C.WomerS.L.Bryner (2005). After-school programs, antisocial behavior, and positive youth development: An exploration of the relationship between program implementation and changes in youth behavior. In J.MahoneyR.LarsonJ.Eccles (Eds.), Organized activities as contexts of development: Extracurricular activities, after-school and community programs (pp. 457-477). Mahwah, NJ: Lawrence Erlbaum.
20. N.A.GonzalesA.M.CauceR.J.FriedmanC.A.Mason (1996). Family, peer, and neighborhood influence on academic achievement among African American adolescents: One year prospective effects. American Journal of Community Psychology, 24, 365-387.
21. S.L.GortmakerR.M.LeeR.S.MozaffarianA.M.SobolT.F.NelsonB.A.RothJ.L.Wiecha (2012). Effect of an after-school intervention on increases in children’s physical activity. Medicine & Science in Sports & Exercise, 44, 450-457.
22. D.GottfredsonS.GerstenblithD.SouléS.WomerS.Lu (2004). Do after school programs reduce delinquency?Prevention Science, 5, 253-266.
23. R.C.Granger (2005). Improving the settings that affect youth. William T. Grant Foundation Annual Report (pp. 6-11). New York, NY. Retrieved from http://wtgrantfoundation.org/WhoWeAre#annual-reports
24. R.C.GrangerJ.A.DurlakN.YohalemE.Reisner (2007). Improving after-school program quality. New York, NY: William T. Grant Foundation.
25. J.B.GrossmanJ.GoldmithJ.SheldonA.J.A.Arbreton (2009). Assessing after-school settings. New Directions for Youth Development, 121, 89-108. doi:10.1002/yd.298
26. T.HalleJ.E.Vick WhittakerR.Anderson (2010). Quality in early childhood care and education settings: A compendium of measures. Washington, DC: Child Trends.
27. B.K.HamreR.C.PiantaA.J.MashburnJ.T.Downer (2007). Building a science classrooms: Application of CLASS framework in over 4,000 U.S. early childhood and elementary classrooms. Retrieved from http://www.icpsr.umich.edu/files/PREK3RD
28. T.HarmsE.JacobsD.White (1996). School-Age Care Environment Rating Scale. New York, NY: Teachers College Press.
29. K.A.KirtlandD.E.PorterC.L.AddyM.J.NeetJ.E.WilliamsP.A.SharpeB.E..Ainsworth (2003). Environmental measures of physical activity supports: Perception versus reality. American Journal of Preventive Medicine, 24, 323-331.
30. G.P.KupermincE.P.SmithC.C.Henrich (2013). Introduction to the special issue on social and motivational processes in after-school settings: Bridging gaps between theory, research, and practice. The Journal of Early Adolescence, 33, 5-16.
31. R.Larson (2000). Toward a psychology of positive youth development. American Psychologist, 55, 170-183.
32. J.L.MahoneyM.E.ParenteH.Lord (2007). After-school program engagement: Links to child competence and program quality and content. The Elementary School Journal, 107, 385-404. doi:10.1086/516670
33. J.L.MahoneyH.Stattin (2000). Leisure activities and adolescent antisocial behavior: The role of structure and social context. Journal of Adolescence, 23, 113-127.
34. J.L.MahoneyH.StattinH.Lord (2004). Unstructured youth recreation centre participation and antisocial peers. International Journal of Behavioral Development, 28, 553-560.
35. J.L.MahoneyE.F.Zigler (2006). Translating science to policy under the No Child Left Behind Act of 2001: Lessons from the national evaluation of the 21st-Century Community Learning Centers. Journal of Applied Developmental Psychology, 27, 282-294.
36. G.J.McCall (1984). Systematic field observation. Annual Review of Sociology, 10, 263-282.
37. K.O.McGrawS.P.Wong (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1, 30-46.
38. M.W.McLaughlinM.A.IrbyJ.Langman (1994). Urban sanctuaries: Neighborhood organizations in the lives and futures of inner-city youth. San Francisco, CA: Jossey-Bass.
39. M.K.NockS.M.S.Kurtz (2005). Direct behavioral observation in school settings: Bringing science to practice. Cognitive and Behavioral Practice, 12, 359-370.
40. J.NunnallyI.Bernstein (1994). Psychometric theory. New York, NY: McGraw-Hill.
41. D.W.OsgoodA.L.Anderson (2004). Unstructured socializing and rates of delinquency. Criminology, 42, 519-550. doi:10.1111/j.1745-9125.2004.tb00528.x
42. D.F.PerkinsL.M.Borden (2003). Key elements of community youth development programs. In F.A.VillarruelD.F.PerkinsL.M.BordenJ.G.Keith (Eds.), Community youth development: Practice, policy, and research (pp. 327-340). Thousand Oaks, CA: Sage.
43. R.C.PiantaJ.P.Allen (2009). Building capacity for positive youth development in secondary school classrooms: Changing teachers’ interactions with students. In M.ShinnH.Yoshikawa (Eds.), Toward positive youth development: Transforming schools and community programs (pp. 21-39). New York, NY: Oxford University Press.
44. R.C.PiantaB.K.Hamre (2009). Conceptualization, measurement, and improvement of classroom processes: Standardized observation can leverage capacity. Educational Researcher, 38, 109-119.
45. K.M.PierceD.BoltD.B.Vandell (2010). Specific features of after-school program quality: Associations with children’s functioning in middle childhood. American Journal of Community Psychology, 45, 381-393.
46. K.M.PierceJ.V.HammD.L.Vandell (1999). Experiences in after-school programs and children’s adjustment in first-grade classrooms. Child Development, 70, 56-767.
47. S.W.RaudenbushA.MartinezH.BloomP.ZhuF.Lin (2011). Studying the reliability of group-level measures with implications for statistical power: A six-step paradigm. Retrieved from http://wtgrantfoundation.org/FocusAreas#youth-social-settings
48. S.W.RaudenbushR.J.Sampson (1999). Ecometrics: Toward a science of assessing ecological settings, with application to the systematic social observation of neighborhoods. Sociological Methodology, 29, 1-41.
49. R.J.ShavelsonN.M.Webb (1991). Generalizability theory: A primer. Newbury Park, CA: Sage.
50. D.J.Shernoff (2010). Engagement in after-school programs as a predictor of social competence and academic performance. American Journal of Community Psychology, 45, 325-337.
51. M.ShinnB.D.Rapkin (2000). Cross-level research without cross-ups in community psychology. In J.RappaportE.Seidman (Eds.), Handbook of community psychology (pp. 669-695). New York, NY: Springer.
52. C.H.SkinnerK.I.DittmerL.A.Howell (2000). Direct observation in school settings: Theoretical issues. In E.S.ShapiroT.R.Kratochwill (Eds.), Behavioral assessment in schools: Theory, research, and clinical foundations (pp. 19-45). New York, NY: Guilford Press.
53. C.SmithC.Hohmann (2005). Full findings from the youth program quality assessment validation study. Ypsilanti, MI: High/Scope Educational Research Foundation.
54. E.P.SmithM.FaulkM.A.Sizer (2013). Exploring the meso-system: The roles of community, family, and peers in adolescent delinquency and positive youth development. Youth & Society. DOI 10.1177/0044118X13491581
55. M.W.StuhlmanB.K.HamreJ.T.DownerR.C.Pianta (2010). A practitioner’s guide to conducting classroom observations: What the research tells us about choosing and using observational systems. Charlottesville: University of Virginia.
56. D.L.VandellE.R.ReisnerB.B.BrownK.M.PierceK.DadismanE.M.Pechman (2004). The study of promising after-school programs: Descriptive report of the promising programs. Retrieved from http://www.wcer.wisc.edu/childcare/statements.html
57. D.L.VandellE.R.ReisnerK.M.Pierce (2007). Outcomes linked to high-quality afterschool programs: Longitudinal findings from the study of promising afterschool programs. Washington, DC: Policy Studies Associates.
58. R.J.VandenbergC.E.Lance (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-69.
59. M.-H.VéronneauT.J.Dishion (2011). Middle school friendships and academic achievement in early adolescence: A longitudinal analysis. The Journal of Early Adolescence, 31, 99-124. doi:10.1177/0272431610384485
60. K.E.WalkerA.J.A.Arbreton (2004). After-school pursuits: An examination of outcomes in the San Francisco beacon initiative. New York, NY: Public/Private Ventures.
61. N.YohalemA.Wilson-Ahlstrom (2010). Inside the black box: Assessing and improving quality in youth programs. American Journal of Community Psychology, 45, 350-357.