Assessing the Effects of a School-Wide Data-Based Decision-Making Intervention on Student Achievement Growth in Primary Schools

American Educational Research Journal

Volumn 53, Issue 2, 2016, Pages 360-394

  van Geel, Marieke ^a   Keuning, Trynke ^a   Visscher, Adrie J ^a   Fox, Jean Paul ^a  

^a UNIVERSITY OF TWENTE   (Netherlands)

Author keywords

data based decision making; intervention; linear mixed model; school improvement; student achievement

Indexed keywords

EID: 84962127824     PISSN: 00028312     EISSN: 19351011     Source Type: Journal    
DOI: 10.3102/0002831216637346     Document Type: Article

References (63)

1. S.AndersonK.LeithwoodT.Strauss (2010). Leading data use in schools: Organizational conditions and practices at the school and district levels. Leadership and Policy in Schools, 9(3), 292–327. doi:10.1080/157007610 03731492
2. D.BatesM.MächlerB.BolkerS.Walker (2014). Fitting linear mixed-effects models using lme4. Retrieved from http://arxiv.org/abs/1406.5823
3. R.E.Bennett (2011). Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice, 18(1), 5–25. doi:10.1080/096959 4X.2010.513678
4. B.F.BirmanL.M.DesimoneA.C.PorterM.S.Garet (2000). Designing professional development that works. Educational Leadership, 57(8), 28–33.
5. B.P.BlackD.Wiliam (1998). Inside the black box: Raising standards through classroom assessment. Assessment in Education, 5(1), 7–74.
6. C.CampbellB.Levin (2008). Using data to support educational improvement. Educational Assessment, Evaluation and Accountability, 21(1), 47–65. doi:10.1007/s11092-008-9063-x
7. D.CarlsonG.D.BormanM.Robinson (2011). A multistate district-level cluster randomized trial of the impact of data-driven reform on reading and mathematics achievement. Educational Evaluation and Policy Analysis, 33(3), 378–398. doi:10.3102/0162373711412765
8. Cito. (2009). Rekenen-Wiskunde Handleiding. Arnhem: Author.
9. C.E.CoburnE.O.Turner (2012). The practice of data use: An introduction. American Journal of Education, 118(2), 99–111. doi:10.1086/663272
10. R.Coe (2002). Evidence on the role and impact of performance feedback in schools. In (Eds.), School improvement through performance feedback (pp. 3–26). London: Routledge.
11. A.DatnowM.Castellano (2000). Teachers’ responses to success for all: How beliefs, experiences, and adaptations shape implementation. American Educational Research Journal, 37(3), 775–799. doi:10.3102/00028312037003775
12. A.DatnowL.HubbardH.Mehan (1998). Educational reform implementation: A co-constructed process. Berkeley, CA: Center for Research on Education, Diversity & Excellence.
13. L.M.Desimone (2002). How can comprehensive school reform models be successfully implemented?Review of Educational Research, 72(3), 433–479. doi:10.3102/00346543072003433
14. L.M.Desimone (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3), 181–199.
15. K.E.DunnD.T.AirolaW.-J.LoM.Garrison (2013). Becoming data driven: The influence of teachers’ sense of efficacy on concerns related to data-driven decision making. The Journal of Experimental Education, 81(2), 222–241. doi:10.1080/00220973.2012.699899
16. L.EarlS.Katz (2006). Leading schools in a data-rich world: Harnessing data for school improvement. Thousans Oaks, CA: Corwin Press.
17. J.M.FaberA.J.Visscher (2014). Digitale leerlingvolgsystemen: een review van de effecten op leerprestaties. 4W. Retrieved from http://www.kennisnet.nl/onderzoek/alle-onderzoeken/digitale-leerlingvolgsystemen-een-review-van-de-effecten-op-leerprestaties/
18. J.M.FerronM.MoeyaertW.Van den NoortgateS.N.Beretvas (2014). Estimating causal effects from multiple-baseline studies: Implications for design and analysis. Psychological Methods, 19(4), 493–510.
19. L.S.FuchsD.Fuchs (1986). Effects of systematic formative evaluation: A meta-analysis. Exceptional Children, 53, 199–208.
20. N.A.GageT.J.Lewis (2014). Hierarchical linear modeling meta-analysis of single-subject design research. The Journal of Special Education, 48, 3–16. doi:10.1177/0022466912443894
21. M.S.GaretA.C.PorterL.M.DesimoneB.F.BirmanK.S.Yoon (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915–945.
22. S.GershensonL.Langbein (2015). the effect of primary school size on academic achievement. Educational Evaluation and Policy Analysis, 37, 135–155. doi:10.3102/0162373715576075
23. J.Gray (2002). Jolts and reactions: Two decades of feeding back information on schools’ performance. In (Eds.), School improvement through performance feedback (pp. 143–162). London: Routledge.
24. J.A.GrissomS.LoebB.Master (2013). Effective instructional time use for school leaders: Longitudinal evidence from observations of principals. Educational Researcher, 42, 433–444. doi:10.3102/0013189X13510020
25. L.HamiltonR.HalversonS.S.JacksonE.B.MandinachJ.A.SupovitzJ.C.Wayman (2009). Using student achievement data to support instructional decision making. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/practice_guides/dddm_pg_092909.pdf
26. E.A.Hanushek (2011). How much is a good teacher worth?Education Next, 11(3), 40–45.
27. J.Hattie (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London: Routledge.
28. J.HattieH.Timperley (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. doi:10.3102/003465430298487
29. G.S.IkemotoJ.A.Marsh (2007). Cutting through the “data-driven” mantra: Different conceptions of data-driven decision making. In (Ed.), Evidence and decision making: Yearbook of the national society of education (pp. 105–131). London: Wiley-Blackwell.
30. Inspectie van het Onderwijs. (2012). Beoordeling van opbrengsten in het basisonderwijs. Utrecht, The Netherlands: Author.
31. Inspectie van het Onderwijs. (2013). De staat van het onderwijs: Onderwijsverslag 2011/2012. Utrecht, The Netherlands: Author.
32. W.R.JensonE.ClarkJ.C.KircherS.D.Kristjansson (2007). Statistical reform: Evidence-based practice, meta-analyses, and single subject designs. Psychology in the Schools, 44(5), 483–493. doi:10.1002/pits.20240
33. F.KamphuisF.Moelands (2000). A student monitoring system. Educational Measurement: Issues and Practice, 19(4), 28–30.
34. N.KingstonB.Nash (2011). Formative assessment: A meta-analysis and a call for research. Educational Measurement: Issues and Practice, 30(4), 28–37. doi:10.1111/j.1745-3992.2011.00220.x
35. A.N.KlugerA.DeNisi (1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological Bulletin, 199(2), 254–284.
36. T.R.KratochwillJ.HitchcockR.H.HornerJ.R.LevinS.L.OdomD.M.RindskopfW.R.Shadish (2010). Single-case design technical documentation. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf
37. M.K.LaiS.McNaughton (2013). Analysis and discussion of classroom and achievement data to raise student achievement. In (Eds.), Data-based decision making in education: Challenges and opportunities (pp. 23–48). New York, NY: Springer.
38. M.K.LaiK.Schildkamp (2013). Data-based decision making: An overview. In (Eds.), Data-based decision making in education: Challenges and opportunities (pp. 9–21). NewYork, NY: Springer.
39. E.A.LockeG.P.Latham (2002). Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist, 57(9), 705–717. doi:10.1037//0003-066X.57.9.705
40. A.T.Lumpe (2007). Research-based professional development: Teachers engaged in professional learning communities. Journal of Science Teacher Education, 18(1), 125–128. doi:10.1007/s10972-006-9018-3
41. E.B.Mandinach (2012). A perfect time for data use: Using data-driven decision making to inform practice. Educational Psychologist, 47(2), 71–85. doi:10.1080/00461520.2012.667064
42. E.B.MandinachE.S.GummerR.D.Muller (2011). The complexities of integrating data-driven decision making into professional preparation in schools of education: It’s harder than you think. Report from an invitational meeting. Alexandria, VA: CNA Analysis & Solutions.
43. J.A.Marsh (2012). Interventions promoting educators use of data: Research insights and gaps. Teachers College Record, 114, 1–47.
44. H.MayJ.HuffE.Goldring (2012). A longitudinal study of principals’ activities and student performance. School Effectiveness and School Improvement, 23, 417–439. doi:10.1080/09243453.2012.678866
45. D.MorisanoE.A.Locke (2013). Goal setting and academic achievement. In (Eds.), International guide to student achievement (pp. 45–48). London: Routledge.
46. I.V.S.MullisM.O.MartinP.FoyA.Arora (2012). TIMSS 2011 international results in mathematics. Chestnut Hill, MA: TIMSS & PIRL International Study Center, Boston College.
47. B.NyeS.KonstantopoulosL.V.Hedges (2004). How large are teacher effects?Educational Evaluation and Policy Analysis, 26(3), 237–257.
48. L.N.OláhN.R.LawrenceM.Riggan (2010, April26). Learning to learn from benchmark assessment data: How teachers analyze results. Peabody Journal of Education, 85, 226–245. doi:10.1080/01619561003688688
49. RCoreTeam. (2013). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. Retrieved from http://www.r-project.org/
50. K.SchildkampM.EhrenM.K.Lai (2012). Editorial article for the special issue on data-based decision making around the world: From policy to practice to results. School Effectiveness and School Improvement, 23(2), 123–131. doi:10.1080/09243453.2011.652122
51. K.SchildkampM.K.LaiL.Earl (Eds.). (2013). Data-based decision making in education: Challenges and opportunities. Dordrecht: Springer Netherlands. http://doi.org/10.1007/978-94-007-4816-3
52. T.A.B.SnijdersR.J.Bosker (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. London: Sage Publishers.
53. L.StamanA.J.VisscherH.Luyten (2013). The effects of training school staff for utilizing student monitoring system data. In (Eds.), 10th IFIP WG 3.7 Conference, ITEM 2012 (pp. 3–14). Bremen, Germany: Springer.
54. L.StamanA.J.VisscherH.Luyten (2014). The effects of professional development on the attitudes, knowledge and skills for data-driven decision making. Studies in Educational Evaluation, 42, 79–90. doi:10.1016/j.stueduc.2013.11.002
55. J.A.Supovitz (2012). Getting at student understanding—The key to teachers’ use of test data. Teachers College Record, 114(11), 1–29.
56. H.Timperley (2008). Teacher professional learning and development. Retrieved from http://edu.aru.ac.th/childedu/images/PDF/benjamaporn/EdPractices_18.pdf.
57. E.O.TurnerC.E.Coburn (2012). Interventions to promote data use: An introduction. Teachers College Record, 114, 13.
58. W.Van den NoortgateP.Onghena (2003). Combining single-case experimental data using hierarchical linear models. School Psychology Quarterly, 18, 325–346.
59. K.Van VeenR.ZwartJ.Meirink (2011). What makes teacher professional development effective? A literature review. In (Eds.), Teacher learning that matters (pp. 3–21). New York, NY: Routledge.
60. J.VanhoofG.VerhaegheP.Van PetegemM.Valcke (2013). Improving data literacy in schools: Lessons from the school feedback project. In (Eds.), Data-based decision making in education: Challenges and opportunities (pp. 113–134). New York, NY: Springer.
61. G.Verhaeghe (2011). School performance feedback systems: Design and implementation issues. Ghent, Belgium: Ghent University.
62. J.C.WaymanV.Cho (2008). Preparing educators to effectively use student data systems. In (Eds.), Handbook on data-based decision-making in education (pp. 89–104). New York, NY: Routledge.
63. J.C.WaymanS.StringfieldM.Yakimowski (2004). Software enabling school improvement through analysis of student data. Baltimore, MD: Johns Hopkins University Center for Research on the Education of Students Placed At Risk.