Examining the Effectiveness of a Multicomponent Reading Comprehension Intervention in Middle Schools: A Focus on Treatment Fidelity

Educational Psychology Review

Volumn 26, Issue 3, 2014, Pages 425-449

  Fogarty, Melissa ^a   Oslund, Eric ^b   Simmons, Deborah ^a   Davis, John ^a   Simmons, Leslie ^a   Anderson, Leah ^a   Clemens, Nathan ^a   Roberts, Greg ^b  

^a TEXAS A AND M UNIVERSITY   (United States)

^b UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

Fidelity of implementation; Middle school intervention; Reading comprehension

Indexed keywords

EID: 84956829374     PISSN: 1040726X     EISSN: 1573336X     Source Type: Journal    
DOI: 10.1007/s10648-014-9270-6     Document Type: Article

References (68)

1. Allinder, R. M., Bolling, R. M., Oats, R. G., & Gagnon, W. A. (2000). Effects of teacher self-monitoring on implementation of curriculum-based measurement and mathematics computation achievement of students with disabilities. Remedial and Special Education, 21, 219–226. doi:10.1177/074193250002100403.
2. Altman, D. G. (1991). Practical statistics for medical research. London: Chapman and Hall.
3. Barnes, M. A., Raghubar, K. P., Faulkner, H., & Denton, C. A. (2014). The construction of visual-spatial situation models in children’s reading and their relation to reading comprehension. Journal of Experimental Child Psychology, 119, 101–111.
4. Beck, I. L., McKeown, M. G., & Kucan, L. (2002). Bringing words to life: Robust vocabulary instruction. New York: Guilford.
5. Benner, G. J., Nelson, J. R., Stage, S. A., & Ralston, N. C. (2011). The influence of fidelity of implementation on the reading outcomes of middle school students experiencing reading difficulties. Remedial and Special Education, 32, 79–88. doi:10.1177/0741932510361265.
6. Browder, D. M., Trela, K., & Jimenez, B. (2007). Training teachers to follow a task analysis to engage middle school students with moderate and severe developmental disabilities in grade-appropriate literature. Focus on Autism and Other Developmental Disabilities, 22(4), 206–219.
7. Brozo, W. G., & Afflerbach, P. P. (2011). Adolescent literacy inventory: grades 6–12. Boston: Pearson.
8. Cantrell, S. C., Almasi, J. F., Carter, J. C., & Rintamaa, M. (2010). Striving readers year 3 evaluation report (Danville, Kentucky). Washington, D.C.: Research report prepared for the U.S. Department of Education.
9. Carlisle, J. F. (2000). Awareness of the structure and meaning of morphologically complex words: impact on reading. Reading and Writing, 12, 169–190.
10. Cash, T. F. (2009). Caveats in the proficient preparation of an APA-style research manuscript for publication. Body Image, 6(1), 1–6. doi:10.1016/j.bodyim.2008.10.003.
11. Catts, H. W., Compton, D., Tomblin, J. B., & Bridges, M. S. (2011). Prevalence and nature of late-emerging poor readers. Journal of Educational Psychology, 104, 166–181. doi:10.1037/a0025323.
12. Chi, M. T. H. (2009). Active-constructive-interactive: a conceptual framework for differentiating learning activities. Topics in Cognitive Science, 1, 73–105. doi:10.1111/j.1756-8765.2008.01005.x.
13. Cromley, J. G., & Azevedo, R. (2007). Testing and refining the direct and inferential mediation model of reading comprehension. Journal of Educational Psychology, 99, 311–325. doi:10.1037/0022-0663.99.2.311.
14. Dane, A. V., & Schneider, B. H. (1998). Program integrity in primary and early secondary prevention: are implementation effects out of control? Clinical Psychology Review, 18(1), 23–45. doi:10.1037/0022-0663.99.2.311.
15. Duke, N. K., Pearson, P. D., Strachan, S. L., & Billman, A. K. (2011). Essential elements of fostering reading comprehension. In S. J. Samuels & A. E. Farstrup (Eds.), What research has to say about reading instruction (4th ed., pp. 51–93). Newark: International Reading Association. Retrieved from http://www.scienceandliteracy.org/sites/scienceandliteracy.org/files/biblio/pdpearson/03-DukePearsonStrachanBillman.prepub.Comprehension.WRST.032311.pdf.
16. Edmonds, M. S., Vaughn, S., Wexler, J., Reutebuch, C., Cable, A., Tackett, K. K., et al. (2009). A synthesis of reading interventions and effects on reading comprehenson outcomes for older struggling readers. Review of Educational Research, 79, 262–300. doi:10.3102/0034654308325998.
17. Elleman, A. M., Lindo, E. J., Morphy, P., & Compton, D. L. (2009). The impact of vocabulary instruction on passage-level comprehension of school-age children: a meta-analysis. Journal of Research on Educational Effectiveness, 2, 1–44. doi:10.1080/19345740802539200.
18. Engelmann, S., Hanner, S., & Johnson, G. (2002). Corrective reading. Columbus: SRA/McGraw-Hill.
19. Fixsen, D., Blasé, K., Metz, A., & Van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79, 213–230.
20. Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38, 915–945. doi:10.3102/00028312038004915.
21. Garet, M. S., Wayne, A. J., Stancavage, F., Taylor, J., Walters, K., Song, M., et al. (2010). Middle school mathematics professional development impact study: Findings after the first year of implementation. NCEE 2010-4009. National Center for Education Evaluation and Regional Assistance.
22. Gersten, R., Fuchs, L. S., Compton, D., Coyne, M., Greenwood, C., & Innocenti, M. S. (2005). Quality indicators for group experimental and quasi-experimental research in special education. Exceptional Children, 71, 149–164.
23. Graves, M. F., & Prenn, M. C. (1986). Costs and benefits of various methods of teaching vocabulary. Journal of Reading, 29, 596–602.
24. Gresham, F. M. (2009). Evolution of the treatment integrity concept: current status and future directions. School Psychology Review, 38, 533–540.
25. Gresham, F. M., MacMillan, D. L., Beebe-Frankenberger, M. E., & Bocian, K. M. (2000). Treatment integrity in learning disabilities intervention research: do we really know how treatments are implemented? Learning Disabilities Research and Practice, 15(4), 198–205. doi:10.1207/SLDRP1504_4.
26. Guthrie, J. T., McRae, A., & Klauda, S. L. (2007). Contributions of concept-oriented reading instruction to knowledge about interventions for motivations in reading. Educational Psychologist, 42, 237–250. doi:10.1080/00461520701621087.
27. Hulleman, C. S., & Cordray, D. S. (2009). Moving from the lab to the field: the role of fidelity and achieved relative intervention strength. Journal of Research on Educational Effectiveness, 2, 88–110. doi:10.1080/19345740802539325.
28. Institute of Education Sciences (2014). Request for applications: Educational research grants (CFDA Number: 84.305A). Retrieved from http://ies.ed.gov/funding/pdf/2014_84305A.pdf. Accesssed 15 Jan 2014.
29. Jones, H. A., Clarke, A. T., & Power, T. J. (2008). Expanding the concept of intervention integrity: a multidimensional model of participant engagement. Balance, 23, 4–5.
30. Kamil, M. L., Borman, G. D., Dole, J., Kral, C. C., Salinger, T., Torgesen, J., et al. (2008). Improving adolescent literacy: Effective classroom and intervention practices: A Practice Guide (NCEE #2008-4027). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Retrieved from http://ies.ed.gov/ncee/wwc. Accessed 1 Aug 2013.
31. Kim, A.-H., Vaughn, S., Wanzek, J., & Wei, S. (2004). Graphic organizers and their effects on the reading comprehension of students with LD: a synthesis of research. Journal of Learning Disabilities, 37, 105–118. doi:10.1177/00222194040370020201.
32. Kim, W., Linan–Thompson, S., & Misquitta, R. (2012). Critical factors in reading comprehension instruction for students with learning disabilities: a research synthesis. Learning Disabilities Research and Practice, 27, 66–78. doi:10.1111/j.1540-5826.2012.00352.x.
33. King, A. (1994). Guiding knowledge construction in the classroom: effects of teaching children how to question and how to explain. American Educational Research Journal, 31, 338–368. doi:10.2307/1163313.
34. King, A. (2002). Structuring peer interaction to promote high-level cognitive processing. Theory Into Practice, 41, 33–39. doi:10.1207/s15430421tip4101_6.
35. Kintsch, W. (1998). The role of knowledge in discourse comprehension: a construction-integration model. Psychological Review, 95, 163–183. doi:10.1037/0033-295X.95.2.163.
36. Kline, P. (2000). The handbook of psychological testing (2nd ed.). London: Routledge.
37. Klingner, J. K., Vaughn, S., Arguelles, M. E., Hughes, M. T., & Leftwich, S. A. (2004). Collaborative strategic reading: “real-world” lessons from classroom teachers. Remedial and Special Education, 25, 291–302. doi:10.1177/07419325040250050301.
38. Kushman, J., Hanita, M., & Raphael, J. (2011). An experimental study of the project CRISS reading program on grade 9 reading achievement in rural high schools. (NCEE 2011-4007). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.
39. Lipsey, M. W., Puzio, K., Yun, C., Hebert, M. A., Steinka-Fry, K., Cole, M. W., et al. (2012). Translating the statistical representation of the effects of education interventions into more readily interpretable forms. Washington, DC: National Center for Special Education Research.
40. MacGinitie, W. H., MacGinitie, R. K., Maria, K., & Dreyer, L. G. (2010). Gates-MacGinitie Reading Tests (4th ed.). Itasca: Riverside Publishing.
41. McCormick, S. (1989). Effects of previews on more skilled and less skilled readers’ comprehension of expository text. Journal of Literacy Research, 21, 219–239. doi:10.1080/10862968909547674.
42. McMaster, K. L., van den Broek, P. V., Espin, C. A., White, M. J., Rapp, D. N., Kendeou, P., et al. (2012). Making the right connections: Differential effects of reading intervention for subgroups of comprehenders. Learning and Individual Differences, 22, 100–111.
43. National Center for Education Statistics. (2013). The nation’s report card: a first look: 2013 mathematics and reading (NCES 2014-451). Washington, DC: U.S. Department of Education, Institute of Education Sciences.
44. National Governors Association Center for Best Practices, Council of Chief State School Officers (2010). Common core state standards for English Language arts and literacy in history/social studies, science, and technical subjects: Appendix B: Text exemplars and sample performance tasks. Washington, DC: National Governors Association Center for Best Practices, Council of Chief State School Officers. Retrieved from http://www.corestandards.org/assets/Appendix_B.pdf. Accessed 25 July 2013.
45. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw Hill.
46. O’Donnell, C. L. (2008). Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research, 78, 33–84. doi:10.3102/0034654307313793.
47. Oakhill, J. V., & Cain, K. (2012). The precursors of reading ability in young readers: evidence from a four-year longitudinal study. Scientific Studies of Reading, 16, 91–121. doi:10.1080/10888438.2010.529219.
48. Penuel, W. R., & Means, B. (2004). Implementation variation and fidelity in an inquiry science program: analysis of GLOBE data reporting patterns. Journal of Research in Science Teaching, 41, 294–315. doi:10.1002/tea.20002.
49. Penuel, W. R., Fishman, B. J., Yamaguchi, R., & Gallagher, L. P. (2007). What makes professional development effective? Strategies that foster curriculum implementation. American Educational Research Journal, 44, 921–958. doi:10.3102/0002831207308221.
50. Penuel, W. R., Riel, M., Krause, A. E., & Frank, K. A. (2009). Analyzing teachers’ professional interactions in a school as social capital: a social network approach. The Teachers College Record, 111, 124–163.
51. Rapp, D. N., van den Broek, P., McMaster, K. L., Kendeou, P., & Espin, C. A. (2007). Higher-order comprehension processes in struggling readers: a perspective for research and intervention. Scientific Studies of Reading, 11, 289–312. doi:10.1080/10888430701530417.
52. Robinson, J. P., Shaver, P. R., & Wrightsman, L. S. (Eds.). (1991). Measures of personality and social psychological attitudes. San Diego: Academic.
53. Roscoe, R. D., & Chi, M. T. H. (2007). Understanding tutor learning: knowledge-building and knowledge-telling in peer tutors’ explanation and questions. Review of Educational Research, 77, 534–574. doi:10.3102/0034654307309920.
54. Sailors, M., & Price, L. R. (2010). Professional development that supports the teaching of cognitive reading strategy instruction. The Elementary School Journal, 110, 301–322. doi:10.1086/648980.
55. Scammacca, N., Roberts, G., Vaughn, S., Edmonds, M., Wexler, J., Reutebuch, C. K., et al. (2007). Interventions for adolescent struggling readers: a meta-analysis with implications for practice. Portsmouth, NH: RMC Research Corporation, Center on Instruction.
56. Scher, L., & O'Reilly, F. (2009). Professional development for K-12 math and science teachers: what do we really know? Journal of Research on Educational Effectiveness, 2, 209–249. doi:10.1080/19345740802641527.
57. Schulte, A. C., Easton, J. E., & Parker, J. (2009). Advances in treatment integrity research: multidisciplinary perspectives on the conceptualization, measurement, and enhancement of treatment integrity. School Psychology Review, 38, 460–475.
58. Sheridan, S. M., Swanger-Gagne, M., Welch, G. W., Kwon, K., & Garbacz, S. A. (2009). Fidelity measurement in consultation: psychometric issues and preliminary examination. School Psychology Review, 38, 476–495.
59. Simmons, D., Fogarty, M., Oslund, E., Simmons, L., Hairrell, A., Davis, J., et al. (2014). Integrating content knowledge-building and student-regulated comprehension practices in secondary English language arts classes. Journal of Research on Educational Effectiveness. doi:10.1080/19345747.2013.836766.
60. Stahl, S. A., & Fairbanks, M. M. (1986). The effects of vocabulary instruction: a model-based meta-analysis. Review of Educational Research, 56, 72–110. doi:10.3102/00346543056001072.
61. Supovitz, J. A., & Turner, H. M. (2000). The effects of professional development on science teaching practices and classroom culture. Journal of Research in Science Teaching, 37, 963–980. doi:10.1002/1098-2736(200011)37:9<963::AID-TEA6>3.0.CO;2-0.
62. Swanson, E., Wanzek, J., Haring, C., Ciullo, S., & McCulley, L. (2013). Intervention fidelity in special and general education research journals. The Journal of Special Education, 47, 3–13. doi:10.1177/0022466911419516.
63. Taboada, A., Townsend, D., & Boynton, M. J. (2013). Mediating effects of reading engagement on the reading comprehension of early adolescent English language learners. Reading and Writing Quarterly, 29, 309–332. doi:10.1080/10573569.2013.741959.
64. Torgesen, J. K. (1977). The role of nonspecific factors in the task performance of learning disabled children: a theoretical assessment. Journal of Learning Disabilities, 10, 27–34. doi:10.1177/002221947701000108.
65. van den Broek, P., Rapp, D. N., & Kendeou, P. (2005). Integrating memory-based and constructionist processes in accounts of reading comprehension. Discourse Processes, 39, 299–316. doi:10.1080/0163853X.2005.9651685.
66. Vaughn, S., Klingner, J. K., Swanson, E. A., Boardman, A. G., Roberts, G., Mohammed, S. S., et al. (2011). Efficacy of collaborative strategic reading with middle school students. American Educational Research Journal, 48, 938–964. doi:10.3102/0002831211410305.
67. Vaughn, S., Roberts, G., Klingner, J. K., Swanson, E. A., Boardman, A., Stillman-Spisak, S. J., et al. (2013). Collaborative strategic reading: Findings from experienced implementers. Journal of Research on Educational Effectiveness, 6, 137–163. doi:10.1080/19345747.2012.741661.
68. Ysseldyke, J., Spicuzza, R., Kosciolek, S., Teelucksingh, E., Boys, C., & Lemkuil, A. (2003). Using a curriculum-based instructional management system to enhance math achievement in urban schools. Journal of Education for Students Placed at Risk, 8, 247–265. doi:10.1207/S15327671ESPR0802_4.