A randomized wait-list controlled analysis of the implementation integrity of team-initiated problem solving processes

Journal of School Psychology

Volumn 50, Issue 4, 2012, Pages 421-441

  Newton, J Stephen ^a   Horner, Robert H ^a   Algozzine, Bob ^b   Todd, Anne W ^a   Algozzine, Kate ^b  

^a UNIVERSITY OF OREGON   (United States)

^b UNIVERSITY OF NORTH CAROLINA AT CHARLOTTE   (United States)

Author keywords

Positive behavior interventions and supports; Problem solving; Teams

Indexed keywords

ACHIEVEMENT; ARTICLE; CLINICAL TRIAL; CONTROLLED CLINICAL TRIAL; CONTROLLED STUDY; GROUP PROCESS; HOSPITAL ADMISSION; HUMAN; PROBLEM SOLVING; RANDOMIZED CONTROLLED TRIAL; SCHOOL;

ACHIEVEMENT; GROUP PROCESSES; HUMANS; PROBLEM SOLVING; SCHOOLS; WAITING LISTS;

EID: 84862505671     PISSN: 00224405     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jsp.2012.04.002     Document Type: Article

References (97)

1. Algozzine, B., Newton, J. S., Horner, R. H., Todd, A. W., & Algozzine, K. (in press). Development and technical characteristics of a team decision-making assessment tool: Decision observation, recording, and analysis (DORA). Journal of Psychoeducational Assessment.
2. Baer D.M., Wolf M.M., Risley T.R. Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis 1968, 1:91-97.
3. Bartels S.M., Mortenson B.P. Enhancing adherence to a problem-solving model for middle-school pre-referral teams: A performance feedback and checklist approach. Journal of Applied School Psychology 2005, 22:109-123.
4. Batsche G.M., Curtis M.J., Dorman C., Castillo J.M., Porter L.J. The Florida problem-solving/response to intervention model: Implementing a statewide initiative. Handbook of response to intervention: The science and practice of assessment and intervention 2007, 378-395. Springer Science, New York, NY. S.R. Jimerson, M.K. Burns, A.M. VanDerHeyden (Eds.).
5. Bergan J.R., Kratochwill T.R. Behavioral consultation and therapy 1990, Plenum, New York, NY.
6. Billingsley F., White O.R., Munson R. Procedural reliability: A rationale and an example. Behavioral Assessment 1980, 2:229-241.
7. Data wise: A step-by-step guide to using assessment results to improve teaching and learning 2005, Harvard Education Press, Cambridge, MA. K.P. Boudett, E.A. City, R.J. Murnane (Eds.).
8. Bransford J.D., Stein B.S. The IDEAL problem solver: A guide for improving thinking, learning, and creativity 1984, W. H. Freeman and Company, New York, NY.
9. Brown-Chidsey R., Steege M.W. Response to intervention: Principles and strategies for effective practice 2005, Guilford Press, New York, NY.
10. Buck G.H., Polloway E.A., Smith-Thomas A., Cook K.W. Prereferral intervention processes: A survey of state practices. Exceptional Children 2003, 69:349-360.
11. Burns M.K., Deno S.L., Jimerson S.R. Toward a unified response-to-intervention model. Handbook of response to intervention: The science and practice of assessment and intervention 2007, 428-440. Springer, New York, NY. S.R. Jimerson, M.K. Burns, A.M. VanDerHeyden (Eds.).
12. Burns M.K., Peters R., Noell G.H. Using performance feedback to enhance implementation fidelity of the problem-solving team process. Journal of School Psychology 2008, 46:537-550.
13. Burns M.K., Symington T. A meta-analysis of prereferral intervention teams: Student and systemic outcomes. Journal of School Psychology 2002, 40:437-447.
14. Burns M.K., Vanderwood M.L., Ruby S. Evaluating the readiness of pre-referral intervention teams for use in a problem solving model. School Psychology Quarterly 2005, 20:89-105.
15. Campbell D.T., Stanley J.C. Experimental and quasi-experimental designs for research 1963, Rand McNally, Chicago, IL.
16. Carroll J.S., Johnson E.J. Decision research: A field guide 1990, Sage, Newbury Park, CA.
17. Carter J., Sugai G. Survey on prereferral practices: Responses from state departments of education. Exceptional Children 1989, 55:298-302.
18. Chalfant J.C., Pysh M.V., Moultrie R. Teacher assistance teams: A model for within-building problem solving. Learning Disability Quarterly 1979, 2:85-96.
19. Coalition for Evidence-Based Policy Identifying and implementing educational practices supported by rigorous evidence: A user-friendly guide 2003, Author, Washington, DC.
20. Cochrane W.S., Laux J.M. A survey investigating school psychologists' measurement of treatment integrity in school-based interventions and their beliefs about its importance. Psychology in the Schools 2008, 45:499-507.
21. Codding R.S., Feinberg A.B., Dunn E.K., Pace G.M. Effects of immediate performance feedback on implementation of behavior support plans. Journal of Applied Behavior Analysis 2005, 38:205-219.
22. Cohen J. Statistical power analysis for the behavioral sciences 1988, Erlbaum, Hillsdale, NJ. 2nd ed.
23. Cohen R., Kincaid D., Childs K.E. Measuring school-wide positive behavior support implementation: Development and validation of the benchmarks of quality. Journal of Positive Behavior Interventions 2007, 9:203-213.
24. Cooper J.O., Heron T.E., Heward W.L. Applied behavior analysis 2007, Pearson, Upper Saddle River, NJ. 2nd ed.
25. Cowan K., Cohn A. School psychology awareness: A positive approach. Communiqué 2009, 38:32-33.
26. D'Zurilla T.J., Nezu A.M. Development and preliminary evaluation of the social problem-solving inventory. Psychological Assessment: A Journal of Consulting and Clinical Psychology 1990, 2:156-163.
27. D'Zurilla T.J., Nezu A.M., Maydeu-Olivares A. Social problem-solving inventory-revised (SPSI-R): Technical manual 2002, Multi-Health Systems, North Tonawanda, NY.
28. Deno S.L. Curriculum-based measurement: The emerging alternative. Exceptional Children 1985, 52:219-232.
29. Deno S.L. Problem-solving assessment. Assessment for intervention: A problem-solving approach 2005, 10-40. Guilford Press, New York, NY. R. Brown-Chidsey (Ed.).
30. DiGennaro F.D., Martens B.K. A comparison of performance feedback procedures on teachers' treatment implementation integrity and students' inappropriate behavior in special education classrooms. Journal of Applied Behavior Analysis 2007, 40:447-461.
31. Doll B., Haack K., Kosse S., Osterloh M., Siemers E. The dilemma of pragmatics: Why schools don't use quality team consultation practices. Journal of Educational and Psychological Consultation 2005, 16:127-155.
32. Duhon G.J., Mesmer E.M., Gregerson L., Witt J.C. Effects of public feedback during RTI team meetings on teacher implementation integrity and student academic performance. Journal of School Psychology 2009, 47:19-37.
33. Durlak J.A., DuPre E.P. Implementation matters: A review of research on the influence of implementation on program outcomes and the factors affecting implementation. American Journal of Community Psychology 2008, 41:327-350.
34. Flugum K.R., Reschly D.J. Prereferral interventions: Quality indices and outcomes. Journal of School Psychology 1994, 32:1-14.
35. Fuchs D., Mock D., Morgan P.L., Young C.L. Responsiveness-to-intervention: Definitions, evidence, and implications for the learning disabilities construct. Learning Disabilities Research and Practice 2003, 18:157-171.
36. Gall M.D., Gall J.P., Borg W.R. Educational research: An introduction 2007, Allyn & Bacon, Inc., Boston, MA. 8th ed.
37. Gansle K.A., McMahon C.M. Component integrity of teacher intervention management behavior using a student self-monitoring treatment: An experimental analysis. Journal of Behavioral Education 1997, 7:405-419.
38. Gelzheiser L.M. Preparing for the future of school psychology: A special educator's view. Journal of Educational and Psychological Consultation 2009, 19:259-266.
39. Gilbert T.F. Human competence: Engineering worthy performance 1978, McGraw-Hill, New York, NY.
40. Graden J.L., Casey A., Christenson S.L. Implementing a prereferral intervention system: I. The model. Exceptional Children 1985, 51:377-384.
41. Gresham F.M., Gansle K.A., Noell G.H. Treatment integrity in applied behavior analysis with children. Journal of Applied Behavior Analysis 1993, 26:257-263.
42. Gresham F.M., Gansle K.A., Noell G.H., Cohen S. Treatment integrity of school-based behavioral intervention studies: 1980-1990. School Psychology Review 1993, 22:254-272.
43. Hamilton L., Halverson R., Jackson S.S., Mandinach E., Supovitz J.A., Wayman J.C. Using student achievement data to support instructional decision making (NCEE 2009-4067) 2009, National Center for Educational Evaluation and Regional Assistance, Institute of Education Sciences, U. S. Department of Education, Washington, DC, Retrieved from http://ies.ed.gov/ncee/wwc/practiceguide.aspx?sid=12.
44. Hedges L.V. Distribution theory for Glass's estimator of effect size and related estimators. Journal of Educational Statistics 1981, 6:107-128.
45. Hedges L.V. Estimation of effect size from a series of independent experiments. Psychological Bulletin 1982, 92:490-499.
46. Hersen M., Barlow D.H. Single-case experimental designs: Strategies for studying behavior change 1976, Pergamon Press, Elmsford, NY.
47. Holcombe A., Wolery M., Snyder E. Effects of two levels of procedural fidelity with constant time delay on children's learning. Journal of Behavioral Education 1994, 4:49-73.
48. Horner R.H., Sugai G., Anderson C.M. Examining the evidence base for school-wide positive behavior support. Focus on Exceptional Children 2010, 42:1-14.
49. Horner R.H., Sugai G., Smolkowski K., Eber L., Nakasato J., Todd A.W., et al. A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions 2009, 11:133-144.
50. Horner R.H., Todd A.W., Lewis-Palmer T., Irvin L.K., Sugai G., Boland J.B. The school-wide evaluation tool (SET): A research instrument for assessing school-wide positive behavior supports. Journal of Positive Behavior Interventions 2004, 6:3-12.
51. Horner R., Todd A., Sampson N., Dickey C., May S., Amedo M., et al. School-wide information system facilitator training manual 2008, Educational and Community Supports, University of Oregon, Eugene, Oregon.
52. Huber G.P. Managerial decision making 1980, Scott, Foresman, Glenview, IL.
53. Individuals with Disabilities Education Improvement Act H.R. 1350, 108th Congress 2004.
54. Irvin L.K., Horner R.H., Ingram K., Todd A.W., Sugai G., Sampson N.K., et al. Using office discipline referral data for decision making about student behavior in elementary and middle schools: An empirical evaluation of validity. Journal of Positive Behavior Interventions 2006, 8:10-23.
55. Joyce B.R., Showers B. Student achievement through staff development 2002, Association for Supervision and Curriculum Development, Alexandria, VA. 3rd ed.
56. Kline R.B. Beyond significance testing: Reforming data analysis methods in behavioral research 2005, American Psychological Association, Washington, DC.
57. Kovaleski J.F., Glew M.C. Bringing instructional support teams to scale: Implications of the Pennsylvania experience. Remedial and Special Education 2006, 27:16-25.
58. Lewis T.J., Jones S.E.L., Horner R.H., Sugai G. School-wide positive behavior support and students with emotional/behavioral disorders: Implications for prevention, identification, and intervention. Exceptionality 2010, 18:82-93.
59. Liaupsin C.J., Scott T.M., Nelson C.M. Functional behavioral assessment and intervention planning: A simplified team process 2006, [CD-ROM], Frederick, Co, Sopris West.
60. Matthews J.N.S. Introduction to randomized controlled clinical trials 2006, Chapman & Hall/CRC, Boca Raton, FL. 2nd ed.
61. May S., Arid W., Todd A.W., Horner R.H., Glasgow A., Sugai G. Schoolwide information system 2003, Educational and Community Supports, University of Oregon, Eugene, Oregon.
62. McDougal J.L., Clonan S.M., Martens B.K. Using organizational change procedures to promote the acceptability of prereferral intervention services: The school-based intervention team project. School Psychology Quarterly 2000, 15:149-171.
63. McIntyre L.L., Gresham F.M., DiGennaro F.D., Reed D.D. Treatment integrity of school-based interventions with children in the Journal of Applied Behavior Analysis 1991-2005. Journal of Applied Behavior Analysis 2007, 40:659-672.
64. Moody P.E. Decision making: Proven methods for better decisions 1983, McGraw-Hill, New York, NY.
65. Moore J.W., Edwards R.P., Sterling-Turner H.E., Riley J., DuBard M., McGeorge A. Teacher acquisition of functional analysis methodology. Journal of Applied Behavior Analysis 2002, 35:73-77.
66. Myers D., Dynarski M. Random assignment in program evaluation and intervention research: Questions and answers 2003, Institute of Education Sciences, Washington, DC.
67. National Association of School Psychologists Model for comprehensive and integrated school psychological services 2010, Washington, DC, Author, Retrieved from http://www.nasponline.org/standards/2010standards/2_PracticeModel.pdf.
68. Newton J.S., Algozzine B., Algozzine K., Horner R.H., Todd A.W. Building local capacity for training and coaching data-based problem solving with positive behavior intervention and support teams. Journal of Applied School Psychology 2011, 27:228-245.
69. Newton J.S., Horner R.H., Algozzine R.F., Todd A.W., Algozzine K.M. Using a problem-solving model to enhance data-based decision making in schools. Handbook of positive behavior support 2009, 551-580. Springer, New York, NY. W. Sailor, G. Dunlap, G. Sugai, R. Horner (Eds.).
70. Newton J.S., Horner R.H., Todd A.W., Algozzine B., Algozzine K.M. A pilot study of a problem-solving model for team decision making. Education and Treatment of Children 2012, 35:25-49.
71. Newton J.S., Todd A.W., Algozzine K.M., Horner R.H., Algozzine B. Team-initiated problem solving training manual 2009, Educational and Community Supports, University of Oregon, Eugene, Oregon.
72. Newton J.S., Todd A.W., Horner R.H., Algozzine B., Algozzine K. Decision observation, recording, and analysis: DORA 2009, Unpublished instrument, Educational and Community Supports, University of Oregon, Eugene, Oregon.
73. Noell G.H. Research examining the relationships among consultation process, treatment integrity, and outcomes. Handbook of research in school consultation 2008, 323-341. Erlbaum, New York, NY. W.P. Erchul, S.M. Sheridan (Eds.).
74. Noell G.H., Gresham F.M., Gansle K.A. Does treatment integrity matter? A preliminary investigation of instructional implementation and mathematics performance. Journal of Behavioral Education 2002, 11:51-67.
75. Noell G.H., Witt J.C., Slider N.J., Connell J.E., Gatti S.L., Williams K.L., et al. Treatment implementation following behavioral consultation in schools: A comparison of three follow-up strategies. School Psychology Review 2005, 34:87-106.
76. O'Donnell C.L. Defining, conceptualizing, and measuring fidelity of implementation and its relationship to outcomes in K-12 curriculum intervention research. Review of Educational Research 2008, 78:33-84.
77. Page T.J., Iwata B.A. Interobserver agreement: History, theory, and current methods. Research methods in applied behavior analysis: Issues and advances 1986, 99-126. Plenum Press, New York, NY. A. Poling, R.W. Fuqua (Eds.).
78. Peterson L., Homer A.L., Wonderlich S.A. The integrity of independent variables in behavior analysis. Journal of Applied Behavior Analysis 1982, 15:477-492.
79. Rankin J.L., Aksamit D.L. Perceptions of elementary, junior high, and high school student assistance team coordinators, team members, and teachers. Journal of Educational and Psychological Consultation 1994, 5:229-256.
80. Rodriguez B.J., Loman S.L., Horner R.H. A preliminary analysis of the effects of coaching feedback on teacher implementation integrity of First Steps to Success. Behavior Analysis in Practice 2009, 2:11-21.
81. Rosenfield S.A., Gravois T.A. Instructional consultation teams: Collaborating for change 1996, Guilford Press, New York, NY.
82. Sanetti L.M.H., Gritter K.L., Dobey L.M. Treatment integrity of interventions with children in the school psychology literature from 1995 to 2008. School Psychology Review 2011, 40:72-84.
83. Shadish W.R., Cook T.D., Campbell D.T. Experimental and quasi-experimental designs for generalized causal inference 2002, Houghton Mifflin Company, Boston, MA.
84. Shriberg D. The school psychologist as leader and change agent in a high-stakes era. Journal of Applied School Psychology 2007, 23:151-166.
85. Sterling-Turner H.E., Watson T.S., Moore J.W. The effects of direct training and treatment integrity on treatment outcomes in school consultation. School Psychology Quarterly 2002, 17:47-77.
86. Sugai G., Horner R.H. A promising approach for expanding and sustaining school-wide positive behavior support. School Psychology Review 2006, 35:245-259.
87. Sugai G., Horner R.H. Defining and describing schoolwide positive behavior support. Handbook of positive behavior support 2009, 307-326. Springer, New York, NY. W. Sailor, G. Dunlap, G. Sugai, R. Horner (Eds.).
88. Sugai G., Horner R.H., Lewis-Palmer T. The team implementation checklist (v. 3.0) 2009, Unpublished instrument, Educational and Community Supports, University of Oregon, Eugene, Oregon.
89. Telzrow C.F., McNamara K., Hollinger C.L. Fidelity of problem-solving implementation and relationship to student performance. School Psychology Review 2000, 29:443-461.
90. Tilly W.D. The evolution of school psychology to a science-based practice: Problem solving and the three-tiered model. Best practices in school psychology V 2008, 17-36. National Association of School Psychologists, Bethesda, MD. J. Grimes, A. Thomas (Eds.).
91. Todd A.W., Horner R.H., Newton J.S., Algozzine R.F., Algozzine K.M., Frank J.L. Effects of team-initiated problem solving on decision making by schoolwide behavior support teams. Journal of Applied School Psychology 2011, 27:42-59.
92. Todd A.W., Newton J.S., Horner R.H., Algozzine B., Algozzine K.M. Decision observation, recording, and analysis (DORA) training manual. Educational and Community Supports 2009, University of Oregon, Eugene, Oregon.
93. Torgerson D.J., Torgerson C.J. Designing randomised trials in health, education, and the social sciences: An introduction 2008, Palmgrave Macmillan, New York, NY.
94. Vollmer T.R., Roane H.S., Ringdahl J.E., Marcus B.A. Evaluating treatment challenges with differential reinforcement of alternative behavior. Journal of Applied Behavior Analysis 1999, 32:9-23.
95. Wilder D.A., Atwell J., Wine B. The effects of varying levels of treatment integrity on child compliance during treatment with a three-step prompting procedure. Journal of Applied Behavior Analysis 2006, 39:369-373.
96. Ysseldyke J., Algozzine B. Effective assessment for students with special needs: A practical guide for every teacher 2006, Corwin, Thousand Oaks, CA.
97. Ysseldyke J., Burns M.K., Rosenfield S. Blueprints on the future of training and practice in school psychology: What do they say about educational and psychological consultation?. Journal of Educational and Psychological Consultation 2009, 19:177-196.