Extensions of a versatile randomization test for assessing single-case intervention effects

Journal of School Psychology

Volumn 49, Issue 1, 2011, Pages 55-79

  Levin, Joel R ^a   Lall, Venessa F ^b   Kratochwill, Thomas R ^c  

^a UNIVERSITY OF ARIZONA   (United States)

^b EDUCATIONAL TESTING SERVICE   (United States)

^c UNIVERSITY OF WISCONSIN MADISON   (United States)

Author keywords

Randomization tests; Single case intervention research; Statistical analysis

Indexed keywords

ARTICLE; CHILD; EDUCATION; HUMAN; METHODOLOGY; PSYCHOLOGICAL MODEL; PSYCHOLOGY; RANDOMIZED CONTROLLED TRIAL (TOPIC); SAMPLE SIZE; STANDARD; STATISTICAL ANALYSIS; STATISTICS;

CHILD; DATA INTERPRETATION, STATISTICAL; EDUCATIONAL MEASUREMENT; HUMANS; MODELS, PSYCHOLOGICAL; PSYCHOLOGY, EDUCATIONAL; RANDOMIZED CONTROLLED TRIALS AS TOPIC; SAMPLE SIZE;

EID: 78650819859     PISSN: 00224405     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jsp.2010.09.002     Document Type: Article

References (56)

1. Baldwin S.A., Murray D.M., Shadish W.R. Empirically supported treatments or type I errors?: Problems with the analysis of data from group-administered treatments. Journal of Consulting and Clinical Psychology 2005, 73:267-285.
2. Borckardt J.J., Nash M.R., Murphy M.D., Moore M., Shaw D., O'Neil P. Clinical practice as natural laboratory for psychotherapy research: A guide to case-based time-series analysis. The American Psychologist 2008, 63:77-95.
3. Bradley J.V. Robustness?. The British Journal of Mathematical and Statistical Psychology 1978, 32:144-152.
4. Busk P.L., Serlin R.C. Meta-analysis for single-case research. Single-case research design and analysis 1992, 187-212. Erlbaum, Hillsdale, NJ. T.R. Kratochwill, J.R. Levin (Eds.).
5. 2 test of goodness of fit. Annals of Mathematical Statistics 1952, 23:315-345.
6. Cohen J. Statistical power analysis for the behavioral sciences 1988, Erlbaum, Hillsdale, NJ. Rev. ed.
7. Edgington E.S. Randomization tests for one-subject operant experiments. The Journal of Psychology 1975, 90:57-68.
8. Edgington E.S., Onghena P. Randomization tests 2007, Chapman & Hall/CRC, Boca Raton, FL. 4th ed.
9. Ferron J., Jones P.K. Tests for the visual analysis of response-guided multiple-baseline data. Journal of Experimental Education 2006, 75:66-81.
10. Ferron J., Sentovich C. Statistical power of randomization tests used with multiple-baseline designs. Journal of Experimental Education 2002, 70:165-178.
11. Ferron J.M., Ware W.B. Analyzing single-case data: The power of randomization tests. Journal of Experimental Education 1995, 63:167-178.
12. Fisher W.W., Kelley M.E., Lomas J.E. Visual aids and structured criteria for improving visual inspection and interpretation of single-case designs. Journal of Applied Behavior Analysis 2003, 36:387-406.
13. Design and analysis of single-case research 1997, Erlbaum, Mahwah, NJ. R.D. Franklin, D.B. Allison, B.S. Gorman (Eds.).
14. Gettinger M. Effects of invented spelling and direct instruction approaches on spelling performance of second-grade boys. Journal of Applied Behavior Analysis 1993, 26:281-291.
15. Glass G.V., Willson V.L., Gottman J.M. Design and analysis of time series experiments 1975, University of Colorado Press, Boulder, CO.
16. Hersen M., Barlow D.H. Single-case experimental designs: Strategies for studying behavior change 1976, Pergamon, New York.
17. Holm S. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 1979, 6:65-70.
18. Huitema B.E. Autocorrelation in applied behavior analysis: A myth. Behavioral Assessment 1985, 7:107-118.
19. Huitema B.E. Autocorrelation: 10years of confusion. Behavioral Assessment 1988, 10:253-297.
20. Jones R.R., Weinrott M.R., Vaught R.S. Effects of serial dependency on the agreement between visual and statistical inference. Journal of Applied Behavior Analysis 1978, 11:277-283.
21. Kazdin A.E. Single-case research designs: Methods for clinical and applied settings 2011, Oxford University Press, New York. 2nd edition.
22. Keppel G., Wickens T.D. Design and analysis: A researcher's handbook 2004, Pearson Prentice Hall, Upper Saddle River, NJ. 4th ed.
23. Kirk R.E. Experimental design: Procedures for the behavioral sciences 1995, Brooks/Cole, Pacific Grove, CA. 3rd ed.
24. Single subject research: Strategies for evaluating change 1978, Academic Press, New York. T.R. Kratochwill (Ed.).
25. Kratochwill T.R. Preparing psychologists for evidence-based school practice: Lessons learned and challenges ahead. The American Psychologist 2007, 62:826-843.
26. Kratochwill T.R., Hitchcock J., Horner R.H., Levin J.R., Odom S.L., Rindskopf D.M., et al. Single case designs technical documentation. What Works Clearinghouse: Procedures and Standards Handbook (Version 2.0) 2010, Retrievable from the What Works Clearinghouse website: http://ies.ed.gov/ncee/wwc/pdf/wwc_procedures_v2_standards_handbook.pdf.
27. Single-case research design and analysis: New directions for psychology and education 1992, Erlbaum, Hillsdale, NJ. T.R. Kratochwill, J.R. Levin (Eds.).
28. Kratochwill T.R., Levin J.R. Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue. Psychological Methods 2010, 15:122-144.
29. Kratochwill, T. R., Levin, J. R., & Swoboda, C. (2010). Visual analysis of single-case research: Methodological and statistical considerations. Unpublished manuscript, University of Wisconsin - Madison.
30. Kratochwill T.R., McDonald L., Levin J.R., Young Bear-Tibbetts H., Demaray M.K. Families and schools together: An experimental analysis of parent-mediated multi-family group program for American Indian children. Journal of School Psychology 2004, 42:359-383.
31. Lall V.F., Levin J.R. An empirical investigation of the statistical properties of generalized single-case randomization tests. Journal of School Psychology 2004, 42:61-86.
32. Levin J.R. Some methodological and statistical "bugs" in research on children's learning. Cognitive learning and memory in children 1985, 205-233. Springer-Verlag, New York. M. Pressley, C.J. Brainerd (Eds.).
33. Levin J.R. Single-case research design and analysis: Comments and concerns. Single-case research design and analysis: New directions for psychology and education 1992, 213-224. Erlbaum, Hillsdale, NJ. T.R. Kratochwill, J.R. Levin (Eds.).
34. Levin J.R. Crafting educational intervention research that's both credible and creditable. Educational Psychology Review 1994, 6:231-243.
35. Levin J.R. Randomized classroom trials on trial. Empirical methods for evaluating educational interventions 2005, 3-27. Elsevier Academic Press, San Diego. G.D. Phye, D.H. Robinson, J.R. Levin (Eds.).
36. Levin J.R. Randomization tests: Statistical tools for assessing the effects of educational interventions when resources are scarce. Real data analysis 2007, 115-123. Information Age, Greenwich, CT. S. Sawilowsky (Ed.).
37. Levin, J. R., O'Donnell, A. M., & Kratochwill, T. R. (2003). Educational/psychological intervention research. In I. B. Weiner (Series Ed.) & W. M. Reynolds & G. E. Miller (Vol. Eds.). Handbook of psychology: Vol. 7. Educational psychology (pp. 557-581). New York: Wiley.
38. Levin J.R., Serlin R.C., Seaman M.A. A controlled, powerful multiple-comparison strategy for several situations. Psychological Bulletin 1994, 115:153-159.
39. Levin J.R., Wampold B.E. Generalized single-case randomization tests: Flexible analyses for a variety of situations. School Psychology Quarterly 1999, 14:59-93.
40. Lewka A., Ysseldyke J. Dissertation research in school psychology: Changes in topics and methodology over the past 25years. Journal of Applied School Psychology 2010, 26:17-37.
41. Marascuilo L.A., Busk P.L. Combining statistics for multiple-baseline AB and replicated ABAB designs across subjects. Behavioral Assessment 1988, 10:1-28.
42. Marquis J.G., Horner R.H., Carr E.G., Turnbull A.P., Thompson M., Behrens G.A., et al. A meta-analysis of positive behavior support. Contemporary special education research: Syntheses of knowledge base on critical instructional issues 2000, 137-178. Erlbaum, Mahwah, NJ. R. Gersten, E.P. Schiller, S. Vaughn (Eds.).
43. Maxwell S.E., Delaney H.D. Designing experiments and analyzing data: A model comparison perspective 2004, Erlbaum, Mahwah, NJ. 2nd ed.
44. Maytas T.A., Greenwood K.M. Visual analysis of single-case time series: Effects of variability, serial dependence, and magnitude of intervention effects. Journal of Applied Behavior Analysis 1990, 23:341-351.
45. Maytas T.A., Greenwood K.M. Problems in the estimation of autocorrelation in brief time series and some implications for behavioral data. Behavioral Assessment 1991, 13:137-157.
46. McAllister L.W., Stachowiak J.G., Baer D.M., Conderman L. The application of operant conditioning techniques in a secondary school classroom. Journal of Applied Behavior Analysis 1969, 2:277-285.
47. Miller P.M. An experimental analysis of retention control training in the treatment of nocturnal enuresis in two institutional adolescents. Behavior Therapy 1973, 4:288-294.
48. Rogers L.A., Graham S. A meta-analysis of single subject design writing intervention research. Journal of Educational Psychology 2008, 100:879-906.
49. SAS/IML software: Usage and reference 1990, SAS Institute, Inc, (Version 6, 1st ed.), SAS Institute, Cary, NC.
50. SAS/IML software (Version 9.1) 2004, SAS Institute, Inc, SAS Institute, Cary, NC.
51. Seaman M.A., Levin J.R., Serlin R.C. New developments in pairwise multiple comparisons: Some powerful and practicable procedures. Psychological Bulletin 1991, 110:577-586.
52. Serlin R.C. Testing for robustness in Monte Carlo studies. Psychological Methods 2000, 5:230-240.
53. Shadish W.R., Cook T.D., Campbell D.T. Experimental and quasi-experimental designs for generalized causal inference 2002, Houghton Mifflin, Boston, NY.
54. Shaffer J.P. Modified sequentially rejective multiple test procedures. Journal of the American Statistical Association 1986, 81:826-831.
55. Todman J.B., Dugard P. Single-case and small-n experimental designs: A practical guide to randomizations tests 2001, Erlbaum, New York.
56. Woodfield T.J. Simulating stationary Gaussian ARMA time series. Computer Science and Statistics: Proceedings of the 20th Symposium on the Interface 1988, 612-617. American Statistical Association, Alexandria, VA.