Single-case experimental designs: Reflections on conduct and analysis

Neuropsychological Rehabilitation

Volumn 24, Issue 3-4, 2014, Pages 634-660

  Manolov, Rumen ^a,b   Gast, David L ^c   Perdices, Michael ^d,e   Evans, Jonathan J ^f  

^a UNIVERSITY OF BARCELONA   (Spain)

^b ESADE BUSINESS SCHOOL   (Spain)

^c UNIVERSITY OF GEORGIA   (United States)

^d ROYAL NORTH SHORE HOSPITAL   (Australia)

^e UNIVERSITY OF SYDNEY   (Australia)

^f UNIVERSITY OF GLASGOW   (United Kingdom)

Author keywords

Analysis; Methodology; Single case designs; Software

Indexed keywords

COMPUTER PROGRAM; HUMAN; METHODOLOGY; PROCEDURES; STANDARDS; STATISTICS; STATISTICS AND NUMERICAL DATA;

HUMANS; RESEARCH DESIGN; SOFTWARE; STATISTICS AS TOPIC;

EID: 84903191953     PISSN: 09602011     EISSN: 14640694     Source Type: Journal    
DOI: 10.1080/09602011.2014.903199     Document Type: Article

References (113)

1. APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61, 271-285.
2. Baek, E., Moeyaert, M., Petit-Bois, M., Beretvas, S. N., Van de Noortgate, W., & Ferron, J. (2014). The use of multilevel analysis for integrating single-case experimental design results within a study and across studies. Neuropsychological Rehabilitation, 24. doi: 10. 1080/09602011.2013. 835740.
3. Barlow, D. H., Nock, M. K., & Hersen, M. (2009). Single case experimental designs: Strategies for studying behavior change (3rd ed.). Boston, MA: Pearson.
4. Bennett, K. D., Ramasamy, R., & Honsberger, T. (2013). The effect of covert audio coaching on teaching clerical skills to adolescents with autism spectrum disorder. Journal of Autism and Developmental Disorders, 43, 585-593.
5. Beretvas, S. N., & Chung, H. (2008). A review of meta-analyses of single-subject experimental designs: Methodological issues and practice. Evidence-Based Communication Assessment and Intervention, 2, 129-141.
6. Borckardt, J., & Nash, M. (2014). Simulation modelling analysis for small sets of single-subject data collected over time. Neuropsychological Rehabilitation, 24. doi: 10.1080/09602011. 2014.895390.
7. Borenstein, M. (2009). Effect sizes for continuous data. In H. Cooper, L. V. Hedges, & J. C. Valentine (Eds.), The handbook of research synthesis and meta-analysis (2nd ed., pp. 221-235). New York, NY: The Russell Sage Foundation.
8. Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. Chichester: John Wiley & Sons.
9. Brossart, D. F., Vannest, K., Davis, J., & Patience, M. (2014). Incorporating nonoverlap indices with visual analysis for quantifying intervention effectiveness insingle-case experimental designs. Neuropsychological Rehabilitation, 24. doi: 10.1080/09602011.2013.868361.
10. Bulte, I. (2013). Being grateful for small mercies: The analysis of single-case data (Unpublished doctoral dissertation). University of Leuven, Belgium.
11. Bulte, I., & Onghena, P. (2008). An R package for single-case randomization tests. Behavior Research Methods, 40, 467-478.
12. Bulte, I., & Onghena, P. (2009). Randomization tests for multiple-baseline designs: An extension of the SCRT-R package. Behavior Research Methods, 41, 477-485.
13. Bulte, I., & Onghena, P. (2012). When the truth hits you between the eyes: A software tool for the visual analysis of single-case experimental data. Methodology, 8, 104-114.
14. Busse, R. T., Kratochwill, T. R., & Elliott, S. N. (1995). Meta-analysis for single-case consultation outcomes: Applications to research and practice. Journal of School Psychology, 33, 269-285.
15. Campbell, J. M. (2003). Efficacy of behavioral interventions for reducing problem behavior in persons with autism: A quantitative synthesis of single-subject research. Research in Developmental Disabilities, 24, 120-138.
16. Cohen, J. (1988). Statistical power analysis for the behavioural sciences. Hillsdale, NJ: Lawrence Erlbaum.
17. Cohen, J. (1994). The Earth is round (p, .05). American Psychologist, 49, 997-1003.
18. Cooper, H. (2010). Research synthesis and meta-analysis (4th ed., Vol. 2). London: Sage.
19. Cortina, J. M., & Landis, R. S. (2011). The Earth is not round (p 1/4 .00). Organizational Research Methods, 14, 332-349.
20. Crawford, J. R., & Garthwaite, P. H. (2005). Testing for suspected impairments and dissociations in single-case studies in neuropsychology: Evaluation of alternatives using Monte Carlo simulations and revised tests for dissociations. Neuropsychology, 19, 318-331.
21. Crawford, J. R., & Garthwaite, P. H. (2012). Single-case research in neuropsychology: A comparison of five forms of t-test for comparing a case to controls. Cortex, 48, 1009-1016.
22. Crawford, J. R., Garthwaite, P. H., & Porter, S. (2010). Point and interval estimates of effect sizes for the case-controls design in neuropsychology: Rationale, methods, implementations, and proposed reporting standards. Cognitive Neuropsychology, 27, 245-260.
23. Crawford, J. R., & Howell, D. C. (1998). Comparing an individual's test score against norms derived from small samples. The Clinical Neuropsychologist, 12, 482-486.
24. Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. London: Routledge.
25. Davis, D. H., Gagne, P., Fredrick, L. D., Alberto, P. A., Waugh, R. E., & Haardorfer, R. (2013). Augmenting visual analysis in single-case research with hierarchical linear modeling. Behavior Modification, 37, 62-89.
26. de Vries, R. M., & Morey, R. D. (2013). Bayesian hypothesis testing for single-subject designs. Psychological Methods, 18, 165-185.
27. Edgington, E. S. (1972). An additive method for combining probability values from independent experiments. Journal of Psychology, 80, 351-363.
28. Fahmie, T. A., & Hanley, G. P. (2008). Progressing toward data intimacy: A review of withinsession data analysis. Journal of Applied Behavior Analysis, 41, 319-331.
29. Faith, M. S., Allison, D. B., & Gorman, B. S. (1996). Meta-analysis of single-case research. In R. D. Franklin, D. B. Allison, & B. S. Gorman (Eds.), Design and analysis of single-case research (pp. 245-277). Mahwah, NJ: Lawrence Erlbaum.
30. Fisch, G. S. (2001). Evaluating data from behavioral analysis: Visual inspection or statistical models? Behavioural Processes, 54, 137-154.
31. Franklin, R. D., Gorman, B. S., Beasley, T. M., & Allison, D. B. (1996). Graphical display and visual analysis. In R. D. Franklin, D. B. Allison, & B. S. Gorman (Eds.), Design and analysis of single-case research (pp. 119-158). Mahwah, NJ: Lawrence Erlbaum.
32. Gage, N. A., & Lewis, T. J. (2012). Hierarchical linear modeling meta-analysis of single-subject design research. The Journal of Special Education, 48, 3-16.
33. Ganz, J. B., Earles-Vollrath, T. L., Heath, A. K., Parker, R. I., Rispoli, M. J., & Duran, J. B. (2012). A meta-analysis of single case research studies on aided augmentative and alternative communication systems with individuals with autism spectrum disorders. Journal of Autism and Developmental Disorders, 42, 60-74.
34. Gast, D. L. (2010a). Replication. In D. L. Gast (Ed.), Single subject research methodology in behavioral sciences (pp. 110-128). London: Routledge.
35. Gast, D. L. (2010b). Single subject research methodology in behavioral sciences. London: Routledge.
36. Gast, D. L., & Ledford, J. (2010). Multiple baseline and multiple probe designs. In D. L. Gast (Ed.), Single subject research methodology in behavioral sciences (pp. 276-328). London: Routledge.
37. Gast, D. L., & Spriggs, A. D. (2010). Visual analysis of graphic data. In D. L. Gast (Ed.), Single subject research methodology in behavioral sciences (pp. 199-233). London: Routledge.
38. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. New York, NY: Academic Press.
39. Heyvaert, M., & Onghena, P. (2014). Analysis of single-case data: Randomisation tests for measures of effect size. Neuropsychological Rehabilitation, 24. doi: 10.1080/09602011. 2013.818564.
40. Holden, G., Bearison, D. J., Rode, D. C., Rosenberg, G., & Fishman, M. (1999). Evaluating the effects of a virtual environment (STARBRIGHT world) with hospitalized children. Research on Social Work Practice, 9, 365-382.
41. Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165-179.
42. Howick, J., Chalmers, I., Glasziou, P., Greenhaigh, T., Heneghan, C., Liberati, A., Thornton, H. (2011). The 2011 Oxford CEBM Evidence Table (Introductory Document). Oxford: Oxford Centre for Evidence-Based Medicine. Retrieved from http://www.cebm.net/index.aspx?o=5653.
43. Huitema, B. E., & McKean, J. W. (1991). Autocorrelation estimation and inference with small samples. Psychological Bulletin, 110, 291-304.
44. Institute of Education Sciences. (2013). Request for applications: Statistical and research methodology in education. Retrieved from http://ies.ed.gov/funding/pdf/2014-84305D.pdf.
45. Jamieson, M., Cullen, B., McGee-Lennon, M., Brewster, S., & Evans, J. J. (2014). The efficacy of cognitive prosthetic technology for people with memory impairments: A systematic review and meta-analysis. Neuropsychological Rehabilitation, 24. doi: 10.1080/09602011. 2013.825632.
46. Jarrett, M. A., & Ollendick, T. H. (2012). Treatment of comorbid attention-deficit/hyperactivity disorder and anxiety in children: A multiple baseline design analysis. Journal of Consulting and Clinical Psychology, 80, 239-244.
47. Jenson, W. R., Clark, E., Kircher, J. C., & Kristjansson, S. D. (2007). Statistical reform: Evidence-based practice, meta-analyses, and single subject designs. Psychology in the Schools, 44, 483-493.
48. Johnston, J. M., & Pennypacker, H. S. (2009). Strategies and tactics of behavioral research (3rd ed.). New York, NY: Routledge.
49. Jones, L. V., & Fiske, D. W. (1953). Models for testing the significance of combined results. Psychological Bulletin, 50, 375-382.
50. Kazdin, A. E. (1978). Methodological and interpretive problems of single-case experimental designs. Journal of Consulting and Clinical Psychology, 46, 629-642.
51. Kazdin, A. E. (2001). Behavior modification in applied settings (6th ed.). Belmont, CA: Wadsworth.
52. Kazdin, A. E. (2011). Single-case research designs: Methods for clinical and applied settings (2nd ed.). Oxford: Oxford University Press.
53. Kennedy, C. H. (2005). Single-case designs for educational research. Boston, MA: Allyn & Bacon.
54. Kratochwill, T. R., Hitchcock, J. H., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single-case designs technical documentation. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc-scd. pdf.
55. Kratochwill, T. R., Hitchcock, J. H., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2013). Single-case intervention research design standards. Remedial and Special Education, 34, 26-38.
56. Kratochwill, T. R., & Levin, J. R. (2010). Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue. Psychological Methods, 15, 124-144.
57. Lane, J., & Gast, D. L. (2014). Visual analysis in single case experimental design studies: Brief review and guidelines. Neuropsychological Rehabilitation, 24. doi:10.1080/09602011.2013. 815636.
58. Ledford, J., & Gast, D. L. (2014). Measuring procedural fidelity in behavioural research. Neuropsychological Rehabilitation, 24. doi: 10.1080/09602011.2013.861352.
59. Lieberman, R. G., Yoder, P. J., Reichow, B., & Wolery, M. (2010). Visual analysis of multiple baseline across participants graphs when change is delayed. School Psychology Quarterly, 25, 28-44.
60. Lipsey, M. W., & Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.
61. Maggin, D. M., & Chafouleas, S. M. (2013). Introduction to the special series: Issues and advance of synthesizing single-case research. Remedial and Special Education, 34, 3-8.
62. Maggin, D. M., Swaminathan, H., Rogers, H. J., O'Keefe, B. V., Sugai, G., & Horner, R. H. (2011). A generalized least squares regression approach for computing effect sizes in single-case research: Application examples. Journal of School Psychology, 49, 301-321.
63. Manolov, R., Guilera, G., & Sierra, V. (2014). Weighting strategies in the meta-analysis of single-case studies. Behavior Research Methods. Advance online publication. doi:10. 3758/s13428-013-0440-0.
64. Manolov, R., & Solanas, A. (2009). Percentage of nonoverlapping corrected data. Behavior Research Methods, 41, 1262-1271.
65. Manolov, R., & Solanas, A. (2012). Assigning and combining probabilities in single-case studies. Psychological Methods, 17, 495-509.
66. Manolov, R., & Solanas, A. (2013). A comparison of mean phase difference and generalized least squares for analyzing single-case data. Journal of School Psychology, 51, 201-215.
67. Matyas, T. A., & Greenwood, K. M. (1990). Visual analysis for single-case time series: Effects of variability, serial dependence, and magnitude of intervention effects. Journal of Applied Behavior Analysis, 23, 341-351.
68. McGrath, R. E., & Meyer, G. J. (2006). When effect sizes disagree: The case of r and d. Psychological Methods, 11, 386-401.
69. Michael, J. (1974). Statistical inference for individual organism research: Mixed blessing or curse? Journal of Applied Behavior Analysis, 7, 647-653.
70. Miller, M. J. (1985). Analyzing client change graphically. Journal of Counseling and Development, 63, 491-494.
71. O'Neill, B., & Findlay, G. (2014). Single case experimental designs in neurobehavioural rehabilitation: Preliminary findings on biofeedback in the treatment of challenging behaviour. Neuropsychological Rehabilitation, 24. doi:10.1080/09602011.2014.903199.
72. Parker, R. I., Brossart, D. F., Vannest, K. J., Long, J. R., Garcia De-Alba, R., Baugh, F. G., & Sullivan, J. R. (2005). Effect sizes in single case research: How large is large? School Psychology Review, 34, 116-132.
73. Parker, R. I., Cryer, J., & Byrns, G. (2006). Controlling baseline trend in single-case research. School Psychology Quarterly, 21, 418-443.
74. Parker, R. I., & Hagan-Burke, S. (2007). Useful effect size interpretations for single case research. Behavior Therapy, 38, 95-105.
75. Parker, R. I., & Vannest, K. J. (2009). An improved effect size for single-case research: Nonoverlap of all pairs. Behavior Therapy, 40, 357-367.
76. Parker, R. I., & Vannest, K. J. (2012). Bottom-up analysis of single-case research designs. Journal of Behavioral Education, 21, 254-265.
77. Parker, R. I., Vannest, K. J., & Brown, L. (2009). The improvement rate difference for singlecase research. Exceptional Children, 75, 135-150.
78. Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S. B. (2011). Combining nonoverlap and trend for single-case research: Tau-U. Behavior Therapy, 42, 284-299.
79. Petersen-Brown, S., Karich, A. C., & Symons, F. J. (2012). Examining estimates of effect using Non-overlap of all pairs in multiple baseline studies of academic intervention. Journal of Behavioral Education, 21, 203-216.
80. Rasquin, S. M. C., Van De Sande, P., Praamstra, A. J., & Van Heugten, C. M. (2009). Cognitivebehavioral intervention for depression after stroke: Five single case studies on effects and feasibility. Neuropsychological Rehabilitation, 19, 208-222.
81. R Core Team. (2013). R: A language and environment for statistical computing. Vienna: R Foundation for Statistical Computing. Retrieved from http://www.R-project.org/.
82. Rindskopf, D. M. (2014). Bayesian analysis of data from single case designs. Neuropsychological Rehabilitation, 24. doi:10.1080/09602011.2013. 866903.
83. Robey, R. R., Schultz, M. C., Crawford, A. B., & Sinner, C. A. (1999). Single-subject clinical outcome research: Designs, data, effect sizes, and analysis. Aphasiology, 13, 445-473.
84. Rogers, H. J., & Swaminathan, H. (2007). A computer program for the statistical analysis of single subject designs. Storrs, CT: University of Connecticut, Measurement, Evaluation, and Assessment Program.
85. Ruscio, J. (2008). A probability-based measure of effect size: Robustness to base rates and other factors. Psychological Methods, 13, 19-30.
86. Schlosser, R. W., Lee, D. L., & Wendt, O. (2008). Application of the percentage of non-overlapping data (PND) in systematic reviews and meta-analyses: A systematic review of reporting characteristics. Evidence-Based Communication Assessment and Intervention, 2, 163-187.
87. Schlosser, R. W., & Sigafoos, J. (2008). Meta-analysis of single-subject experimental designs: Why now? Evidence-Based Communication Assessment and Intervention, 2, 117-119.
88. Scruggs, T. E., & Mastropieri, M. A. (2013). PND at 25: Past, present, and future trends in summarizing single-subject research. Remedial and Special Education, 34, 9-19.
89. Shadish, W. R., Brasil, I. C. C., Illingworth, D. A., White, K. D., Galindo, R., Nagler, E. D., & Rindskopf, D. M. (2009). Using UnGraph to extract data from image files: Verification of reliability and validity. Behavior Research Methods, 41, 177-183.
90. Shadish, W. R, Hedges, L. V., & Pustejovsky, J. E. (2013). Analysis and meta-analysis of single-case designs with a standardized mean difference statistic: A primer and applications. Journal of School Psychology, 52, 123-147.
91. Shadish, W. R, Hedges, L. V, Pustejovsky, J. E, Boyajian, J. G., Sullivan, K. J., Andrade, A, & Barrientos, J. L. (2014). A d-statistic for single-case designs that is equivalent to the usual between-groups d-statistic. Neuropsychological Rehabilitation, 24. 10.1080/09602011. 2013.819021.
92. Shadish, W. R., Kyse, E. N., & Rindskopf, D. M. (2013). Analyzing data from single-case designs using multilevel models: New applications and some agenda items for future research. Psychological Methods, 18, 385-405.
93. Shadish, W. R., & Sullivan, K. J. (2011). Characteristics of single-case designs used to assess intervention effects in 2008. Behavior Research Methods, 43, 971-980.
94. Sidman, M. (1960). Tactics of scientific research: Evaluating experimental data in psychology. New York, NY: Basic Books.
95. Skinner, B. F. (1938). The behavior of organisms: An experimental analysis. New York, NY: Appleton-Century.
96. Smith, J. D. (2012). Single-case experimental designs: A systematic review of published research and current standards. Psychological Methods, 17, 510-550.
97. Solanas, A., Manolov, R., & Onghena, P. (2010). Estimating slope and level change in N1/41 designs. Behavior Modification, 34, 195-218.
98. Solanas, A., Manolov, R., & Sierra, V. (2010). Lag-one autocorrelation in short series: Estimation and hypothesis testing. Psicologica, 31, 357-381.
99. Solmi, F., & Onghena, P. (2014). Combining p-values in replicated single-case experiments with multivariate outcome. Neuropsychological Rehabilitation, 24. doi:10.1080/09602011.2014.881747.
100. Strube, M. J. (1985). Combining and comparing significance levels form nonindependent hypothesis tests. Psychological Bulletin, 97, 334-341.
101. Svanberg, J., & Evans, J. J. (2014). Impact of SenseCam on memory, identity and mood in Korsakoff's syndrome: A single case experimental design study. Neuropsychological Rehabilitation, 24. doi: 10.1080/09602011.2013.814573.
102. Swaminathan, H., Rogers, H. J., Horner, R., Sugai, G., & Smolkowski, K. (2014). Regression models for the analysis of single case designs. Neuropsychological Rehabilitation, 24. doi: 10.1080/09602011.2014.887586.
103. Tate, R. L., Perdices, M., McDonald, S., Togher, L., & Rosenkoetter, U. (2014). The conduct and report of single-case research: Strategies to improve the quality of the neurorehabilitation literature. Neuropsychological Rehabilitation, 24. doi: 10.1080/09602011.2013.875043.
104. Tate, R. L., Perdices, M., Rosenkoetter, U., Wakima, D., Godbee, K., Togher, L., & McDonald, S. (2013). Revision of a method quality rating scale for single-case experimental designs and n-of-1 trials: The 15-item Risk of Bias in N-of-1 Trials (RoBiNT) Scale. Neuropsychological Rehabilitation, 23, 619-638.
105. Tate, R. L., Togher, L., Perdices, M., McDonald, S., & Rosenkoetter, U. (2012, July). Developing reporting guidelines for single-case experimental designs: The SCRIBE project. Paper presented at the 9th Conference of the Neuropsychological Rehabilitation Special Interest Group of the World Federation for Neurorehabilitation, Bergen, Norway.
106. Tukey, J. W. (1977). Exploratory data analysis. London: Addison-Wesley.
107. Vannest, K. J., Parker, R. I., & Gonen, O. (2011). Single case research: web based calculators for SCR analysis (Version 1.0) [Web-based application]. College Station, TX: Texas A&M University. Retrieved from http://singlecaseresearch.org.
108. Wampold, B. E., & Furlong, M. J. (1981). Randomization tests in single-subject designs: Illustrative examples. Journal of Behavioral Assessment, 3, 329-341.
109. White, O. R. (1972). The split-middle: A quickie method of trend analysis. Eugene, OR: Regional Resource Center for Handicapped Children.
110. Whitlock, M. C. (2005). Combining probability from independent tests: The weighted Z-method is superior to Fisher's approach. Journal of Evolutionary Biology, 18, 1368-1373.
111. Wilkinson, L., & The Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 694-704.
112. Winkens, I., Ponds, R., Pouwels-van den Nieuwenhof, C., Eilander, H., & van Heugten, C. (2014). Using single-case experimental design methodology to evaluate the effects of the ABC method for nursing staff on verbal aggressive behaviour after acquired brain injury. Neuropsychological Rehabilitation, 24. doi: 10.1080/09602011.2014.901229.
113. Wolery, M., Busick, M., Reichow, B., & Barton, E. E. (2010). Comparison of overlap methods for quantitatively synthesizing single-subject data. The Journal of Special Education, 44, 18-29.