Quality appraisal of single-subject experimental designs: An overview and comparison of different appraisal tools

Education and Treatment of Children

Volumn 35, Issue 2, 2012, Pages 235-268

  Wendt, Oliver ^a   Miller, Bridget ^a  

^a PURDUE UNIVERSITY   (United States)

Author keywords

Appraisal tool; Critical appraisal; Evidence based practice; Quality assessment; Single case design; Single subject experiment; Single subject research; Systematic review

Indexed keywords

EID: 84860434235     PISSN: 07488491     EISSN: 19348924     Source Type: Journal    
DOI: 10.1353/etc.2012.0010     Document Type: Article

References (46)

1. Agency for Healthcare Research and Quality (2009). Testimony on comparative efficacy research. Retrieved from http://www.ahrq.gov/about/nac/autsp_tnf.htm.
2. Agency for Healthcare Research and Quality (2012). "Best evidence" approaches. Retrieved from http://www.ahrq.gov/clinic/tp/bestevtp.htm.
3. American Psychological Association (2002). Criteria for evaluating treatment guidelines. American Psychologist, 57, 1052-1059.
4. Anderson, N. H. (2001). Empirical direction in design and analysis. Mahwah, NJ: Erlbaum.
5. Atkins, C. F., & Sampson, J. (2002, June). Critical appraisal guidelines for single case study research. Paper presented at the 10th European Conference of Information Systems (ECIS), Gdansk, Poland. Paper retrieved from http://is2.lse.ac.uk/asp/aspecis/20020011.pdf.
6. Backman, C. L., & Harris, S. R. (1999). Case studies, single-subject research, and N of 1 randomized trials. Comparisons and contrasts. American Journal of Physical Medicine and Rehabilitation, 78, 170-176.
7. Barlow, D. H., Nock, M. K., & Hersen, M. (2009). Single-case experimental designs: Strategies for studying behavior change (3rd ed.). Boston, MA: Allyn & Bacon.
8. Beeson, P. M., & Robey, R. R. (2006). Evaluating single-subject treatment research: Lessons learned from the aphasia literature. Neuropsychological Review, 16, 161-169.
9. Carr, J. E. (2005). Recommendations for reporting multiple-baseline designs across participants. Behavioral Interventions, 20, 219-224.
10. Chambless, D. L., Baker, M. J., Baucom, D. H., Beutler, L. E., Calhoun, K. S., Crits-Christoph, P., Woody, S. R. (1998). Update on empirically validated therapies, II. The Clinical Psychologist, 51, 3-16.
11. Cicchetti, D. V. (2011). On the reliability and accuracy of the Evaluative Method for identifying evidence-based practices in autism. In B. Reichow, P. Doehring, D. V. Cichetti, & F. R. Volkmar (Eds.), Evidence-based practices and treatments for children with autism (pp. 41-51). New York, NY: Springer.
12. Crombie, I. K. (1996). The pocket guide to critical appraisal. London, United Kingdom: BMJ Publishing Group.
13. Crozier, S., & Tincani, M., J. (2005). Using a modified social story to decrease disruptive behavior of a child with autism. Focus on Autism and Other Developmental Disabilities, 20, 150-157.
14. Di Noia, J., & Tripodi, T. (2008). Single-case design for clinical social workers (2nd ed.). Washington, DC: NASW Press.
15. Ganz, J. B., & Sigafoos, J. (2005). Self-monitoring: Are young adults with MR and autism able to utilize cognitive strategies independently? Education and Training in Developmental Disabilites, 40, 24-33.
16. Gast, D. L. (2010). Single subject research methodology in behavioral sciences. New York, NY: Routledge.
17. Gast, D. L., & Hammond, D. (2010). Withdrawal and reversal designs. In D. L. Gast (Ed.), Single subject research methodology in behavioral sciences (pp. 234-275). New York, NY: Routledge.
18. Gast, D. L., & Ledford, J. (2010). Multiple baseline and multiple probe designs. In D. L. Gast (Ed.), Single subject research methodology in behavioral sciences (pp. 276-328). New York, NY: Routledge.
19. Graham, S. (2005). Criteria for evidence-based practice in special education [special issue editorial]. Exceptional Children, 71(2), 135.
20. Hancock, G. R., & Mueller, R. O. (2010). The reviewer's guide to quantitative methods in the social sciences. New York, NY: Routledge.
21. Hootman, J. M., Driban, J. B., Sitler, M. R., Harris, K. P., & Cattano, N. M. (2011). Reliability and validity of three quality rating instruments for systematic reviews of observational studies. Research Synthesis Methods, 2, 110-118.
22. Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005). The use of single-subject research to identify evidencebased practice in special education. Exceptional Children, 71, 165-179.
23. Interagency Autism Coordinating Committee (2011). 2011 IACC Strategic Plan for Autism Spectrum Disorder Research. Retrieved from http://iacc.hhs.gov/strategic-plan/2011/index.shtml
24. Katrak, P., Bialocerkowski, A. E., Massy-Westropp, N., Kumar, S., & Grimmer, K. A. (2004). A systematic review of the content of critical appraisal tools. BMC Medical Research Methodology, 4:22. doi:10.1186/1471-2288-4-22
25. Kennedy, C. H. (2005). Single-case designs for educational research. Boston, MA: Allyn & Bacon.
26. Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W.R. (2010). Single-case designs technical documentation. Retrieved from What Works Clearinghouse website: http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf.
27. Logan, L. R., Hickman, R. R., Harris, S. R., & Heriza, C. B. (2008). Single-subject research design: Recommendations for levels of evidence and quality rating. Developmental Medicine & Child Neurology, 50, 99-103.
28. Ozdemir, S. (2008). The effectiveness of social stories on decreasing disruptive behavior on children with autism: Three case studies. Journal of Autism and Developmental Disorders, 38, 1689-1696.
29. Perdices, M., & Tate, R. L. (2009). Single-subject designs as a tool for evidence-based clinical practice: Are they unrecognized and undervalued? Neuropsychological Rehabilitation, 19, 904-927.
30. Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Malden, MA: Blackwell Publishing.
31. Reichow, B., Volkmar, F. R., & Cicchetti, D. V. (2008). Development of the evaluative method for evaluating and determining evidence-based practices in autism. Journal of Autism and Developmental Disorders, 38, 1311-1319.
32. Schlosser, R. W. (2003). The efficacy of augmentative and alternative communication: Toward evidence-based practice. San Diego, CA: Academic Press.
33. Schlosser, R. W. (2009). The role of single-subject experimental designs in evidence-based practice times. FOCUS, 22, 1-8.
34. Schlosser, R. W. (2011). EVIDAAC Single-Subject Scale. Retrieved October 19, 2011 from http://www.evidaac.com/ratings/Single_Sub_Scale.pdf.
35. Schlosser, R. W., & Sigafoos, J. (2008). Identifying "evidence-based practice" versus "empirically supported treatment." Evidence-Based Communication Assessment and Intervention, 2, 61-62.
36. Schlosser, R. W., Sigafoos, J., & Belfiore, P. (2009). EVIDAAC Comparative Single-Subject Experimental Design Scale (CSSEDARS). Retrieved from http://www.evidaac.com/ratings/CSSEDARS.pdf.
37. Smith, V., Jelen, M., & Patterson, S. (2010). Video Modeling to improve play skills in a child with autism: A procedure to examine single-subject experimental research. Evidence-based Practice Briefs, 4, 1-11.
38. Simeonsson, R., & Bailey, D. (1991). Evaluating programme impact: Levels of certainty. In D. Mitchell, & R. Brown (Eds.), Early intervention studies for young children with special needs (pp. 280-296). London, United Kingdom: Chapman and Hall.
39. Straus, S. E., Richardson, W. S., Glasziou, P., & Haynes, R. B. (2005). Evidence-based medicine: How to practice and teach EBM (3rd ed.). Edinburgh, United Kingdom: Elsevier Science.
40. Tankersley, M., Cook, B. G., & Cook, L. (2008). A preliminary examination to identify the presence of quality indicators in singlesubject research. Education and Treatment of Children, 31, 523-548.
41. Tate, R. L., McDonald, S., Perdices, M., Togher, L., Schultz, R., & Savage, S. (2008). Rating the methodological quality of singlesubject designs and n-of-1 trials: Introducing the single-case experimental design (SCED) scale. Neuropsychological Rehabilitation, 18, 385-401.
42. Tincani, M. (2004). Comparing the Picture Exchange Communication System and sign language training for children with autism. Focus on Autism and Other Developmental Disabilities, 19, 152-163.
43. United Kingdom Civil Service (2011). What is a Rapid Evidence Assessment? Retrieved from http://www.civilservice.gov.uk/networks/gsr/resources-and-guidance/rapid-evidence-assessment/what-is.
44. Wendt, O. (2006). Critically Appraised Topics: An approach to critical appraisal of evidence. Perspectives on Augmentative and Alternative Communication, 15, 24-26.
45. Wendt, O. (2007). The effectiveness of augmentative and alternative communication for individuals with autism spectrum disorders: A systematic review and meta-analysis. Dissertation Abstracts International, 68(2), 526-A.
46. Wendt, O. (2009). Research on the use of graphic symbols and manual signs. In P. Mirenda & T. Iacono (Eds.), Autism Spectrum Disorders and AAC (pp. 83-139). Baltimore, MD: Paul H. Brookes.