Estimating causal effects from multiple-baseline studies: Implications for design and analysis

Psychological Methods

Volumn 19, Issue 4, 2014, Pages 493-510

  Ferron, John M ^a   Moeyaert, Mariola ^b   Van den Noortgate, Wim ^b   Natasha Beretvas, S ^c  

^a UNIVERSITY OF SOUTH FLORIDA   (United States)

^b UNIVERSITY OF LEUVEN   (Belgium)

^c UNIVERSITY OF TEXAS AT AUSTIN   (United States)

Author keywords

Causal effect; Multilevel model; Multiple baseline; Randomization; Single case

Indexed keywords

EPIDEMIOLOGY; HUMAN; METHODOLOGY; MULTILEVEL ANALYSIS; RANDOMIZED CONTROLLED TRIAL (TOPIC); STATISTICS;

CAUSALITY; HUMANS; MULTILEVEL ANALYSIS; RANDOMIZED CONTROLLED TRIALS AS TOPIC; RESEARCH DESIGN; STATISTICS AS TOPIC;

EID: 84925441096     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0037038     Document Type: Article

References (56)

1. Baek, E., & Ferron, J. M. (2013). Multilevel models for multiple-baseline data: Modeling across participant variation in autocorrelation and residual variance. Behavior Research Methods, 45, 65-74. doi:10.3758/s13428-012-0231-z
2. Baer, D. M., Wolf, M. M., & Risley, T. R. (1968). Some current dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1, 91-97. doi:10.1901/jaba.1968.1-91
3. Barlow, D. H., Nock, M. K., & Hersen, M. (2009). 6Single case experimental design: Strategies for studying behavior change (3rd ed.). Boston, MA: Pearson.
4. Campbell, D. T., & Stanley, J. C. (1966). Experimental and quasiexperimental designs for research. Boston, MA: Houghton Mifflin.
5. Christ, T. J. (2007). Experimental control and threats to internal validity of concurrent and nonconcurrent multiple baseline designs. Psychology in the Schools, 44, 451-459. doi:10.1002/pits.20237
6. de Vries, R. M., & Morey, R. D. (2013). Bayesian hypothesis testing for single-subject design. Psychological Methods, 18, 165-185. doi: 10.1037/a0031037
7. Edgington, E. S. (1980). Validity of randomization tests for one-subject experiments. Journal of Educational Statistics, 5, 235-251. doi:10.2307/1164966
8. Edgington, E. S. (1996). Randomized single-subject experimental designs. Behaviour Research and Therapy, 34, 567-574. doi:10.1016/0005-7967(96)00012-5
9. Edgington, E. S., & Onghena, P. (2007). Randomization tests (4th ed.). Boca Raton, FL: Chapman & Hall.
10. Ferron, J. M., Bell, B. A., Hess, M. R., Rendina-Gobioff, G., & Hibbard, S. T. (2009). Making treatment effect inferences from multiple-baseline data: The utility of multilevel modeling approaches. Behavior Research Methods, 41, 372-384. doi:10.3758/BRM.41.2.372
11. Ferron, J., Farmer, J., & Owens, C. (2010). Estimating individual treatment effects from multiple-baseline data: A Monte Carlo study of multilevel modeling approaches. Behavior Research Methods, 42, 930-943. doi: 10.3758/BRM.42.4.930
12. Ferron, J., & Jones, P. K. (2006). Tests for the visual analysis of responseguided multiple-baseline data. Journal of Experimental Education, 75,66-81. doi:10.3200/JEXE.75.1.66-81
13. France, K. G., Blampied, N. M., & Wilkinson, P. (1999). A multiplebaseline, double-blind evaluation of the effects of trimeprazine tartrate on infant sleep disturbance, Experimental and Clinical Psychopharmacology, 7, 502-513. doi:10.1037/1064-1297.7.4.502
14. Hedges, L. V., Pustejovsky, J. E., & Shadish, W. R. (2012). A standardized mean difference effect size for single case designs. Research Synthesis Methods, 3, 224-239. doi:10.1002/jrsm.1052
15. Hembry, I., Bunuan, R., Beretvas, S. N., Ferron, J. M., & Van den Noortgate, W. (in press). Estimation of a nonlinear intervention phase trajectory for multiple-baseline design data. Journal of Experimental Education.
16. Hilgard, J. R. (1933). The effect of early and delayed practice on memory and motor performances studied by the method of co-twin control. Genetic Psychology Monographs, 14, 493-567.
17. Holland, P. W. (1986). Statistical and causal inference. Journal of the American Statistical Association, 81, 945-960. doi:10.1080/01621459.1986.10478354
18. Horner, R. H., Swaminathan, H., Sugai, G., & Smolkowski, K. (2012). Considerations for the systematic analysis and use of single-case research. Education & Treatment of Children, 35, 269-290. doi:10.1353/etc.2012.0011
19. Huitema, B. E., & McKean, J. W. (1998). Irrelevant autocorrelation in least-squares intervention models. Psychological Methods, 3, 104-116. doi:10.1037/1082-989X.3.1.104
20. Kazdin, A. E. (2011). Single-case research designs: Method for clinical and applied settings (2nd ed.). New York, NY: Oxford University Press.
21. Kenward, M., & Roger, J. (1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 53, 983-997. doi: 10.2307/2533558
22. Koehler, M., & Levin, J. (1998). Regulated randomization: A potentially sharper analytical tool for the multiple-baseline design. Psychological Methods, 3, 206-217. doi:10.1037/1082-989X.3.2.206
23. Koutsoftas, A. D., Harmon, M. T., & Gray, S. (2009). The effect of Tier 2 intervention for phonemic awareness in a response-to-intervention model in low-income preschool classrooms. Language, Speech, and Hearing Services in Schools, 40, 116-130. doi:10.1044/0161-1461(2008/07-0101)
24. Kratochwill, T. R. (1978). Single subject research: Strategies for evaluating change. New York, NY: Academic Press.
25. Kratochwill, T. R., Hitchcock, J., Horner, R. H., Levin, J. R., Odom, S. L., Rindskopf, D. M., & Shadish, W. R. (2010). Single-case designs technical documentation. Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc_scd.pdf
26. Kratochwill, T. R., & Levin, J. R. (2010). Enhancing the scientific credibility of single-case intervention research: Randomization to the rescue. Psychological Methods, 15, 124-144. doi:10.1037/a0017736
27. Little, R. J., & Rubin, D. B. (2000). Causal effects in clinical and epidemiological studies via potential outcomes: Concepts and analytical approaches. Annual Review of Public Health, 21, 121-145. doi:10.1146/annurev.publhealth.21.1.121
28. Maggin, D. M., Swaminathan, H., Rogers, H. J., O'Keeffe, B. V., Sugai, G., & Horner, R. H. (2011). A generalized least squares regression approach for computing effect sizes in single-case research: Application examples. Journal of School Psychology, 49, 301-321. doi:10.1016/j.jsp .2011.03.004
29. Marascuilo, L., & Busk, P. (1988). Combining statistics for multiplebaseline AB and replicated ABAB designs across subjects. Behavioral Assessment, 10, 1-28.
30. McKnight, S. D., McKean, J. W., & Huitema, B. E. (2000). A double bootstrap method to analyze linear models with autoregressive error terms. Psychological Methods, 5, 87-101. doi:10.1037/1082-989X.5.1.87
31. Moeyaert, M., Ugille, M., Ferron, J., Beretvas, T., & Van den Noortgate, W. (2013a). Modeling external events in the three-level analysis of multiplebaseline across participants designs: A simulation study. Behavior Research Methods, 45, 547-559. doi:10.3758/s13428-012-0274-1
32. Moeyaert, M., Ugille, M., Ferron, J., Beretvas, S. N., & Van den Noortgate, W. (2013b). The three-level synthesis of standardized single-subject experimental data: A Monte Carlo simulation study. Multivariate Behavioral Research, 48, 719-748. doi:10.1080/00273171.2013.816621
33. Moeyaert, M., Ugille, M., Ferron, J., Beretvas, T., & Van den Noortgate, W. (2014). Three-level analysis of single-case experimental data: Empirical validation. Journal of Experimental Education, 82, 1-21. doi: 10.1080/00220973.2012.745470
34. Molenaar, P. C. M. (2013). On the necessity to use person-specific data analysis approaches in psychology. European Journal of Developmental Psychology, 10, 29-39. doi:10.1080/17405629.2012.747435
35. Owens, C. M., & Farmer, J. L. (2013, April). Analyzing multiple baseline data using multilevel modeling with various residual distributions: A Monte Carlo study. Paper presented at the American Educational Research Association Annual Meeting, San Francisco, CA.
36. Parker, R. I., & Vannest, K. (2009). An improved effect size for single-case research: Nonoverlap of all pairs. Behavior Therapy, 40, 357-367. doi:10.1016/j.beth.2008.10.006
37. Petit-Bois, M., Baek, E. K., & Ferron, J. M. (2013, April). The effect of error structure specification on the meta-analysis of single-case studies: A Monte Carlo study. Poster presented at the American Educational Research Association Annual Meeting, San Francisco, CA.
38. Rantz, W. G., Dickinson, A. M., Sinclair, G. A., & Van Houten, R. (2009). The effect of feedback on the accuracy of checklist completion during instrument flight training. Journal of Applied Behavior Analysis, 42,497-509. doi:10.1901/jaba.2009.42-497
39. Revusky, S. H. (1967). Some statistical treatments compatible with individual organism methodology. Journal of the Experimental Analysis of Behavior, 10, 319-330. doi:10.1901/jeab.1967.10-319
40. Rindskopf, D., & Ferron, J. (2014). Using multilevel models to analyze singlecase design data. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case intervention research: Methodological and data-analysis advances (pp. 221-246). Washington, DC: American Psychological Association.
41. Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66,688-701. doi:10.1037/h0037350
42. Shadish, W. R. (2010). Campbell and Rubin: A primer and comparison of their approaches to causal inference in field settings. Psychological Methods, 15,3-17. doi:10.1037/a0015916
43. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
44. Shadish, W. R., Kyse, E. N., & Rindskopf, D. M. (2013). Analyzing data from single-case designs using multilevel models: New applications and some agenda items for future research. Psychological Methods, 18,385-405. doi:10.1037/a0032964
45. Shadish, W. R., & Rindskopf, D. M. (2007). Methods for evidence-based practice: Quantitative synthesis of single-subject designs. New Directions for Evaluation, 113, 95-109. doi:10.1002/ev.217
46. Shadish, W. R., Rindskopf, D. M., & Hedges, L. V. (2008). The state of the science in the meta-analysis of single-case experimental designs. Evidence-Based Communication Assessment and Intervention, 2, 188-196. doi:10.1080/17489530802581603
47. Shadish, W. R., & Sullivan, K. J. (2011). Characteristics of single-case designs used to assess intervention effects in 2008. Behavior Research Methods, 43, 971-980. doi:10.3758/s13428-011-0111-y
48. Singer, J. D. (1998). Using SAS PROC MIXED to fit multilevel models, hierarchical models, and individual growth models. Journal of Educational and Behavioral Statistics, 23, 323-355. doi:10.2307/1165280
49. Van den Noortgate, W., & Onghena, P. (2003a). Combining single-case experimental data using hierarchical linear models. School Psychology Quarterly, 18, 325-346. doi:10.1521/scpq.18.3.325.22577
50. Van den Noortgate, W., & Onghena, P. (2003b). Hierarchical linear models for the quantitative integration of effect sizes in single-case research. Behavior Research Methods, Instruments & Computers, 35, 1-10. doi: 10.3758/BF03195492
51. Van den Noortgate, W., & Onghena, P. (2007). The aggregation of singlecase results using hierarchical linear models. The Behavior Analyst Today, 8, 196-209.
52. Van den Noortgate, W., & Onghena, P. (2008). A multilevel meta-analysis of single-subject experimental designs. Evidence-Based Communication Assessment and Intervention, 2, 142-151. doi:10.1080/17489530802505362
53. Velicer, W. F., & Molenaar, P. C. (2012). Time series analysis for psychological research. In J. A. Schinka, W. F. Velicer, & I. B. Weiner (Eds.), Handbook of psychology: Vol. 2. Research methods in psychology (pp. 628-660). New York, NY: Wiley.
54. Wampold, B., & Worsham, N. (1986). Randomization tests for multiplebaseline designs. Behavioral Assessment, 8, 135-143.
55. West, S. G., & Thoemmes, F. (2010). Campbell's and Rubin's perspectives on causal inference. Psychological Methods, 15, 18-37. doi:10.1037/a0015917
56. Wong, V. C., Wing, C., Steiner, P. M., Wong, M., & Cook, T. D. (2012). Research designs for program evaluation. In J. A. Schinka, W. F. Velicer, & I. B. Weiner (Eds.), Handbook of psychology: Vol. 2. Research methods in psychology (pp. 316-341). New York, NY: Wiley.