Estimating individual treatment effects from multiple-baseline data: A Monte Carlo study of multilevel-modeling approaches

Behavior Research Methods

Volumn 42, Issue 4, 2010, Pages 930-943

  Ferron, John M ^a   Farmer, Jennie L ^a   Owens, Corina M ^a  

^a UNIVERSITY OF SOUTH FLORIDA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CONFIDENCE INTERVAL; MONTE CARLO METHOD; MULTILEVEL ANALYSIS; STATISTICAL MODEL;

CONFIDENCE INTERVALS; MODELS, STATISTICAL; MONTE CARLO METHOD; MULTILEVEL ANALYSIS;

EID: 78650700512     PISSN: 1554351X     EISSN: None     Source Type: Journal    
DOI: 10.3758/BRM.42.4.930     Document Type: Article

References (54)

1. Allsopp, D. H., McHatton, P. A., Ray, S. N. E., & Farmer, J. L. (2010). Mathematics RTI: A problem-solving approach to creating an effective model. Horsham, PA: LRP.
2. American Psychological Association (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association.
3. Barlow, D. H., & Hersen, M. (1984). Single case experimental designs: Strategies for studying behavior change (2nd ed.). Needam Heights, MA: Allyn & Bacon.
4. Box, G. E. P., Jenkins, G. M., & Reinsel, G. C. (1994). Time series analysis, forecasting, and control. San Francisco: Holden-Day.
5. Busk, P. L., & Marascuilo, L. A. (1988). Autocorrelation in singlesubject research: A counterargument to the myth of no autocorrelation. Behavioral Assessment, 10, 229-242.
6. Busk, P. L., & Serlin, R. C. (1992). Meta-analysis for single-case research. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis (pp. 187-212). Hillsdale, NJ: Erlbaum.
7. Candel, M. J. J. M., & Winkens, B. (2003). Performance of empirical Bayes estimators of Level-2 random parameters in multilevel analysis: A Monte Carlo study for longitudinal designs. Journal of Educational & Behavioral Statistics, 28, 169-194. doi:10.3102/10769986028002169
8. Center, B. A., Skiba, R. J., & Casey, A. (1985). A methodology for the quantitative synthesis of intra-subject design research. Journal of Special Education, 19, 387-400. doi:10.1177/002246698501900404
9. Edgington, E. S. (1980). Validity of randomization tests for one-subject experiments. Journal of Educational Statistics, 5, 235-251. doi:10.2307/1164966
10. Fai, A. H.-T., & Cornelius, P. L. (1996). Approximate F-tests of multiple degree of freedom hypotheses in generalized least squares analyses of unbalanced split-plot experiments. Journal of Statistical Computation & Simulation, 54, 363-378.
11. Ferron, J. M., Bell, B. A., Hess, M. R., Rendina-Gobioff, G., & Hibbard, S. T. (2009). Making treatment effect inferences from multiple-baseline data: The utility of multilevel modeling approaches. Behavior Research Methods, 41, 372-384.
12. Ferron, J. [M.], Dailey, R., & Yi, Q. (2002). Effects of misspecifying the first-level error structure in two-level models of change. Multivariate Behavioral Research, 37, 379-403. doi:10.1207/S15327906MBR3703_4
13. Ferron, J. [M.], & Jones, P. K. (2002, April). Visual tests for the analysis of multiple-baseline data. Paper presented at the annual conference of the American Educational Research Association, New Orleans, LA.
14. Ferron, J. [M.], & Jones, P. K. (2006). Tests for the visual analysis of response-guided multiple-baseline data. Journal of Experimental Education, 75, 66-81. doi:10.3200/JEXE.75.1.66-81
15. Ferron, J. [M.], & Sentovich, C. (2002). Statistical power of randomization tests used with multiple-baseline designs. Journal of Experimental Education, 70, 165-178.
16. Fouladi, R. T., & Shieh, Y.-Y. (2004). A comparison of two general approaches to mixed model longitudinal analyses under small sample size conditions. Communications in Statistics: Simulation & Computation, 33, 807-824. doi:10.1081/SAC-200033260
17. Gadbury, G. L., & Iyer, H. K. (2000). Unit-treatment interaction and its practical consequences. Biometrics, 56, 882-885.
18. Glover, T. A., & DiPerna, J. C. (2007). Service delivery for response to intervention: Core components and directions for future research. School Psychology Review, 36, 526-540.
19. Gomez, E. V., Schaalje, G. B., & Fellingham, G. W. (2005). Performance of the Kenward-Roger method when the covariance matrix is selected using AIC and BIC. Communications in Statistics: Simulation & Computation, 34, 377-392.
20. Greenwood, K. M., & Matyas, T. A. (1990). Problems with the application of interrupted time series analysis for brief single-subject data. Behavioral Assessment, 12, 355-370.
21. Gresham, F. M., McIntyre, L. L., Olson-Tinker, H., Dolstra, L., McLaughlin, V., & Van, M. (2004). Relevance of functional behavioral assessment research for school-based interventions and positive behavioral support. Research in Developmental Disabilities, 25, 19-37. doi:10.1016/j.ridd.2003.04.003
22. Harville, D. A., & Jeske, D. R. (1992). Mean squared error of estimation or prediction under a general linear model. Journal of the American Statistical Association, 87, 724-731.
23. Holland, P. W. (1986). Statistical and causal inference. Journal of the American Statistical Association, 81, 945-960.
24. Huitema, B. E. (1985). Autocorrelation in applied behavior analysis: A myth. Behavioral Assessment, 7, 107-118.
25. Huitema, B. E., & McKean, J. W. (1998). Irrelevant autocorrelation in least-squares intervention models. Psychological Methods, 3, 104-116. doi:10.1037/1082-989X.3.1.104
26. Huitema, B. E., & McKean, J. W. (2000). Design specification issues in time-series intervention models. Educational & Psychological Measurement, 60, 38-58. doi:10.1177/00131640021970358
27. Kazdin, A. E., & Kopel, S. A. (1975). On resolving ambiguities of the multiple-baseline design: Problems and recommendations. Behavior Therapy, 6, 601-608.
28. Kenward, M. G., & Roger, J. H. (1997). Small sample inference for fixed effects from restricted maximum likelihood. Biometrics, 53, 983-997.
29. Kenward, M. G., & Roger, J. H. (2009). An improved approximation to the precision of fixed effects from restricted maximum likelihood. Computational Statistics & Data Analysis, 53, 2583-2595.
30. Koehler, M. J., & Levin, J. R. (1998). Regulated randomization: A potentially sharper analytical tool for the multiple-baseline design. Psychological Methods, 3, 206-217. doi:10.1037/1082-989X.3.2.206
31. Kowalchuk, R. K., Keselman, H. J., Algina, J., & Wolfinger, R. D. (2004). The analysis of repeated measurements with mixed-model adjusted F tests. Educational & Psychological Measurement, 64, 224-242. doi:10.1177/0013164403260196
32. Kratochwill, T., Alden, K., Demuth, D., Dawson, D., Panicucci, C., Arntson, P., et al. (1974). A further consideration in the application of an analysis-of-variance model for the intrasubject replication design. Journal of Applied Behavior Analysis, 7, 629-633.
33. Kwok, O., West, S. G., & Green, S. B. (2007). The impact of misspecifying the within-subject covariance structure in multiwave longitudinal multilevel models: A Monte Carlo study. Multivariate Behavioral Research, 42, 557-592.
34. Matyas, T. A., & Greenwood, K. M. (1997). Serial dependency in single-case time series. In R. D. Franklin, D. B. Allison, & B. S. Gorman (Eds.), Design and analysis of single-case research (pp. 215-243). Mahwah, NJ: Erlbaum.
35. Morgan, D. L., & Morgan, R. K. (2001). Single-participant research design: Bringing science to managed care. American Psychologist, 56, 119-127. doi:10.1037/0003-066X.56.2.119
36. Murphy, D. L., & Pituch, K. A. (2009). The performance of multilevel growth curve models under an autoregressive moving average process. Journal of Experimental Education, 77, 255-282.
37. Neuman, S. B., & McCormick, S. (Eds.) (1995). Single-subject experimental research: Applications for literacy. Newark, DE: International Reading Association.
38. Nugent, W. R. (1996). Integrating single-case and group comparison designs for evaluation research. Journal of Applied Behavioral Science, 32, 209-226. doi:10.1177/0021886396322007
39. Parsonson, B. S., & Baer, D. M. (1992). The visual analysis of data, and current research into the stimuli controlling it. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis: New directions for psychology and education (pp. 15-40). Hillsdale, NJ: Erlbaum.
40. Prasad, N. G. N., & Rao, J. N. K. (1990). The estimation of the mean squared error of small-area estimators. Journal of the American Statistical Association, 85, 163-171.
41. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage.
42. Rubin, D. B. (1974). Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of Educational Psychology, 66, 688-701. doi:10.1037/h0037350
43. SAS Institute (2004). SAS/STAT 9.1 user's guide. Carey, NC: SAS Institute.
44. SAS Institute (2005). SAS, release 9.12 [Computer program]. Carey, NC: SAS Institute.
45. Satterthwaite, F. E. (1941). Synthesis of variance. Psychometrika, 6, 309-316. doi:10.1007/BF02288586
46. Schaalje, G. B., McBride, J. B., & Fellingham, G. W. (2001). Approximations to distributions of test statistics in complex mixed linear models using SAS Proc MIXED. In Proceedings of the SAS Users Group International 26th Annual Conference (paper 262-26). Available at http://support.sas.com/events/sasglobalforum/previous/index.html.
47. Shadish, W. R., & Rindskopf, D. M. (2007). Methods for evidencebased practice: Quantitative synthesis of single-subject designs. New Directions for Evaluation, 113, 95-109.
48. Sharpley, C. (1981). Time series analysis of counseling research. Measurement & Evaluation in Guidance, 14, 149-157.
49. Skinner, C. H. (2004). Single-subject designs: Procedures that allow school psychologists to contribute to the intervention evaluation and validation process. Journal of Applied School Psychology, 20, 1-10.
50. Swanson, H. L., & Sachse-Lee, C. (2000). A meta-analysis of singlesubject-design intervention research for students with LD. Journal of Learning Disabilities, 33, 114-136.
51. Toothaker, L. E., Banz, M., Noble, C., Camp, J., & Davis, D. (1983). N=1 designs: The failure of ANOVA-based tests. Journal of Educational Statistics, 8, 289-309.
52. Van den Noortgate, W., & Onghena, P. (2003a). Combining singlecase experimental data using hierarchical linear models. School Psychology Quarterly, 18, 325-346. doi:10.1521/scpq.18.3.325.22577
53. Van den Noortgate, W., & Onghena, P. (2003b). Hierarchical linear models for the quantitative integration of effect sizes in single-case research. Behavior Research Methods, Instruments, & Computers, 35, 1-10.
54. Van den Noortgate, W., & Onghena, P. (2007). The aggregation of single-case results using hierarchical linear models. Behavior Analyst Today, 8, 52-75.