The relationship between visual analysis and five statistical analyses in a simple AB single-case research design

Behavior Modification

Volumn 30, Issue 5, 2006, Pages 531-563

  Brossart, Daniel F ^a   Parker, Richard I ^a   Olson, Elizabeth A ^a   Mahadevan, Lakshmi ^a  

^a TEXAS A AND M UNIVERSITY   (United States)

Author keywords

Statistical analysis; Visual analysis

Indexed keywords

ANALYTIC METHOD; ARTICLE; BINOMIAL DISTRIBUTION; DECISION MAKING; HEALTH PRACTITIONER; INTERMETHOD COMPARISON; MEDICAL STAFF; METHODOLOGY; RELIABILITY; RESEARCH; STATISTICAL ANALYSIS; VALIDATION PROCESS; VISUAL ANALYSIS;

CASE-CONTROL STUDIES; DATA INTERPRETATION, STATISTICAL; HUMANS; JUDGMENT; MODELS, PSYCHOLOGICAL; OBSERVER VARIATION; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN;

EID: 33746577407     PISSN: 01454455     EISSN: 15524167     Source Type: Journal    
DOI: 10.1177/0145445503261167     Document Type: Article

References (94)

1. Allison, D. B. (1992). When cyclicity is a concern: A caveat regarding phase change criteria in single-case designs. Comprehensive Mental Health Care, 2, 131-149.
2. Allison, D. B., & Gorman, B. S. (1993). Calculating effect sizes formeta-analysis: The case of the single case. Behavior, Research, and Therapy, 31, 621-631.
3. Baer, D. M. (1977). Perhaps it would be better not to know everything. Journal of Applied Behavior Analysis, 10, 167-172.
4. Ballard, K. D. (1983). The visual analysis of time series data: Issues affecting the assessment of behavioral interventions. New Zealand Journal of Psychology, 12, 69-73.
5. Barlow, D. H., & Hersen, M. (Eds.). (1984). Single case experimental designs: Strategies for studying behavior change (2nd ed.). Oxford, England: Pergamon.
6. Berry, W. D., & Lewis-Beck, M. S. (1986). Interrupted time series. In W. D. Berry & M. S. Lewis-Beck (Eds.), New tools for social scientists: Advances and applications in research methods (pp. 209-240). Beverly Hills, CA: Sage.
7. Box, G. E. P., & Jenkins, G. M. (1970). Time-series analysis: Forecasting and control. New York: Cambridge University Press.
8. Busk, P. L., & Marascuilo, L. A. (1988). Autocorrelation in single-subject research: A counter-argument to the myth of no autocorrelation. Behavioral Assessment, 10, 229-242.
9. Busk, P. L., & Marascuilo, L. A. (1992). Statistical analysis in single-case research: Issues, procedures, and recommendations, with applications to multiple behaviors. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis: New directions for psychology and education (pp. 159-185). Hillsdale, NJ: Lawrence Erlbaum.
10. Busk, P. L., & Serlin, R. C. (1992). Meta-analysis for single-case research. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis: New directions for psychology and education (pp. 187-212). Hillsdale, NJ: Lawrence Erlbaum.
11. Carver, R. (1978). The case against statistical significance testing. Harvard Educational Review, 48, 378-399.
12. Center, B. A., Skiba, R. J., & Casey, A. (1985-1986). A methodology for the quantitative synthesis of intra-subject design research. Journal of Special Education, 19, 387-400.
13. Cochrane, D., & Orcutt, G. H. (1949). Application of least-squares regression to relationships containing autocorrelated error terms. Journal of the American Statistical Association, 44, 32-61.
14. Cohen, J. (1968). Multiple regression as a general data-analytic system. Psychological Bulletin, 70, 426-443.
15. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
16. Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-1312.
17. Cohen, J., & Cohen, P. (1983). Applied multiple regression/ correlation analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
18. Crosbie, J. (1987). The inability of the binomial test to control Type I error with single-subject data. Behavioral Assessment, 9, 141-150.
19. Crosbie, J. (1993). Interrupted time-series analysis with brief single-subject data. Journal of Consulting and Clinical Psychology, 61, 966-974.
20. Crosbie, J. (1995). Interrupted time-series analysis with short series: Why it is problematic; how it can be improved. In J. M. Gottman (Ed.), The analysis of change (pp. 361-395). Mahwah, NJ: Lawrence Erlbaum.
21. Dar, R., Serlin, R. C., & Orner, H. (1994). Misuse of statistical tests in three decades of psychotherapy research. Journal of Consulting and Clinical Psychology, 62, 75-82.
22. Darlington, R. B., & Carlson, P. M. (1987). Behavioral statistics: Logic & methods. New York: Free Press.
23. DeProspero, A., & Cohen, S. (1979). Inconsistent visual analyses of intrasubject data. Journal of Applied Behavior Analysis, 12, 573-579.
24. Faith, M. S., Allison, D. B., & Gorman, B. S. (1996). Meta-analysis of single-case research. In R. D. Franklin, D. B. Allison, & B. S. Gorman (Eds.), Design and analysis of single-case research (pp. 245-277). Mahwah, NJ: Lawrence Erlbaum.
25. Fox, J. (1991). Regression diagnostics. Newbury Park, CA: Sage.
26. Franklin, R. D., Allison, D. B., & Gorman, B. S. (Eds.). (1996). Design and analysis of single-case research. Mahwah, NJ: Lawrence Erlbaum.
27. Franklin, R. D., Gorman, B. S., Beasley, T. M., & Allison, D. B. (1996). Graphical display and visual analysis. In R. D. Franklin, D. B. Allison, & B. S. Gorman (Eds.), Design and analysis of single-case research (pp. 119-158). Mahwah, NJ: Lawrence Erlbaum.
28. Furlong, M. J., & Wampold, B. E. (1982). Intervention effects and relative variation as dimensions in experts' use of visual inference. Journal of Applied Behavior Analysis, 15, 415-421.
29. Gibson, G., & Ottenbacker, K. (1988). Characteristics influencing the visual analysis of single-subject data: An empirical analysis. Journal of Applied Behavioral Science, 24, 298-314.
30. Glass, G. V., McGaw, B., & Smith, W. (1981). Meta-analysis in social research. Beverly Hills, CA: Sage.
31. Glass, G. V., Willson, V. L., & Gottman, J. M. (1975). Design and analysis of time-series experiments. Boulder: Colorado Associated University Press.
32. Gorman, B. S., & Allison, D. B. (1996). Statistical alternatives for single-case designs. In R. D. Franklin, D. B. Allison, & B. S. Gorman (Eds.), Design and analysis of single-case research (pp. 159-214). Mahwah, NJ: Lawrence Erlbaum.
33. Gorsuch, R. L. (1983). Three methods for analyzing time-series (N of 1) data. Behavioral Assessment, 5, 141-154.
34. Gottman, J. M., & Glass, G. V. (1978). Analysis of interrupted time-series experiments. In T. R. Kratochwill (Ed.), Single subject research: Strategies for evaluating change (pp. 197-235). New York: Academic Press.
35. Greenspan, P., & Fisch, G. S. (1992). Visual inspection of data: A statistical analysis of behavior. Paper presented at the annual meeting of the American Statistical Association, Boston, MA.
36. Greenwald, A. G. (1976). Within-subjects designs: To use or not to use? Psychological Bulletin, 8, 314-320.
37. Harbst, K. B., Ottenbacher, K. J., & Harris, S. R. (1991). Interrater reliability of therapists' judgments of graphed data. Physical Therapy, 71, 107-115.
38. Hartmann, D. P., Gottman, J. M., Jones, R. R., Gardner, W., Kazdin, A. E., & Vaught, R. S. (1980). Interrupted time-series analysis and its application to behavioral data. Journal of Applied Behavior Analysis, 13, 543-559.
39. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. Orlando, FL: Academic Press.
40. Hintze, J. (2000). NCSS 2000 [Computer software]. Kaysville, UT: NCSS Statistical Software.
41. Hojem, M. A., & Ottenbacker, K. J. (1988). Empirical investigation of visual-inspection versus trend-line analysis of single-subject data. Journal of the American Physical Therapy Association, 68, 983-988.
42. Holtzman, W. H. (1963). Statistical methods for the study of change in the single case. In C. W. Harris (Ed.), Problems in measuring change (pp. 199-211). Madison: University of Wisconsin Press.
43. Home, G. P., Yang, M. C. K., & Ware, W. B. (1982). Time series analysis for single-subject designs. Psychological Bulletin, 91, 178-189.
44. Huitema, B. E. (1985). Autocorrelation in applied behavior analysis: A myth. Behavioral Assessment, 7, 107-118.
45. Huitema, B. E. (1986). Autocorrelation in behavioral research: Wherefore art thou? In A. Poling & R. W. Fuqua (Eds.), Research methods in applied behavior analysis: Issues and advances (pp. 187-208). New York: Plenum.
46. Huitema, B. E., & McKean, J. W. (1991). Autocorrelation estimation and inference with small samples. Psychological Bulletin, 110, 291-304.
47. Huitema, B. E., & McKean, J. W. (1998). Irrelevant autocorrelation in least-squares intervention models. Psychological Methods, 3, 104-116.
48. Jaccard, J., & Wan, C. K. (1993). Statistical analysis of temporal data with many observations: Issues for behavioral medicine data. Annals of Behavioral Medicine, 15, 41-50.
49. Jacobson, N. S., Follette, W. C., & Revenstorf, D. (1984). Psychotherapy outcome research: Methods for reporting variability and evaluating clinical significance. Behavior Therapy, 15, 336-352.
50. Jones, R. R., Vaught, R. S., & Weinrott, M. (1977). Time-series analysis in operant research. Journal of Applied Behavior Analysis, 10, 151-166.
51. Jones, R. R., Weinrott, M. R., & Vaught, R. S. (1978). Effects of serial dependency on the agreement between visual and statistical inference. Journal of Applied Behavior Analysis, 11, 277-283.
52. Kazdin, A. (1984). Statistical analysis for single-case experiments designs. In D. H. Barlow & M. Hersen (Eds.), Single case experimental designs (2nd ed., pp. 285-324). New York: Pergamon.
53. Kazdin, A. E. (1982). Single-case research designs: Methods for clinical and applied settings. New York: Oxford University Press.
54. Keppel, G. (1982). Design and analysis: A researcher's handbook (2nd ed.). Englewood Cliffs, NJ: Prentice Hall.
55. Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746-759.
56. Knapp, T. J. (1983). Behavior analysts' visual appraisal of behavior change in graphic display. Behavioral Assessment, 5, 155-164.
57. Kratochwill, T. R., & Brody, G. H. (1978). Single subject designs: A perspective on the controversy over employing statistical inference and implications for research and training in behavior modification. Behavior Modification, 2, 291-307.
58. Kratochwill, T. R., & Levin, J. R. (Eds.). (1992). Single-case research design and analysis: New directions for psychology and education. Hillsdale, NJ: Lawrence Erlbaum.
59. Kromrey, J. D., & Foster-Johnson, L. (1996). Determining the efficacy of intervention: The use of effect sizes for data analysis in single-subject research. The Journal of Experimental Education, 65, 73-93.
60. Kupfersmid, J. (1988). Improving what is published: A model in search on an editor. American Psychologist, 43, 635-642.
61. Matyas, T. A., & Greenwood, K. M. (1990). Visual analysis of single-case time series: Effects of variability, serial dependence, and magnitude of intervention effects. Journal of Applied Behavior Analysis, 23, 341-351.
62. Matyas, T. A., & Greenwood, K. M. (1996). Serial dependency in single-case time series. In R. D. Franklin, D. B. Allison, & B. S. Gorman (Eds.), Design and analysis of single-case research (pp. 215-243). Mahwah, NJ: Lawrence Erlbaum.
63. Maxwell, S. E., Camp, C. J., & Avery, R. D. (1981). Measures of strength of association: A comparative examination. Journal of Applied Psychology, 66, 525-534.
64. Maxwell, S. E., & Delaney, H. D. (1990). Designing experiments and analyzing data: A model comparison perspective. Belmont, CA: Wadsworth.
65. Michael, J. L. (1974). Statistical inference for individual organism research: Mixed blessing or curse? Journal of Applied Behavior Analysis, 7, 647-653.
66. Mitchell, C., & Hartmann, D. P. (1981). A cautionary note on the use of Omega squared to evaluate the effectiveness of behavioral treatments. Behavioral Assessment, 3, 93-100.
67. Nourbakhsh, M. R., & Ottenbacher, K. J. (1994). The statistical analysis of single-subject data: A comparative examination. Physical Therapy, 74, 80-88.
68. Nunnally, J. C. (1978). Psychometric theory (2nd ed.). New York: McGraw-Hill.
69. Ostrom, C. W., Jr. (1978). Time series analysis: Regression techniques. Beverly Hills, CA: Sage.
70. Ostrom, C. W., Jr. (1990). Time series analysis: Regression techniques (2nd ed.). Newbury Park, CA: Sage.
71. Ottenbacher, K. J. (1986a). An analysis of serial dependency in occupational therapy research. Occupational Therapy Journal of Research, 6, 211-216.
72. Ottenbacher, K. J. (1986b). Reliability and accuracy of visually analyzing graphed data from single-subject designs. American Journal of Occupational Therapy, 40, 464-469.
73. Ottenbacher, K. J. (1990a). Visual inspection of single-subject data: An empirical analysis. Mental Retardation, 28, 283-290.
74. Ottenbacher, K. J. (1990b). When is a picture worth a thousand p values? A comparison of visual and quantitative methods to analyze single subject data. Journal of Special Education, 23, 436-449.
75. Park, H., Marascuilo, L., & Gaylord-Ross, R. (1990). Visual inspection and statistical analysis of single-case designs. Journal of Experimental Education, 58, 311-320.
76. Parsonson, B. S., & Baer, D. M. (1978). The analysis and presentation of graphic data. In T. R. Kratochwill (Ed.), Single-subject research: Strategies for evaluating change (pp. 101-165). New York: Academic Press.
77. Parsonson, B. S., &Baer, D. M. (1986). The graphic analysis of data. In A. Poling & R. W. Fuqua (Eds.), Research methods in applied behavior analysis: Issues and advances (pp. 157-186). New York: Plenum.
78. Parsonson, B. S., & Baer, D. M. (1992). The visual analysis of data, and current research into the stimuli controlling it. In T. R. Kratochwill & J. R. Levin (Eds.), Single-case research design and analysis (pp. 15-40). Hillsdale, NJ: Lawrence Erlbaum.
79. Prentice, D. A., & Miller, D. T. (1992). When small effects are impressive. Psychological Bulletin, 112, 160-164.
80. Rojahn, J., & Schulze, H. (1985). The linear regression line as a judgmental aid in the visual analysis of serially dependent A-B time-series data. Journal of Psychopathology and Behavioral Assessment, 7, 191-206.
81. Rosenthal, R. (1984). Meta-analytic procedures for social research. Beverly Hills, CA: Sage.
82. Rosenthal, R. (1991). Meta-analytic procedures for social research (Rev. ed., Vol. 6). Newbury Park, CA: Sage.
83. Rosnow, R., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276-1284.
84. Scheff, H. (1959). The analysis of variance. New York: John Wiley.
85. Sharpley, C. F., & Alavosius, M. P. (1988). Autocorrelation in behavioral data: An alternative perspective. Behavioral Assessment, 10, 243-251. (The autocorrelation debate)
86. Shaver, J. P. (1991). Quantitative reviewing of research. In J. P. Shaver (Ed.), Handbook of research on social studies teaching and learning (pp. 83-97). New York: Macmillan.
87. Skiba, R., Deno, S., Marston, D., & Casey, A. (1989). Influence of trend estimation and subject familiarity on practitioners judgements of intervention effectiveness. Journal of Special Education, 22, 433-446.
88. Suen, H. K., & Ary, D. (1987). Autocorrelation in applied behavior analysis: Myth or reality? Behavioral Assessment, 9, 125-130.
89. Suen, H. K., & Ary, D. (1989). Analyzing quantitative behavioral observation data. Hillsdale, NJ: Lawrence Erlbaum.
90. Toothaker, L. E., Banz, M., Noble, C., Camp, J., & Davis, D. (1983). N = 1 designs: The failure of ANOVA-based tests. Journal of Educational Statistics, 8, 289-309.
91. Wampold, B. E., & Furlong, M. J. (1981). The heuristics of visual inference. Behavioral Assessment, 3, 79-92.
92. White, D. M., Rusch, F. R., Kazdin, A. E., & Hartmann, D. P. (1989). Applications of meta-analysis in individual subject research. Behavioral Assessment, 11, 281-296.
93. White, O. R., & Haring, N. G. (1980). Exceptional teaching (2nd ed.). Columbus, OH: Merrill.
94. Wolf, F. M. (1986). Meta-analysis: Quantitative methods for research synthesis (Vol. 59). Beverly Hills, CA: Sage.