Assigning and combining probabilities in single-case studies

Psychological Methods

Volumn 17, Issue 4, 2012, Pages 495-509

  Manolov, Rumen ^a   Solanas, Antonio ^a  

^a UNIVERSITY OF BARCELONA   (Spain)

Author keywords

Compound probability; Effect size; Quantitative integration; Single case

Indexed keywords

ARTICLE; HUMAN; META ANALYSIS (TOPIC); PROBABILITY; STATISTICAL ANALYSIS;

DATA INTERPRETATION, STATISTICAL; HUMANS; META-ANALYSIS AS TOPIC; PROBABILITY;

EID: 84874431176     PISSN: 1082989X     EISSN: None     Source Type: Journal    
DOI: 10.1037/a0029248     Document Type: Article

References (95)

1. Allison, D. B., & Gorman, B. S. (1993). Calculating effect sizes for meta-analysis: The case of the single case. Behaviour Research and Therapy, 31, 621-631. doi:10.1016/0005-7967(93)90115-B
2. APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology. American Psychologist, 61, 271-285. doi:10.1037/0003-066X.61.4.271
3. Barlow, D. H., & Hersen, M. (1973). Single-case experimental designs: Uses in applied clinical research. Archives of General Psychiatry, 29, 319-325. doi:10.1001/archpsyc.1973.04200030017003
4. Barlow, D. H., Nock, M. K., & Hersen, M. (Eds.). (2009). Single case experimental designs: Strategies for studying behavior change (3rd ed.). Boston, MA: Pearson.
5. Becker, B. J. (1987). Applying tests of combined significance in metaanalysis. Psychological Bulletin, 102, 164-171. doi:10.1037/00332909.102.1.164
6. Beretvas, S. N., & Chung, H. (2008). A review of meta-analyses of single-subject experimental designs: Methodological issues and practice. Evidence-Based Communication Assessment and Intervention, 2, 129- 141. doi:10.1080/17489530802446302
7. Blampied, N. M. (2000). Single-case research designs: A neglected alternative. American Psychologist, 55, 960. doi:10.1037/0003-066X.55.8.960
8. Borckardt, J. J., Nash, M. R., Murphy, M. D., Moore, M., Shaw, D., & O'Neil, P. (2008). Clinical practice as natural laboratory for psychotherapy research: A guide to case-based time-series analysis. American Psychologist, 63, 77-95. doi:10.1037/0003-066X.63.2.77
9. Bradley, J. V. (1977). A common situation conducive to bizarre distribution shapes. American Statistician, 31, 147-150.
10. Busse, R. T., Kratochwill, T. R., & Elliott, S. N. (1995). Meta-analysis for single-case consultation outcomes: Applications to research and practice. Journal of School Psychology, 33, 269-285. doi:10.1016/00224405(95)00014-D
11. Center, B. A., Skiba, R. J., & Casey, A. (1985). A methodology for the quantitative synthesis of intra-subject design research. The Journal of Special Education, 19, 387-400. doi:10.1177/002246698501900404
12. Chaterjee, S., & Price, B. (1977). Regression analysis by example. New York, NY: Wiley.
13. Cohen, J. (1990). Things I have learned (so far). American Psychologist, 45, 1304-1312. doi:10.1037/0003-066X.45.12.1304
14. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159. doi:10.1037/0033-2909.112.1.155
15. Cohen, J. (1994). The earth is round (p<05). American Psychologist, 49, 997-1003. doi:10.1037/0003-066X.49.12.997
16. Cortina, J. M., & Landis, R. S. (2011). The earth is not round (p =.00). Organizational Research Methods, 14, 332-349. doi:10.1177/1094428110391542
17. Cumming, G. (2008). Replication and p intervals: p values predict the future only vaguely, but confidence intervals do much better. Perspectives on Psychological Science, 3, 286-300. doi:10.1111/j.1745-6924.2008.00079.x
18. Cumming, G. (2012). Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis. London, England: Routledge.
19. Darlington, R. B., & Hayes, A. F. (2000). Combining independent p values: Extensions of the Stouffer and binomial methods. Psychological Methods, 5, 496-515. doi:10.1037/1082-989X.5.4.496
20. Edgington, E. S. (1972a). An additive method for combining probability values from independent experiments. Journal of Psychology, 80, 351- 363. doi:10.1080/00223980.1972.9924813
21. Edgington, E. S. (1972b). A normal curve method for combining probability values from independent experiments. Journal of Psychology, 82, 85-89. doi:10.1080/00223980.1972.9916971
22. Edgington, E. S., & Haller, O. (1984). Combining probabilities from discrete probability distributions. Educational and Psychological Measurement, 44, 265-274. doi:10.1177/0013164484442009
23. Evans, J. J., Emslie, H., & Wilson, B. A. (1998). External cueing systems in the rehabilitation of executive impairments of action. Journal of the International Neuropsychological Society, 4, 399-408.
24. Fahmie, T. A., & Hanley, G. P. (2008). Progressing toward data intimacy: A review of within-session data analysis. Journal of Applied Behavior Analysis, 41, 319-331. doi:10.1901/jaba.2008.41-319
25. Ferron, J. M., & Ware, W. (1995). Analyzing single-case data: The power of randomization tests. Journal of Experimental Education, 63, 167- 178. doi:10.1080/00220973.1995.9943820
26. Gast, D. L., & Spriggs, A. D. (2009). Visual analysis of graphic data. In D. L. Gast (Ed.), Single subject research methodology in behavioral sciences (pp. 199-233). London, England: Routledge.
27. Gottman, J. M. (1981). Time-series analysis: A comprehensive introduction for social scientists. Cambridge, England: Cambridge University Press.
28. Greenwald, A. G. (1976). Within-subject designs: To use or not to use? Psychological Bulletin, 83, 314-320. doi:10.1037/0033-2909.83.2.314
29. Greenwood, K. M., & Matyas, T. A. (1990). Problems with application of interrupted time series analysis for brief single-subject data. Behavioral Assessment, 12, 355-370.
30. Grissom, R. J., & Kim, J. J. (2012). Effect size for research: Univariate and multivariate applications (2nd ed.). London, England: Routledge.
31. Hallahan, M., & Rosenthal, R. (1996). Statistical power: Concepts, procedures, and applications. Behaviour Research and Therapy, 34, 489-499. doi:10.1016/0005-7967(95)00082-8
32. Harrop, J. W., & Velicer, W. F. (1985). A comparison of alternative approaches to the analysis of interrupted time-series. Multivariate Behavioral Research, 20, 27-44. doi:10.1207/s15327906mbr2001_2
33. Hedges, L. V., Cooper, H., & Bushman, B. J. (1992). Testing the null hypothesis in meta-analysis: A comparison of combined probability and confidence interval procedures. Psychological Bulletin, 111, 188-194. doi:10.1037/0033-2909.111.1.188
34. Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. New York, NY: Academic Press.
35. Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S. L., & Wolery, M. (2005). The use of single-subject research to identify evidence-based practice in special education. Exceptional Children, 71, 165-179.
36. Huitema, B. E., & McKean, J. W. (1991). Autocorrelation estimation and inference with small samples. Psychological Bulletin, 110, 291-304. doi:10.1037/0033-2909.110.2.291
37. Huitema, B. E., & McKean, J. W. (2000). Design specification issues in time-series intervention models. Educational and Psychological Measurement, 60, 38-58. doi:10.1177/00131640021970358
38. Huitema, B. E., & McKean, J. W. (2007). An improved portmanteau test for autocorrelated errors in interrupted time-series regression models. Behavior Research Methods, 39, 343-349. doi:10.3758/BF03193002
39. Huitema, B. E., McKean, J. W., & McKnight, S. (1999). Autocorrelation effects on least-squares intervention analysis of short time series. Educational and Psychological Measurement, 59, 767-786. doi:10.1177/00131649921970134
40. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Thousand Oaks, CA: Sage.
41. Jenson, W. R., Clark, E., Kircher, J. C., & Kristjansson, S. D. (2007). Statistical reform: Evidence-based practice, meta-analyses, and single subject designs. Psychology in the Schools, 44, 483-493. doi:10.1002/pits.20240
42. Johnston, J. M., & Pennypacker, H. S., Jr. (2009). Strategies and tactics of behavioral research (3rd ed.). New York, NY: Routledge.
43. Jones, L. V., & Fiske, D. W. (1953). Models for testing the significance of combined results. Psychological Bulletin, 50, 375-382. doi:10.1037/h0057844
44. Kazdin, A. E. (2008). Evidence-based treatment and practice: New opportunities to bridge clinical research and practice, enhance the knowledge base, and improve patient care. American Psychologist, 63, 146-159. doi:10.1037/0003-066X.63.3.146
45. Killeen, P. R. (2005). An alternative to null-hypothesis significance tests. Psychological Science, 16, 345-353. doi:10.1111/j.0956-7976.2005.01538.x
46. Kirk, R. E. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746-759. doi: 10.1177/0013164496056005002
47. Kromrey, J. D., & Foster-Johnson, L. (1996). Determining the efficacy of intervention: The use of effect sizes for data analysis in single-subject research. Journal of Experimental Education, 65, 73-93. doi:10.1080/00220973.1996.9943464
48. Ma, H. H. (2006). An alternative method for quantitative synthesis of single-subject research: Percentage of data points exceeding the median. Behavior Modification, 30, 598-617. doi:10.1177/0145445504272974
49. Mace, F. C., & Kratochwill, T. R. (1986). The individual subject in behavior analysis research. In J. Valsiner (Ed.), The individual subject and scientific psychology (pp. 153-180). London, England: Plenum Press.
50. Maggin, D. M., Swaminathan, H., Rogers, H. J., O'Keefe, B. V., Sugai, G., & Horner, R. H. (2011). A generalized least squares regression approach for computing effect sizes in single-case research: Application examples. Journal of School Psychology, 49, 301-321. doi:10.1016/j.jsp.2011.03.004
51. Manolov, R., Arnau, J., Solanas, A., & Bono, R. (2010). Regression-based techniques for statistical decision making in single-case designs. Psicothema, 22, 1026-1032.
52. Manolov, R., & Solanas, A. (2009). Percentage of nonoverlapping corrected data. Behavior Research Methods, 41, 1262-1271. doi:10.3758/BRM.41.4.1262
53. Manolov, R., Solanas, A., Sierra, V., & Evans, J. J. (2011). Choosing among techniques for quantifying single-case intervention effectiveness. Behavior Therapy, 42, 533-545. doi:10.1016/j.beth.2010.12.003
54. Mastropieri, M. A., & Scruggs, T. E. (1985). Early intervention for socially withdrawn children. Journal of Special Education, 19, 429-441. doi: 10.1177/002246698501900407
55. Mathew, T., Sinha, B. K., & Zhou, L. (1993). Some statistical procedures for combining independent tests. Journal of the American Statistical Association, 88, 912-919.
56. Matyas, T. A., & Greenwood, K. M. (1990). Visual analysis for single-case time series: Effects of variability, serial dependence, and magnitude of intervention effects. Journal of Applied Behavior Analysis, 23, 341-351. doi:10.1901/jaba.1990.23-341
57. McKnight, S. D., McKean, J. W., & Huitema, B. E. (2000). A double bootstrap method to analyze linear models with autoregressive error terms. Psychological Methods, 5, 87-101. doi:10.1037/1082989X.5.1.87
58. Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156-166. doi:10.1037/00332909.105.1.156
59. Noble, J. H. (2006). Meta-analysis: Methods, strengths, weaknesses, and political uses. Journal of Laboratory and Clinical Medicine, 147, 7-20. doi:10.1016/j.lab.2005.08.006
60. Noreen, E. W. (1989). Computer-intensive methods for testing hypotheses: An introduction. New York, NY: Wiley.
61. Onghena, P., & Edgington, E. S. (2005). Customization of pain treatments: Single-case design and analysis. Clinical Journal of Pain, 21, 56-68. doi:10.1097/00002508-200501000-00007
62. Parker, R. I. (2006). Increased reliability for single-case research results: Is bootstrap the answer? Behavior Therapy, 37, 326-338. doi:10.1016/j.beth.2006.01.007
63. Parker, R. I., Brossart, D. F., Vannest, K. J., Long, J. R., Garcia De-Alba, R., Baugh, F. G., & Sullivan, J. R. (2005). Effect sizes in single case research: How large is large? School Psychology Review, 34, 116-132.
64. Parker, R. I., & Vannest, K. J. (2009). An improved effect size for single-case research: Nonoverlap of all pairs. Behavior Therapy, 40, 357-367. doi:10.1016/j.beth.2008.10.006
65. Parker, R. I., Vannest, K. J., & Brown, L. (2009). The improvement rate difference for single-case research. Exceptional Children, 75, 135-150.
66. Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S. B. (2011). Combining nonoverlap and trend for single-case research: Tau-U. Behavior Therapy, 42, 284-299. doi:10.1016/j.beth.2010.08.006
67. Parsonson, B. S., & Baer, D. M. (1986). The graphic analysis of data. In A. Poling & R. W. Fuqua (Eds.), Research methods in applied behavior analysis: Issues and advances (pp. 157-186). New York, NY: Plenum Press.
68. Perdices, M., & Tate, R. L. (2009). Single-subject designs as a tool for evidence-based clinical practice: Are they unrecognised and undervalued? Neuropsychological Rehabilitation, 19, 904-927. doi:10.1080/09602010903040691
69. Reynhout, G., & Carter, M. (2006). Social stories for children with disabilities. Journal of Autism and Developmental Disorders, 36, 445-469. doi:10.1007/s10803-006-0086-1
70. Rice, W. R. (1990). A consensus combined p-value test and the familywide significance of component tests. Biometrics, 46, 303-308. doi: 10.2307/2531435
71. Robey, R. R., & Barcikowski, R. S. (1992). Type I error and the number of iterations in Monte Carlo studies of robustness. British Journal of Mathematical and Statistical Psychology, 45, 283-288. doi:10.1111/j.2044-8317.1992.tb00993.x
72. Rosenthal, R. (1978). Combining results of independent studies. Psychological Bulletin, 85, 185-193. doi:10.1037/0033-2909.85.1.185
73. Rosenthal, R. (1979). The "file drawer problem" and tolerance for null results. Psychological Bulletin, 86, 638-641. doi:10.1037/00332909.86.3.638
74. Rosenthal, R., & Rubin, D. B. (1985). Statistical analysis: Summarizing evidence versus establishing facts. Psychological Bulletin, 97, 527-529. doi:10.1037/0033-2909.97.3.527
75. Rosnow, R. L., & Rosenthal, R. (1989). Statistical procedures and the justification of knowledge in psychological science. American Psychologist, 44, 1276-1284. doi:10.1037/0003-066X.44.10.1276
76. Rosnow, R. L., & Rosenthal, R. (2009). Effect sizes: Why, when, and how to use them. Zeitschrift für Psychologie/Journal of Psychology, 217, 6-14.
77. Sawilowsky, S. S., & Blair, R. C. (1992). A more realistic look at the robustness and Type II error properties of the t test to departures from population normality. Psychological Bulletin, 111, 352-360. doi: 10.1037/0033-2909.111.2.352
78. Schlosser, R. W., Lee, D. L., & Wendt, O. (2008). Application of the percentage of non-overlapping data (PND) in systematic reviews and meta-analyses: A systematic review of reporting characteristics. Evidence-Based Communication Assessment and Intervention, 2, 163- 187. doi:10.1080/17489530802505412
79. Schmidt, F. L. (1996). Statistical significance testing and cumulative knowledge in psychology: Implications for training of researchers. Psychological Methods, 1, 115-129. doi:10.1037/1082-989X.1.2.115
80. Scruggs, T. E., & Mastropieri, M. A. (1998). Summarizing single-subject research: Issues and applications. Behavior Modification, 22, 221-242. doi:10.1177/01454455980223001
81. Shadish, W. R., Rindskopf, D. M., & Hedges, L. V. (2008). The state of the science in the meta-analysis of single-case experimental designs. Evidence-Based Communication Assessment and Intervention, 2, 188- 196. doi:10.1080/17489530802581603
82. Shadish, W. R., & Sullivan, K. J. (2011). Characteristics of single-case designs used to assess intervention effects in 2008. Behavior Research Methods, 43, 971-980. doi:10.3758/s13428-011-0111-y
83. Sidman, M. (1960). Tactics of scientific research. New York, NY: Basic Books.
84. Simonton, D. K. (1977). Cross-sectional time-series experiments: Some suggested statistical analyses. Psychological Bulletin, 84, 489-502. doi: 10.1037/0033-2909.84.3.489
85. Solanas, A., Manolov, R., & Onghena, P. (2010). Estimating slope and level change in N = 1 designs. Behavior Modification, 34, 195-218. doi:10.1177/0145445510363306
86. Solanas, A., Manolov, R., & Sierra, V. (2010). Lag-one autocorrelation in short series: Estimation and hypothesis testing. Psicológica, 31, 357- 381.
87. Strube, M. J. (1985). Combining and comparing significance levels from nonindependent hypothesis tests. Psychological Bulletin, 97, 334-341. doi:10.1037/0033-2909.97.2.334
88. Strube, M. J., & Miller, R. H. (1986). Comparison of power rates for combined probability procedures: A simulation study. Psychological Bulletin, 99, 407-415. doi:10.1037/0033-2909.99.3.407
89. Thompson, B. (2001). Significance, effect sizes, stepwise methods, and other issues: Strong arguments move the field. Journal of Experimental Education, 70, 80-93. doi:10.1080/00220970109599499
90. Vevea, J. L., & Woods, C. M. (2005). Publication bias in research synthesis: Sensitivity analysis using a priori weight functions. Psychological Methods, 10, 428-443. doi:10.1037/1082-989X.10.4.428
91. Wampold, B. E., & Furlong, M. J. (1981). Randomization tests in singlesubject designs: Illustrative examples. Journal of Psychopathology and Behavioral Assessment, 3, 329-341.
92. Wampold, B. E., Goodheart, C., & Levant, R. (2007). Clarification and elaboration on evidence-based practice in psychology. American Psychologist, 62, 616-618. doi:10.1037/0003-066X62.6.616
93. White, D. M., Rusch, F. R., Kazdin, A. E., & Hartmann, D. P. (1989). Applications of meta-analysis in individual subject research. Behavioral Assessment, 11, 281-296.
94. Whitlock, M. C. (2005). Combining probability from independent tests: The weighted Z-method is superior to Fisher's approach. Journal of Evolutionary Biology, 18, 1368-1373. doi:10.1111/j.1420-9101.2005.00917.x
95. Wilkinson, L., & the Task Force on Statistical Inference. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54, 594-604. doi:10.1037/0003-066X.54.8.594