Statistical, practical, clinical, and personal significance: Definitions and applications in speech-language pathology

American Journal of Speech-Language Pathology

Volumn 20, Issue 3, 2011, Pages 233-242

  Bothe, Anne K ^a   Richardson, Jessica D ^b  

^a UNIVERSITY OF GEORGIA   (United States)

^b UNIVERSITY OF SOUTH CAROLINA   (United States)

Author keywords

Clinical significance; Efficacy; Treatment outcomes

Indexed keywords

EID: 79961212796     PISSN: 10580360     EISSN: 15589110     Source Type: Journal    
DOI: 10.1044/1058-0360(2011/10-0034)     Document Type: Article

References (111)

1. Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., for the CONSORT Group. (2001). The revised CONSORT statement for reporting randomized trials: Explanation and elaboration. Annals of Internal Medicine, 134, 663-694.
2. American Psychological Association. (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.
3. American Psychological Association. (2009). Publication manual of the American Psychological Association (6th ed.). Washington, DC: Author.
4. American Speech-Language-Hearing Association. (2010). Code of ethics. Available from: www.asha.org/policy.
5. Anderson, E. M., & Lambert, M. J. (2001). A survival analysis of clinically significant change in outpatient psychotherapy. Journal of Clinical Psychology, 57, 875-888.
6. Anderson, W. T., Fox, E. P., & Fulcher, D. G. (1976). Bank selection decision and market segmentation. Journal of Marketing, 40, 40-45.
7. Ankuta, G. Y., & Abeles, N. (1993). Client satisfaction, clinical significance, and meaningful change in psychotherapy. Professional Psychology: Research and Practice, 24, 70-74.
8. Apel, K., & Masterson, J. J. (2001). Theory-guided spelling assessment and intervention: A case study. Language, Speech, and Hearing Services in Schools, 32, 182-195.
9. Baer, D. M. (1988). If you know why you're changing a behavior, you'll know when you've changed it enough. Behavioral Assessment, 10, 219-223.
10. Bain, B. A., & Dollaghan, C. A. (1991). Treatment efficacy: The notion of clinically significant change. Language, Speech, and Hearing Services in Schools, 22, 264-270.
11. Bakan, D. (1966). The test of significance in psychological research. Psychological Bulletin, 66, 423-437.
12. Barlow, D. H., & Hersen, M. (1984). Single case experimental designs: Strategies for studying behavior change (2nd ed.). New York, NY: Pergamon.
13. Battle, C. C., Imber, S. D., Hoehn-Saric, R., Stone, A. R., Nash, K. R., & Frank, J. D. (1966). Target complaints as criteria of improvement. American Journal of Psychotherapy, 20, 184-192.
14. Beal, D., & Duckro, P. (2003). Empirically documenting clinically significant change in rational emotive behavior therapy. Journal of Rational-Emotive and Cognitive-Behavior Therapy, 21(2), 75-88.
15. Berkson, J. (1938). Some difficulties of interpretation encountered in the application of the chi-square test. Journal of the American Statistical Association, 33, 526-536.
16. Beutler, L. E., & Hamblin, D. L. (1986). Individualized outcome measures of internal change: Methodological considerations. Journal of Consulting and Clinical Psychology, 54, 48-53.
17. Black, D. A. K., & Pole, J. D. (1975). Priorities in biomedical research: Indices of burden. British Journal of Preventative and Social Medicine, 29, 222-227.
18. Boring, E. G. (1919). Mathematical vs. scientific importance. Psychological Bulletin, 16, 335-338.
19. Bothe, A. K. (2003). Evidence-based treatment of stuttering: V. The art of clinical practice and the future of clinical research. Journal of Fluency Disorders, 28, 247-258.
20. Brookshire, R. H. (1986). An introduction to aphasia (3rd ed.). Minneapolis, MN: BRK.
21. Carver, R. (1978). The case against statistical significance testing. Harvard Educational Review, 48, 378-399.
22. Cella, D., Yount, S., Rothrock, N., Gershon, R., Cook, K., Reeve, B., on behalf of the PROMIS cooperative group. (2007). The Patient Reported Outcomes Measurement Information System (PROMIS): Progress of an NIH Roadmap Cooperative Group during its first two years. Medical Care, 45(5), S3-S11.
23. Christensen, L., & Mendoza, J. L. (1986). A method of assessing change in a single subject: An alteration of the RC index. Behavior Therapy, 17, 305-308.
24. Cleary, P. D., & McNeil, B. J. (1988). Patient satisfaction as an indicator of quality care. Inquiry: A Journal of Medical Care Organization, Provision and Financing, 25, 25-36.
25. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
26. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155-159.
27. Copay, A. G., Subach, B. R., Glassman, S. D., Polly, D. W., Jr., & Schuler, T. C. (2007). Understanding the minimum clinically important difference: A review of concepts and methods. The Spine Journal, 7, 541-546.
28. Cordes, A. K. (1994). The reliability of observational data: I. Theories and methods for speech-language pathology. Journal of Speech and Hearing Research, 37, 264-278.
29. Cortina, J. M., & Landis, R. S. (2009). When small effect sizes tell a big story, and when large effect sizes don't. In C. E. Lance & R. J. Vandenberg (Eds.), Statistical and methodological myths and urban legends: Doctrine, verity and fable in the organizational and social sciences (pp. 287-308). New York, NY: Routledge.
30. Costello, J. M., & Ingham, R. J. (1984). Assessment strategies for stuttering. In R. F. Curlee & W. H. Perkins (Eds.), Nature and treatment of stuttering: New directions (pp. 303-333). San Diego, CA: College Hill Press.
31. Donabedian, A.(1988). The qualityofcare: How canitbeassessed? JAMA: The Journal of the American Medical Association, 260, 1743-1748.
32. Elman, R. J., & Bernstein-Ellis, E. (1999). The efficacy of group communication treatment in adults with chronic aphasia. Journal of Speech, Language, and Hearing Research, 42, 411-419.
33. Emanuel, E. J., & Emanuel, L. L. (1996). What is accountability in health care? Annals of Internal Medicine, 124, 229-239.
34. Evans, C., Margison, F., & Barkham, M. (1998). The contribution of reliable and clinically significant change methods to evidence-based mental health. Evidence-Based Mental Health, 1(3), 70-72.
35. Fayers, P. M., & Machin, D. (2007). Quality of life: The assessment, analysis and interpretation of patient-reported outcomes (2nd ed.). Chichester, England: Wiley.
36. Finn, P. (2003). Evidence-based treatment of stuttering: 2. Clinical significance of behavioral stuttering treatments. Journal of Fluency Disorders, 28, 209-218.
37. Fitzpatrick, R., Fletcher, A., Gore, S., Jones, D., Spiegelhalter, D., & Cox, D. (1992). Quality of life measures in health care: 1. Applications and issues in assessment. British Medical Journal, 305, 1074-1077.
38. Follette, W. C., & Callaghan, G. M. (2001). The evolution of clinical significance. Clinical Psychology: Science and Practice, 8, 431-435.
39. Ford, R. D., Bach, S. A., & Fottler, M. D. (1997). Methods of measuring patient satisfaction in health care organizations. Health Care Management Review, 22, 74-89.
40. Foster, S. L., & Mash, E. J. (1999). Assessing social validity in clinical treatment research: Issues and procedures. Journal of Consulting and Clinical Psychology, 67, 308-319.
41. Franic, D. M., & Bothe, A. K. (2008). Psychometric evaluation of condition-specific instruments used to assess health-related quality of life, attitudes, and related constructs in stuttering. American Journal of Speech-Language Pathology, 17, 60-80.
42. Franic, D. M., Haddock, S. M., Tucker, L. T., & Wooten, N. (2008). Pharmacy patronage: Identifying key factors in the decision-making process using the determinant attribute approach. Journal of the American Pharmacists Association, 48, 71-85.
43. Freeman, M., deMarrais, K., Preissle, J., Roulston, K., & St. Pierre, E. A. (2007). Standards of evidence in qualitative research: An incitement to discourse. Educational Researcher, 36, 25-32.
44. Friel-Patti, S., DesBarres, K., & Thibodeau, L. (2001). Case studies of children using Fast For Word. American Journal of Speech-Language Pathology, 10, 203-215.
45. Gardner, M. J., & Altman, D. G. (1986). Confidence intervals rather than P values: Estimation rather than hypothesis testing. British Medical Journal, 292, 746-750.
46. Gillam, R. B., Crofford, J. A., Gale, M. A., & Hoffman, L. M. (2001). Language change following computer-assisted language instruction with Fast For Word or Laureate Learning Systems software. American Journal of Speech-Language Pathology, 10, 231-247.
47. Hageman, W. J., & Arrindell, W. A. (1999). Establishing clinically significant change: Increment of precision and the distinction between individual and group level of analysis. Behavioral Research Therapy, 37, 1169-1193.
48. Holland, A. H., Frattali, C., & Fromm, D. (1999). Communication Activities of Daily Living, Second Edition. Austin, TX: Pro-Ed.
49. Houle, T. T., & Stump, D. A. (2008). Statistical significance versus clinical significance. Seminars in Cardiothoracic and Vascular Anesthesia, 12, 5-6.
50. Hugdahl, K., & Ost, L. (1981). On the difference between statistical and clinical significance. Behavioral Assessment, 3, 289-295.
51. Ingham, J. C. (1990). Issues of treatment efficacy: Design and experimental control. In L. B. Olswang, C. K. Thompson, S. F. Warren, & N. J. Minghetti (Eds.), Treatment efficacy research in communication disorders (pp. 51-61). Rockville, MD: American Speech-Language-Hearing Foundation.
52. Ingham, R. J., & Cordes, A. K. (1997). Self-measurement and evaluating stuttering treatment efficacy. In R. F. Curlee & G. S. Siegel (Eds.), Nature and treatment of stuttering: New directions (Rev. ed., pp. 413-437). Boston, MA: Allyn & Bacon.
53. Jacobson, N., Follette, W. C., & Revenstorf, D. (1984). Psychotherapy outcome research: Methods of reporting variability and evaluating clinical significance. Behavior Therapy, 15, 336-352.
54. Jacobson, N. S., Roberts, L. J., Berns, S. B., & McGlinchey, J. B. (1999). Methods for defining and determining the clinical significance of treatment effects: Description, application, and alternatives. Journal of Consulting and Clinical Psychology, 67, 300-307.
55. Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, 12-19.
56. Jaeschke, R., Singer, J., & Guyatt, G. H. (1989). Measurement of health status: Ascertaining the minimal clinically important difference. Controlled Clinical Trials, 10, 407-415.
57. Johnson, C. J. (2006). Getting started in evidence-based practice for childhood speech-language disorders. American Journal of Speech-Language Pathology, 15, 20-35.
58. Juniper, E. F., Guyatt, G. H., Streiner, D. L., & King, D. R. (1997). Clinical impact versus factor analysis for quality of life questionnaire construction. Journal of Clinical Epidemiology, 50, 233-238.
59. Kagan, A., & LeBlanc, K. (2002). Motivating for infrastructure change: Toward a communicatively accessible, participation-based stroke care system for all those affected by aphasia. Journal of Communication Disorders, 35(2), 153-169.
60. Katz, R. C., & Wertz, R. T. (1997). The efficacy of computer-provided reading treatment for chronic aphasic adults. Journal of Speech, Language, and Hearing Research, 40, 493-507.
61. Kazdin, A. E. (1977). Assessing the clinical or applied importance of behavior change through social validation. Behavior Modification, 1, 427-452.
62. Kazdin, A. E. (1982). Single-case research designs: Methods for clinical and applied settings. New York, NY: Oxford.
63. Kazdin, A. E. (1999). The measurement and meanings of clinical significance. Journal of Consulting and Clinical Psychology, 67, 332-339.
64. Kazdin, A. E., & Kendall, P. C. (1998). Current progress and future plans for developing effective treatments: Comments and perspectives. Journal of Clinical Child Psychology, 27, 217-226.
65. Kearns, K. P., & Simmons, N. N. (1988). Interobserver reliability and perceptual ratings. Journal of Speech and Hearing Research, 31, 131-136.
66. Kendall, P. C., & Norton-Ford, J. D. (1982). Therapy outcome research methods. In P. C. Kendall & J. N. Butcher (Eds.), Handbook of research methods in clinical psychology (pp. 429-460). New York, NY: Wiley.
67. Keppel, G., & Wickens, T. D. (2004). Design and analysis: A researcher's handbook (4th ed.). Upper Saddle River, NJ: Pearson Prentice Hall.
68. Kirk, R. (1996). Practical significance: A concept whose time has come. Educational and Psychological Measurement, 56, 746-759.
69. Kirk, R. E. (2001). Promoting good statistical practices: Some suggestions. Educational and Psychological Measurement, 61, 213-218.
70. Lauridsen, H. H., Hartvigsen, J., Manniche, C, Korsholm, L., & Grunnet-Nilsson, N. (2006). Responsiveness and minimal clinically important difference for pain and disability instruments in low back pain patients. BMC Musculoskeletal Disorders, 7, 82.doi:10.1186/1471-2474-7-82.
71. Lenth, R. V. (2001). Some practical guidelines for effective sample size determination. The American Statistician, 55, 187-193.
72. LPAA Project Group. (2001). Life participation approach to aphasia: A statement of values for the future. In R. Chapey (Ed.), Language intervention strategies in aphasia and related neurogenic communication disorders (4th ed., pp. 235-245). Baltimore, MD: Lippincott Williams & Wilkins.
73. Mazor, K. M., Clauser, B. E., Field, T., Yood, R. A., & Gurwitz, J. H. (2002). A demonstration of the impact of response bias on the results of patient satisfaction surveys. HSR: Health Services Research, 37, 1403-1417.
74. McCauley, R. J., & Swisher, L. (1984). Psychometric review of language and articulation tests for preschool children. Journal of Speech and Hearing Disorders, 49, 34-42.
75. McHorney, C. A., & Tarlov, A. R. (1995). Individual-patient monitoring in clinical practice: Are available health status surveys adequate? Quality of Life Research, 4, 293-307.
76. Meline, T. (2010). A research primer for communication sciences and disorders. Boston, MA: Pearson.
77. Meline, T., & Paradiso, T. (2003). Evidence-based practice in schools: Evaluating research and reducing barriers. Language, Speech, and Hearing Services in Schools, 34, 273-283.
78. Meline, T., & Schmitt, J. F. (1997). Case studies for evaluating statistical significance in group designs. American Journal of Speech-Language Pathology, 6(1), 33-41.
79. Meline, T., & Wang, B. (2004). Effect-size reporting practices in AJSLP and other ASHA journals, 1999-2003. American Journal of Speech-Language Pathology, 13, 202-207.
80. Mintz, J. (1972). What is success in psychotherapy? Journal of Abnormal Psychology, 80, 11-19.
81. Morrison, D. E., & Henkel, R. E. (Eds.). (1970). The significance test controversy. Chicago, IL: Aldine.
82. Murray, C. J. L. (1994). Quantifying the burden of disease: The technical basis for disability adjusted life years. Bulletin of the World Health Organization, 72, 429-445.
83. Nickerson, R. S. (2000). Null hypothesis significance testing: A review of an old and continuing controversy. Psychological Methods, 5, 241-301.
84. Nunnally, J. C, & Kotsche, W. E. (1983). Studies of individual subjects: Logic and methods of analysis. The British Journal of Clinical Psychology, 22, 83-93.
85. Ogles, B. M., Lunnen, K. M., & Bonesteel, K. (2001). Clinical significance: History, application, and current practice. Clinical Psychology Review, 21, 421-446.
86. Onslow, M., Jones, M., O'Brian, S., Menzies, R., & Packman, A. (2008). Defining, identifying, and evaluating clinical trials of stuttering treatments: A tutorial for clinicians. American Journal of Speech-Language Pathology, 17, 401-415.
87. Peterson, L. S. (2008, February). "Clinical" significance: "Clinical" significance and "practical" significance are NOT the same things. Paper presented at the annual meeting of the Southwest Educational Research Association, New Orleans, LA.
88. Rosen, A., & Proctor, E. K. (1981). Distinctions between treatment outcomes and their implications for treatment evaluation. Journal of Consulting and Clinical Psychology, 49, 418-425.
89. Rosenbek, J. C, LaPointe, L. L., & Wertz, R. T. (1989). Aphasia: A clinical approach. Boston, MA: College Hill Press.
90. Schiavetti, N., Metz, D. E., & Orlikoff, R. F. (2011). Evaluating research in communicative disorders (6th ed.). Boston, MA: Pearson.
91. Schuele, C, & Justice, L. (2006, August 15). The importance of effect sizes in the interpretation of research: Primer on research: Part 3. The ASHA Leader. Retrieved from www.asha.org/Publications/leader/2006/060815/f060815d/.
92. Shriberg, L. D., & Lof, G. L. (1991). Reliability studies in broad and narrow phonetic transcription. Clinical Linguistics and Phonetics, 5, 225-279.
93. Shumak, I. C. (1955). A speech rating sheet for stutterers. In W. Johnson & R. R. Leutenegger (Eds.), Stuttering in children and adults: Thirty years of research at the University of Iowa (pp. 341-347). Minneapolis: University of Minnesota Press.
94. Sloan, J., Symonds, T., Vargas-Chanes, D., & Fridley, B. (2003). Practical guidelines for assessing the clinical significance of health-related quality of life changes within clinical trials. Drug Information Journal, 37, 23-31.
95. Straus, S. E., Richardson, W. S., Glasziou, P., & Haynes, R. B. (2005). Evidence-based medicine: How to practice and teach EBM (3rd ed.). Edinburgh, United Kingdom: Elsevier Churchill Livingstone.
96. Strupp, H. H., & Hadley, S. W. (1977). A tripartite model of mental health and therapeutic outcome: With special reference to negative effects in psychotherapy. American Psychologist, 32, 187-196.
97. Swanson, L. A., Fey, M. E., Mills, C. E., & Hood, L. S. (2005). Use of narrative-based language intervention with children who have specific language impairment. American Journal of Speech-Language Pathology, 14, 131-143.
98. Thompson, B. (2002). "Statistical," "practical," and "clinical": How many kinds of significance do counselors need to consider? Journal of Counseling & Development, 80, 64-71.
99. Thompson, M. S. (1986). Willingness to pay and accept risks to cure chronic disease. American Journal of Public Health, 76, 392-396.
100. Tingey, R. C, Lambert, M. J., Burlingame, G. M., & Hansen, N. B. (1996). Clinically significant change: Practical indicators for evaluating psychotherapy outcome. Psychotherapy Research, 6(2), 144-153.
101. Tukey, J. W. (1991). The philosophy of multiple comparisons. Statistical Science, 6, 100-116.
102. Wade, D. T., Smeets, R. J. E. M., & Verbunt, J. A. (2010). Research in rehabilitation medicine: Methodological challenges. Journal of Clinical Epidemiology, 63, 699-704.
103. Walker, S., & Rosser, R. (1993). Quality of life assessment: Key issues in the 1990s. Boston, MA: Kluwer Academic.
104. Walker-Batson, D., Curtis, S., Natarajan, R., Ford, J., Dronkers, N., Salmeron, E., Unwin, D. H. (2001). A double-blind, placebo-controlled study of the use of amphetamine in the treatment of aphasia. Stroke, 32, 2093-2098.
105. Wiklund, I. (2004). Assessment of patient-reported outcomes in clinical trials: The example of health-related quality of life. Fundamental & Clinical Pharmacology, 18, 351-363.
106. Wolf, M. M. (1978). Social validity: The case of subjective measurement or how applied behavior analysis is finding its heart. Journal of Applied Behavior Analysis, 11, 203-214.
107. Woolf, G. (1967). The assessment of stuttering as struggle, avoidance, and expectancy. British Journal of Disorders of Communication, 2, 158-171.
108. World Health Organization. (1980). Classification of impairments, disabilities and handicaps. Geneva, Switzerland: Author.
109. World Health Organization. (2001). International classification of functioning, disability, and health. Geneva, Switzerland: Author.
110. Young, M. A. (1976). Application of regression analysis concepts to retrospective research in speech pathology. Journal of Speech and Hearing Research, 19, 5-18.
111. Young, M. A. (1993). Supplementing tests of statistical significance: Variation accounted for. Journal of Speech and Hearing Research, 36, 644-656.