Toward improved evidence standards and methods for rehabilitation: Recommendations and challenges

Archives of Physical Medicine and Rehabilitation

Volumn 93, Issue 8 SUPPL., 2012, Pages

  Johnston, Mark V ^a   Dijkers, Marcel P ^b  

^a UNIVERSITY OF WISCONSIN MILWAUKEE   (United States)

^b MOUNT SINAI SCHOOL OF MEDICINE   (United States)

Author keywords

Chronic disease; Disabled persons; Evidence based practice; Practice guidelines as topic; Rehabilitation; Research design; Systematic review as topic

Indexed keywords

ARTICLE; CLINICAL EVALUATION; CLINICAL PRACTICE; CLINICAL PROTOCOL; CONFLICT OF INTEREST; CONTROLLED STUDY; DATA SYNTHESIS; DISABILITY; EFFECT SIZE; EVIDENCE BASED PRACTICE; HEALTH CARE QUALITY; HUMAN; META ANALYSIS (TOPIC); OUTCOME ASSESSMENT; PEER REVIEW; PRACTICE GUIDELINE; PROFESSIONAL STANDARD; QUALITY OF LIFE; RANDOMIZED CONTROLLED TRIAL; REGRESSION ANALYSIS; REHABILITATION CARE; REHABILITATION MEDICINE; REHABILITATION RESEARCH; RELIABILITY; RISK REDUCTION; SYSTEMATIC REVIEW (TOPIC); TREATMENT OUTCOME;

EID: 84864203017     PISSN: 00039993     EISSN: 1532821X     Source Type: Journal    
DOI: 10.1016/j.apmr.2011.12.011     Document Type: Note

References (130)

1. M.V. Johnston, G.C. Vanderheiden, M.D. Farkas The challenge of evidence in disability and rehabilitation research and practice: a position paper 2009 National Center for Dissemination of Disability Research, Southwest Educational Development Laboratory Austin
2. M. Dijkers The value of "traditional" reviews in the era of systematic reviewing Am J Phys Med Rehabil 88 2009 423 430
3. Institute of Medicine Knowing what works in health care: a roadmap for the nation J. Eden, B. Wheatley, B. McNeil, H. Sox Committee on reviewing evidence to identify highly effective clinical services 2008 National Academies Pr Washington (DC)
4. D. Atkins, M. Eccles, S. Flottorp Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches The GRADE Working Group BMC Health Serv Res 4 2004 38
5. W. Edlund, G. Gronseth, Y. So, G. Franklin American Academy of Neurology clinical practice guideline process manual 2004 American Academy of Neurology St. Paul
6. J.P.T. Higgins, S. Green Cochrane handbook for systematic reviews of interventions 2009 Cochrane Collaboration Chichester
7. G.H. Guyatt, A.D. Oxman, G.E. Vist GRADE: an emerging consensus on rating quality of evidence and strength of recommendations BMJ 336 2008 924 926
8. S.G. West, V. King, T. Carey Systems to rate the strength of scientific evidence Evidence Report/Technology Assessment No. 47 2002 Agency for Healthcare Research and Quality Rockville AHRQ Publication No 02-E016
9. Agency for Health Care Research and Quality U.S. Preventive Services Task Force Procedure Manual AHRQ Publication No. 08-05118-EF, July 2008 2008 U.S. Government Rockville http://www.ahrq.gov/clinic/prevenix.htm Accessed March 8, 2012
10. J. Eden, L. Levit, A. Berg, S. Morton Committee on Standards for Systematic Reviews of Comparative Effectiveness Research Finding what works in health care: standards for systematic reviews editors 2011 Institute of Medicine, National Academies Pr Washington (DC)
11. R. Graham, M. Mancher, D.M. Wolman, S. Greenfield, E. Steinberg Clinical practice guidelines we can trust 2011 Institute of Medicine, National Academies Pr Washington (DC)
12. R. Rosenthal Interpersonal expectations: effects of the experimenter's hypothesis R.L. Rosnow, R. Rosenthal, People studying people: artifacts and ethics in behavioral research 2009 W.H. Freeman New York 138 210
13. D.L. Chambless, T.H. Ollendick Empirically supported psychological interventions: controversies and evidence Annu Rev Psychol 52 2001 685 716
14. G. Gronseth, L.K. Moses, T.S. Getchius Clinical practice guideline process manual: 2011 edition 2011 American Academy of Neurology St. Paul
15. G.H. Guyatt, A.D. Oxman, R. Kunz Incorporating considerations of resources use into grading recommendations BMJ 336 2008 1170 1173
16. J.L. Brozek, E.A. Akl, P. Alonso-Coello Grading quality of evidence and strength of recommendations in clinical practice guidelines Part 1 of 3. An overview of the GRADE approach and grading quality of evidence about interventions Allergy 64 2009 669 677
17. G. Guyatt, A.D. Oxman, E.A. Akl GRADE guidelines: 1 Introduction-GRADE evidence profiles and summary of findings tables J Clin Epidemiol 64 2011 383 394
18. G.H. Guyatt, A.D. Oxman, H.J. Schunemann, P. Tugwell, A. Knottnerus GRADE guidelines: a new series of articles in the Journal of Clinical Epidemiology J Clin Epidemiol 64 2011 380 382
19. M.P. Dijkers, S.L. Murphy, J. Krellman Evidence-based practice for rehabilitation professionals: concepts and controversies Arch Phys Med Rehabil 93 8 Suppl 2 2012 S164 S176
20. D.L. Sackett, S.E. Straus, W.S. Richardson, W.M. Rosenberg, R.B. Haynes Evidence-based medicine: how to practice and teach EBM 2000 Churchill Livingstone Edinburgh
21. R.B. Fitzpatrick PEDro: a physiotherapy evidence database Med Ref Serv Q 27 2008 189 198
22. D.T. Campbell, J.C. Stanley, N.L. Gage Experimental and quasi-experimental designs for research 1966 R. McNally Chicago
23. W.R. Shadish, T.D. Cook, D.T. Campbell Experimental and quasi-experimental designs for generalized causal inference 2002 Houghton Mifflin Boston
24. T.D. Cook, D.T. Campbell Quasi-experimentation: design and analysis issues for field settings 1979 Houghton Mifflin Boston
25. M.V. Johnston, K.J. Ottenbacher, C.S. Reichardt Strong quasi-experimental designs for research on the effectiveness of rehabilitation Am J Phys Med Rehabil 74 1995 383 392
26. G. Imbens, T. Lemieux National Bureau of Economic Research Regression discontinuity designs: a guide to practice 2007 National Bureau of Economic Research Cambridge
27. P. Schochet, T. Cook, J. Deke et al. Standards for regression discontinuity designs Institute for Educational Sciences, What Works Clearinghouse, Department of Education June 2010 Institute for Educational Sciences Washington (DC) editors http://ies.ed.gov/ncee/wwc/documentsum.aspx? sid=231 Accessed March 8, 2012
28. J.L. Brozek, E.A. Akl, E. Compalati Grading quality of evidence and strength of recommendations in clinical practice guidelines part 3 of 3 The GRADE approach to developing recommendations Allergy 66 2011 588 595
29. G.H. Guyatt, R.B. Haynes, R.Z. Jaeschke Users' Guides to the Medical Literature: XXV Evidence-based medicine: principles for applying the Users' Guides to patient care. Evidence-Based Medicine Working Group JAMA 284 2000 1290 1296
30. L.G. Portney, M.P. Watkins Foundations of clinical research: applications to practice 3rd ed. 2009 Pearson/Prentice Hall Upper Saddle River
31. R.W. Schlosser Synthesizing efficacy research in AAC R. Schlosser, L. Lloyd, R.W. Schlodber, The efficacy of augmentative and alternative communication 2003 Academic Pr San Diego 230 257
32. R.W. Schlosser The role of single-subject experimental designs in evidence-based practice times 2009 SEDL, National Center for Dissemination of Rehabilitation Research Austin, TX Technical Brief No. 22 2009
33. R.L. Tate, S. McDonald, M. Perdices, L. Togher, R. Schultz, S. Savage Rating the methodological quality of single-subject designs and n-of-1 trials: introducing the Singe-Case Experimental Design (SCED) Scale Neuropsychol Rehabil 18 2008 385 401
34. C.L. Backman, S.R. Harris, J.A. Chisholm, A.D. Monette Single-subject research in rehabilitation: a review of studies using AB, withdrawal, multiple baseline, and alternating treatments designs Arch Phys Med Rehabil 78 1997 1145 1153
35. J.E. Janosky, S.L. Leininger, M.P. Hoerger, T.M. Libkuman Single subject designs in biomedicine 2009 Springer Dordrecht
36. M.V. Johnston, R.O. Smith Single subject designs: current methodologies and future directions Occup Ther J Res 30 2010 4 10
37. T.R. Kratochwill, J. Hitchcock, R.H. Horner, J.R. Levin, S.L. Odom, D.M. Rindskopf, W.R. Shadish Single-case designs technical documentation 2010 What Works Clearinghouse, Institute of Educational Sciences, U.S. Department of Education, 2010 http://ies.ed.gov/ncee/wwc/pdf/wwc-scd.pdf Accessed February 8, 2012
38. D.L. Morgan, R.K. Morgan Single-case research methods for the behavioral and health sciences 2009 SAGE Los Angeles
39. A.E. Kazdin Single-case research designs: methods for clinical and applied settings 2nd ed. 2011 Oxford Univ Pr New York
40. J.D. Cryer, K.S. Chan Time series analysis with applications in R 2nd ed. 2008 Springer New York
41. L.K. Matowe, C.A. Leister, C. Crivera, J.M. Korth-Bradley Interrupted time series analysis in clinical research Ann Pharmacother 37 2003 1110 1116
42. C.R. Ramsay, L. Matowe, R. Grilli, J.M. Grimshaw, R.E. Thomas Interrupted time series designs in health technology assessment: lessons from two systematic reviews of behavior change strategies Int J Technol Assess Health Care 19 2003 613 623
43. K.J. Ottenbacher, S.R. Hinderer Evidence-based practice: methods to evaluate individual patient improvement Am J Phys Med Rehabil 80 2001 786 796
44. J.K. Lindsey Statistical analysis of stochastic processes in time 2004 Cambridge Univ Pr Cambridge
45. R. Harris, M. Helfand, S. Woolf Current methods of the US Preventive Services Task Force: a review of the process Am J Preventive Medicine 20 2001 21 35 (3 (Suppl)
46. B.L. Schneider American Educational Research Association Estimating causal effects: using experimental and observational designs: a think tank white paper 2007 American Educational Research Association Washington (DC)
47. D.F. Stroup, J.A. Berlin, S.C. Morton Meta-analysis of observational studies in epidemiology: a proposal for reporting JAMA 283 2000 2008 2012
48. J. Wang, P.T. Donnan, D. Steinke, T.M. MacDonald The multiple propensity score for analysis of dose-response relationships in drug safety studies Pharmacoepidemiol Drug Saf 10 2001 105 111
49. E. von Elm, D.G. Altman, M. Egger, S.J. Pocock, P.C. Gotzsche, J.P. Vandenbroucke Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies BMJ 335 2007 806 808
50. D.B. Rubin Estimating causal effects from large data sets using propensity scores Ann Intern Med 127 1997 757 763
51. B.M. Psaty, T.D. Koepsell, D. Lin Assessment and control for confounding by indication in observational studies J Am Geriatr Soc 47 1999 749 754
52. National Research Council (U.S.) Transportation Research Board Statistical methods 2008 Transportation Research Board, National Academy of Sciences Washington (DC)
53. R.J. Little, D.B. Rubin Causal effects in clinical and epidemiological studies via potential outcomes: concepts and analytical approaches Annu Rev Public Health 21 2000 121 145
54. R. Kunz Randomized trials and observational studies: still mostly similar results, still crucial differences J Clin Epidemiol 61 2008 207 208
55. A. Jahn, O. Razum Observational studies for intervention assessment Lancet 357 2001 2141
56. J.W. Hogan, T. Lancaster Instrumental variables and inverse probability weighting for causal inference from longitudinal observational studies Stat Methods Med Res 13 2004 17 48
57. A. Hartz, S. Bentler, M. Charlton Assessing observational studies of medical treatments Emerg Themes Epidemiol 2 2005 8
58. H. Comber, I.J. Perry Observational studies for intervention assessment Lancet 357 2001 2141 2142
59. N. Black Why we need observational studies to evaluate the effectiveness of health care BMJ 312 1996 1215 1218
60. A. Basu, J.J. Heckman, S. Navarro-Lozano, S. Urzua Use of instrumental variables in the presence of heterogeneity and self-selection: an application to treatments of breast cancer patients Health Econ 16 2007 1133 1157
61. M. Rutter Proceeding from observed correlation to causal inference: the use of natural experiments Perspect Psychol Sci 2 2007 377 395
62. J. Odgaard-Jensen, G. Vist, A. Timmer Randomisation to protect against selection bias in healthcare trials Cochrane Database Syst Rev 4 2011 Apr 13 MR000012
63. P.R. Rosenbaum Observational studies 2nd ed. 2010 Springer New York
64. J. Heckman, S. Navarro-Lozano Using matching, instrumental variables, and control functions to estimate economic choice models Rev Econ Stat 86 2004 30 57
65. K. Benson, A.J. Hartz A comparison of observational studies and randomized controlled trials N Engl J Med 342 2000 1878 1886
66. P.M. Steiner, T.D. Cook, W.R. Shadish, M.H. Clark The importance of covariate selection in controlling for selection bias in observational studies Psychol Methods 15 2010 250 267
67. S.G. West, N. Duan, W. Pequegnat Alternatives to the randomized controlled trial Am J Public Health 98 2008 1359 1366
68. M.S. Cepeda, R. Boston, J.T. Farrar, B.L. Strom Comparison of logistic regression versus propensity score when the number of events is low and there are multiple confounders Am J Epidemiol 158 2003 280 287
69. E.P. Martens, W.R. Pestman, A. de Boer, S.V. Belitser, O.H. Klungel Instrumental variables: application and limitations Epidemiol 17 2006 260 267
70. J.P. Newhouse, M. McClellan Econometrics in outcomes research: the use of instrumental variables Annu Rev Pub Health 19 1998 17 34
71. A. Liberati, D.G. Altman, J. Tetzlaff The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration J Clin Epidemiol 62 2009 e1 e34
72. D. Moher, A. Liberati, J. Tetzlaff, D.G. Altman Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement J Clin Epidemiol 62 2009 1006 1012
73. H.M. Cooper, L.V. Hedges, J.C. Valentine The handbook of research synthesis and meta-analysis 2nd ed. 2009 Russell Sage Foundation New York
74. E.S. Robert Best evidence synthesis: an intelligent alternative to meta-analysis J Clin Epidemiol 48 1995 9 18
75. GRADEpro. [Computer program]. Version 3.2 for Windows. Jan Brozek, Andrew Oxman, Holger Schünemann, 2008.
76. R.E. Slavin Best evidence synthesis: an intelligent alternative to meta-analysis J Clin Epidemiol 48 1995 9 18
77. M.V. Johnston, K.J. Ottenbacher, J.E. Graham, P.A. Findley, A.C. Hansen Systematically assessing and improving the quality and outcomes of medical rehabilitation programs W.R. Frontera, J.A. DeLisa, DeLisa's physical medicine & rehabilitation: principles and practice 5th ed. 2010 Wolters Kluwer/Lippincott Williams & Wilkins Health Philadelphia 325 355
78. R.A. Keith Treatment strength in rehabilitation Arch Phys Med Rehabil 78 1997 1298 1304
79. J.M. Hartman, J.W. Forsen Jr, M.S. Wallace, J.G. Neely Tutorials in clinical research: part IV: recognizing and controlling bias Laryngoscope 112 2002 23 31
80. M.V. Johnston, M. Sherer, J. Whyte Applying evidence standards to rehabilitation research: an overview Am J Phys Med Rehabil 85 2006 292 309
81. L. Wood, M. Egger, L.L. Gluud Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study BMJ 336 2008 601 605
82. A.A. Stone, J.S. Turkkan, C.A. Bachrach, J.B. Jobe, H.S. Kurtzman, V.S. Cain The science of self-report: implications for research and practice 2000 Lawrence Erlbaum Mahwah
83. D.A. Dillman, J.D. Smyth, L.M. Christian Internet, mail, and mixed-mode surveys: the tailored design method 3rd ed. 2008 Wiley Hoboken
84. T.F. van de Mortel Faking it: social desirability response bias in self-report research Austrailan J Adv Nursing 25 2008 40 48
85. E. Stokes Rehabilitation outcome measures 2011 Churchill-Livingston Phildelphia
86. J. Lexchin Those who have the gold make the evidence: how the pharmaceutical industry biases the outcomes of clinical trials of medications Sci Eng Ethics 2011 Feb 15 [Epub ahead of print]
87. J. Lexchin, L.A. Bero, B. Djulbegovic, O. Clark Pharmaceutical industry sponsorship and research outcome and quality: systematic review BMJ 326 2003 1167 1170
88. American Educational Research Association, American Psychological Association, National Council on Measurement in Education, Joint Committee on Standards for Educational and Psychological Testing (U.S.) Standards for educational and psychological testing 1999 American Educational Research Association Washington (DC)
89. M.V. Johnston, R.A. Keith, S.R. Hinderer Measurement standards for interdisciplinary medical rehabilitation Arch Phys Med Rehabil 73 12-S 1992 S3 S23
90. K.J. Conrad International conference on objective measurement: applications of Rasch analysis in health care Med Care 42 Suppl 1 2004 1 6
91. C.A. McHorney, P.O. Monahan Postscript: applications of Rasch analysis in health care Med Care 42 1 Suppl 2004 I73 I78
92. E.M. Andresen Criteria for assessing the tools of disability outcomes research Arch Phys Med Rehabil 81 12 Suppl 2 2000 S15 S20
93. M.V. Johnston, D. Graves, M. Greene The uniform postacute assessment tool: systematically evaluating the quality of measurement evidence Arch Phys Med Rehabil 88 2007 1505 1512
94. M.V. Johnston, D.E. Graves Towards guidelines for evaluation of measures: an introduction with application to spinal cord injury J Spinal Cord Med 31 2008 13 26
95. D.T. Campbell, J.C. Stanley, N.L. Gage Experimental and quasi-experimental designs for research 1981 Houghton Mifflin Boston
96. G.H. Guyatt, A.D. Oxman, R. Kunz, G.E. Vist, Y. Falck-Ytter, H.J. Schunemann What is "quality of evidence" and why is it important to clinicians? BMJ 336 2008 995 998
97. S. Hillier, K. Grimmer-Somers, T. Merlin FORM: an Australian method for formulating and grading recommendations in evidence-based clinical guidelines BMC Med Res Methodol 11 2011 23
98. L. Friedman, C.D. Furberg, D. DeMets Fundamentals of clinical trials 4th ed. 2010 Springer New York
99. S.M. Haley, W.J. Coster, K. Binda-Sundberg Measuring physical disablement: the contextual challenge Phys Ther 74 1994 443 451
100. I. Boutron, D. Moher, D.G. Altman, K.F. Schulz, P. Ravaud Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration Ann Intern Med 148 2008 295 309
101. P. Craig, P. Dieppe, S. Macintyre, S. Michie, I. Nazareth, M. Petticrew Developing and evaluating complex interventions: the new Medical Research Council guidance BMJ 337 2008 1655
102. M.P. Dijkers External validity in research on rehabilitation interventions: issues for knowledge translation Focus Technical Brief No. 33 2011 National Center for Dissemination of Disability Research, SEDL Austin, TX http://www.ncddr.org/kt/products/focus/focus33/Focus33.pdf Accessed February 8, 2012
103. K.S. Dobson, K.D. Craig Empirically supported therapies: best practice in professional psychology 1998 Sage Thousand Oaks
104. S. Rollnick, W.R. Miller, C.C. Butler Motivational interviewing in health care; helping patients change behavior 2008 Guilford Pr New York
105. J.C. Muran, J.P. Barber The therapeutic alliance: an evidence-based guide to practice 2010 Guilford Pr New York
106. G. Kielhofner Conceptual foundations of occupational therapy practice 4th ed. 2009 F.A. Davis Co Philadelphia
107. Cochrane Collaboration Glossary of terms in The Cochrane Collaboration http://www.cochrane.org/sites/default/files/uploads/glossary.pdf 2005 Accessed July 13, 2011
108. S.R. Tunis, D.B. Stryer, C.M. Clancy Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy JAMA 290 2003 1624 1632
109. G.A. Fransen, C.J. van Marrewijk, S. Mujakovic Pragmatic trials in primary care Methodological challenges and solutions demonstrated by the DIAMOND-study BMC Med Res Methodol 7 2007 16
110. T.D. Cook A quasi-sampling theory of the generalization of causal relationships New Directions for Program Evaluation 1993 1993 39 82
111. R.L. Brennan Generalizability theory 2001 Springer New York
112. R.J. Shavelson, N.M. Webb Generalizability theory: a primer 1991 Sage Publications Newbury Park
113. S.D. Horn, J. Gassaway Practice based evidence: incorporating clinical heterogeneity and patient-reported outcomes for comparative effectiveness research Med Care 48 6 Suppl 2010 S17 S22
114. L.J. Damschroder, D.C. Aron, R.E. Keith, S.R. Kirsh, J.A. Alexander, J.C. Lowery Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science Implement Sci 4 2009 50
115. A. Qaseem, V. Snow, D.K. Owens, P. Shekelle The development of clinical practice guidelines and guidance statements of the American College of Physicians: summary of methods Ann Intern Med 153 2010 194 199
116. M.M. Haby, M. Donnelly, J. Corry, T. Vos Cognitive behavioural therapy for depression, panic disorder and generalized anxiety disorder: a meta-regression of factors that may predict outcome Aust N Z J Psychiatry 40 2005 9 19
117. A. Roth, P. Fonagy What works for whom? A critical review of the psychotherapy research 2nd ed. 2005 Guilford Pr New York
118. B. Arroll, C.R. Elley, T. Fishman Antidepressants versus placebo for depression in primary care Cochrane Database Syst Rev 3 2009 Jul 8 CD007954
119. G.H. Guyatt, A.D. Oxman, R. Kunz Going from evidence to recommendations BMJ 336 2008 1049 1051
120. Institute of Medicine Crossing the quality chasm: a new health care system for the 21st century 2001 National Academy Pr Washington (DC)
121. G.C. Smith, J.P. Pell Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials BMJ 327 2003 1459 1461
122. R.W. Schlosser, R. Koul, J. Costello Asking well-built questions for evidence-based practice in augmentative and alternative communication J Commun Disord 40 2007 225 238
123. M.J. Scherer Assistive technology: matching device and consumer for successful rehabilitation 1st ed. 2002 American Psychological Association Washington (DC)
124. A.M. Cook, S.M. Hussey Assistive technologies: principles and practice 2nd ed. 2002 Mosby St. Louis
125. M.J. Fuhrer Assistive technology outcomes research: challenges met and yet unmet Am J Phys Med Rehabil 80 2001 528 535
126. W.C. Mann, K.J. Ottenbacher, L. Fraas, M. Tomita, C.V. Granger Effectiveness of assistive technology and environmental interventions in maintaining independence and reducing home care costs for the frail elderly A randomized controlled trial Arch Fam Med 8 1999 210 217
127. M.V. Johnston, J. Case-Smith Development and testing of interventions in occupational therapy: towards a new generation of research in occupational therapy Occup Ther J Res 29 2009 4 13
128. M.W. Lipsey Theory as method: small theories of treatment New Directions for Program Evaluation 57 1993 5 38
129. D.P. MacKinnon Introduction to statistical mediation analysis 2008 Lawrence Erlbaum Associates New York
130. J. Whyte Using treatment theories to refine the designs of brain injury rehabilitation treatment effectiveness studies J Head Trauma Rehabil 21 2006 99 106