The grace checklist for rating the quality of observational studies of comparative effectiveness: A tale of hope and caution

Journal of Managed Care Pharmacy

Volumn 20, Issue 3, 2014, Pages 301-308

  Dreyer, Nancy A ^a   Velentgas, Priscilla ^a   Westrich, Kimberly ^b   Dubois, Robert ^c  

^a IQVIA   (United States)

^b Health Services Engineering   (United States)

^c National Pharmaceutical Council   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CHECKLIST; COMPARATIVE EFFECTIVENESS; DECISION SUPPORT SYSTEM; DRUG EFFICACY; DRUG SAFETY; GOOD RESEARCH FOR COMPARATIVE EFFECTIVENESS CHECKLIST; MEDICAL DEVICE; MEDICAL PROCEDURES; OBSERVATIONAL STUDY; PREDICTIVE VALUE; QUALITY CONTROL; SENSITIVITY ANALYSIS; VALIDITY;

CHECKLIST; COMPARATIVE EFFECTIVENESS RESEARCH; DECISION MAKING; HUMANS; OBSERVATIONAL STUDY AS TOPIC; RESEARCH DESIGN;

EID: 84896282254     PISSN: 10834087     EISSN: None     Source Type: Journal    
DOI: 10.18553/jmcp.2014.20.3.301     Document Type: Article

References (38)

1. Greene JA, Podolsky SH. Reform, regulation, and pharmaceuticals-the Kefauver-Harris Amendments at 50. N Engl J Med. 2012;367(16):1481-83. Available at: http://www.nejm.org/doi/full/10.1056/NEJMp1210007. Accessed November 14, 2013.
2. Dreyer NA, Tunis SR, Berger M, Ollendorf D, Mattox P, Gliklich R. Why observational studies should be among the tools used in comparative effectiveness research. Health Aff (Millwood). 2010;29(10):1818-25.
3. Dreyer NA, Schneeweiss S, McNeil B, et al. GRACE principles: recognizing high-quality observational studies of comparative effectiveness. Am J Manag Care. 2010;16(6):467-71.
4. Wells GA, Shea B, O'Connell D, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Available at: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp. Accessed November 14, 2013.
5. Agency for Healthcare Research and Quality. Assessing confounding, the risk of bias and precision of observational studies of interventions or exposures: further development of the RTI Item Bank. Evidence-Based Practice Center. Draft Methods Research Report. May 2012. Available at: http://www.effectivehealthcare.ahrq.gov/ehc/products/414/1272/Observational-studiesrefinement_ DraftReport_20120927.pdf. Accessed January 8, 2014.
6. Thomas L, Peterson ED. The value of statistical analysis plans in observational research: defining high-quality research from the start. JAMA. 2012;308(8):773-74.
7. Luce BR, Drummond MF, Dubois RW, et al. Principles for planning and conducting comparative effectiveness research. J Comp Eff Res. 2012;1(5):431-40.
8. Velentgas P, Dreyer NA, Nourjah P, Smith SR, Torchia MM, eds. Developing a Protocol for Observational Comparative Effectiveness Research: A User's Guide. AHRQ Publication No. 12(13)-EHC099. Rockville, MD: Agency for Healthcare Research and Quality; January 2013.
9. Willke RJ, Mullins CD. "Ten Commandments" for conducting comparative effectiveness research using "real-world data." J Manag Care Pharm. 2011;17 (9 Suppl A):S10-S15. Available at: http://amcp.org/WorkArea/ DownloadAsset.aspx?id=13714.
10. Cox E, Martin BC, Van Staa T, Garbe E, Siebert U, Johnson ML. Good research practices for comparative effectiveness research: approaches to mitigate bias and confounding in the design of nonrandomized studies of treatment effects using secondary data sources: the International Society for Pharmacoeconomics and Outcomes Research Good Research Practices for Retrospective Database Analysis Task Force Report-Part II. Value Health. 2009;12(8):1053-61.
11. Johnson ML, Crown W, Martin BC, Dormuth CR, Siebert U. Good research practices for comparative effectiveness research: analytic methods to improve causal inference from nonrandomized studies of treatment effects using secondary data sources: The ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report-Part III. Value Health. 2009;12(8):1062-73.
12. Berger ML, Mamdani M, Atkins D, Johnson ML. Good research practices for comparative effectiveness research: defining, reporting and interpreting nonrandomized studies of treatment effects using secondary data sources: the ISPOR Good Research Practices for Retrospective Database Analysis Task Force Report-Part I. Value Health. 2009;12(8):1044-52.
13. Berger ML, Dreyer N, Anderson F, Towse A, Sedrakyan A, Normand SL. Prospective observational studies to assess comparative effectiveness: the ISPOR Good Research Practices Task Force Report. Value Health. 2012;15(2):217-30.
14. Appendix D: research questions and PICO(TS): the effective health care program stakeholder guide. July 2011. Agency for Healthcare Research and Quality. Rockville, MD. Available at: http://www.ahrq.gov/research/findings/evidence-based-reports/stakeholderguide/appendixd.html. Accessed November 14, 2013.
15. Patient-Centered Outcomes Research Institute. PCORI methodology standards. December 14, 2012. Available at: http://www.pcori.org/assets/PCORI-Methodology-Standards.pdf. Accessed November 14, 2013.
16. Agency for Healthcare Research and Quality. Rating the strength of scientific research findings. AHRQ Publication No. 02-P022. April 2002. Available at: http://archive.ahrq.gov/clinic/epcsums/strenfact.htm. Accessed November 14, 2013.
17. Owens DK, Lohr KN, Atkins D, et al. AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions- Agency for Healthcare Research and Quality and the Effective Health-Care Program. J Clin Epidemiol. 2010;63(5):513-23.
18. West S, King V, Carey TS, et al. Systems to rate the strength of scientific evidence. Evidence Report/Technology Assessment No. 47 (Prepared by the Research Triangle Institute-University of North Carolina Evidence-based Practice Center under Contract No. 290-97-0011). AHRQ Publication No. 02-E016. April 2002. Rockville, MD: Agency for Healthcare Research and Quality. April 2002. Available at: http://www.thecre.com/pdf/ahrq-systemstrength. pdf. Accessed January 8, 2014.
19. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-26. Available at: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2335261/. Accessed November 14, 2013.
20. Sharma T, Nyong J, Shaw E. S22-Using and adapting GRADE methodology in an area of low-quality evidence: an example from a national guideline on ablative therapies for the treatment of Barrett's esophagus. Otolaryngol Head Neck Surg. 2010;143(1 Suppl 1):S21.
21. Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis Of Observational Studies in Epidemiology (MOOSE) group. JAMA. 2000;283(15):2008-12.
22. Wong WC, Cheung CS, Hart GJ. Development of a quality assessment tool for systematic reviews of observational studies (QATSO) of HIV prevalence in men having sex with men and associated risk behaviours. Emerg Themes Epidemiol. 2008;17(5):23. Available at: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2603000/. Accessed November 14, 2013.
23. Tseng TY, Breau RH, Fesperman SF, Vieweg J, Dahm P. Evaluating the evidence: the methodological and reporting quality of comparative observational studies of surgical interventions in urological publications. BJU Int. 2009;103(8):1026-31.
24. Von Elm E, Altman DG, Egger M, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61(4):344-49.
25. Dubois RW, Kindermann SL. Demystifying comparative effectiveness research: a case study learning guide. National Pharmaceutical Council. November 2009. Available at: http://www.npcnow.org/publication/demystifying- comparative-effectiveness-research-case-study-learning-guide. Accessed November 14, 2013.
26. Jadad AR, Moore RA, Carroll D, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary?. Control Clin Trials. 1996;17(1):1-12.
27. Streiner DL, Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. 4th ed. New York: Oxford University Press; 2008.
28. McDonagh MS, Peterson K, Carson S, Fu R, Thakurta S. Drug class review: atypical antipsychotic drugs. Final Update 3 Report [Internet]. Portland, OR: Oregon Health and Science University; July 2010.
29. Humphrey LL, Chan BK, Sox HC. Postmenopausal hormone replacement therapy and the primary prevention of cardiovascular disease. Ann Intern Med. 2002;137(4):273-84.
30. Nelson HD, Humphrey LL, Nygren P, Teutsch SM, Allan JD. Postmenopausal hormone replacement therapy: scientific review. JAMA. 2002;288(7):872-81.
31. Ip S, Bonis P, Tatsioni A, et al. Comparative effectiveness of management strategies for gastroesophageal reflux disease. Comparative Effectiveness Review No. 1. (Prepared by Tufts-New England Medical Center Evidencebased Practice Center under Contract No. 290-02-0022). Rockville, MD: Agency for Healthcare Research and Quality. AHRQ Publication No. 06-EHC003-1 December 2005. Available at: http://effectivehealthcare.ahrq. gov/ehc/products/1/43/GERD%20Final%20Report.pdf. Accessed November 14, 2013.
32. Yank V, Tuohy CV, Logan AC, et al.. Comparative effectiveness of inhospital use of recombinant factor VIIa for off-label indications vs. usual care. Comparative Effectiveness Review No. 21. (Prepared by Stanford-UCSF Evidence-based Practice Center under Contract No. #290-02-0017). Rockville, MD: Agency for Healthcare Research and Quality. AHRQ Publication No. 10-EHC030-EF May 2010. Available at: http://www.effectivehealthcare. ahrq.gov/ehc/products/20/450/Final%20Report_CER21_Factor7.pdf. Accessed November 14, 2013.
33. Donahue KE, Gartlehner G, Jonas DE, et al.. Comparative effectiveness of drug therapy for rheumatoid arthritis and psoriatic arthritis in adults. Comparative Effectiveness Review No. 11. (Prepared by RTI-University of North Carolina Evidence-based Practice Center under Contract No. 290-02-0016). Rockville, MD: Agency for Healthcare Research and Quality. November 2007. Available at: http://www.effectivehealthcare.ahrq.gov/ehc/products/14/68/RheumArthritisFinal.pdf. Accessed November 14, 2013.
34. Lang S, Kleijnen J. Quality assessment tools for observational studies: lack of consensus. Int J Evid Based Healthc. 2010;8(4):247.
35. Sanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol. 2007;36(3):666-76.
36. Dreyer NA. Using observational studies for comparative effectiveness: finding quality with GRACE. J Comp Eff Res. 2013;2(5):413-18.
37. Da Costa BR, Cevallos M, Altman DG, Rutjes AW, Egger M. Uses and misuses of the STROBE statement: bibliographic study. BMJ Open. 2011;1(1):e000048. Available at: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3191404/. Accessed November 14, 2013.
38. Neyarapally GA, Hammad TA, Pinheiro SP, Iyasu S. Review of quality assessment tools for the evaluation of pharmacopidemiological safety studies. BMJ Open. 2012;2(5):e001362. Available at: http://www.ncbi.nlm.nih. gov/pmc/articles/PMC3467649/. Accessed November 14, 2013.