Clinical knowledge from observational studies: Everything you wanted to know but were afraid to ask

American Journal of Respiratory and Critical Care Medicine

Volumn 198, Issue 7, 2018, Pages 859-867

  Gershon, Andrea S ^a   Jafarzadeh, S Reza ^b   Wilson, Kevin C ^b,c   Walkey, Allan J ^b  

^a SUNNYBROOK RESEARCH INSTITUTE   (Canada)

^b BOSTON UNIVERSITY SCHOOL OF MEDICINE   (United States)

^c Pulmonary Center   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLINICAL COMPETENCE; CLINICAL DECISION MAKING; COMPARATIVE STUDY; DOCTOR PATIENT RELATIONSHIP; EVIDENCE BASED MEDICINE; FEMALE; HEALTH CARE DELIVERY; HUMAN; MALE; OBSERVATIONAL STUDY; ORGANIZATION AND MANAGEMENT; OUTCOME ASSESSMENT; PATIENT CARE; RANDOMIZED CONTROLLED TRIAL (TOPIC); RISK ASSESSMENT; UNITED STATES;

CLINICAL COMPETENCE; CLINICAL DECISION-MAKING; DELIVERY OF HEALTH CARE; EVIDENCE-BASED MEDICINE; FEMALE; HUMANS; MALE; OBSERVATIONAL STUDIES AS TOPIC; OUTCOME ASSESSMENT (HEALTH CARE); PATIENT-CENTERED CARE; PHYSICIAN-PATIENT RELATIONS; RANDOMIZED CONTROLLED TRIALS AS TOPIC; RISK ASSESSMENT; UNITED STATES;

EID: 85054233444     PISSN: 1073449X     EISSN: 15354970     Source Type: Journal    
DOI: 10.1164/rccm.201801-0118PP     Document Type: Review

References (81)

1. Institute of Medicine. Integrating research and practice: health system leaders working toward high-value care: workshop summary. Washington, D.C.: The National Academies Press; 2015.
2. Califf RM, Sherman R. What we mean when we talk about data. 2015 Dec 10 [accessed 2018 Mar 15]. Available from: https://blogs.fda.gov/ fdavoice/?s=What1we1mean1when1we1talk1about1data.
3. Agency for Healthcare Research and Quality. AHRQ research summit on learning health systems. 2017 Sept 15 [accessed 2018 Mar 15]. Available from: https://www.ahrq.gov/news/events/ahrq-researchsummit- learning-health-system.html.
4. Mansournia MA, Higgins JP, Sterne JA, Hernán MA. Biases in randomized trials: a conversation between trialists and epidemiologists. Epidemiology 2017;28:54-59.
5. Manson JE, Shufelt CL, Robins JM. The potential for postrandomization confounding in randomized clinical trials. JAMA 2016:315:2273-2274.
6. Schmidt GA, Girard TD, Kress JP, Morris PE, Ouellette DR, Alhazzani W, et al.; ATS/CHEST Ad Hoc Committee on Liberation from Mechanical Ventilation in Adults. Official executive summary of an American Thoracic Society/American College of Chest Physicians clinical practice guideline: liberation from mechanical ventilation in critically ill adults. Am J Respir Crit Care Med 2017;195:115-119.
7. Fan E, Del Sorbo L, Goligher EC, Hodgson CL, Munshi L, Walkey AJ, et al.; American Thoracic Society, European Society of Intensive Care Medicine, and Society of Critical Care Medicine. An official American Thoracic Society/European Society of Intensive Care Medicine/Society of Critical Care Medicine clinical practice guideline: mechanical ventilation in adult patients with acute respiratory distress syndrome. Am J Respir Crit Care Med 2017;195:1253-1263.
8. Rochwerg B, Brochard L, Elliott MW, Hess D, Hill NS, Nava S, et al. Official ERS/ATS clinical practice guidelines: noninvasive ventilation for acute respiratory failure. Eur Respir J 2017;50:1602426.
9. Esteban A, Alía I, Gordo F, Fernández R, Solsona JF, Vallverdú I, et al.; The Spanish Lung Failure Collaborative Group. Extubation outcome after spontaneous breathing trials with T-tube or pressure support ventilation. Am J Respir Crit Care Med 1997;156:459-465.
10. Brochard L, Rauss A, Benito S, Conti G, Mancebo J, Rekik N, et al. Comparison of three methods of gradual withdrawal from ventilatory support during weaning from mechanical ventilation. Am J Respir Crit Care Med 1994;150:896-903.
11. Albert RK. "Lies, damned lies ⋯" and observational studies in comparative effectiveness research. Am J Respir Crit Care Med 2013;187:1173-1177.
12. Anglemyer A, Horvath HT, Bero L. Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials. Cochrane Database Syst Rev 2014;4:MR000034.
13. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000;342:1887-1892.
14. Ioannidis JP, Haidich AB, Pappa M, Pantazis N, Kokori SI, Tektonidou MG, et al. Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA 2001;286:821-830.
15. Franklin JM, Dejene S, Huybrechts KF, Wang SV, Kulldorff M, Rothman KJ. A bias in the evaluation of bias comparing randomized trials with nonexperimental studies. Epidemiol Methods 2017;6:20160018.
16. Stampfer MJ, Colditz GA, Willett WC, Manson JE, Rosner B, Speizer FE, et al. Postmenopausal estrogen therapy and cardiovascular disease: ten-year follow-up from the nurses' health study. N Engl J Med 1991;325:756-762.
17. Manson JE, Hsia J, Johnson KC, Rossouw JE, Assaf AR, Lasser NL, et al.; Women's Health Initiative Investigators. Estrogen plus progestin and the risk of coronary heart disease. N Engl J Med 2003; 349:523-534.
18. Falagas ME, Makris GC, Matthaiou DK, Rafailidis PI. Statins for infection and sepsis: a systematic review of the clinical evidence. J Antimicrob Chemother 2008;61:774-785.
19. Truwit JD, Bernard GR, Steingrub J, Matthay MA, Liu K, Albertson TE, et al. SAILS: statins for acutely injured lungs (ARDS) from sepsis [abstract]. Am J Respir Crit Care Med 2014;189:A5088.
20. Christ-Crain M, Jaccard-Stolz D, Bingisser R, Gencay MM, Huber PR, Tamm M, et al. Effect of procalcitonin-guided treatment on antibiotic use and outcome in lower respiratory tract infections: clusterrandomised, single-blinded intervention trial. Lancet 2004;363: 600-607.
21. Christ-Crain M, Stolz D, Bingisser R, Müller C, Miedinger D, Huber PR, et al. Procalcitonin guidance of antibiotic therapy in communityacquired pneumonia: a randomized trial. Am J Respir Crit Care Med 2006;174:84-93.
22. Schuetz P, Christ-Crain M, Thomann R, Falconnier C, Wolbers M, Widmer I, et al.; ProHOSP Study Group. Effect of procalcitoninbased guidelines vs standard guidelines on antibiotic use in lower respiratory tract infections: the ProHOSP randomized controlled trial. JAMA 2009;302:1059-1066.
23. de Jong E, van Oers JA, Beishuizen A, Vos P, Vermeijden WJ, Haas LE, et al. Efficacy and safety of procalcitonin guidance in reducing the duration of antibiotic treatment in critically ill patients: a randomised, controlled, open-label trial. Lancet Infect Dis 2016;16:819-827.
24. Nobre V, Harbarth S, Graf JD, Rohner P, Pugin J. Use of procalcitonin to shorten antibiotic treatment duration in septic patients: a randomized trial. Am J Respir Crit Care Med 2008;177:498-505.
25. Bouadma L, Luyt C-E, Tubach F, Cracco C, Alvarez A, Schwebel C, et al.; PRORATA trial group. Use of procalcitonin to reduce patients' exposure to antibiotics in intensive care units (PRORATA trial): a multicentre randomised controlled trial. Lancet 2010;375:463-474.
26. Hochreiter M, Köhler T, Schweiger AM, Keck FS, Bein B, von Spiegel T, et al. Procalcitonin to guide duration of antibiotic therapy in intensive care patients: a randomized prospective controlled trial. Crit Care 2009;13:R83.
27. Kopterides P, Siempos II, Tsangaris I, Tsantes A, Armaganidis A. Procalcitonin-guided algorithms of antibiotic therapy in the intensive care unit: a systematic review and meta-analysis of randomized controlled trials. Crit Care Med 2010;38:2229-2241.
28. Hohn A, Schroeder S, Gehrt A, Bernhardt K, Bein B, Wegscheider K, et al. Procalcitonin-guided algorithm to reduce length of antibiotic therapy in patients with severe sepsis and septic shock. BMC Infect Dis 2013;13:158.
29. Kalil AC, La Rosa SP. Effectiveness and safety of drotrecogin alfa (activated) for severe sepsis: a meta-analysis and metaregression. Lancet Infect Dis 2012;12:678-686.
30. Lai PS, Matteau A, Iddriss A, Hawes JC, Ranieri V, Thompson BT. An updated meta-analysis to understand the variable efficacy of drotrecogin alfa (activated) in severe sepsis and septic shock. Minerva Anestesiol 2013;79:33-43.
31. Young P, Bailey M, Beasley R, Henderson S, Mackle D, McArthur C, et al.; SPLIT Investigators; ANZICS CTG. Effect of a buffered crystalloid solution vs saline on acute kidney injury among patients in the intensive care unit: the SPLIT randomized clinical trial. JAMA 2015;314:1701-1710.
32. Semler MW, Self WH, Wanderer JP, Ehrenfeld JM, Wang L, Byrne DW, et al.; SMART Investigators and the Pragmatic Critical Care Research Group. Balanced crystalloids versus saline in critically ill adults. N Engl J Med 2018;378:829-839.
33. Higgins JPT, Altman DG, Sterne JAC, editors. Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions. Version 5.1.0 [updated 2011 Mar; accessed 2018 Aug 27]. The Cochrane Collaboration; 2011. Available from: www.handbook.cochrane.org.
34. Kor DJ, Carter RE, Park PK, Festic E, Banner-Goodspeed VM, Hinds R, et al.; US Critical Illness and Injury Trials Group: Lung Injury Prevention with Aspirin Study Group (USCIITG: LIPS-A). Effect of aspirin on development of ARDS in at-risk patients presenting to the emergency department: the LIPS-A randomized clinical trial. JAMA 2016;315:2406-2414.
35. Bradford MA, Lindenauer PK, Wiener RS, Walkey AJ. Do-not-resuscitate status and observational comparative effectiveness research in patients with septic shock. Crit Care Med 2014;42:2042-2047.
36. Hernán MA, Alonso A, Logan R, Grodstein F, Michels KB, Willett WC, et al. Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease. Epidemiology 2008;19:766-779.
37. Greenland S, Robins JM. Identifiability, exchangeability, and epidemiological confounding. Int J Epidemiol 1986;15:413-419.
38. Suissa S. Effectiveness of inhaled corticosteroids in chronic obstructive pulmonary disease: immortal time bias in observational studies. Am J Respir Crit Care Med 2003;168:49-53.
39. Suissa S. Immortal time bias in pharmaco-epidemiology. Am J Epidemiol 2008;167:492-499.
40. Rothman KJ, Gallacher JE, Hatch EE. Why representativeness should be avoided. Int J Epidemiol 2013;42:1012-1014.
41. Gershon AS, Hwee J, Croxford R, Aaron SD, To T. Patient and physician factors associated with pulmonary function testing for COPD: a population study. Chest 2014;145:272-281.
42. Halpin DMG, Kerkhof M, Soriano JB, Mikkelsen H, Price DB. Eligibility of real-life patients with COPD for inclusion in trials of inhaled longacting bronchodilator therapy. Respir Res 2016;17:120.
43. Lindenauer PK, Shieh MS, Stefan MS, Fisher KA, Haessler SD, Pekow PS, et al. Hospital procalcitonin testing and antibiotic treatment of patients admitted for COPD exacerbation. Ann Am Thorac Soc 2017; 14:1779-1785.
44. Chu DC, Mehta AB, Walkey AJ. Practice patterns and outcomes associated with procalcitonin use in critically ill patients with sepsis. Clin Infect Dis 2017;64:1509-1515.
45. Fisher KA, Landyn V, Lindenauer PK, Walkey AJ. procalcitonin test availability: a survey of acute care hospitals in Massachusetts. Ann Am Thorac Soc 2017;14:1489-1491.
46. Juurlink DN, Mamdani MM, Lee DS, Kopp A, Austin PC, Laupacis A, et al. Rates of hyperkalemia after publication of the Randomized Aldactone Evaluation Study. N Engl J Med 2004;351:543-551.
47. Kent DM, Kitsios G. Against pragmatism: on efficacy, effectiveness and the real world. Trials 2009;10:48.
48. Rothman KJ. Six persistent research misconceptions. J Gen Intern Med 2014;29:1060-1064.
49. Fawzy A, Evans SR, Walkey AJ. Practice patterns and outcomes associated with choice of initial vasopressor therapy for septic shock. Crit Care Med 2015;43:2141-2146.
50. De Backer D, Aldecoa C, Njimi H, Vincent JL. Dopamine versus norepinephrine in the treatment of septic shock: a meta-analysis. Crit Care Med 2012;40:725-730.
51. Mansournia MA, Hernán MA, Greenland S. Matched designs and causal diagrams. Int J Epidemiol 2013;42:860-869.
52. Keiding N, Clayton D. Standardization and control for confounding in observational studies: a historical perspective. Stat Sci 2014;29: 529-558.
53. Vansteelandt S, Daniel RM. On regression adjustment for the propensity score. Stat Med 2014;33:4053-4072.
54. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika 1983;70: 41-55.
55. Austin PC. The performance of different propensity-score methods for estimating differences in proportions (risk differences or absolute risk reductions) in observational studies. Stat Med 2010;29: 2137-2148.
56. Naimi AI, Cole SR, Kennedy EH. An introduction to g methods. Int J Epidemiol 2017;46:756-762.
57. Daniel RM, Cousens SN, De Stavola BL, Kenward MG, Sterne JA. Methods for dealing with time-dependent confounding. Stat Med 2013;32:1584-1618.
58. Joffe MM, Hoover DR, Jacobson LP, Kingsley L, Chmiel JS, Visscher BR. Effect of treatment with zidovudine on subsequent incidence of Kaposi's sarcoma. Clin Infect Dis 1997;25:1125-1133.
59. Hernán MA, Robins JM. Instruments for causal inference: an epidemiologist's dream? Epidemiology 2006;17:360-372.
60. Stukel TA, Fisher ES, Wennberg DE, Alter DA, Gottlieb DJ, Vermeulen MJ. Analysis of observational studies in the presence of treatment selection bias: effects of invasive cardiac management on AMI survival using propensity score and instrumental variable methods. JAMA 2007;297:278-285.
61. Joffe MM. Confounding by indication: the case of calcium channel blockers. Pharmacoepidemiol Drug Saf 2000;9:37-41.
62. Moscoe E, Bor J, Bärnighausen T. Regression discontinuity designs are underutilized in medicine, epidemiology, and public health: a review of current and best practice. J Clin Epidemiol 2015;68:122-133.
63. Walkey AJ, Cordella N, Drainoni ML, Bor J. Advancing quality improvement with regression discontinuity designs. Ann Am Thorac Soc 2018;15:523-529.
64. Walkey AJ, Drainoni ML, Cordella N, Bor J. Advancing quality improvement with regression discontinuity designs. Ann Am Thorac Soc 2018;15:523-529.
65. Penfold RB, Zhang F. Use of interrupted time series analysis in evaluating health care quality improvements. Acad Pediatr 2013;13: S38-S44.
66. Wing C, Simon K, Bello-Gomez RA. Designing difference in difference studies: best practices for public health policy research. Annu Rev Public Health 2018;39:453-469.
67. Van der Weele TJ, Ding P. Sensitivity analysis in observational research: introducing the E-value. Ann Intern Med 2017;167:268-274.
68. Mathur MB, Ding P, Riddell CA, Vander Weele TJ. Website and R package for computing E-values. Epidemiology [online ahead of print] 14 Jun 2018 DOI: 10.1097/EDE.0000000000000864.
69. Gershon AS, Warner L, Cascagnette P, Victor JC, To T. Lifetime risk of developing chronic obstructive pulmonary disease: a longitudinal population study. Lancet 2011;378:991-996.
70. Gershon AS, Wang C, Guan J, Vasilevska-Ristovska J, Cicutto L, To T. Identifying individuals with physcian diagnosed COPD in health administrative databases. COPD 2009;6:388-394.
71. Gershon AS, Campitelli MA, Croxford R, Stanbrook MB, To T, Upshur R, et al. Combination long-acting b-agonists and inhaled corticosteroids compared with long-acting b-agonists alone in older adults with chronic obstructive pulmonary disease. JAMA 2014;312:1114-1121.
72. Corbin M, Haslett S, Pearce N, Maule M, Greenland S. A comparison of sensitivity-specificity imputation, direct imputation and fully Bayesian analysis to adjust for exposure misclassification when validation data are unavailable. Int J Epidemiol 2017;46:1063-1072.
73. Sterne JAC, Hernán MA, Reeves BC, Savović J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016;355: i4919.
74. Good Research for Comparative Effectiveness (GRACE) [accessed 2018 Apr 12]. Available from: https://www.graceprinciples.org.
75. Shamliyan TA, Kane RL, Ansari MT, Raman G, Berkman ND, Grant M, et al. Development of quality criteria to evaluate nontherapeutic studies of incidence, prevalence, or risk factors of chronic diseases: pilot study of new checklists. Rockville, MD: Agency for Healthcare Research and Quality (US); 2011 Jan [accessed 2018 Aug 27]. Appendix A, Methodological evaluation of observational research. Available from: https://www.ncbi.nlm.nih.gov/books/ NBK53279/.
76. Hirst C, Calingaert B, Stanford R, Castellsague J. Use of long-acting beta-agonists and inhaled steroids in asthma: meta-analysis of observational studies. J Asthma 2010;47:439-446.
77. Loymans RJ, Gemperli A, Cohen J, Rubinstein SM, Sterk PJ, Reddel HK, et al. Comparative effectiveness of long term drug treatment strategies to prevent asthma exacerbations: network meta-analysis. BMJ 2014;348:g3009.
78. Kitsios GD, Dahabreh IJ, Callahan S, Paulus JK, Campagna AC, Dargin JM. Can we trust observational studies using propensity scores in the critical care literature? A systematic comparison with randomized clinical trials. Crit Care Med 2015;43:1870-1879.
79. Institute of Medicine. Clinical practice guidelines we can trust. Washington, D.C.: The National Academies Press; 2011.
80. Institute of Medicine. Finding what works in health care: standards for systematic reviews. Washington, D.C.: The National Academies Press; 2011.
81. Schoenberg NC, Barker AF, Bernardo J, Deterding RR, Ellner JJ, Hess DR, et al. A comparative analysis of pulmonary and critical care medicine guideline development methodologies. Am J Respir Crit Care Med 2017;196:621-627.