False Dichotomies and Health Policy Research Designs: Randomized Trials Are Not Always the Answer

Journal of General Internal Medicine

Volumn 32, Issue 2, 2017, Pages 204-209

  Soumerai, Stephen B ^a   Ceccarelli, Rachel ^a   Koppel, Ross ^b  

^a HARVARD MEDICAL SCHOOL   (United States)

^b UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

health interventions; quasi experimental design; randomization; research design

Indexed keywords

ARTICLE; CLINICAL EVALUATION; CLINICAL RESEARCH; DRUG ABUSE; DRUG MARKETING; EXPERIMENTAL DESIGN; HEALTH CARE POLICY; HEALTH INSURANCE; HOSPITAL CARE; HUMAN; MEDICAL EDUCATION; PATIENT SAFETY; PUBLIC HEALTH; QUASI EXPERIMENTAL STUDY; RANDOMIZED CONTROLLED TRIAL (TOPIC); SELECTION BIAS; STUDY DESIGN; SYSTEMATIC REVIEW (TOPIC); TIME SERIES ANALYSIS; CONTROLLED CLINICAL TRIAL (TOPIC); EPIDEMIOLOGY; METHODOLOGY; STANDARDS;

HEALTH POLICY; HUMANS; INTERRUPTED TIME SERIES ANALYSIS; NON-RANDOMIZED CONTROLLED TRIALS AS TOPIC; RANDOMIZED CONTROLLED TRIALS AS TOPIC; RESEARCH DESIGN;

EID: 84991716553     PISSN: 08848734     EISSN: 15251497     Source Type: Journal    
DOI: 10.1007/s11606-016-3841-9     Document Type: Article

References (40)

1. Ioannidis JP. Are mortality differences detected by administrative data reliable and actionable? JAMA. 2013;309(13):1410–1.
2. Ioannidis JP. Why most published research findings are false. PLoS Med. 2005;2(8):e124.
3. Lehrer J. The Truth Wears Off. The New Yorker; 2010. http://www.newyorker.com/magazine/2010/12/13/the-truth-wears-off. Accessed June 14, 2016.
4. Freedman DH. Lies, damned lies and medical science. Atlantic. 2010.
5. Shadish W, Cook T, Campbell D. Experimental and quasi-experimental designs for generalized causal inference. Belmont: Wadsworth Cengage Learning; 2002.
6. Soumerai SB, Starr D, Majumdar SR. How do you know which health care effectiveness research you can trust? A guide to study design for the perplexed. Prev Chronic Dis. 2015;12:E101.
7. Fretheim A, Zhang F, Ross-Degnan D, et al. A reanalysis of cluster randomized trials showed interrupted time-series studies were valuable in health system evaluation. J Clin Epidemiol. 2015;68(3):324–33.
8. Briesacher BA, Madden JM, Zhang F, et al. Did Medicare Part D affect national trends in health outcomes or hospitalizations? A time-series analysis. Ann Intern Med. 2015;162(12):825–33.
9. Soumerai SB, Ross-Degnan D, Gortmaker S, Avorn J. Withdrawing payment for nonscientific drug therapy. Intended and unexpected effects of a large-scale natural experiment. JAMA. 1990;263(6):831–9.
10. Epperlein T. The Impact of the 65 MPH Speed Limit in Arizona. Arizona: Arizona Statistical Analysis Center; 1989.
11. Wharam JF, Landon BE, Galbraith AA, Kleinman KP, Soumerai SB, Ross-Degnan D. Emergency department use and subsequent hospitalizations among members of a high-deductible health plan. JAMA. 2007;297(10):1093–102.
12. Ornstein C. New hepatitis C drugs are costing Medicare billions. The Washington Post; 2015. https://www.washingtonpost.com/national/health-science/medicare-spent-45-billion-on-new-hepatitis-c-drugs-last-year-data-shows/2015/03/29/66952dde-d32a-11e4-a62f-ee745911a4ff_story.html. Accessed June 14, 2016.
13. Roehrig C. The Impact of New Hepatitis C Drugs on National Health Spending. 2016: Health Affairs Blog; 2015.
14. Arnold SR, Straus SE. Interventions to improve antibiotic prescribing practices in ambulatory care. Cochrane Database Syst Rev. 2005;4:CD003539.
15. Serumaga B, Ross-Degnan D, Avery AJ, et al. Effect of pay for performance on the management and outcomes of hypertension in the United Kingdom: interrupted time series study. BMJ. 2011;342:d108.
16. Wagner AK, Ross-Degnan D, Gurwitz JH, et al. Effect of New York State regulatory action on benzodiazepine prescribing and hip fracture rates. Ann Intern Med. 2007;146(2):96–103.
17. Campbell D, Stanley J. Experimental and quasi-experimental designs for research. Boston: Houghton Mifflin Company; 1963.
18. Cook TD, Campbell DT. Quasi-Experimentation: Design & Analysis Issues for Field Settings. Boston: Houghton Mifflin; 1979.
19. McCall WA. How to experiment in education. New York: The Macmillian Company; 1926.
20. Lee GM, Kleinman K, Soumerai SB, et al. Effect of nonpayment for preventable infections in US hospitals. N Engl J Med. 2012;367(15):1428–37.
21. Soumerai SB, McLaughlin TJ, Ross-Degnan D, Casteris CS, Bollini P. Effects of a limit on Medicaid drug-reimbursement benefits on the use of psychotropic agents and acute mental health services by patients with schizophrenia. N Engl J Med. 1994;331(10):650–5.
22. Effective Practice and Organisation of Care (EPOC). What study designs should be included in an EPOC review? EPOC Resources for review authors. http://epoc.cochrane.org/epoc-specific-resources-review-authors. Accessed June 14, 2016.
23. Amarasingham R, Plantinga L, Diener-West M, Gaskin DJ, Powe NR. Clinical information technologies and inpatient outcomes: a multiple hospital study. Arch Intern Med. 2009;169(2):108–14.
24. Cebul RD, Love TE, Jain AK, Hebert CJ. Electronic health records and quality of diabetes care. N Engl J Med. 2011;365(9):825–33.
25. Sanghavi P, Jena AB, Newhouse JP, Zaslavsky AM. Outcomes of basic versus advanced life support for out-of-hospital medical emergencies. Ann Intern Med. 2015;163(9):681–90.
26. Soumerai S, Koppel R. A major glitch for digitized health-care records. Wall Street J. 2012. http://www.wsj.com/articles/SB10000872396390443847404577627041964831020. Accessed June 14, 2016.
27. Soumerai SB, Koppel R. Avoiding expensive and consequential health care decisions based on weak research design August 31 ed: Health Aff Blog. 2015.
28. Koppel R, Soumerai S. Designing good research to support good policy: the case of health IT vol 2015: Health Aff Blog; 2015.
29. Ackermann RT, Kenrik Duru O, Albu JB, et al. Evaluating diabetes health policies using natural experiments: the natural experiments for translation in diabetes study. Am J Prev Med. 2015;48(6):747–54.
30. Zhang F, Wagner AK, Soumerai SB, Ross-Degnan D. Methods for estimating confidence intervals in interrupted time series analyses of health interventions. J Clin Epidemiol. 2009;62(2):143–8.
31. Gillings D, Makuc D, Siegel E. Analysis of interrupted time series mortality trends: an example to evaluate regionalized perinatal care. Am J Public Health. 1981;71(1):38–46.
32. Wagner AK, Soumerai SB, Zhang F, Ross-Degnan D. Segmented regression analysis of interrupted time series studies in medication use research. J Clin Pharm Ther. 2002;27(4):299–309.
33. Soumerai SB, Avorn J, Ross-Degnan D, Gortmaker S. Payment restrictions for prescription drugs under Medicaid. Effects on therapy, cost, and equity. N Engl J Med. 1987;317(9):550–6.
34. Soumerai SB, Ross-Degnan D, Avorn J, McLaughlin T, Choodnovskiy I. Effects of Medicaid drug-payment limits on admission to hospitals and nursing homes. N Engl J Med. 1991;325(15):1072–7.
35. Soumerai SB, Ross-Degnan D, Fortess EE, Walser BL. Determinants of change in Medicaid pharmaceutical cost sharing: does evidence affect policy? Milbank Q. 1997;75(1):11–34.
36. Tunis SR. Letter from chief medical officer at the Center for Medicare & Medicaid Services, DHHS, to Stephen Soumerai. 2005.
37. National Institutes of Health. Enhancing Reproducibility through Rigor and Transparency. 2015; http://grants.nih.gov/grants/guide/notice-files/NOT-OD-15-103.html. Accessed June 14, 2016.
38. Urquhart C, Currell R, Grant MJ, Hardiker NR. Nursing record systems: effects on nursing practice and healthcare outcomes. Cochrane Database Syst Rev. 2009;1:CD002099.
39. Ioannidis JP. Implausible results in human nutrition research. BMJ. 2013;347:f6698.
40. β-Blocker Heart Attack Trial Research Group. A randomized trial of propranolol in patients with acute myocardial infarction. I. Mortality results. JAMA. 1982;247(12):1707–14.