Study design elements for rigorous quasi-experimental comparative effectiveness research

Journal of Comparative Effectiveness Research

Volumn 2, Issue 2, 2013, Pages 159-173

  MacIejewski, Matthew L ^a,b   Curtis, Lesley H ^c   Dowd, Bryan ^d  

^a DURHAM VA MEDICAL CENTER   (United States)

^b DUKE UNIVERSITY SCHOOL OF MEDICINE   (United States)

^c DUKE CLINICAL RESEARCH INSTITUTE   (United States)

^d UNIVERSITY OF MINNESOTA   (United States)

Author keywords

comparative effectiveness research; control group; external validity; health services research; internal validity; nonequivalent outcomes; outcomes research; quasi experiment; study design; treatment effect

Indexed keywords

BETA ADRENERGIC RECEPTOR BLOCKING AGENT;

BARIATRIC SURGERY; CLINICAL DECISION MAKING; CLINICAL EFFECTIVENESS; CLINICAL EXAMINATION; COMORBIDITY; COMPARATIVE EFFECTIVENESS; DISEASE SEVERITY; ETHNIC DIFFERENCE; EXTERNAL VALIDITY; HEALTH CARE AVAILABILITY; HEALTH CARE COST; HEALTH CARE PLANNING; HEART FAILURE; HIGH RISK POPULATION; HORMONE SUBSTITUTION; HUMAN; INTERNAL VALIDITY; ISCHEMIC HEART DISEASE; MARRIAGE; OBESITY; OUTCOME ASSESSMENT; PATIENT ATTITUDE; PATIENT SELECTION; PRIORITY JOURNAL; PROGNOSIS; QUASI EXPERIMENTAL STUDY; RANDOMIZATION; REVIEW; RISK ASSESSMENT; RISK BENEFIT ANALYSIS; STUDY DESIGN; SURGICAL MORTALITY; TREATMENT DURATION;

EID: 84886945664     PISSN: 20426305     EISSN: 20426313     Source Type: Journal    
DOI: 10.2217/cer.13.7     Document Type: Review

References (74)

1. Berger ML, Dreyer N, Anderson F, Towse A, Sedrakyan A, Normand SL. Prospective observational studies to assess comparative effectiveness: the ISPOR good research practices task force report. Value Health 15(2), 217-230 (2012).
2. Ioannidis JP, Haidich AB, Pappa M et al. Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA 286(7), 821-830 (2001). (Pubitemid 33015576)
3. Naik G. Analytical trend troubles scientists. Wall Street Journal, 3 May (2012).
4. Kling JR. CBO's use of evidence in analysis of budget and economic policies. Presented at: Annual Fall Research Conference of the Association for Public Policy Analysis & Management. Baltimore, MD, USA, 2-5 November 2011.
5. Grady D, Rubin SM, Petitti DB et al. Hormone therapy to prevent disease and prolong life in postmenopausal women. Ann. Intern. Med. 117(12), 1016-1037 (1992).
6. Grodstein F, Manson JE, Colditz GA, Willett WC, Speizer FE, Stampfer MJ. A prospective, observational study of postmenopausal hormone therapy and primary prevention of cardiovascular disease. Ann. Intern. Med. 133(12), 933-941 (2000). (Pubitemid 32003509)
7. Manson JE, Hsia J, Johnson KC et al. Estrogen plus progestin and the risk of coronary heart disease. N. Engl. J. Med. 349(6), 523-534 (2003). (Pubitemid 36951364)
8. Hernan MA, Alonso A, Logan R et al. Observational studies analyzed like randomized experiments: an application to postmenopausal hormone therapy and coronary heart disease. Epidemiology 19(6), 766-779 (2008).
9. Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N. Engl. J. Med. 342(25), 1887-1892 (2000). (Pubitemid 30419054)
10. Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. N. Engl. J. Med. 342(25), 1878-1886 (2000). (Pubitemid 30419053)
11. Furlan AD, Tomlinson G, Jadad AA, Bombardier C. Methodological quality and homogeneity influenced agreement between randomized trials and nonrandomized studies of the same intervention for back pain. J. Clin. Epidemiol. 61(3), 209-231 (2008).
12. Abraham NS, Byrne CJ, Young JM, Solomon MJ. Meta-analysis of well-designed nonrandomized comparative studies of surgical procedures is as good as randomized controlled trials. J. Clin. Epidemiol. 63(3), 238-245 (2010).
13. Imai K, King G, Stuart EA. Misunderstandings between experimentalists and observationalists about causal inference. J. R. Stat. Soc. Series A 171(2), 481-502 (2008).
14. Kravitz RL, Duan N, Braslow J. Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Q 82(4), 661-687 (2004). (Pubitemid 40143102)
15. Austin PC. An Introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav. Res. 46(3), 399-424 (2011).
16. Newhouse JP, McClellan M. Econometrics in outcomes research: the use of instrumental variables. Annu. Rev. Public Health 19, 17-34 (1998). (Pubitemid 28279922)
17. Brookhart MA, Rassen JA, Schneeweiss S. Instrumental variable methods in comparative safety and effectiveness research. Pharmacoepidemiol. Drug Saf. 19(6), 537-554 (2010).
18. Shadish WR, Galindo R, Wong VC, Steiner PM, Cook TD. A randomized experiment comparing random and cutoff-based assignment. Psychol. Methods 16(2), 179-191 (2011).
19. Dreyer NA, Schneeweiss S, McNeil BJ et al. GRACE principles: recognizing high-quality observational studies of comparative effectiveness. Am. J. Manag. Care 16(6), 467-471 (2010).
20. von Elm E, Altman DG, Egger M, Pocock SJ, Gotzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann. Intern. Med. 147(8), 573-577 (2007). (Pubitemid 351664504)
21. Cox E, Martin BC, Van Staa T, Garbe E, Siebert U, Johnson ML. Good research practices for comparative effectiveness research: approaches to mitigate bias and confounding in the design of nonrandomized studies of treatment effects using secondary data sources: the International Society for Pharmacoeconomics and Outcomes Research Good Research Practices for Retrospective Database Analysis Task Force Report-Part II. Value Health 12(8), 1053-1061 (2009).
22. Greenfield S, Platt R. Can observational studies approximate RCTs? Value Health 15(2), 215-216 (2012).
23. Maciejewski ML, Weaver EM, Hebert PL. Synonyms in health services research methodology. Med. Care Res. Rev. 68(2), 156-176 (2011).
24. Rothwell PM, Warlow CP. Prediction of benefit from carotid endarterectomy in individual patients: a risk-modelling study. European Carotid Surgery Trialists' Collaborative Group. Lancet 353(9170), 2105-2110 (1999). (Pubitemid 29278479)
25. Hayward RA, Kent DM, Vijan S, Hofer TP. Reporting clinical trial results to inform providers, payers, and consumers. Health Aff. 24, 1571-1581 (2005).
26. Basu A. Economics of individualization in comparative effectiveness research and a basis for a patient-centered health care. J. Health Econ. 30(3), 549-559 (2011).
27. Brookhart MA, Patrick AR, Dormuth C et al. Adherence to lipid-lowering therapy and the use of preventive health services: an investigation of the healthy user effect. Am. J. Epidemiol. 166(3), 348-354 (2007). (Pubitemid 47216808)
28. Petri H, Urquhart J. Channeling bias in the interpretation of drug effects. Stat. Med. 10(4), 577-581 (1991).
29. Brookhart MA, Sturmer T, Glynn RJ, Rassen J, Schneeweiss S. Confounding control in healthcare database research: challenges and potential approaches. Med. Care 48(Suppl. 6), S114-S120 (2010).
30. Shrank WH, Patrick AR, Brookhart MA. Healthy user and related biases in observational studies of preventive interventions: a primer for physicians. J. Gen. Intern. Med. 26(5), 546-550 (2011).
31. Rosenbaum PR. Observational Studies (2nd Edition). Springer Series in Statistics, NY, USA (2002).
32. Rubin DB. Estimating causal effects of treatments in randomized and nonrandomized studies. J. Educational Psychol. 66, 688-701 (1974).
33. Schneeweiss S, Patrick AR, Sturmer T et al. Increasing levels of restriction in pharmacoepidemiologic database studies of elderly and comparison with randomized trial results. Med. Care 45(10 Suppl. 2), S131-S142 (2007).
34. Cook TD, Shadish WR, Wong VC. Three conditions under which experiments and observational studies produce comparable causal estimates: new findings from within-study comparisons. J. Policy Anal. Manage. 27(4), 724-250 (2008).
35. Wacholder S, McLaughlin JK, Silverman DT, Mandel JS. Selection of controls in case-control studies. I. Principles. Am. J. Epidemiol. 135(9), 1019-1028 (1992).
36. Mingrone G, Panunzi S, De Gaetano A et al. Bariatric surgery versus conventional medical therapy for Type 2 diabetes. N. Engl. J. Med. 366(17), 1577-1585 (2012).
37. Schauer PR, Kashyap SR, Wolski K et al. Bariatric surgery versus intensive medical therapy in obese patients with diabetes. N. Engl. J. Med. 366(17), 1567-1576 (2012).
38. Flum DR, Dellinger EP. Impact of gastric bypass operation on survival: a population-based analysis. J. Am. Coll. Surg. 199(4), 543-551 (2004). (Pubitemid 39420204)
39. Finkelstein EA, Allaire BT, Burgess SM, Hale BC. Financial implications of coverage for laparoscopic adjustable gastric banding. Surg. Obes. Relat. Dis. 7(3), 295-303 (2011).
40. Schneeweiss S, Wang PS, Avorn J, Glynn RJ. Improved comorbidity adjustment for predicting mortality in Medicare populations. Health Serv. Res. 38(4), 1103-1120 (2003). (Pubitemid 37048302)
41. Stam PJ, van Vliet RC, van de Ven WP. Diagnostic, pharmacy-based, and self-reported health measures in risk equalization models. Med. Care 48(5), 448-457 (2010).
42. Young BA, Lin E, Von Korff M et al. Diabetes complications severity index and risk of mortality, hospitalization, and healthcare utilization. Am. J. Manag. Care 14(1), 15-23 (2008).
43. Jones RC, Donaldson GC, Chavannes NH et al. Derivation and validation of a composite index of severity in chronic obstructive pulmonary disease: the DOSE Index. Am. J. Respir. Crit. Care Med. 180(12), 1189-1195 (2009).
44. Shadish WR, Cook TD, Campbell DT. Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin Co., Boston, MA, USA (2002).
45. Steiner PM, Cook TD, Shadish WR, Clark MH. The importance of covariate selection in controlling for selection bias in observational studies. Psychol. Methods 15(3), 250-267 (2010).
46. Cook TD, Steiner PM. Case matching and the reduction of selection bias in quasi-experiments: the relative importance of pretest measures of outcome, of unreliable measurement, and of mode of data analysis. Psychol. Methods 15(1), 56-68 (2010).
47. Garber AM, MaCurdy TE, McClellan MA. Persistence of Medicare expenditures among elderly beneficiaries. In: Frontiers in Health Policy Research 1. Garber AM (Ed.). MIT Press, Cambridge, MA, USA, 153-180 (1998).
48. Maciejewski ML, Farley JF, Parker J, Wansink D. Copayment reductions generate greater medication adherence in targeted patients. Health Aff. (Millwood) 29(11), 2002-2008 (2010).
49. Mauri L, Silbaugh TS, Wolf RE et al. Long-term clinical outcomes after drug-eluting and bare-metal stenting in Massachusetts. Circulation 118(18), 1817-1827 (2008).
50. Califf RM, Filerman GL, Murray RK, Rosenblatt M. The Clinical Trials Enterprise in the United States: a Call for Disruptive Innovation. Institute of Medicine, Washington, DC, USA (2012).
51. Heiat A, Gross CP, Krumholz HM. Representation of the elderly, women, and minorities in heart failure clinical trials. Arch. Intern. Med. 162(15), 1682-1688 (2002).
52. Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA 290(12), 1624-1632 (2003). (Pubitemid 37430387)
53. Maciejewski ML, Livingston EH, Smith VA et al. Survival among high risk patients following bariatric surgery. JAMA 305(23), 2419-2426 (2011).
54. Ray WA. Evaluating medication effects outside of clinical trials: new-user designs. Am. J. Epidemiol. 158(9), 915-920 (2003). (Pubitemid 37323253)
55. Sidney S, Petitti DB, Quesenberry CP Jr. Myocardial infarction and the use of estrogen and estrogen-progestogen in postmenopausal women. Ann. Intern. Med. 127(7), 501-508 (1997). (Pubitemid 27419441)
56. Palmer JR, Shapiro S. A case-control study of myocardial infarction in relation to use of estrogen supplements. Am. J. Epidemiol. 137(1), 54-63 (1993). (Pubitemid 23077308)
57. Sourander L, Rajala T, Räihä I, Mäkinen J, Erkkola R, Helenius H. Cardiovascular and cancer morbidity and mortality and sudden cardiac death in postmenopausal women on oestrogen replacement therapy (ERT). Lancet 352(9145), 1965-1969 (1998). (Pubitemid 29000862)
58. Danaei G, Tavakkoli M, Hernan MA. Bias in observational studies of prevalent users: lessons for comparative effectiveness research from a meta-analysis of statins. Am. J. Epidemiol. 175(4), 250-262 (2012).
59. Hernandez AF, Hammill BG, O'Connor CM, Schulman KA, Curtis LH, Fonarow GC. Clinical effectiveness of beta-blockers in heart failure: findings from the OPTIMIZE-HF (Organized Program to Initiate Lifesaving Treatment in Hospitalized Patients with Heart Failure) Registry. J. Am. Coll. Cardiol. 53(2), 184-192 (2009).
60. Gardarsdottir H, Heerdink ER, Egberts AC. Potential bias in pharmacoepidemiological studies due to the length of the drug free period: a study on antidepressant drug use in adults in the Netherlands. Pharmacoepidemiol. Drug Saf. 15(5), 338-343 (2006). (Pubitemid 43797121)
61. Abbas S, Ihle P, Koster I, Schubert I. Estimation of disease incidence in claims data dependent on the length of follow-up: a methodological approach. Health Serv. Res. 47(2), 746-755 (2012).
62. Maciejewski ML, Bryson CL, Wang V, Perkins M, Liu CF. Potential bias in medication adherence studies of prevalent users. Health Serv. Res. (2013) (In Press).
63. Freiberger E, Haberle L, Spirduso WW, Zijlstra GA. Long-term effects of three multicomponent exercise interventions on physical performance and fall-related psychological outcomes in community-dwelling older adults: a randomized controlled trial. J. Am. Geriatr. Soc. 60(3), 437-446 (2012).
64. Nguyen HQ, Maciejewski ML, Gao S, Lin E, Williams B, Logerfo JP. Health care use and costs associated with use of a health club membership benefit in older adults with diabetes. Diabetes Care 31(8), 1562-1567 (2008).
65. West SL, Strom BL, Poole C. Validity of pharmacoepidemiology data and diagnosis data. In: Pharmacoepidemiology (3rd Edition). Strom BL (Ed.). John Wiley & Sons, Chichester, UK, 661-705 (2000).
66. Quan H, Parsons GA, Ghali WA. Validity of information on comorbidity derived rom ICD-9-CCM administrative data. Med. Care 40(8), 675-685 (2002).
67. Hammill BG, Hernandez AF, Peterson ED, Fonarow GC, Schulman KA, Curtis LH. Linking inpatient clinical registry data to Medicare claims data using indirect identifiers. Am. Heart J. 157(6), 995-1000 (2009).
68. Douglas I, Evans S, Smeeth L. Effect of statin treatment on short term mortality after pneumonia episode: cohort study. BMJ 342, d1642 (2012).
69. Glasgow RE. Commentary: electronic health records for comparative effectiveness research. Med. Care 50(Suppl.), S19-S20 (2012).
70. Glasgow RE, Kaplan RM, Ockene JK, Fisher EB, Emmons KM. Patient-reported measures of psychosocial issues and health behavior should be added to electronic health records. Health Aff. (Millwood) 31(3), 497-504 (2012).
71. Estabrooks PA, Boyle M, Emmons KM et al. Harmonized patient-reported data elements in the electronic health record: supporting meaningful use by primary care action on health behaviors and key psychosocial factors. J. Am. Med. Inform. Assoc. 19(4), 575-582 (2012).
72. Wu J, Roy J, Stewart WF. Prediction modeling using EHR data: challenges, strategies, and a comparison of machine learning approaches. Med. Care 48(Suppl. 6), S106-S113 (2010).
73. Kennedy EH, Wiitala WL, Hayward RA, Sussman JB. Improved cardiovascular risk prediction using nonparametric regression and electronic health record data. Med. Care doi:10.1097/MLR.0b013e31827da594 (2012) (Epub ahead of print).
74. Morgan SL, Winship C. Counterfactuals and Causal Inference: Methods and Principles for Social Research. Cambridge University Press, NY, USA (2007).