Opportunities and challenges in developing risk prediction models with electronic health records data: A systematic review

Journal of the American Medical Informatics Association

Volumn 24, Issue 1, 2017, Pages 198-208

  Goldstein, Benjamin A ^a,b   Navar, Ann Marie ^b,c   Pencina, Michael J ^a,b   Ioannidis, John P A ^d,e  

^a DUKE UNIVERSITY SCHOOL OF MEDICINE   (United States)

^b DUKE CLINICAL RESEARCH INSTITUTE   (United States)

^c DUKE UNIVERSITY MEDICAL CENTER   (United States)

^d STANFORD UNIVERSITY   (United States)

^e STANFORD UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

Electronic medical record; Review; Risk assessment

Indexed keywords

CONTROLLED CLINICAL TRIAL; CONTROLLED STUDY; ELECTRONIC HEALTH RECORD; ELECTRONIC MEDICAL RECORD; FOLLOW UP; HOSPITALIZATION; HUMAN; MEDLINE; MORTALITY; MULTICENTER STUDY; PREDICTION; PREDICTOR VARIABLE; PUBLICATION; REPRODUCIBILITY; RISK ASSESSMENT; SAMPLE SIZE; STATISTICS; STUDY DESIGN; SYSTEMATIC REVIEW; VALIDATION PROCESS; PROCEDURES; STATISTICAL MODEL;

ELECTRONIC HEALTH RECORDS; HUMANS; MODELS, STATISTICAL; RISK ASSESSMENT;

EID: 85014666740     PISSN: 10675027     EISSN: 1527974X     Source Type: Journal    
DOI: 10.1093/jamia/ocw042     Document Type: Article

References (140)

1. Charles D, Gabriel M, Searcy T. Adoption of electronic health record systems among U.S. non-federal acute care hospitals: 2008-2014. 2015 https://www.healthit.gov/sites/default/files/data-brief/2014HospitalAdop tionDataBrief.pdf.
2. Rothman B, Leonard JC, Vigoda MM. Future of electronic health records: implications for decision support. Mt Sinai J Med NY. 2012;79(6): 757-768.
3. Wilson PW, D'Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97(18):1837-1847.
4. Hersh WR, Weiner MG, Embi PJ, et al. Caveats for the use of operational electronic health record data in comparative effectiveness research. Med Care. 2013;51(8 Suppl 3):S30-S37.
5. Richesson RL, Rusincovitch SA, Wixted D, et al. A comparison of phenotype definitions for diabetes mellitus. J Am Med Inform Assoc. 2013;20(e2):e319-e326.
6. Tangri N, Inker L, Levey AS. A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods. J Clin Epidemiol. 2013;66(6):697.
7. Mallett S, Royston P, Waters R, Dutton S, Altman DG. Reporting performance of prognostic models in cancer: a review. BMC Med. 2010;8:21.
8. Counsell C, Dennis M. Systematic review of prognostic models in patients with acute stroke. Cerebrovasc Dis Basel Switz. 2001;12(3): 159-170.
9. Collins GS, Mallett S, Omar O, Yu L-M. Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting. BMC Med. 2011;9:103.
10. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009;62(10):e1-e34.
11. Ingui BJ, Rogers MA. Searching for clinical prediction rules in MEDLINE. J Am Med Inform Assoc. 2001;8(4):391-397.
12. Geersing G-J, Bouwmeester W, Zuithoff P, et al. Search filters for finding prognostic and diagnostic prediction studies in Medline to enhance systematic reviews. PloS One. 2012;7(2):e32844.
13. Hippisley-Cox J, Stables D, Pringle M. QRESEARCH: a new general practice database for research. Inform Prim Care. 2004;12(1):49-50.
14. Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. J Clin Epidemiol. 2015;68(2):134-143.
15. Moons KGM, Altman DG, Reitsma JB, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med. 2015;162(1): W1-W73.
16. D'Agostino RB, Pencina MJ, Massaro JM, Coady S. Cardiovascular Disease Risk Assessment: Insights from Framingham. Glob Heart. 2013;8(1): 11-23.
17. Alvarez CA, Clark CA, Zhang S, et al. Predicting out of intensive care unit cardiopulmonary arrest or death using electronic medical record data. BMC Med Inform Decis Mak. 2013;13:28.
18. Amarasingham R, Moore BJ, Tabak YP, et al. An automated model to identify heart failure patients at risk for 30-day readmission or death using electronic medical record data. Med Care. 2010;48(11):981-988.
19. Atchison CM, Arlikar S, Amankwah E, et al. Development of a new risk score for hospital-associated venous thromboembolism in noncritically ill children: findings from a large single-institutional case-control study. J Pediatr. 2014;165(4):793-798.
20. Ayyagari R, Vekeman F, Lefebvre P, et al. Pulse pressure and stroke risk: development and validation of a new stroke risk model. Curr Med Res Opin. 2014;30(12):2453-2460.
21. Baillie CA, VanZandbergen C, Tait G, et al. The readmission risk flag: using the electronic health record to automatically identify patients at risk for 30-day readmission. J Hosp Med Off Publ Soc Hosp Med. 2013;8(12):689-695.
22. Barrett TW, Martin AR, Storrow AB, et al. A clinical prediction model to estimate risk for 30-day adverse events in emergency department patients with symptomatic atrial fibrillation. Ann Emerg Med. 2011;57(1):1-12.
23. Billings J, Georghiou T, Blunt I, Bardsley M. Choosing a model to predict hospital admission: an observational study of new variants of predictive models for case finding. BMJ Open. 2013;3(8):e003352.
24. Busch AB, Neelon B, Zelevinsky K, He Y, Normand S-LT. Accurately predicting bipolar disorder mood outcomes: implications for the use of electronic databases. Med Care. 2012;50(4):311-319.
25. Carter EM, Potts HWW. Predicting length of stay from an electronic patient record system: a primary total knee replacement example. BMC Med Inform Decis Mak. 2014;14:26.
26. Casarett DJ, Farrington S, Craig T, et al. The art versus science of predicting prognosis: can a prognostic index predict short-term mortality better than experienced nurses do J Palliat Med. 2012;15(6):703-708.
27. Chang Y-J, Yeh M-L, Li Y-C, et al. Predicting hospital-acquired infections by scoring system with simple parameters. PloS One. 2011;6(8):e23137.
28. Cheng P, Neugaard B, Foulis P, Conlin PR. Hemoglobin A1c as a predictor of incident diabetes. Diabetes Care. 2011;34(3):610-615.
29. Cholleti S, Post A, Gao J, et al. Leveraging derived data elements in data analytic models for understanding and predicting hospital readmissions. AMIA Annu Symp Proc AMIA Symp AMIA Symp. 2012;2012:103-111.
30. Choudhry SA, Li J, Davis D, Erdmann C, Sikka R, Sutariya B. A publicprivate partnership develops and externally validates a 30-day hospital readmission risk prediction model. Online J Public Health Inform. 2013;5(2):219.
31. Churpek MM, Yuen TC, Park SY, Gibbons R, Edelson DP. Using electronic health record data to develop and validate a prediction model for adverse outcomes in the wards*, Crit Care Med. 2014;42(4):841-848.
32. Crane SJ, Tung EE, Hanson GJ, Cha S, Chaudhry R, Takahashi PY. Use of an electronic administrative database to identify older community dwelling adults at high-risk for hospitalization or emergency department visits: the elders risk assessment index. BMC Health Serv Res. 2010;10: 338.
33. Dai W, Brisimi TS, Adams WG, Mela T, Saligrama V, Paschalidis IC. Prediction of hospitalization due to heart diseases by supervised learning methods. Int J Med Inf. 2015;84(3):189-197.
34. Eapen ZJ, Liang L, Fonarow GC, et al. Validated, electronic health record deployable prediction models for assessing patient risk of 30-day rehospitalization and mortality in older heart failure patients. JACC Heart Fail. 2013;1(3):245-251.
35. Escobar GJ, Gardner MN, Greene JD, Draper D, Kipnis P. Risk-adjusting hospital mortality using a comprehensive electronic record in an integrated health care delivery system. Med Care. 2013;51(5):446-453.
36. Escobar GJ, LaGuardia JC, Turk BJ, Ragins A, Kipnis P, Draper D. Early detection of impending physiologic deterioration among patients who are not in intensive care: development of predictive models using data from an automated electronic medical record. J Hosp Med Off Publ Soc Hosp Med. 2012;7(5):388-395.
37. Goldstein BA, Chang TI, Mitani AA, Assimes TL, Winkelmayer WC. Near-term prediction of sudden cardiac death in older hemodialysis patients using electronic health records. Clin J Am Soc Nephrol. 2014;9(1): 82-91.
38. Gultepe E, Green JP, Nguyen H, Adams J, Albertson T, Tagkopoulos I. From vital signs to clinical outcomes for patients with sepsis: a machine learning basis for a clinical decision support system. J Am Med Inform Assoc. 2014;21(2):315-325.
39. Gupta S, Tran T, Luo W, et al. Machine-learning prediction of cancer survival: a retrospective study using electronic administrative records and a cancer registry. BMJ Open. 2014;4(3):e004007.
40. Hao S, Jin B, Shin AY, et al. Risk prediction of emergency department revisit days post discharge: a prospective study. PloS One. 2014;9(11): e112944.
41. Hebert C, Du H, Peterson LR, Robicsek A. Electronic health record-based detection of risk factors for Clostridium difficile infection relapse. Infect Control Hosp Epidemiol Off J Soc Hosp Epidemiol Am. 2013;34(4): 407-414.
42. Hebert C, Shivade C, Foraker R, et al. Diagnosis-specific readmission risk prediction using electronic health data: a retrospective cohort study. BMC Med Inform Decis Mak. 2014;14:65.
43. Himes BE, Dai Y, Kohane IS, Weiss ST, Ramoni MF. Prediction of chronic obstructive pulmonary disease (COPD) in asthma patients using electronic medical records. J AmMed Inform Assoc. 2009;16(3):371-379.
44. Hivert M-F, Dusseault-Bélanger F, Cohen A, Courteau J, Vanasse A. Modified metabolic syndrome criteria for identification of patients at risk of developing diabetes and coronary heart diseases: longitudinal assessment via electronic health records. Can J Cardiol. 2012;28(6): 744-749.
45. Hong KJ, Shin SD, Ro YS, Song KJ, Singer AJ. Development and validation of the excess mortality ratio-based Emergency Severity Index. Am J Emerg Med. 2012;30(8):1491-1500.
46. Huang SH, LePendu P, Iyer SV, Tai-Seale M, Carrell D, Shah NH. Toward personalizing treatment for depression: predicting diagnosis and severity. J Am Med Inform Assoc. 2014;21(6):1069-1075.
47. Huang Y, Hanauer DA. Patient no-show predictive model development using multiple data sources for an effective overbooking approach. Appl Clin Inform. 2014;5(3):836-860.
48. Jin SJ, Kim M, Yoon JH, Song YG. A new statistical approach to predict bacteremia using electronic medical records. Scand J Infect Dis. 2013;45(9):672-680.
49. Johnson ES, Weinstein JR, Thorp ML, et al. Predicting the risk of hyperkalemia in patients with chronic kidney disease starting lisinopril. Pharmacoepidemiol Drug Saf. 2010;19(3):266-272.
50. Karnik S, Tan SL, Berg B, et al. Predicting atrial fibrillation and flutter using electronic health records. Conf Proc Annu Int Conf IEEE Eng Med Biol Soc IEEE Eng Med Biol Soc Conf. 2012;2012:5562-5565.
51. Kawaler E, Cobian A, Peissig P, Cross D, Yale S, Craven M. Learning to predict post-hospitalization VTE risk from EHR data. AMIA Annu Symp Proc AMIA Symp AMIA Symp. 2012;2012:436-445.
52. Kennedy EH, Wiitala WL, Hayward RA, Sussman JB. Improved cardiovascular risk prediction using nonparametric regression and electronic health record data. Med Care. 2013;51(3):251-258.
53. Khan A, Malone ML, Pagel P, Vollbrecht M, Baumgardner DJ. An electronic medical record-derived real-time assessment scale for hospital readmission in the elderly. WMJ Off Publ State Med Soc Wis. 2012;111(3):119-123.
54. Kontio E, Airola A, Pahikkala T, et al. Predicting patient acuity from electronic patient records. J Biomed Inform. 2014;51:35-40.
55. Kor DJ, Warner DO, Alsara A, et al. Derivation and diagnostic accuracy of the surgical lung injury prediction model. Anesthesiology. 2011;115(1):117-128.
56. Liu V, Kipnis P, Gould MK, Escobar GJ. Length of stay predictions: improvements through the use of automated laboratory and comorbidity variables. Med Care. 2010;48(8):739-744.
57. Mani S, Chen Y, Elasy T, Clayton W, Denny J. Type 2 diabetes risk forecasting from EMR data using machine learning. AMIA Annu Symp Proc AMIA Symp AMIA Symp. 2012;2012:606-615.
58. Mani S, Ozdas A, Aliferis C, et al. Medical decision support using machine learning for early detection of late-onset neonatal sepsis. J Am Med Inform Assoc. 2014;21(2):326-336.
59. Matheny ME, Miller RA, Ikizler TA, et al. Development of inpatient risk stratification models of acute kidney injury for use in electronic health records. Med Decis Mak Int J Soc Med Decis Mak. 2010;30(6): 639-650.
60. Mathias JS, Agrawal A, Feinglass J, Cooper AJ, Baker DW, Choudhary A. Development of a 5 year life expectancy index in older adults using predictive mining of electronic health record data. J Am Med Inform Assoc. 2013;20(e1):e118-e124.
61. Meyfroidt G, Guïza F, Cottem D, et al. Computerized prediction of intensive care unit discharge after cardiac surgery: development and validation of a Gaussian processes model. BMC Med Inform Decis Mak. 2011;11:64.
62. Monsen KA, Swanberg HL, Oancea SC, Westra BL. Exploring the value of clinical data standards to predict hospitalization of home care patients. Appl Clin Inform. 2012;3(4):419-436.
63. Navarro AE, Enguidanos S, Wilber KH. Identifying risk of hospital readmission among Medicare aged patients: an approach using routinely collected data. Home Health Care Serv Q. 2012;31(2): 181-195.
64. Nijhawan AE, Clark C, Kaplan R, Moore B, Halm EA, Amarasingham R. An electronic medical record-based model to predict 30-day risk of readmission and death among HIV-infected inpatients. J Acquir Immune Defic Syndr 1999. 2012;61(3):349-358.
65. O'Leary EA, Desale SY, Yi WS, et al. Letting the sun set on small bowel obstruction: can a simple risk score tell us when nonoperative care is inappropriate Am Surg. 2014;80(6):572-579.
66. Perkins RM, Rahman A, Bucaloiu ID, et al. Readmission after hospitalization for heart failure among patients with chronic kidney disease: a prediction model. Clin Nephrol. 2013;80(6):433-440.
67. Puttkammer N, Zeliadt S, Balan JG, et al. Development of an electronic medical record based alert for risk of HIV treatment failure in a low-resource setting. PloS One. 2014;9(11):e112261.
68. Ramchandran KJ, Shega JW, Von Roenn J, et al. A predictive model to identify hospitalized cancer patients at risk for 30-day mortality based on admission criteria via the electronic medical record. Cancer. 2013;119(11):2074-2080.
69. Rana S, Tran T, Luo W, Phung D, Kennedy RL, Venkatesh S. Predicting unplanned readmission after myocardial infarction from routinely collected administrative hospital data. Aust Health Rev Publ Aust Hosp Assoc. 2014;38(4):377-382.
70. Rapsomaniki E, Shah A, Perel P, et al. Prognostic models for stable coronary artery disease based on electronic health record cohort of 102 023 patients. Eur Heart J. 2014;35(13):844-852.
71. Robbins GK, Johnson KL, Chang Y, et al. Predicting virologic failure in an HIV clinic. Clin Infect Dis Off Publ Infect Dis Soc Am. 2010;50(5): 779-786.
72. Robicsek A, Beaumont JL, Wright M-O, Thomson RB, Kaul KL, Peterson LR. Electronic prediction rules for methicillin-resistant Staphylococcus aureus colonization. Infect Control Hosp Epidemiol Off J Soc Hosp Epidemiol Am. 2011;32(1):9-19.
73. Rothman MJ, Rothman SI, Beals J. Development and validation of a continuous measure of patient condition using the Electronic Medical Record. J Biomed Inform. 2013;46(5):837-848.
74. Roubinian NH, Murphy EL, Swain BE, et al.; NHLBI Recipient Epidemiology and Donor Evaluation Study-III (REDS-III), and Northern California Kaiser Permanente DOR Systems Research Initiative. Predicting red blood cell transfusion in hospitalized patients: role of hemoglobin level, comorbidities, and illness severity. BMC Health Serv Res. 2014; 14:213.
75. Sairamesh J, Rajagopal R, Nemana R, Argenbright K. Early warning and risk estimation methods based on unstructured text in electronic medical records to improve patient adherence and care. AMIA Annu Symp Proc AMIA Symp AMIA Symp. 2009;2009:553-557.
76. Sho S, Neal MD, Sperry J, Hackam DJ. A novel scoring system to predict the development of necrotizing enterocolitis totalis in premature infants. J Pediatr Surg. 2014;49(7):1053-1056.
77. Singal AG, Rahimi RS, Clark C, et al. An automated model using electronic medical record data identifies patients with cirrhosis at high risk for readmission. Clin Gastroenterol Hepatol Off Clin Pract J Am Gastroenterol Assoc. 2013;11(10):1335-1341.e1.
78. Skevofilakas M, Zarkogianni K, Karamanos BG, Nikita KS. A hybrid Decision Support System for the risk assessment of retinopathy development as a long term complication of type 1 diabetes mellitus. Conf Proc Annu Int Conf IEEE Eng Med Biol Soc IEEE Eng Med Biol Soc Annu Conf. 2010;2010:6713-6716.
79. Smith DH, Johnson ES, Thorp ML, et al. Predicting poor outcomes in heart failure. Perm J. 2011;15(4):4-11.
80. Smits FT, Brouwer HJ, van Weert HCP, Schene AH, ter Riet G. Predictability of persistent frequent attendance: a historic 3-year cohort study. Br J Gen Pract JR Coll Gen Pract. 2009;59(559):e44-e50.
81. Smolin B, Levy Y, Sabbach-Cohen E, Levi L, Mashiach T. Predicting mortality of elderly patients acutely admitted to the Department of Internal Medicine. Int J Clin Pract. 2015;69(4):501-508.
82. Still CD, Wood GC, Benotti P, et al. Preoperative prediction of type diabetes remission after Roux-en-Y gastric bypass surgery: a retrospective cohort study. Lancet Diabetes Endocrinol. 2014;2(1): 38-45.
83. Sun J, McNaughton CD, Zhang P, et al. Predicting changes in hypertension control using electronic health records from a chronic disease management program. J Am Med Inform Assoc. 2014;21(2): 337-344.
84. Tabak YP, Sun X, Hyde L, Yaitanes A, Derby K, Johannes RS. Using enriched observational data to develop and validate age-specific mortality risk adjustment models for hospitalized pediatric patients. Med Care. 2013;51(5):437-445.
85. Tabak YP, Sun X, Johannes RS, Hyde L, Shorr AF, Lindenauer PK. Development and validation of a mortality risk-adjustment model for patients hospitalized for exacerbations of chronic obstructive pulmonary disease. Med Care. 2013;51(7):597-605.
86. Tabak YP, Sun X, Nunez CM, Johannes RS. Using electronic health record data to develop inpatient mortality predictive model: Acute Laboratory Risk of Mortality Score (ALaRMS). J Am Med Inform Assoc. 2014;21(3):455-463.
87. Tescher AN, Branda ME, Byrne TJO, Naessens JM. All at-risk patients are not created equal: analysis of Braden pressure ulcer risk scores to identify specific risks. J Wound Ostomy Cont Nurs Off Publ Wound Ostomy Cont Nurses Soc. 2012;39(3):282-291.
88. Tran T, Luo W, Phung D, et al. Risk stratification using data from electronic medical records better predicts suicide risks than clinician assessments. BMC Psychiatry. 2014;14:76.
89. Uyar A, Bener A, Ciray HN. Predictive modeling of implantation outcome in an in vitro fertilization setting: an application of machine learning methods. Med Decis Mak Int J Soc Med Decis Mak. 2014.
90. Wang L, Porter B, Maynard C, et al. Predicting risk of hospitalization or death among patients with heart failure in the veterans health administration. Am J Cardiol. 2012;110(9):1342-1349.
91. Wang L, Porter B, Maynard C, et al. Predicting risk of hospitalization or death among patients receiving primary care in the Veterans Health Administration. Med Care. 2013;51(4):368-373.
92. Wells BJ, Roth R, Nowacki AS, et al. Prediction of morbidity and mortality in patients with type 2 diabetes. Peer J. 2013;1:e87.
93. Westra BL, Savik K, Oancea C, Choromanski L, Holmes JH, Bliss D. Predicting improvement in urinary and bowel incontinence for home health patients using electronic health record data. J Wound Ostomy Cont Nurs Off Publ Wound Ostomy Cont Nurses Soc. 2011;38(1): 77-87.
94. Woller SC, Stevens SM, Jones JP, et al. Derivation and validation of a simple model to identify venous thromboembolism risk in medical patients. Am J Med. 2011;124(10):947-954.e2.
95. Wu J, Roy J, Stewart WF. Prediction modeling using EHR data: challenges, strategies, and a comparison of machine learning approaches. Med Care. 2010;48(6 Suppl):S106-S113.
96. Zhai H, Brady P, Li Q, et al. Developing and evaluating a machine learning based algorithm to predict the need of pediatric intensive care unit transfer for newly hospitalized children. Resuscitation. 2014;85(8): 1065-1071.
97. Zhao D, Weng C. Combining PubMed knowledge and EHR data to develop a weighted bayesian network for pancreatic cancer prediction. J Biomed. Inform. 2011;44(5):859-868.
98. ChurpekMM, Yuen TC, Edelson DP. Predicting clinical deterioration in the hospital: the impact of outcome selection. Resuscitation. 2013;84(5): 564-568.
99. Churpek MM, Yuen TC, Park SY, Meltzer DO, Hall JB, Edelson DP. Derivation of a cardiac arrest prediction model using ward vital signs*. Crit Care Med. 2012;40(7):2102-2108.
100. Churpek MM, Yuen TC, Winslow C, et al. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med. 2014;190(6):649-655.
101. Kotz D, Simpson CR, Viechtbauer W, van Schayck OCP, Sheikh A. Development and validation of a model to predict the 10-year risk of general practitioner-recorded COPD. NPJ Prim Care Respir Med. 2014; 24:14011.
102. Lipsky BA, Kollef MH, Miller LG, Sun X, Johannes RS, Tabak YP. Predicting bacteremia among patients hospitalized for skin and skin-structure infections: derivation and validation of a risk score. Infect Control Hosp Epidemiol. 2010;31(8):828-837.
103. Lipsky BA, Weigelt JA, Sun X, Johannes RS, Derby KG, Tabak YP. Developing and validating a risk score for lower-extremity amputation in patients hospitalized for a diabetic foot infection. Diabetes Care. 2011;34(8):1695-1700.
104. Liu V, Kipnis P, Gould MK, Escobar GJ. Length of stay predictions: improvements through the use of automated laboratory and comorbidity variables. Med Care. 2010;48(8):739-744.
105. Puopolo KM, Draper D, Wi S, et al. Estimating the probability of neonatal early-onset infection on the basis of maternal risk factors. Pediatrics. 2011;128(5):e1155-e1163.
106. Saltzman JR, Tabak YP, Hyett BH, Sun X, Travis AC, Johannes RS. A simple risk score accurately predicts in-hospital mortality, length of stay, and cost in acute upper GI bleeding. Gastrointest Endosc. 2011;74(6): 1215-1224.
107. Tabak YP, Sun X, Derby KG, Kurtz SG, Johannes RS. Development and validation of a disease-specific risk adjustment system using automated clinical data. Health Serv Res. 2010;45(6 Pt 1):1815-1835.
108. Hippisley-Cox J, Coupland C, Brindle P. Derivation and validation of QStroke score for predicting risk of ischaemic stroke in primary care and comparison with other risk scores: a prospective open cohort study. BMJ. 2013;346:f2573.
109. Hippisley-Cox J, Coupland C, Robson J, Brindle P. Derivation, validation, and evaluation of a new QRISK model to estimate lifetime risk of cardiovascular disease: cohort study using QResearch database. BMJ. 2010;341:c6624.
110. Hippisley-Cox J, Coupland C, Robson J, Sheikh A, Brindle P. Predicting risk of type 2 diabetes in England and Wales: prospective derivation and validation of QDScore. BMJ. 2009;338:b880.
111. Hippisley-Cox J, Coupland C. Predicting risk of osteoporotic fracture in men and women in England and Wales: prospective derivation and validation of QFractureScores. BMJ. 2009;339:b4229.
112. Hippisley-Cox J, Coupland C. Predicting the risk of chronic Kidney Disease in men and women in England and Wales: prospective derivation and external validation of theQKidney Scores. BMC FamPract. 2010;11:49.
113. Hippisley-Cox J, Coupland C. Development and validation of risk prediction algorithm (QThrombosis) to estimate future risk of venous thromboembolism: prospective cohort study. BMJ. 2011;343:d4656.
114. Hippisley-Cox J, Coupland C. Identifying patients with suspected gastro- oesophageal cancer in primary care: derivation and validation of an algorithm. Br J Gen Pract J R Coll Gen Pract. 2011;61(592):e707-e714.
115. Hippisley-Cox J, Coupland C. Identifying patients with suspected lung cancer in primary care: derivation and validation of an algorithm. Br J Gen Pract J R Coll Gen Pract. 2011;61(592):e715-e723.
116. Hippisley-Cox J, Coupland C. Derivation and validation of updated QFracture algorithm to predict risk of osteoporotic fracture in primary care in the United Kingdom: prospective open cohort study. BMJ. 2012;344:e3427.
117. Hippisley-Cox J, Coupland C. Identifying women with suspected ovarian cancer in primary care: derivation and validation of algorithm. BMJ. 2012;344:d8009.
118. Hippisley-Cox J, Coupland C. Identifying patients with suspected colorectal cancer in primary care: derivation and validation of an algorithm. Br J Gen Pract J R Coll Gen Pract. 2012;62(594):e29-e37.
119. Hippisley-Cox J, Coupland C. Identifying patients with suspected pancreatic cancer in primary care: derivation and validation of an algorithm. Br J Gen Pract J R Coll Gen Pract. 2012;62(594):e38-e45.
120. Hippisley-Cox J, Coupland C. Identifying patients with suspected renal tract cancer in primary care: derivation and validation of an algorithm. Br J Gen Pract J R Coll Gen Pract. 2012;62(597):e251-e260.
121. Hippisley-Cox J, Coupland C. Predicting risk of emergency admission to hospital using primary care data: derivation and validation of QAdmissions score. BMJ Open. 2013;3(8):e003482.
122. Hippisley-Cox J, Coupland C. Symptoms and risk factors to identify men with suspected cancer in primary care: derivation and validation of an algorithm. Br J Gen Pract J R Coll Gen Pract. 2013;63(606):e1-e10.
123. Hippisley-Cox J, Coupland C. Symptoms and risk factors to identify women with suspected cancer in primary care: derivation and validation of an algorithm. Br J Gen Pract J R Coll Gen Pract. 2013; 63(606):e11-e21.
124. Hippisley-Cox J, Coupland C. Predicting risk of upper gastrointestinal bleed and intracranial bleed with anticoagulants: cohort study to derive and validate the QBleed scores. BMJ. 2014;349:g4606.
125. Siontis GCM, Tzoulaki I, Castaldi PJ, Ioannidis JPA. External validation of new risk prediction models is infrequent and reveals worse prognostic discrimination. J Clin Epidemiol. 2015;68(1):25-34.
126. Wiens J, Guttag J, Horvitz E. A study in transfer learning: leveraging data from multiple hospitals to enhance hospital-specific predictions. J Am Med Inform Assoc. 2014;21(4):699-706.
127. Liu J, Hong Y, D'Agostino RB, et al. Predictive value for the Chinese population of the Framingham CHD risk assessment tool compared with the Chinese Multi-Provincial Cohort Study. JAMA. 2004;291(21): 2591-2599.
128. Fleurence RL, Curtis LH, Califf RM, Platt R, Selby JV, Brown JS. Launching PCORnet, a national patient-centered clinical research network. J Am Med Inform Assoc. 2014;21(4):578-582.
129. Henderson R, Diggle P, Dobson A. Joint modelling of longitudinal measurements and event time data. Biostat Oxf Engl. 2000;1(4):465-480.
130. Kharrazi H, Wang C, Scharfstein D. Prospective EHR-based clinical trials: the challenge of missing data. J Gen Intern Med. 2014;29(7): 976-978.
131. Rusanov A,WeiskopfNG, Wang S, Weng C. Hidden in plain sight: bias towards sick patients when sampling patients with sufficient electronic health record data for research.BMCMed InformDecisMak. 2014;14:51.
132. Goldstein BA, Bhavsar NA, Phelan M, Pencina MJ. Controlling for informed presence bias due to the number of health encounters in an Electronic Health Record. Am J Epidemiol. (in press).
133. Cook NR. Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. Clin Chem. 2008;54(1):17-23.
134. Mandic S, Go C, Aggarwal I, Myers J, Froelicher VF. Relationship of predictive modeling to receiver operating characteristics. J Cardiopulm Rehabil Prev. 2008;28(6):415-419.
135. Gottesman O, Kuivaniemi H, Tromp G, et al.; eMERGE Network. The Electronic Medical Records and Genomics (eMERGE) Network: past, present, and future. Genet Med Off J Am Coll Med Genet. 2013;15(10): 761-771.
136. Goldstein BA, Knowles JW, Salfati E, Ioannidis JPA, Assimes TL. Simple, standardized incorporation of genetic risk into non-genetic risk prediction tools for complex traits: coronary heart disease as an example. Front Genet. 2014;5:254.
137. Kho AN, Rasmussen LV, Connolly JJ, et al. Practical challenges in integrating genomic data into the electronic health record. Genet Med Off J Am Coll Med Genet. 2013;15(10):772-778.
138. Ury AG. Storing and interpreting genomic information in widely deployed electronic health record systems. Genet Med Off J Am Coll Med Genet. 2013;15(10):779-785.
139. Van Driest SL, Wells QS, Stallings S, et al. Association of arrhythmia-related genetic variants with phenotypes documented in electronic medical records. JAMA. 2016;315(1):47-57.
140. Feero WG. Establishing the clinical validity of arrhythmia-related genetic variations using the electronic medical record: a valid take on precision medicine JAMA. 2016;315(1):33-35.