Prediction models for diagnosis and prognosis of covid-19: Systematic review and critical appraisal

The BMJ

Volumn 369, Issue , 2020, Pages

  Wynants, Laure ^a,b   Van Calster, Ben ^b,c   Collins, Gary S ^d,e   Riley, Richard D ^f   Heinze, Georg ^g   Schuit, Ewoud ^h   Bonten, Marc M J ^h   Damen, Johanna A A ^h   Debray, Thomas P A ^h   De Vos, Maarten ^b   Dhiman, Paula ^d,e   Haller, Maria C ^g,i   Harhay, Michael O ^j,k   Henckaerts, Liesbet ^l,m   Kreuzberger, Nina ⁿ   Lohmann, Anna ^c   Luijken, Kim ^c   Ma, Jie ^e   Andaur Navarro, Constanza L ^h   Reitsma, Johannes B ^h   Sergeant, Jamie C ^o   Shi, Chunhu ^p   Skoetz, Nicole ⁿ   Smits, Luc J M ^a   Snell, Kym I E ^f   Sperrin, Matthew ^p   Spijker, René ^h   Steyerberg, Ewout W ^c   Takada, Toshihiko ^d   Van Kuijk, Sander M J ^q   Van Royen, Florien S ^h   Wallisch, Christine ^g,r,s   Hooft, Lotty ^h   Moons, Karel G M ^h   Van Smeden, Maarten ^h   more..

^a MAASTRICHT UNIVERSITY   (Netherlands)

^b UNIVERSITY OF LEUVEN   (Belgium)

^c LEIDEN UNIVERSITY MEDICAL CENTER   (Netherlands)

^d UNIVERSITY OF OXFORD   (United Kingdom)

^e JOHN RADCLIFFE HOSPITAL   (United Kingdom)

^f KEELE UNIVERSITY   (United Kingdom)

^g MEDICAL UNIVERSITY OF VIENNA   (Austria)

^h UNIVERSITY MEDICAL CENTER UTRECHT   (Netherlands)

ⁱ Hospital of Elisabethinen   (Austria)

^j UNIVERSITY OF PENNSYLVANIA   (United States)

^k UNIVERSITY OF PENNSYLVANIA   (United States)

^l REGA INSTITUTE   (Belgium)

^m UNIVERSITY HOSPITALS LEUVEN   (Belgium)

ⁿ UNIVERSITY HOSPITAL OF COLOGNE   (Germany)

^o UNIVERSITY OF MANCHESTER   (United Kingdom)

^p UNIVERSITY OF MANCHESTER   (United Kingdom)

^q MAASTRICHT UNIVERSITY MEDICAL CENTER   (Netherlands)

^r CHARITÉ UNIVERSITÄTSMEDIZIN BERLIN   (Germany)

^s BERLIN INSTITUTE OF HEALTH   (Germany)

more..

Author keywords

[No Author keywords available]

Indexed keywords

C REACTIVE PROTEIN; LACTATE DEHYDROGENASE;

AGE; ARTICLE; BODY TEMPERATURE; COMPUTER ASSISTED TOMOGRAPHY; CORONAVIRUS DISEASE 2019; CORONAVIRUS INFECTION; DISEASE SEVERITY; HOSPITAL ADMISSION; HOSPITALIZATION; HUMAN; LACTATE DEHYDROGENASE BLOOD LEVEL; LENGTH OF STAY; LYMPHOCYTE COUNT; MORTALITY RISK; PHYSICAL DISEASE BY BODY FUNCTION; PREDICTION; PRIORITY JOURNAL; PROGNOSIS; PROTEIN BLOOD LEVEL; SEX; SYSTEMATIC REVIEW; VIRUS PNEUMONIA; CORONAVIRINAE; DISEASE EXACERBATION; MULTIVARIATE ANALYSIS; PANDEMIC; THEORETICAL MODEL;

CORONAVIRUS; CORONAVIRUS INFECTIONS; DISEASE PROGRESSION; HOSPITALIZATION; HUMANS; MODELS, THEORETICAL; MULTIVARIATE ANALYSIS; PANDEMICS; PNEUMONIA, VIRAL; PROGNOSIS;

EID: 85083072038     PISSN: 09598146     EISSN: 17561833     Source Type: Journal    
DOI: 10.1136/bmj.m1328     Document Type: Article

References (71)

1. Dong E, Du H, Gardner L. An interactive web-based dashboard to track COVID-19 in real time. Lancet Infect Dis 2020:S1473-3099(20)30120-1. doi:10.1016/S1473-3099(20)30120-1
2. Arabi YM, Murthy S, Webb S. COVID-19: a novel coronavirus and a novel challenge for critical care. Intensive Care Med 2020. doi:10.1007/s00134-020-05955-1
3. Grasselli G, Pesenti A, Cecconi M. Critical care utilization for the COVID-19 outbreak in Lombardy, Italy: early experience and forecast during an emergency response. JAMA 2020. doi:10.1001/jama.2020.4031
4. Xie J, Tong Z, Guan X, Du B, Qiu H, Slutsky AS. Critical care crisis and some recommendations during the COVID-19 epidemic in China. Intensive Care Med 2020. doi:10.1007/s00134-020-05979-7
5. Wellcome Trust. Sharing research data and fndings relevant to the novel coronavirus (COVID-19) outbreak 2020. https://wellcome.ac.uk/press-release/sharing-research-data-and-fndings-relevant-novel-coronavirus-covid-19-outbreak.
6. Institute of Social and Preventive Medicine. Living evidence on COVID-19 2020. https://ispmbern.github.io/covid-19/living-review/index.html.
7. Xie J, Hungerford D, Chen H, et al. Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19. medRxiv [Preprint] 2020. doi:10.1101/2020.03.28.20045997
8. DeCaprio D, Gartner J, Burgess T, et al. Building a COVID-19 vulnerability index. arXiv e-prints [Preprint] 2020. https://ui.adsabs. harvard.edu/abs/2020arXiv200307347D.
9. Bai X, Fang C, Zhou Y, et al. Predicting COVID-19 malignant progression with AI techniques. medRxiv [Preprint] 2020. doi:10.1101/2020.03.20.20037325
10. Feng C, Huang Z, Wang L, et al. A novel triage tool of artifcial intelligence assisted diagnosis aid system for suspected covid-19 pneumonia in fever clinics. medRxiv [Preprint] 2020. doi:10.1101/2020.03.19.20039099
11. Jin C, Chen W, Cao Y, et al. Development and evaluation of an AI system for covid-19 diagnosis. medRxiv [Preprint] 2020. doi:10.1101/2020.03.20.20039834
12. Meng Z, Wang M, Song H, et al. Development and utilization of an intelligent application for aiding COVID-19 diagnosis. medRxiv [Preprint] 2020. doi:10.1101/2020.03.18.20035816
13. Moons KG, de Groot JA, Bouwmeester W, et al. Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist. PLoS Med 2014;11:e1001744. doi:10.1371/journal.pmed.1001744
14. Moons KGM, Wolf RF, Riley RD, et al. PROBAST: a tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration. Ann Intern Med 2019;170:W1-33. doi:10.7326/M18-1377
15. Moons KGM, Altman DG, Reitsma JB, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 2015;162:W1-73. doi:10.7326/M14-0698
16. Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. Springer US, 2019. doi:10.1007/978-3-030-16399-0
17. Liberati A, Altman DG, Tetzlaf J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 2009;6:e1000100. doi:10.1371/journal.pmed.1000100
18. Caramelo F, Ferreira N, Oliveiros B. Estimation of risk factors for COVID-19 mortality-preliminary results. medRxiv [Preprint] 2020. doi:10.1101/2020.02.24.20027268
19. Lu J, Hu S, Fan R, et al. ACP risk grade: a simple mortality index for patients with confrmed or suspected severe acute respiratory syndrome coronavirus 2 disease (COVID-19) during the early stage of outbreak in Wuhan, China. medRxiv [Preprint] 2020. doi:10.1101/2020.02.20.20025510
20. Qi X, Jiang Z, YU Q, et al. Machine learning-based CT radiomics model for predicting hospital stay in patients with pneumonia associated with SARS-CoV-2 infection: a multicenter study. medRxiv [Preprint] 2020. doi:10.1101/2020.02.29.20029603
21. Yan L, Zhang H-T, Xiao Y, et al. Prediction of criticality in patients with severe Covid-19 infection using three clinical features: a machine learning-based prognostic model with clinical data in Wuhan. medRxiv [Preprint] 2020. doi:10.1101/2020.02.27.20028027
22. Yuan M, Yin W, Tao Z, Tan W, Hu Y. Association of radiologic fndings with mortality of patients infected with 2019 novel coronavirus in Wuhan, China. PLoS One 2020;15:e0230548. doi:10.1371/journal. pone.0230548
23. Song Y, Zheng S, Li L, et al. Deep learning enables accurate diagnosis of novel coronavirus (covid-19) with CT images. medRxiv [Preprint] 2020. doi:10.1101/2020.02.23.20026930
24. Yu H, Shao J, Guo Y, et al. Data-driven discovery of clinical routes for severity detection in covid-19 pediatric cases. medRxiv [Preprint] 2020. doi:10.1101/2020.03.09.20032219
25. Gozes O, Frid-Adar M, Greenspan H, et al. Rapid AI development cycle for the coronavirus (covid-19) pandemic: initial results for automated detection & patient monitoring using deep learning CT image analysis. arXiv e-prints [Preprint] 2020. https://ui.adsabs.harvard. edu/abs/2020arXiv200305037G
26. Chen J, Wu L, Zhang J, et al. Deep learning-based model for detecting 2019 novel coronavirus pneumonia on high-resolution computed tomography: a prospective study. medRxiv [Preprint] 2020. doi:10.1101/2020.02.25.20021568
27. Xu X, Jiang X, Ma C, et al. Deep learning system to screen coronavirus disease 2019 pneumonia. arXiv e-prints [Preprint] 2020. https://ui.adsabs.harvard.edu/abs/2020arXiv200209334X
28. Shan F, Gao Y, Wang J, et al. Lung infection quantifcation of covid-19 in CT images with deep learning. arXiv e-prints 2020. https://ui.adsabs.harvard.edu/abs/2020arXiv200304655S
29. Wang S, Kang B, Ma J, et al. A deep learning algorithm using CT images to screen for corona virus disease (covid-19). medRxiv [Preprint] 2020. doi:10.1101/2020.02.14.20023028
30. Song C-Y, Xu J, He J-Q, et al. COVID-19 early warning score: a multi-parameter screening tool to identify highly suspected patients. medRxiv [Preprint] 2020. doi:10.1101/2020.03.05.20031906
31. Barstugan M, Ozkaya U, Ozturk S. Coronavirus (COVID-19) classifcation using CT images by machine learning methods. arXiv e-prints [Preprint] 2020. https://ui.adsabs.harvard.edu/abs/2020arXiv200309424B
32. Gong J, Ou J, Qiu X, et al. A tool to early predict severe 2019-novel coronavirus pneumonia (covid-19): a multicenter study using the risk nomogram in Wuhan and Guangdong, China. medRxiv [Preprint] 2020. doi:10.1101/2020.03.17.20037515
33. Jin S, Wang B, Xu H, et al. AI-assisted CT imaging analysis for COVID-19 screening: building and deploying a medical AI system in four weeks. medRxiv [Preprint] 2020. doi:10.1101/2020.03.19.20039354
34. Li L, Qin L, Xu Z, et al. Artifcial intelligence distinguishes covid-19 from community acquired pneumonia on chest CT. Radiology 2020:200905. doi:10.1148/radiol.2020200905
35. Lopez-Rincon A, Tonda A, Mendoza-Maldonado L, et al. Accurate identifcation of SARS-CoV-2 from viral genome sequences using deep learning. bioRxiv [Preprint] 2020. doi:10.1101/2020.03.13.990242
36. Shi F, Xia L, Shan F, et al. Large-scale screening of covid-19 from community acquired pneumonia using infection size-aware classifcation. arXiv e-prints [Preprint] 2020. https://ui.adsabs. harvard.edu/abs/2020arXiv200309860S
37. Shi Y, Yu X, Zhao H, Wang H, Zhao R, Sheng J. Host susceptibility to severe COVID-19 and establishment of a host risk score: fndings of 487 cases outside Wuhan. Crit Care 2020;24:108. doi:10.1186/s13054-020-2833-7
38. Zheng C, Deng X, Fu Q, et al. Deep learning-based detection for covid-19 from chest CT using weak label. medRxiv [Preprint] 2020. doi:10.1101/2020.03.12.20027185
39. ClosedLoop.ai. Covid-19 vulnerability index (CV19 index) 2020. https://closedloop.ai/cv19index/.
40. Chinese PLA General Hospital. Suspected covid-19 pneumonia diagnosis aid system 2020. https://intensivecare.shinyapps.io/COVID19/.
41. Renmin Hospital of Wuhan University & Wuhan EndoAngel Medical Technology Co. AI diagnostic system for 2019-nCoV 2020. http://121.40.75.149/znyx-ncov/index.
42. National Supercomputing Center of Tianjin. Peunomnia CT 2020. https://ai.nscc-tj.cn/thai/deploy/public/pneumonia_ct.
43. Sun Yat-sen University. Discriminating covid-19 pneumonia from CT images 2020. http://biomed.nscc-gz.cn/server/Ncov2019.
44. Riley RD, Ensor J, Snell KIE, et al. Calculating the sample size required for developing a clinical prediction model. BMJ 2020;368:m441. doi:10.1136/bmj.m441
45. Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW, Topic Group 'Evaluating diagnostic tests and prediction models' of the STRATOS initiative. Calibration: the Achilles heel of predictive analytics. BMC Med 2019;17:230. doi:10.1186/s12916-019-1466-7
46. Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. Springer US, 2009. doi:10.1007/978-0-387-77244-8
47. Austin PC, Lee DS, Fine J P. Introduction to the analysis of survival data in the presence of competing risks. Circulation 2016;133:601-9. doi:10.1161/CIRCULATIONAHA.115.017719
48. Riley RD, Ensor J, Snell KI, et al. External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: opportunities and challenges [correction: BMJ 2019;365:l4379]. BMJ 2016;353:i3140. doi:10.1136/bmj.i3140
49. Debray TP, Riley RD, Rovers MM, Reitsma JB, Moons KG, Cochrane IPD Meta-analysis Methods group. Individual participant data (IPD) meta-analyses of diagnostic and prognostic modeling studies: guidance on their use. PLoS Med 2015;12:e1001886. doi:10.1371/journal. pmed.1001886
50. Steyerberg EW, Harrell FE Jr. Prediction models need appropriate internal, internal-external, and external validation. J Clin Epidemiol 2016;69:245-7. doi:10.1016/j.jclinepi.2015.04.005
51. Wynants L, Kent DM, Timmerman D, Lundquist CM, Van Calster B. Untapped potential of multicenter studies: a review of cardiovascular risk prediction models revealed inappropriate analyses and wide variation in reporting. Diagn Progn Res 2019;3:6. doi:10.1186/s41512-019-0046-9
52. Wynants L, Riley RD, Timmerman D, Van Calster B. Random-efects meta-analysis of the clinical utility of tests and prediction models. Stat Med 2018;37:2034-52. doi:10.1002/sim.7653
53. Zhou F, Yu T, Du R, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet 2020;395:1054-62. doi:10.1016/S0140-6736(20)30566-3
54. Li K, Wu J, Wu F, et al. The clinical and chest CT features associated with severe and critical covid-19 pneumonia. Invest Radiol 2020. doi:10.1097/RLI.0000000000000672
55. Li B, Yang J, Zhao F, et al. Prevalence and impact of cardiovascular metabolic diseases on COVID-19 in China. Clin Res Cardiol 2020. doi:10.1007/s00392-020-01626-9
56. Jain V, Yuan J-M. Systematic review and meta-analysis of predictive symptoms and comorbidities for severe COVID-19 infection. medRxiv [Preprint] 2020. doi:10.1101/2020.03.15.20035360
57. Rodriguez-Morales AJ, Cardona-Ospina JA, Gutiérrez-Ocampo E, et al, Latin American Network of Coronavirus Disease 2019-COVID-19 Research (LANCOVID-19). Electronic address: https://www. lancovid.org. Clinical, laboratory and imaging features of COVID-19: A systematic review and meta-analysis. Travel Med Infect Dis 2020:101623. doi:10.1016/j.tmaid.2020.101623
58. Lippi G, Plebani M, Henry BM. Thrombocytopenia is associated with severe coronavirus disease 2019 (COVID-19) infections: a meta-analysis. Clin Chim Acta 2020;506:145-8. doi:10.1016/j. cca.2020.03.022
59. Zhao X, Zhang B, Li P, et al. Incidence, clinical characteristics and prognostic factor of patients with covid-19: a systematic review and meta-analysis. medRxiv [Preprint] 2020. doi:10.1101/2020.03.17.20037572
60. Johansson MA, Saderi D. Open peer-review platform for COVID-19 preprints. Nature 2020;579:29. doi:10.1038/d41586-020-00613-4
61. Xu B, Kraemer MU, Gutierrez B, et al. Open access epidemiological data from the COVID-19 outbreak. Lancet Infect Dis 2020. doi:10.1016/s1473-3099(20)30119-5
62. Società Italiana di Radiologia Medica e Interventistica. COVID-19 database 2020. https://www.sirm.org/category/senza-categoria/covid-19/.
63. Kaggle. COVID-19 Kaggle community contributions 2020. https://www.kaggle.com/covid-19-contributions.
64. Cohen JP, Morrison P, Dao L. COVID-19 image data collection. arXiv [Preprint] 2020. doi:2003.11597, https://github.com/ieee8023/covid-chestxray-dataset.
65. Dutch CardioVascular Alliance. European registry of patients with covid-19 including cardiovascular risk and complications 2020. https://capacity-covid.eu/.
66. World Health Organization. Coronavirus disease (COVID-19) technical guidance: early investigations protocols 2020. https://www.who.int/emergencies/diseases/novel-coronavirus-2019/technical-guidance/early-investigations.
67. Infervision. Infervision launches hashtag#AI-based hashtag#Covid-19 solution in Europe 2020. https://www. linkedin.com/posts/infervision_ai-covid-medicine-activity-6650772755031613440-TqLJ.
68. Surgisphere Corporation. COVID-19 response center 2020. https://surgisphere.com/covid-19-response-center/.
69. Enfeld K, Miller R, Rice T, et al. Limited utility of SOFA and APACHE II prediction models for ICU triage in pandemic Influenza. Chest 2011;140:913A. doi:10.1378/chest.1118087
70. Van Calster B, Vickers AJ. Calibration of risk prediction models: impact on decision-analytic performance. Med Decis Making 2015;35:162-9. doi:10.1177/0272989X14547233
71. van Smeden M, Moons KG, de Groot JA, et al. Sample size for binary logistic prediction models: beyond events per variable criteria. Stat Methods Med Res 2019;28:2455-74. doi:10.1177/0962280218784726