GIST 2.0: A scalable multi-trait metric for quantifying population representativeness of individual clinical studies

Journal of Biomedical Informatics

Volumn 63, Issue , 2016, Pages 325-336

  Sen, Anando ^a   Chakrabarti, Shreya ^a   Goldstein, Andrew ^a,b   Wang, Shuang ^a   Ryan, Patrick B ^a,c   Weng, Chunhua ^a  

^a Columbia University ^*  (United States)

^b NEW YORK UNIVERSITY   (United States)

^c JANSSEN RESEARCH AND DEVELOPMENT   (United States)

Author keywords

Clinical trials; Eligibility criteria; Generalizability; Population representativeness; Trait dependencies

Indexed keywords

COMPUTER APPLICATIONS;

CLINICAL TRIAL; ELIGIBILITY CRITERION; GENERALIZABILITY; POPULATION REPRESENTATIVENESS; TRAIT DEPENDENCIES;

ITERATIVE METHODS;

ALGORITHM; ANALYTICAL PARAMETERS; ARTICLE; EMPIRICAL RESEARCH; GENERALIZABILITY INDEX FOR STUDY TRAIT 2.0; HUMAN; IRON DEFICIENCY ANEMIA; NON INSULIN DEPENDENT DIABETES MELLITUS; PATIENT SELECTION; POPULATION REPRESENTATIVENESS; PRIORITY JOURNAL; QUANTITATIVE ANALYSIS; RANDOMIZED CONTROLLED TRIAL (TOPIC); SCORING SYSTEM; STUDY DESIGN; THEORETICAL STUDY; TRAIT SELECTION; TRAIT SUMMARIZING TECHNIQUE; TRIAL SETTING; VALIDATION PROCESS; MEDICAL RESEARCH; SOFTWARE;

BIOMEDICAL RESEARCH; HUMANS; PATIENT SELECTION; RANDOMIZED CONTROLLED TRIALS AS TOPIC; SOFTWARE;

EID: 84987985196     PISSN: 15320464     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jbi.2016.09.003     Document Type: Article

References (39)

1. [1] Green, L.W., Glasgow, R.E., Evaluating the relevance, generalization, and applicability of research issues in external validation and translation methodology. Eval. Health Professions 29:1 (2006), 126–153.
2. [2] Kukull, W.A., Generalizability the trees, the forest, and the low-hanging fruit. Neurology 78:23 (2012), 1886–1891.
3. [3] Dubey, A., What researchers mean by generalizability. At Work, 45, 2006, 2.
4. [4] Weng, C., Tu, S., Sim, I., Richesson, R., Formal representation of eligibility criteria: a literature review. J. Biomed. Inform. 43:3 (2010), 451–467.
5. [5] Schoenmaker, N., Gool, W.A.V., The age gap between patients in clinical studies and in the general population: a pitfall for dementia research. Lancet Neurol. 3 (2004), 627–630.
6. [6] Masoudi, F.A., Havranek, E.P., Wolfe, P., Gross, C.P., Rathore, S.S., Steiner, J.F., Ordin, D.L., Krumholz, H.M., Most hospitalized older persons do not meet the enrollment criteria for clinical trials in heart failure. Am. Heart J. 146 (2003), 250–257.
7. [7] Ma, H., Weng, C., Identification of questionable exclusion criteria in mental disorder clinical trials using a medical encyclopedia. Pacific Symp. Biocomput. 21 (2016), 219–230.
8. [8] Musen, M., Rohn, J., Fagan, L., Shortliffe, E., Knowledge engineering for a clinical trial advice system: Uncovering errors in protocol specification. Bull. Cancer 74:3 (1987), 291–296.
9. [9] J. Ross, S. Tu, S. Carini, I. Sim, Analysis of eligibility criteria complexity in clinical trials, in: AMIA Summits Translational Science Proceedings, 2010, pp. 46–50.
10. [10] Hao, T., Rusanov, A., Boland, M., Weng, C., Clustering clinical trials with similar eligibility criteria features. J. Biomed. Inform. 52 (2014), 112–120.
11. [11] Rubin, D., Gennari, J., Musen, M., Knowledge representation and tool support for critiquing clinical trial protocols. Proceedings of AMIA Annual Symposium, 2000, 724–728.
12. [12] A.L. Buchanan, M.G. Hudgens, S.R. Cole, K. Mollan, P.E. Sax, E. Daar, A.A. Adimora, J. Eron, M. Mugavero, Generalizing evidence from randomized trials using inverse probability of sampling weights, Tech. rep., 2015.
13. [13] Weng, C., Optimizing clinical research participant selection with informatics. Trends Pharmacol. Sci. 36:11 (2015), 706–709.
14. [14] He, Z., Ryan, P., Hoxha, J., Wang, S., Carini, S., Sim, I., Weng, C., Multivariate analysis of the population representativeness of related clinical studies. J. Biomed. Inform. 60 (2016), 66–76.
15. [15] Kennedy-Martin, T., Curtis, S., Faries, D., Robinson, S., Johnston, J., A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results. Trials 16:1 (2015), 1–14.
16. [16] Bress, A.P., Tanner, R.M., Hess, MPH.R., Colantonio, L.D., Shimbo, D., Muntner, P., Generalizability of results from the systolic blood pressure intervention trial (SPRINT) to the us adult population. J. Am. College Cardiol. 67:5 (2016), 463–472.
17. [17] Pressler, T.R., Kaizar, E.E., The use of propensity scores and observational data to estimate randomized controlled trial generalizability bias. Stat. Med. 32 (2013), 3552–3568.
18. [18] Greenhouse, J.B., Kaizar, E.E., Kelleher, K., Seltman, H., Gardner, W., Generalizing from clinical trial data: a case study. the risk of suicidality among pediatric antidepressant users. Stat. Med. 27 (2008), 1801–1813.
19. [19] Bleeker, S.E., Moll, H.A., Steyerberg, E.W., Donders, A.R., Lubsen-Derksen, G., Grobbee, D.E., Moons, K.G., External validation is necessary in prediction research: a clinical example. J. Clin. Epidemiol. 56:9 (2003), 826–832.
20. [20] Wang, T.S., Hellkamp, A.S., Patel, C.B., Ezekowitz, J.A., Fonarow, G.C., Hernandez, A.F., Representativeness of relax-ahf clinical trial population in acute heart failure. Circ.: Cardiovasc. Qual. Outcomes 7:2 (2014), 259–268.
21. [21] Cole, S.R., Stuart, E.A., Generalizing evidence from randomized clinical trials to target populations the actg 320 trial. Am. J. Epidemiol. 172:1 (2010), 107–115.
22. [22] Weng, C., Li, Y., Ryan, P., Zhang, Y., Liu, F., Gao, J., Bigger, J., Hripcsak, G., A distribution-based method for assessing the differences between clinical trial target populations and patient populations in electronic health records. Appl. Clin. Inform. 5:2 (2014), 463–479.
23. [23] He, Z., Chander, P., Ryan, P., Weng, C., Simulation-based evaluation of the generalizability index for study traits. AMIA Annual Symposium Proceedings, 2015, 594–602.
24. [24] “External validity.” < https://en.wikipedia.org/wiki/External_validity> (accessed: 2016-02-16).
25. [25] Saeed, M., Villarroel, M., Reisner, A.T., Clifford, G., Lehman, L.-W., Moody, G., Heldt, T., Kyaw, T.H., Moody, B., Mark, R.G., Multiparameter intelligent monitoring in intensive care II (MIMIC-II): a public-access intensive care unit database. Crit. Care Med. 39:5 (2011), 952–960.
26. [26] Weiskopf, N.G., Hripcsak, G., Swaminathan, S., Weng, C., Defining and measuring completeness of electronic health records for secondary use. J. Biomed. Inform. 46 (2013), 830–836.
27. [27] Hao, T., Liu, H., Weng, C., Valx: a system for extracting and structuring numeric lab test comparison statements from text. Methods Inform. Med. 55:3 (2016), 266–275.
28. [28] Goddard, A.F., McIntyre, A., Scott, B.B., Guidelines for the management of iron deficiency anaemia. Gut 46:suppl. 4 (2000), iv1–iv5.
29. [29] Johnson, S.B., Generic data modeling for clinical repositories. J. Am. Med. Inform. Assoc., 3(5), 1996, 328.
30. [30] Fort, D., Weng, C., Bakken, S., Wilcox, A.B., Considerations for using research data to verify clinical data accuracy. AMIA Summits on Translational Science Proceedings, vol. 201, 2014, 211.
31. [31] Paulson, M., Weng, C., Desiderata for major eligibility criteria in breast cancer trials. AMIA Annual Symposium Proceedings, 2015, 2025–2034.
32. [32] Tukey, J.W., Comparing individual means in the analysis of variance. Biometrics 5:2 (1949), 99–114.
33. [33] Friedman, M., The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J. Am. Stat. Assoc. 32:200 (1937), 675–701.
34. [34] Sen, A., Ryan, P.B., Goldstein, A., Chakrabarti, S., Wang, S., Koski, E., Weng, C., Correlating eligibility criteria generalizability and adverse events using Big Data for patients and clinical trials. Ann. N. Y. Acad. Sci.(September), 2016, 10.1111/nyas.13195.
35. [35] Huang, L.-W., Inouye, S.K., Jones, R.N., Fong, T.G., Rudolph, J.L., O'Connor, M.G., Metzger, E.D., Crane, P.K., Marcantonio, E.R., Identifying indicators of important diagnostic features of delirium. J. Am. Geriatr. Soc. 60:6 (2012), 1044–1050.
36. [36] Weiskopf, N., Rusanov, A., Weng, C., Sick patients have more data: the non-random completeness of electronic health records. AMIA Annual Symposium Proceedings, 2013, 1472–1477.
37. [37] Ren, L., Zhu, D., Wei, Y., Pan, X., Liang, L., Xu, J., Zhong, Y., Xue, Z., Jin, L., Zhan, S., Enhanced recovery after surgery (eras) program attenuates stress and accelerates recovery in patients after radical resection for colorectal cancer: a prospective randomized controlled trial. World J. Surgery 36:2 (2012), 407–414.
38. [38] Vira, T., Colquhoun, M., Etchells, E., Reconcilable differences: correcting medication errors at hospital admission and discharge. Qual. Safety Health Care 15:2 (2006), 122–126.
39. [39] Idorn, T., Knop, F.K., Jørgensen, M.B., Jensen, T., Resuli, M., Hansen, P.M., Christensen, K.B., Holst, J.J., Hornum, M., Feldt-Rasmussen, B., Safety and efficacy of liraglutide in patients with type 2 diabetes and end-stage renal disease: an investigator-initiated, placebo-controlled, double-blind, parallel-group, randomized trial. Diabetes Care 39:2 (2016), 206–213.