Quantitative imaging biomarkers: A review of statistical methods for technical performance assessment

Statistical Methods in Medical Research

Volumn 24, Issue 1, 2015, Pages 27-67

  Raunig, David L ^a   McShane, Lisa M ^b   Pennello, Gene ^c   Gatsonis, Constantine ^d   Carson, Paul L ^e   Voyvodic, James T ^f   Wahl, Richard L ^g   Kurland, Brenda F ^h   Schwarz, Adam J ⁱ   Gönen, Mithat ^j   Zahlmann, Gudrun ^k   Kondratovich, Marina V ^c   O'Donnell, Kevin ^l   Petrick, Nicholas ^c   Cole, Patricia E ^m   Garra, Brian ^c   Sullivan, Daniel C ⁿ  

^a ICON Medical Imaging   (United States)

^b NATIONAL CANCER INSTITUTE   (United States)

^c CENTER FOR DRUG EVALUATION AND RESEARCH   (United States)

^d BROWN UNIVERSITY   (United States)

^e UNIVERSITY OF MICHIGAN HEALTH SYSTEM   (United States)

^f DUKE UNIVERSITY   (United States)

^g DEPARTMENT OF PATHOLOGY   (United States)

^h UNIVERSITY OF PITTSBURGH   (United States)

ⁱ ELI LILLY AND COMPANY   (United States)

^j MEMORIAL SLOAN KETTERING CANCER CENTER   (United States)

^k F HOFFMANN LA ROCHE LTD   (Switzerland)

^l Toshiba Medical Research Institute   (United States)

^m Takaeda ^*  (United States)

ⁿ DUKE UNIVERSITY SCHOOL OF MEDICINE   (United States)

more..

Author keywords

agreement; bias; imaging biomarkers; linearity; precision; quantitative imaging; reliability; repeatability; reproducibility

Indexed keywords

BIOLOGICAL MARKER;

CORRELATION COEFFICIENT; DIAGNOSTIC IMAGING; HUMAN; MEASUREMENT REPEATABILITY; METRIC SYSTEM; PROBABILITY; QUANTITATIVE DIAGNOSIS; REFERENCE VALUE; REPRODUCIBILITY; REVIEW; SAMPLE SIZE; STATISTICAL ANALYSIS; STATISTICAL MODEL; STUDY DESIGN; CLINICAL TRIAL (TOPIC); METHODOLOGY; NOMENCLATURE; STATISTICAL BIAS; STATISTICS;

BIAS (EPIDEMIOLOGY); BIOMARKERS; CLINICAL TRIALS AS TOPIC; DIAGNOSTIC IMAGING; HUMANS; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN; STATISTICS AS TOPIC; TERMINOLOGY AS TOPIC;

EID: 84988215602     PISSN: 09622802     EISSN: 14770334     Source Type: Journal    
DOI: 10.1177/0962280214537344     Document Type: Review

References (62)

1. Mueller DK, Kutscherenko A, Bartel H, et al. Phantom-less QCT BMD system as screening tool for osteoporosis without additional radiation. Eur J Radiol. 2011 ; 79: 375-381
2. Habte F, Budhiraja S, Keren S, et al. In situ study of the impact of inter-and intra-reader variability on region of interest (ROI) analysis in preclinical molecular imaging. Am J Nucl Med Mol Imaging. 2013 ; 3: 175-175
3. Bhavane R, Badea C, Ghaghada KB, et al. Dual-energy computed tomography imaging of atherosclerotic plaques in a mouse model using a liposomal-iodine nanoparticle contrast agent. Circ Cardiovasc Imaging. 2013 ; 6: 285-294
4. Hamburg MA, Collins FS. The path to personalized medicine. N Engl J Med. 2010 ; 363: 301-304
5. Atkinson AJ, Colburn WA, DeGruttola VG, et al. Biomarkers and surrogate endpoints: Preferred definitions and conceptual framework. Clin Pharmacol Therapeut. 2001 ; 69: 89-95
6. Atkinson AJ, Colburn WA, DeGruttola VG, et al. Biomarkers and surrogate endpoints: Preferred definitions and conceptual framework∗. Clin Pharmacol Therapeut. 2001 ; 69: 89-95
7. Kessler LG, Barnhart HX, Buckler AJ, et al. The emerging science of quantitative imaging biomarkers terminology and definitions for scientific studies and regulatory submissions. Stat Methods Med Res 2015; 24: 9-26.
8. Obuchowski NA, Reeves AP, Huang EP, et al. Quantitative imaging biomarkers: A review of statistical methods for computer algorithm comparisons. Stat Methods Med Res. 2015 ; 24: 68-106
9. Sullivan DBL, Linda B, Seto B, et al. Introduction to metrology series. Stat Methods Med Res. 2015 ; 24: 3-8
10. McShane L. Meta-analysis of the technical performance of an imaging procedure: Guidelines and statistical methodology. Stat Methods Med Res. 2015 ; 24: 141-174
11. http://qibawiki.rsna.org/index.php?title=Main-Page/. QIBA Main Page (accessed 21 December 2013).
12. ISO/IEC. International vocabulary of metrology-Basic and general concepts and associated terms (VIM). ISO/IEC Guide 99. Geneva: International Organization for Standardization, 2007.
13. http://qibawiki.rsna.org/index.php?title=What-Are-Profiles-and-Protocols%3F. What are Profiles and Protocols (accessed 21 December 2013).
14. http://qibawiki.rsna.org/index.php?title=Main-Page. QIBA Wiki Main Page. (accessed 21 December 2013).
15. Barnhart HX, Haber MJ, Lin LI. An overview on assessing agreement with continuous measurements. J Biopharm Statist. 2007 ; 17: 529-569
16. Kenward MG, Roger JH. Small sample inference for fixed effects from restricted maximum likelihood. Biometrics. 1997 ; 53 (3). 983-997
17. QI-Bench, 10 October 2013.
18. Natrella M. NIST/SEMATECH e-handbook of statistical methods. NIST/SEMATECH, 2010, http://www itl nist gov/div898/handbook.
19. Evaluation of the linearity of quantitative measurement procedures: A statistical approach; approved guideline. CLSI/NCCLS document. Wayne, PA: NCCLS ; 2010 :
20. Czichos H, Saito T, Smith LE Springer handbook of metrology and testing. Berlin Heidelberg: Springer-Verlag ; 2011 :
21. ISO. Statistical interpretation of data-Part 4: Detection and treatment of outliers ISO 16269-4. 2010.
22. (2002) C. Evaluation of precision performance of quantitative measurement methods; Approved guideline-second edition. CLSI document. EP5-A2. 2002.
23. CLSI. Measurement procedure comparison and bias estimation using patient samples: Approved guideline-third edition. CLSI Document EP09-A3. Wayne, PA: Clinical and Laboratory Standards Institute, 2013.
24. Penny KI, Jolliffe IT. A comparison of multivariate outlier detection methods for clinical laboratory safety data. J R Stat Soc Ser D (Statist). 2001 ; 50: 295-307
25. Rosner B. Percentage points for a generalized ESD many-outlier procedure. Technometrics. 1983 ; 25: 165-172
26. Snedecor G, Cochran WG Statistical methods. Iowa: Ames ; 1989 :
27. Montgomery DC, Peck EA, Vining GG Introduction to linear regression analysis. Hoboken, NJ: Wiley ; 2012 :
28. Levene H Contributions to probability and statistics: Essays in honor of Harold hotelling. Olkin I, ed. Redwood City, CA: Stanford University Press ; 1960 :
29. Carroll RJ, Ruppert D Transformation and weighting in regression. New York, NY: Chapman & Hall A International Thomson Publishing Company ; 1988 :
30. Barnhart HX, Barboriak DP. Applications of the repeatability of quantitative imaging biomarkers: A review of statistical analysis of repeat data sets. Transl Oncol. 2009 ; 2: 231-231
31. http://qibawiki.rsna.org/index.php?title=VolCT---Group-3A. QIBA 3A study. (accessed 21 December 2013).
32. Berkson J. Are there two regressions?. J Am Stat Assoc. 1950 ; 45: 164-180
33. Haeckel R, Wosniok W, Klauke R. Comparison of ordinary linear regression, orthogonal regression, standardized principal component analysis, Deming and Passing-Bablok approach for method validation in laboratory medicine. Laboratoriumsmedizin. 2013 ; 37: 147-163
34. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986 ; 327: 307-310
35. Demonstrator Q-BQ. s-rdg-Test-retest-reliability-study.csv. 2 January 2014.
36. Neter J, Kutner MH, Nachtsheim CJ, et al. Applied linear regression models. Chicago, USA: Irwin, 1996, p. 1050.
37. Fitzmaurice GM, Laird NM, Ware JH Applied longitudinal analysis. Hoboken, NJ: Wiley ; 2012 :
38. Streiner DL. Learning how to differ: Agreement and reliability statistics in psychiatry. Can J Psychiatry. 1995 ; 40: 60-66
39. Shrout PE, Fleiss JL. Intraclass correlations: Uses in assessing rater reliability. Psychol Bull. 1979 ; 86: 420-428
40. Chen C-C, Barnhart HX. Comparison of ICC and CCC for assessing agreement for data without and with replications. Comput Statist Data Anal. 2008 ; 53: 554-564
41. Johnson NL, Kotz S and Balakrishnan N. Continuous univariate distributions, vol. 1. Hoboken, NJ: Wiley & Sons, 1994.
42. Vangel MG. Confidence intervals for a normal coefficient of variation. Am Statist. 1996 ; 50: 21-26
43. I. Accuracy (trueness and precision) of measurement methods and results. Part 1: General principles and definitions. ISO 5725-1: Cor 1:1998. Geneva, 1994.
44. I. Accuracy (trueness and precision) of measurement methods and results-Part 2: Basic method for the determination of repeatability and reproducibility of a standard measurement method. ISO 5725-2 1994.
45. Jobling D, Snell EJ. The use of the coefficient of reproducibility in attitude scaling. Incorp Statist. 1961 ; 11: 110-118
46. (BIPM). BIdPeM. International vocabulary of metrology-Basic and general concepts and associated terms (VIM, 3rd edition, JCGM 200:2008) and Corrigendum (May 2010).
47. Kimothi SK, Kimothi S The uncertainty of measurements: Physical and chemical metrology: Impact and analysis. ASQ Press, Milwaukee, WI ; 2002 :
48. R D. Vocabulary for use in measurement procedures and description of reference materials in laboratory medicine. Eur J Clin Chem Clin Biochem. 1997 ; 35: 141-173
49. Statistics-Vocabulary and symbols-Part 2: Applied statistics. ISO-3534-2-2006. 2006 :
50. Linnet K, Boyd JC Tietz textbook of clinical chemistry and molecular diagnostics. 4 th ed. St Louis: Elsevier Saunders, 2006, pp. 7-47. Burtis CA Ashwood ER Bruns DE, ed. St Louis: Elsevier Saunders ; 2006: 7-47.
51. Graybill FA, Wang C-M. Confidence intervals on nonnegative linear combinations of variances. J Am Statist Assoc. 1980 ; 75: 869-873
52. Searle S, Casella G, McCulloch C Variance components. New York: John Wiley and Sons ; 1992 :
53. Milliken GA, Johnson DE Analysis of messy data volume 1: Designed experiments. Chapman and Hall/CRC, Boca Raton, FL ; 2009 :
54. Burdick R, Graybill F Confidence intervals on variance components. New York: Marcel Dekker Inc ; 1992 :
55. Lawrence I, Lin K. A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989 ; 45: 255-268
56. Graybill FA An introduction to linear statistical models. New York: McGraw-Hill ; 1961 :
57. Draper NP, Smith H Applied regression analysis. 1998 :
58. Crawford SB, Kosinski AS, Lin H-M, et al. Computer programs for the concordance correlation coefficient. Comput Methods Prog Biomed. 2007 ; 88: 62-74
59. Jacene HA, Leboulleux S, Baba S, et al. Assessment of interobserver reproducibility in quantitative 18F-FDG PET and CT measurements of tumor response to therapy. J Nucl Med. 2009 ; 50: 1760-1769
60. Velasquez LM, Boellaard R, Kollia G, et al. Repeatability of 18F-FDG PET in a multicenter phase I study of patients with advanced gastrointestinal malignancies. J Nucl Med. 2009 ; 50: 1646-1654
61. Kottner J, Audigé L, Brorson S, et al. Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. Int J Nurs Stud. 2011 ; 48: 661-671
62. Rudd JH, Myers KS, Bansilal S, et al. Atherosclerosis inflammation imaging with 18F-FDG PET: Carotid, iliac, and femoral uptake reproducibility, quantification methods, and recommendations. J Nucl Med. 2008 ; 49: 871-878