Waste, leaks, and failures in the biomarker pipeline

Clinical Chemistry

Volumn 63, Issue 5, 2017, Pages 963-972

  Ioannidis, John P A ^a   Bossuyt, Patrick M M ^b  

^a STANFORD UNIVERSITY   (United States)

^b UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

BIOLOGICAL MARKER;

CLINICAL CHEMISTRY; HUMAN; REVIEW; CLINICAL STUDY; MEDICAL RESEARCH; STANDARDS; TRENDS;

BIOMARKERS; BIOMEDICAL RESEARCH; CLINICAL STUDIES AS TOPIC; HUMANS;

EID: 85018364165     PISSN: 00099147     EISSN: 15308561     Source Type: Journal    
DOI: 10.1373/clinchem.2016.254649     Document Type: Review

References (81)

1. Annesley T, Diamandis E, Bachmann L, Hanash S, Hart B, Javahery R, et al. A spectrum of views on clinical mass spectrometry. Clin Chem 2016;62: 30-6.
2. Tzoulaki I, Ebbels TM, Valdes A, Elliott P, Ioannidis JP. Design and analysis of metabolomics studies in epidemiologic research: A primer on -omic technologies. Am J Epidemiol 2014;180:129 -39.
3. Nair VS, Pritchard CC, Tewari M, Ioannidis JP. Design and analysis for studying microRNAs in human disease: A primer on -omic technologies. Am J Epidemiol 2014; 180:140 -52.
4. Phillip Law W, Miles KA. Incorporating prognostic imaging biomarkers into clinical practice. Cancer Imaging 2013;13:332- 41.
5. Crutchfield CA, Thomas SN, Sokoll LJ, Chan DW. Advances in mass spectrometry-based clinical biomarker discovery. Clin Proteomics 2016;13:1.
6. Diamandis EP. The failure of protein cancer biomarkers to reach the clinic: Why, and what can be done to address the problem? BMC Med 2012;10:87.
7. Sturgeon C, Hill R, Hortin GL, Thompson D. Taking a new biomarker into routine use -A perspective from the routine clinical biochemistry laboratory. Proteomics Clin Appl 2010;4:892-903.
8. Malottki K, Biswas M, Deeks JJ, Riley RD, Craddock C, Johnson P, Billingham L. Stratified medicine in European Medicines Agency licensing: A systematic review of predictive biomarkers. BMJ Open 2014;4:e004188.
9. Makawita S, Diamandis EP. The bottleneck in the cancer biomarker pipeline and protein quantification through mass spectrometry-based approaches: current strategies for candidate verification. Clin Chem 2010;56: 212-22.
10. Hlatky MA, Greenland P, Arnett DK, Ballantyne CM, Criqui MH, Elkin MS, et al. Criteria for evaluation of novel markers of cardiovascular risk: A scientific statement from the American Heart Association. Circ Cardiovasc Qual Outcomes 2009;119:2408-16.
11. Ransohoff DF. Challenges and opportunities in evaluating diagnostic tests. J Clin Epidemiol 2002;55: 1178 - 82.
12. Ioannidis JP. Biomarker failures. Clin Chem 2013;59: 202- 4.
13. Bossuyt PM. Where are all the new omics-based tests? Clin Chem 2014;60:1256 -7.
14. Bossuyt PM. The thin line between hope and hype in biomarker research. JAMA 2011;305:2229-30.
15. Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet 2009; 374:86 -9.
16. Macleod MR, Michie S, Roberts I, Dirnagl U, Chalmers I, Ioannidis JP, et al. Biomedical research: increasing value, reducing waste. Lancet 2014;383:101- 4.
17. Begley CG, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature 2012;483: 531-3.
18. Ransohoff DF. Lessons from controversy: ovarian cancer screening and serum proteomics. J Natl Cancer Inst 2005;97:315-9.
19. Ransohoff DF. How to improve reliability and efficiency of research about molecular markers: Roles of phases, guidelines, and study design. J Clin Epidemiol 2007; 60:1205-19.
20. Rifai N, Altman DG, Bossuyt PM. Reporting bias in diagnostic and prognostic studies: Time for action. Clin Chem 2008;54:1101-3.
21. Hayden JA, Cote P, Bombardier C. Evaluation of the quality of prognosis studies in systematic reviews. Ann Intern Med 2006;144:427-37.
22. Kyzas PA, Denaxa-Kyza D, Ioannidis JP. Quality of reporting of cancer prognostic marker studies: Association with reported prognostic effect. J Natl Cancer Inst 2007;99: 236-43.
23. Mallett S, Timmer A, Sauerbrei W, Altman DG. Reporting of prognostic studies of tumour markers: A review of published articles in relation to REMARK guidelines. Br J Cancer 2010;102:173-80.
24. Sigounas DE, Tatsioni A, Christodoulou DK, Tsianos EV, Ioannidis JP. New prognostic markers for outcome of acute pancreatitis: overview of reporting in 184 studies. Pancreas 2011;40:522-32.
25. Pepe MS, Feng Z. Improving biomarker identification with better designs and reporting. Clin Chem 2011;57: 1093-5.
26. Pepe MS, Li CI, Feng Z. Improving the quality of biomarker discovery research: The right samples and enough of them. Cancer Epidemiol Biomarkers Prev 2015;24: 944-50.
27. Korevaar DA, Bossuyt PM, Hooft L. Infrequent and incomplete registration of test accuracy studies: Analysis of recent study reports. BMJ Open 2014;4:e004596.
28. Korevaar DA, Ochodo EA, Bossuyt PM, Hooft L. Publication and reporting of test accuracy studies registered in Clinical Trials.gov. Clin Chem 2014;60:651-9.
29. Korevaar DA, van Enst WA, Spijker R, Bossuyt PM, Hooft L. Reporting quality of diagnostic accuracy studies: A systematic review and meta-Analysis of investigations on adherence to STARD. Evid Based Med 2014;19: 47-54.
30. Ochodo EA, de Haan MC, Reitsma JB, Hooft L, Bossuyt PM, Leeflang MM. Overinterpretation and misreporting of diagnostic accuracy studies: Evidence of spin. Radiology 2013;267:581- 8.
31. Kyzas PA, Denaxa-Kyza D, Ioannidis JP. Almost all articles on cancer prognostic markers report statistically significant results. Eur J Cancer 2007;43: 2559-79.
32. Andre F, McShane LM, Michiels S, Ransohoff DF, Altman DG, Reis-Filho JS, et al. Biomarker studies: A call for a comprehensive biomarker study registry. Nat Rev Clin Oncol 2011;8:171-6.
33. Dal-Ré R, Ioannidis JP, Bracken MB, Buffler PA, Chan AW, Franco EL, et al. Making prospective registration of observational research a reality. Sci Transl Med 2014; 6: 2241.
34. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, et al. STARD 2015: An updated list of essential items for reporting diagnostic accuracy studies. Clin Chem 2015;61:1446 -52.
35. Altman DG, McShane LM, Sauerbrei W, Taube SE. Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK): Explanation and elaboration. PLoS Med 2012;9:e1001216.
36. Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): Explanation and elaboration. Ann Intern Med 2015;162:W1-W73.
37. Janssens AC, Ioannidis JP, Bedrosian S, Boffetta P, Dolan SM, Dowling N, et al. Strengthening the reporting of genetic risk prediction studies (GRIPS): Explanation and elaboration. Eur J Clin Invest 2011;41:1010-35.
38. Brazma A, Hingamp P, Quackenbush J, Sherlock G, Spellman P, Stoeckert C, et al. Minimum information about a microarray experiment (MIAME)-Toward standards for microarray data. Nat Genet 2001;29:365-71.
39. Ioannidis JP, Khoury MJ. Improving validation practices in "omics" research. Science 2011;334:1230 -2.
40. Chau CH, Rixe O, McLeod H, Figg WD. Validation of analytic methods for biomarkers used in drug development. Clin Cancer Res 2008;14:5967-76.
41. Rifai N, Gillette MA, Carr SA. Protein biomarker discovery and validation: The long and uncertain path to clinical utility. Nat Biotechnol 2006;24:971-83.
42. Riley RD, Burchill SA, Abrams KR, Heney D, Lambert PC, Jones DR, et al. A systematic review and evaluation of the use of tumour markers in paediatric oncology: Ewing's sarcoma and neuroblastoma. Health Technol Assess 2003;7:1-162.
43. Castaldi PJ, Dahabreh IJ, Ioannidis JP. An empirical assessment of validation practices for molecular classifiers. Brief Bioinform 2011;12:189 -202.
44. Riley RD, Hayden JA, Steyerberg EW, Moons KG, Abrams K, Kyzas PA, et al. Prognosis Research Strategy (PROGRESS) 2: prognostic factor research. PLoS Med 2013;10:e1001380.
45. Altman DG. Systematic reviews of evaluations of prognostic variables. BMJ 2001;323:224-8.
46. Hemingway H, Philipson P, Chen R, Fitzpatrick NK, Damant J, Shipley M, et al. Evaluating the quality of research into a single prognostic biomarker: A systematic review and meta-Analysis of 83 studies of C-reactive protein in stable coronary artery disease. PLoS Med 2010;7:e1000286.
47. Ioannidis JP, Panagiotou OA. Comparison of effect sizes associated with biomarkers reported in highly cited individual articles and in subsequent metaanalyses. JAMA 2011;305:2200-10.
48. Kyzas PA, Loizou KT, Ioannidis JP. Selective reporting biases in cancer prognostic factor studies. J Natl Cancer Inst 2005;97:1043-55.
49. Ransohoff DF. Bias as a threat to the validity of cancer molecular-marker research. Nat Rev Cancer 2005;52: 142-9.
50. Vickers AJ, Lilja H. Cutpoints in clinical chemistry: Time for fundamental reassessment. Clin Chem 2009;55: 15-7.
51. Assmann SF, Pocock SJ, Enos LE, Kasten LE. Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet 2000;355:1064-9.
52. Altman DG, Lausen B, Sauerbrei W, Schumacher M. Dangers of using "optimal" cutpoints in the evaluation of prognostic factors. J Natl Cancer Inst 1994;86:829-35.
53. Holländer N, Sauerbrei W, Schumacher M. Confidence intervals for the effect of a prognostic factor after selection of an 'optimal' cutpoint. Stat Med 2004;23:1701-13.
54. Dupuy A, Simon RM. Critical review of published microarray studies for cancer outcome and guidelines on statistical analysis and reporting. J Natl Cancer Inst 2007;99:147-57.
55. Kondo T. Inconvenient truth: cancer biomarker development by using proteomics. Biochim Biophys Acta 2014;1844:861-5.
56. Serghiou S, Kyriakopoulou A, Ioannidis JP. Long noncoding RNAs as novel predictors of survival in human cancer: A systematic review and meta-Analysis. Mol Cancer 2016;15:50.
57. Mischak H, Allmaier G, Apweiler R, Attwood T, Baumann M, Benigni A, et al. Recommendations for bio-marker identification and qualification in clinical proteomics. Sci Transl Med 2010;2:4642.
58. Alkhalaf A, Zü rbig P, Bakker SJ, Bilo HJ, Cerna M, Fischer C, et al. Multicentric validation of proteomic biomarkers in urine specific for diabetic nephropathy. PLoS One 2010;5:e13421.
59. Wichmann HE, Kuhn KA, Waldenberger M, Schmelcher D, Schuffenhauer S, Meitinger T, et al. Comprehensive catalog of European biobanks. Nat Biotechnol 2011; 29:795-7.
60. Bell AW, Deutsch EW, Au CE, Kearney RE, Beavis R, Sechi S, et al., and HUPO Test Sample Working Group. A HUPO test sample study reveals common problems in mass spectrometry-based proteomics. Nat Methods 2009;6:423-30.
61. Janssens AC, Gwinn M, Bradley LA, Oostra BA, van Duijn CM, Khoury MJ. A critical appraisal of the scientific basis of commercial genomic profiles used to assess health risks and personalize health interventions. Am J Hum Genet 2008;82:593-9.
62. Ioannidis JP. Integration of evidence from multiple meta-Analyses: A primer on umbrella reviews, treatment networks and multiple treatments meta-Analyses. CMAJ 2009;181:488-93.
63. Netherlands Care Institute. http://www.zorginstituutnederland.nl/binaries/content/documents/zinl-/documenten/publicaties/rapporten-enstandpunten/2011/1101-medische-Tests-beoordelingstand-van-de-wetenschap-en-praktijk/1101-medischetests-beoordeling-stand-van-de-wetenschap-en-pra (Accessed October 2016).
64. Ioannidis JP, Tzoulaki I. What makes a good predictor?: The evidence applied to coronary artery calcium score. JAMA 2010;303:1646 -7.
65. Siontis KC, Siontis GC, Contopoulos-Ioannidis DG, Ioannidis JP. Diagnostic tests often fail to lead to changes in patient outcomes. J Clin Epidemiol 2014;67:612-21.
66. Ferrante di Ruffano L, Davenport C, Eisinga A, Hyde C, Deeks JJ. A capture-recapture analysis demonstrated that randomized controlled trials evaluating the impact of diagnostic tests on patient outcomes are rare. J Clin Epidemiol 2012;65:282-7.
67. Ransohoff DF, Khoury MJ. Personal genomics: information can be harmful. Eur J Clin Invest 2010;40: 64-8.
68. Le Tourneau C, Delord JP, Goncalves A, Gavoille C, Dubot C, Isambert N, et al. Molecularly targeted therapy based on tumour molecular profiling versus conventional therapy for advanced cancer (SHIVA): A multicentre, open-label, proof-of-concept, randomised, controlled phase 2 trial. Lancet Oncol 2015;16:1324-34.
69. Bossuyt PM, Lijmer JG, Mol BW. Randomised comparisons of medical tests: sometimes invalid, not always efficient. Lancet 2000;356:1844 -7.
70. Merlin T, Lehman S, Hiller JE, Ryan P. The linked evidence approach to assess medical tests: A critical analysis. Int J Technol Assess Health Care 2013;29:343-50.
71. Riley RD, Abrams KR, Sutton AJ, Lambert PC, Jones DR, Heney D, Burchill SA. Reporting of prognostic markers: current problems and development of guidelines for evidence-based practice in the future. Br J Cancer 2003;88: 1191-8.
72. Chang CQ, Tingle SR, Filipski KK, Khoury MJ, Lam TK, Schully SD, Ioannidis JP. An overview of recommendations and translational milestones for genomic tests in cancer. Genet Med 2015;17:431-40.
73. Mischak H, Ioannidis JP, Argiles A, Mischak H, Ioannidis JP, Argiles A, et al. Implementation of proteomic biomarkers: making it work. Eur J Clin Invest 2012;42: 1027-36.
74. Horvath AR, Lord SJ, StJohn A, Sandberg S, Cobbaert CM, Lorenz S, et al. From biomarkers to medical tests: The changing landscape of test evaluation. Clin Chim Acta 2014;427: 49-57. Organizational author: Test Evaluation Working Group of the European Federation of Clinical Chemistry and Laboratory Medicine.
75. Prasad V, Cifu A, Ioannidis JP. Reversals of established medical practices: Evidence to abandon ship. JAMA 2012;307: 37-8.
76. Moyer VA. Screening for prostate cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med 2012;157:120-34. The author is on behalf of the U.S. Preventive Services Task Force.
77. Prasad V, Ioannidis JPA. Evidence-based deimplementation for contradicted, unproven, and aspiring healthcare practices. Implement Sci 2014;9:1.
78. Diamandis EP. Cancer biomarkers: can we turn recent failures into success? J Natl Cancer Inst 2010;102: 1462-7.
79. Bossuyt PM, Irwig L, Craig J, Glasziou P. Comparative accuracy: Assessing new tests against existing diagnostic pathways. BMJ 2006;332: 1089-92.
80. Krastins B, Prakash A, Sarracino DA, Nedelkov D, Niederkofler EE, Kiernan UA, et al. Rapid development of sensitive, high-Throughput, quantitative and highly selective mass spectrometric targeted immunoassays for clinically important proteins in human plasma and serum. Clin Biochem 2013;46: 399-410.
81. Monaghan PJ, Lord SJ, John A, Sandberg S, Cobbaert CM, Lennartz L, et al. Biomarker development targeting unmet clinical needs. Clin Chim Acta 2016;460: 211-9.