How to improve reliability and efficiency of research about molecular markers: roles of phases, guidelines, and study design

Journal of Clinical Epidemiology

Volumn 60, Issue 12, 2007, Pages 1205-1219

  Ransohoff, David F ^a  

^a UNIVERSITY OF NORTH CAROLINA SCHOOL OF MEDICINE   (United States)

Author keywords

Diagnosis; Genomics; Molecular markers; Prognosis; Proteomics; Screening

Indexed keywords

MOLECULAR MARKER; RNA;

ARTICLE; CANCER; CANCER DIAGNOSIS; CLINICAL RESEARCH; DIAGNOSTIC ACCURACY; DIAGNOSTIC VALUE; EXPECTATION; GENOMICS; HUMAN; METHODOLOGY; PLANNING; PRACTICE GUIDELINE; PRIORITY JOURNAL; PROGNOSIS; PROTEOMICS; RELIABILITY; REPRODUCIBILITY;

EID: 35848962931     PISSN: 08954356     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jclinepi.2007.04.020     Document Type: Note

References (105)

1. Ransohoff D.F. Bias as a threat to the validity of cancer molecular-marker research. Nat Rev Cancer 5 2 (2005) 142-149
2. Ransohoff D.F. Rules of evidence for cancer molecular-marker discovery and validation. Nat Rev Cancer 4 4 (2004) 309-314
3. Ransohoff D.F. Developing molecular biomarkers for cancer. Science 299 5613 (2003) 1679-1680
4. Petricoin E.F., Ardekani A.M., Hitt B.A., Levine P.J., Fusaro V.A., Steinberg S.M., et al. Use of proteomic patterns in serum to identify ovarian cancer. Lancet 359 9306 (2002) 572-577
5. Zhu W., Wang X., Ma Y., Rao M., Glimm J., and Kovach J.S. Detection of cancer-specific markers amid massive mass spectral data. Proc Natl Acad Sci USA 100 25 (2003) 14666-14671
6. van de Vijver M.J., He Y.D., van't Veer L.J., Dai H., Hart A.A., Voskuil D.W., et al. A gene-expression signature as a predictor of survival in breast cancer. N Engl J Med 347 25 (2002) 1999-2009
7. Huang E., Cheng S.H., Dressman H., Pittman J., Tsou M.H., Horng C.F., et al. Gene expression predictors of breast cancer outcomes. Lancet 361 9369 (2003) 1590-1596
8. Ramaswamy S., and Perou C.M. DNA microarrays in breast cancer: the promise of personalised medicine. Lancet 361 9369 (2003) 1576-1577
9. Bernards R., and Weinberg R.A. A progression puzzle. Nature 418 6900 (2002) 823
10. Baggerly K.A., Morris J.S., Edmonson S.R., and Coombes K.R. Signal in noise: evaluating reported reproducibility of serum proteomic tests for ovarian cancer. J Natl Cancer Inst 97 4 (2005) 307-309
11. Frantz S. An array of problems. Nat Rev Drug Discov 4 (2005) 362-363
12. Ioannidis J.P. Microarrays and molecular research: noise discovery?. Lancet 365 9458 (2005) 454-455
13. Michiels S., Koscielny S., and Hill C. Prediction of cancer outcome with microarrays: a multiple random validation strategy. Lancet 365 9458 (2005) 488-492
14. Clark G.M. Interpreting and integrating risk factors for patients with primary breast cancer. J Natl Cancer Inst Monogr 30 (2001) 17-21
15. Hayes D.F., Bast R.C., Desch C.E., Fritsche Jr. H., Kemeny N.E., Jessup J.M., et al. Tumor marker utility grading system: a framework to evaluate clinical utility of tumor markers. J Natl Cancer Inst 88 20 (1996) 1456-1466
16. Sackett D.L., and Haynes R.B. The architecture of diagnostic research. Br Med J 324 7336 (2002) 539-541
17. Ransohoff D.F. Challenges and opportunities in evaluating diagnostic tests. J Clin Epidemiol 55 (2002) 1178-1182
18. Concato J. Challenges in prognostic analysis. Cancer 91 8 Suppl (2001) 1607-1614
19. Knottnerus J.A., and van Weel C. General introduction: evaluation of diagnostic procedures. In: Knottnerus J.A. (Ed). The evidence base of clinical diagnosis (2002), BMJ Books, London 1-17
20. Petricoin III E.F., Ornstein D.K., Paweletz C.P., Ardekani A., Hackett P.S., Hitt B.A., et al. Serum proteomic patterns for detection of prostate cancer. J Natl Cancer Inst 94 20 (2002) 1576-1578
21. Adam B.L., Qu Y., Davis J.W., Ward M.D., Clements M.A., Cazares L.H., et al. Serum protein fingerprinting coupled with a pattern-matching algorithm distinguishes prostate cancer from benign prostate hyperplasia and healthy men. Cancer Res 62 13 (2002) 3609-3614
22. Vlahou A., Schellhammer P.F., Mendrinos S., Patel K., Kondylis F.I., Gong L., et al. Development of a novel proteomic approach for the detection of transitional cell carcinoma of the bladder in urine. Am J Pathol 158 (2001) 1491-1502
23. Drake R.R., Manne U., Bao-Ling A., Ahn C., Cazares L., Semmes O.J., et al. SELDI-TOF-MS profiling of serum for early detection of colorectal cancer. Gastroenterology 124 Suppl 1 (2003) A650
24. Pollack A. New cancer test stirs hope and concern. New York Times (February 3, 2004) [D1, D6]
25. Marcus A. Testing for ovarian cancer is on the way. Wall St J (October 1, 2002) [D1, D2]
26. Check E. Running before we can walk?. Nature 429 (2004) 496-497
27. McCullough M. Hope-and hype-in the cancer war. Phila Inquirer (2005) [Sect. A1, A12-A13]
28. Diamandis E.P. Re: Serum proteomic patterns for detection of prostate cancer. J Natl Cancer Inst 95 6 (2003) 489-490 [author reply 90-91]
29. Diamandis E.P. Point: proteomic patterns in biological fluids: do they represent the future of cancer diagnostics?. Clin Chem 49 (2003) 1272-1275
30. Diamandis E.P. OvaCheck: doubts voiced soon after publication. Nature 430 7000 (2004) 611
31. Diamandis E.P. Mass spectrometry as a diagnostic and a cancer biomarker discovery tool: opportunities and potential limitations. Mol Cell Proteomics 3 4 (2004) 367-378
32. Garber K. Debate rages over proteomic patterns. J Natl Cancer Inst 96 11 (2004) 816-818
33. Baggerly K.A., Coombes K.R., and Morris J.S. Bias, randomization, and ovarian proteomic data: a reply to "Producers and consumers". Cancer Inform 1 (2005) 9-14
34. Baggerly K.A., Morris J.S., and Coombes K.R. Reproducibility of SELDI-TOF protein patterns in serum: comparing datasets from different experiments. Bioinformatics 20 (2004) 777-785
35. Baggerly K.A., Edmonson S.R., Morris J.S., and Coombes K.R. High-resolution serum proteomic patterns for ovarian cancer detection. [letter to the editor]. Endocr Relat Cancer 11 (2004) 583-584
36. Ransohoff D.F. Lessons from controversy: ovarian cancer screening and serum proteomics. J Natl Cancer Inst 97 4 (2005) 315-319
37. Liotta L.A., Lowenthal M., Mehta A., Conrads T.P., Veenstra T.D., Fishman D.A., et al. Importance of communication between producers and consumers of publicly available experimental data. J Natl Cancer Inst 97 4 (2005) 310-314
38. Villanueva J., Shaffer D.R., Philip J., Chaparro C.A., Erdjument-Bromage H., Olshen A.B., et al. Differential exoprotease activities confer tumor-specific serum peptidome patterns. J Clin Invest 116 (2006) 271-284
39. Liotta L.A., and Petricoin E.F. Serum peptidome for cancer detection: spinning biologic trash into diagnostic gold. J Clin Invest 116 1 (2006) 26-30
40. Diamandis E.P., Kulasingam V., and Sardana G. Letter to the editor about differential exoprotease activities confer tumor-specific serum peptidome. Accessed February 21, 2006. Available at. http://www.jci.org/cgi/eletters/116/1/271
41. Diamandis E.P., Kulasingam V., and Sardana G. Letter to the editor about differential exoprotease activities confer tumor-specific serum peptidome. Accessed February 21, 2006. Available at. J Clin Invest (2006)
42. Tempst P., Villanueva J., Shaffer D.R., Lilja H., and Scher H.I. Response to 'Letter to the Editor' by E.P. Diamandis, V. Kulasingam, G. Sardana. Accessed February 21, 2006. Available at. http://www.jci.org/cgi/eletters/116/1/271
43. Tempst P., Villanueva J., Shaffer D.R., Lilja H., and Scher H.I. Response to 'Letter to the Editor' by E.P. Diamandis, V. Kulasingam, G. Sardana. Accessed February 21, 2006. Available at. J Clin Invest (2006)
44. Zhang Z., Bast Jr. R.C., Yu Y., Li J., Sokoll L.J., Rai A.J., et al. Three biomarkers identified from serum proteomic analysis for the detection of early stage ovarian cancer. Cancer Res 64 16 (2004) 5882-5890
45. Mor G., Visintin I., Lai Y., Zhao H., Schwartz P., Rutherford T., et al. Serum protein markers for early detection of ovarian cancer. Proc Natl Acad Sci USA 102 21 (2005) 7677-7682
46. Kolata G. Breast cancer: genes are tied to death rates. New York Times (December 19, 2002) [A1]
47. Ransohoff D.F. Gene-expression signatures in breast cancer. N Engl J Med 348 (2003) 1715-1717 [author reply-7]
48. Buyse M., Loi S., van't Veer L., Viale G., Delorenzi M., Glas A.M., et al. Validation and clinical utility of a 70-gene prognostic signature for women with node-negative breast cancer. J Natl Cancer Inst 98 17 (2006) 1183-1192
49. Marshall E. Getting the noise out of gene arrays. Science 306 (2004) 630-631
50. Tibshirani R. Immune signatures in follicular lymphoma. N Engl J Med 352 (2005) 1496-1497 [author reply-7]
51. Paik S., Shak S., Tang G., Kim C., Baker J., Cronin M., et al. A multigene assay to predict recurrence of tamoxifen-treated, node-negative breast cancer. N Engl J Med 351 27 (2005) 2817-2826
52. Aliferis C.F., Statnikov A., Tsamardinos I., Schildcrout J.S., Shepherd B.E., and Harrell Jr. F.E. Gene expression microarrays do predict outcome. Technical Report (2007)
53. Grizzle W.E., Semmes O.J., Bigbee W., Zhu L., Malik G., Oelschlager D.K., et al. The need for review and understanding of SELDI/MALDI mass spectroscopy data prior to analysis. Cancer Bioinform 1 (2005) 86-97
54. Mitchell B.L., Yasui Y., Li C.I., Fitzpatrick A.L., and Lampe P.D. Impact of freeze-thaw cycles and storage time on plasma samples used in mass spectroscopy based biomarker discovery projects. Cancer Bioinform 1 (2005) 25-31
55. Simon R., Radmacher M.D., Dobbin K., and McShane L.M. Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. J Natl Cancer Inst 95 1 (2003) 14-18
56. Burman P. A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods. Biometrika 76 (1989) 503-514
57. Breiman L., and Spector P. Submodel selection and evaluation in regression. The X-random case. Int Stat Rev 60 (1992) 291-319
58. Molinaro A.M., Simon R., and Pfeiffer R.M. Prediction error estimation: a comparison of resampling methods. Bioinformatics 21 15 (2005) 3301-3307
59. Ntzani E.E., and Ioannidis J.P. Predictive ability of DNA microarrays for cancer outcomes and correlates: an empirical assessment. Lancet 362 9394 (2003) 1439-1444
60. Simon R. Roadmap for developing and validating therapeutically relevant genomic classifiers. J Clin Oncol 23 29 (2005) 7332-7341
61. Petricoin E.F., Zoon K.C., Kohn E.C., Barrett J.C., and Liotta L.A. Clinical proteomics: translating benchside promise into bedside reality. Nat Rev Drug Discov 1 9 (2002) 683-695
62. Imperiale T.F., Ransohoff D.F., Itzkowitz S.H., Turnbull B.A., and Ross M.E. Fecal DNA versus fecal occult blood for colorectal-cancer screening in an average-risk population. N Engl J Med 351 (2004) 2704-2714
63. Food and Drug Administration. Innovation or stagnation: challenge and opportunity on the critical path to new medical products. Available at (2004). http://www.fda.gov/oc/initiatives/criticalpath/whitepaper.html (accessed Aug 25, 2007)
64. Cohen F.J. Macro trends in pharmaceutical innovation. Nat Rev Drug Discov 4 1 (2005) 78-84
65. Williams D. The critical path to medical innovation. Med Device Technol 15 5 (2004) 8-10
66. Couzin J. The new math of clinical trials. Science 303 5659 (2004) 784-786
67. Hulley S.B., Cummings S.R., Browner W.S., Grady D., Hearst N., and Newman T.B. Designing clinical research: an epidemiologic approach. 2nd edition (2001), Lippincott Williams & Wilkins, Philadelphia, PA
68. Sullivan Pepe M., Etzioni R., Feng Z., Potter J.D., Thompson M.L., Thornquist M., et al. Phases of biomarker development for early detection of cancer. J Natl Cancer Inst 93 14 (2001) 1054-1061
69. Altman D.G., Schulz K.F., Moher D., Egger M., Davidoff F., Elbourne D., et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 134 8 (2001) 663-694
70. Rennie D. How to report randomized controlled trials. The CONSORT statement. JAMA 276 8 (1996) 649
71. Begg C., Cho M., Eastwood S., Horton R., Moher D., Olkin I., et al. Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA 276 8 (1996) 637-639
72. U.S. Food and Drug Administration. FDA regulations relating to good clinical practice and clinical trials. Available at. http://www.fda.gov/oc/gcp/regulations.html (accessed Aug 25, 2007)
73. Buntinx F., and Knottnerus J.A. Are we at the start of a new era in diagnostic research?. J Clin Epidemiol 59 4 (2006) 325-326
74. In: Higgins J.P.T., and Green S. (Eds). Cochrane handbook for systematic reviews of interventions 4.2.6 (2006). http://www.cochrane.org/resources/handbook/hbook.htm Updated September 2006. Accessed February 22, 2007. Available at
75. Pepe M.S. The statistical evaluation of medical tests for classification and prediction (2003), Oxford University Press, Oxford, England
76. In: Knottnerus J.A. (Ed). The evidence base of clinical diagnosis (2002), BMJ Books, London
77. Feinstein A.R. Clinical epidemiology: the architecture of clinical research (1985), WB Saunders, Philadelphia, PA
78. Sackett D.L., and Haynes R.B. The architecture of diagnostic research. In: Knottnerus J.A. (Ed). The evidence base of clinical diagnosis (2002), BMJ Books, London 19-38
79. Knottnerus J.A., and Muris J.W. Assessment of the accuracy of diagnostic tests: the cross-sectional study. In: Knottnerus J.A. (Ed). The evidence base of clinical diagnosis (2002), BMJ Books, London 39-59
80. Feng Z., Prentice R., and Srivastava S. Research issues and strategies for genomic and proteomic biomarker discovery and validation: a statistical perspective. Pharmacogenomics 5 (2004) 709-719
81. Bossuyt P.M., Reitsma J.B., Bruns D.E., Gatsonis C.A., Glasziou P.P., Irwig L.M., et al. Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Ann Intern Med 138 1 (2003) 40-44
82. McShane L.M., Altman D.G., Sauerbrei W., Taube S.E., Gion M., and Clark G.M. Reporting recommendations for tumor marker prognostic studies (REMARK). J Natl Cancer Inst 97 16 (2005) 1180-1184
83. Brazma A., Hingamp P., Quackenbush J., Sherlock G., Spellman P., Stoeckert C., et al. Minimum information about a microarray experiment (MIAME)-toward standards for microarray data. Nat Genet 29 4 (2001) 365-371
84. Irwig L., Macaskill P., Glasziou P., and Fahey M. Meta-analytic methods for diagnostic test accuracy. [Review; 32 refs]. J Clin Epidemiol 48 (1995) 119-130
85. Knottnerus J.A., and Muris J.W. Assessment of the accuracy of diagnostic tests: the cross-sectional study. J Clin Epidemiol 56 (2003) 1118-1128
86. Whiting P., Rutjes A.W., Dinnes J., Reitsma J.B., Bossuyt P.M., and Kleijnen J. A systematic review finds that diagnostic reviews fail to incorporate quality despite available tools. J Clin Epidemiol 58 1 (2005) 1-12
87. Baker S.G., Kramer B.S., McIntosh M., Patterson B.H., Shyr Y., and Skates S. Evaluating markers for the early detection of cancer: overview of study designs and methods. Clin trials (London, England) 3 1 (2006) 43-56
88. Bogardus S.T., Concato J., and Feinstein A.R. Clinical epidemiological quality in molecular genetic research: the need for methodological standards. JAMA 281 (1999) 1919-1926
89. Dupuy A., and Simon R.M. Critical review of published microarray studies for cancer outcome and guidelines on statistical analysis and reporting. J Natl Cancer Inst 99 2 (2007) 147-157
90. Hede K. $104 million proteomics initiative gets green light. J Natl Cancer Inst 97 (2005) 1324-1325
91. Goldberg K.B. Advisors reject NCI's $89 million plan for proteomics as too much, too soon. The Cancer Lett 31 10 (2005) 1-10
92. Martin R.L. Breakthrough ideas for 2005: seek validity, not reliability. Harv Bus Rev 83 2 (2005) 23-24 32
93. Semmes O.J., Feng Z., Adam B.L., Banez L.L., Bigbee W.L., Campos D., et al. Evaluation of serum protein profiling by surface-enhanced laser desorption/ionization time-of-flight mass spectrometry for the detection of prostate cancer: I. Assessment of platform reproducibility. Clin Chem 51 (2005) 102-112
94. Bogaerts J., Cardoso F., Buyse M., Braga S., Loi S., Harrison J.A., et al. Gene signature evaluation as a prognostic tool: challenges in the design of the MINDACT trial. Nat Clin Pract Oncol 3 10 (2006) 540-551
95. Mauriac L., Debled M., and MacGrogan G. When will more useful predictive factors be ready for use?. Breast 14 (2005) 617-623
96. Platt J.R. Strong inference. Science 146 3642 (1964) 347-353
97. Ransohoff D.F. Research opportunity at the interface of molecular biology and clinical epidemiology. Gastroenterology 122 (2002) 1199
98. Herdman R.K. Testimony concerning the roles of the SEC and the FASB in establishing GAAP; before the House Subcommittee on Capital Markets, Insurance, and Government Sponsored Enterprises, Committee on Financial Services. Accessed February 11, 2007. Available at (2002). http://www.sec.gov/news/testimony/051402tsrkh.htm
99. Prorok P.C., Andriole G.L., Bresalier R.S., Buys S.S., Chia D., Crawford E.D., et al. Design of the prostate, lung, colorectal and ovarian (PLCO) cancer screening trial. Control Clin Trials 21 6 Suppl (2000) 273S-309S
100. Manson J.E., Hsia J., Johnson K.C., Rossouw J.E., Assaf A.R., Lasser N.L., et al. Estrogen plus progestin and the risk of coronary heart disease. N Engl J Med 349 6 (2003) 523-534
101. Skates S.J., Horick N.K., Moy J.M., Minihan A.M., Seiden M.V., Marks J.R., et al. Pooling of case specimens to create standard serum sets for screening cancer biomarkers. Cancer Epidemiol Biomarkers Prev 16 (2007) 334-341
102. Mandel J.S., Bond J.H., Church T.R., Snover D.C., Bradley G.M., Schuman L.M., et al. Reducing mortality from colorectal cancer by screening for fecal occult blood. N Engl J Med 328 (1993) 1365-1371
103. Hardcastle J.D., Chamberlain J.O., Robinson M.H.E., Moss S.M., Amar S.S., Balfour T.W., et al. Randomised controlled trial of faecal-occult-blood screening for colorectal cancer. Lancet 348 (1996) 1472-1477
104. Kronborg O., Fenger C., Olsen J., Jorgensen O.D., and Sondergaard O. Randomised study of screening for colorectal cancer with faecal-occult-blood test. Lancet 348 (1996) 1467-1471
105. Winawer S.J., Fletcher R.H., Miller L., Godlee F., Stolar M.H., Mulrow C.D., et al. Colorectal cancer screening: clinical guidelines and rationale. Gastroenterology 112 (1997) 594-642