Avoiding pitfalls in applying prediction models, as illustrated by the example of prostate cancer diagnosis

Clinical Chemistry

Volumn 57, Issue 11, 2011, Pages 1490-1498

  Cammann, Henning ^a   Jung, Klaus ^a,b   Meyer, Hellmuth A ^a   Stephan, Carsten ^a,b  

^a CHARITÉ UNIVERSITÄTSMEDIZIN BERLIN   (Germany)

^b Berlin Institute for Urologic Research   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

BIOMETRY; CALIBRATION; CANCER DIAGNOSIS; CANCER RISK; CLINICAL PRACTICE; DIGITAL RECTAL EXAMINATION; GOLD STANDARD; MATHEMATICAL MODEL; MEDICAL DECISION MAKING; PROSTATE BIOPSY; PROSTATE CANCER; SHORT SURVEY; ULTRASOUND; VALIDATION STUDY;

BIOPSY, NEEDLE; DECISION SUPPORT TECHNIQUES; DIGITAL RECTAL EXAMINATION; ENDPOINT DETERMINATION; HUMANS; MALE; MODELS, BIOLOGICAL; MULTIVARIATE ANALYSIS; PREDICTIVE VALUE OF TESTS; PROSTATE; PROSTATE-SPECIFIC ANTIGEN; PROSTATIC NEOPLASMS; REFERENCE STANDARDS; RISK ASSESSMENT; TUMOR BURDEN;

EID: 80055065816     PISSN: 00099147     EISSN: 15308561     Source Type: Journal    
DOI: 10.1373/clinchem.2011.166959     Document Type: Short Survey

References (43)

1. Diamandis EP. Prostate cancer screening with prostate-specific antigen testing: more answers or more confusion? Clin Chem 2010;56:345-51.
2. Guillonneau B. Ceteris paribus and nomograms in medicine. Eur Urol 2007;52:1287-9. (Pubitemid 47445996)
3. Ross RW, Kantoff PW. Predicting outcomes in prostate cancer: how many more nomograms do we need? J Clin Oncol 2007;25:3563-4. (Pubitemid 47372589)
4. Schroder F, Kattan MW. The comparability of models for predicting the risk of a positive prostate biopsy with prostate-specific antigen alone: a systematic review. Eur Urol 2008;54:274 -90.
5. Stephan C, Cammann H, Bender M, Miller K, Lein M, Jung K, et al. Internal validation of an artificial neural network for prostate biopsy outcome. Int J Urol 2010;17:62-8.
6. Stephan C, Siemßen K, Cammann H, Friedersdorff F, Deger S, Schrader M, et al. Between-method differences in prostate-specific antigen assays affect prostate cancer risk prediction by nomograms. Clin Chem 2011;57:995-1004.
7. Nam RK, Toi A, Trachtenberg J, Jewett MA, Klotz L, Fleshner N, et al. Variation in patterns of practice in diagnosing screen-detected prostate cancer. BJU Int 2004;94:1239-44. (Pubitemid 40052085)
8. Campos-Fernandes JL, Bastien L, Nicolaiew N, Robert G, Terry S, Vacherot F, et al. Prostate cancer detection rate in patients with repeated extended 21-sample needle biopsy. Eur Urol 2009;55:600-6.
9. Ornstein DK, Smith DS, Rao GS, Basler JW, Ratliff TL, Catalona WJ. Biological variation of total, free and percent free serum prostate specific antigen levels in screening volunteers. J Urol 1997;157:2179-82. (Pubitemid 27208667)
10. Soletormos G, Semjonow A, Sibley PE, Lamerz R, Petersen PH, Albrecht W, et al. Biological variation of total prostate-specific antigen: a survey of published estimates and consequences for clinical practice. Clin Chem 2005;51:1342-51. (Pubitemid 41060864)
11. Jones GR. Critical difference calculations revised: inclusion of variation in standard deviation with analyte concentration. Ann Clin Biochem 2009;46:517-9.
12. Stephan C, Klaas M, Muller C, Schnorr D, Loening SA, Jung K. Interchangeability of measurements of total and free prostate-specific antigen in serum with 5 frequently used assay combinations: an update. Clin Chem 2006;52:59-64. (Pubitemid 43032467)
13. Rafferty B, Rigsby P, Rose M, Stamey T, Gaines DR. Reference reagents for prostate-specific antigen (PSA): establishment of the first international standards for free PSA and PSA (90:10). Clin Chem 2000;46:1310-7.
14. Kort SA, Martens F, Vanpoucke H, van Duijnhoven HL, Blankenstein MA. Comparison of 6 automated assays for total and free prostatespecific antigen with special reference to their reactivity toward the WHO 96/670 reference preparation. Clin Chem 2006;52:1568-74. (Pubitemid 44148320)
15. Bates TS, Reynard JM, Peters TJ, Gingell JC. Determination of prostatic volume with transrectal ultrasound: a study of intra-observer and interobserver variation. J Urol 1996;155:1299-300. (Pubitemid 26081223)
16. Choi YJ, Kim JK, Kim HJ, Cho KS. Interobserver variability of transrectal ultrasound for prostate volume measurement according to volume and observer experience. AJR Am J Roentgenol 2009;192:444-9.
17. Sech S, Montoya J, Girman CJ, Rhodes T, Roehrborn CG. Interexaminer reliability of transrectal ultrasound for estimating prostate volume. J Urol 2001;166:125-9. (Pubitemid 32565360)
18. Smith DS, Catalona WJ. Interexaminer variability of digital rectal examination in detecting prostate cancer. Urology 1995;45:70-4.
19. Gosselaar C, Kranse R, Roobol MJ, Roemeling S, Schroder FH. The interobserver variability of digital rectal examination in a large randomized trial for the screening of prostate cancer. Prostate 2008;68:985-93. (Pubitemid 351794163)
20. Kawamura K, Suzuki H, Kamiya N, Imamoto T, Yano M, Miura J, et al. Development of a new nomogram for predicting the probability of a positive initial prostate biopsy in Japanese patients with serum PSA levels less than 10 ng/mL. Int J Urol 2008;15:598-603. (Pubitemid 351989334)
21. Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21:128-38.
22. Utsumi T, Kawamura K, Suzuki H, Kamiya N, Imamoto T, Miura J, et al. External validation and head-to-head comparison of Japanese and Western prostate biopsy nomograms using Japanese data sets. Int J Urol 2009;16:416-9.
23. Wichard JD, Cammann H, Stephan C, Tolxdorff T. Classification models for early detection of prostate cancer. J Biomed Biotechnol 2008;2008:218097.
24. Anagnostou T, Remzi M, Lykourinas M, Djavan B. Artificial neural networks for decision-making in urologic oncology. Eur Urol 2003;43:596-603. (Pubitemid 36648597)
25. Dreiseitl S, Ohno-Machado L. Logistic regression and artificial neural network classification models: a methodology review. J Biomed Inform 2002;35:352-9. (Pubitemid 36951935)
26. Rodvold DM, McLeod DG, Brandt JM, Snow PB, Murphy GP. Introduction to artificial neural networks for physicians: taking the lid off the black box. Prostate 2001;46:39-44. (Pubitemid 32157774)
27. Kattan MW. Factors affecting the accuracy of prediction models limit the comparison of rival prediction models when applied to separate data sets. Eur Urol 2011;59:566-7.
28. Rodvold DM. Validation and regulation of medical neural networks. Mol Urol 2001;5:141-5. (Pubitemid 34038833)
29. Schwarzer G, Vach W, Schumacher M. On the misuses of artificial neural networks for prognostic and diagnostic classification in oncology. Stat Med 2000;19:541-61. (Pubitemid 30126883)
30. Shariat SF, Kattan MW, Vickers AJ, Karakiewicz PI, Scardino PT. Critical review of prostate cancer predictive tools. Future Oncol 2009;5:1555-84.
31. Stephan C, Cammann H, Meyer HA, Muller C, Deger S, Lein M, et al. An artificial neural network for five different assay systems of prostatespecific antigen in prostate cancer diagnostics. BJU Int 2008;102:799-805.
32. Finne P, Finne R, Auvinen A, Juusela H, Aro J, Maattanen L, et al. Predicting the outcome of prostate biopsy in screen-positive men by a multilayer perceptron network. Urology 2000;56:418-22. (Pubitemid 30658171)
33. Finne P, Finne R, Bangma C, Hugosson J, Hakama M, Auvinen A, et al. Algorithms based on prostate-specific antigen (PSA), free PSA, digital rectal examination and prostate volume reduce false-positive PSA results in prostate cancer screening. Int J Cancer 2004;111:310-5. (Pubitemid 38924712)
34. Obuchowski NA. ROC analysis. AJR Am J Roentgenol 2005;184:364-72.
35. Cook NR. Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. Clin Chem 2008;54:17-23. (Pubitemid 351182592)
36. Gjengsto P, Paus E, Halvorsen OJ, Eide J, Akslen LA, Wentzel-Larsen T, et al. Predictors of prostate cancer evaluated by receiver operating characteristics partial area index: a prospective institutional study. J Urol 2005;173:425-8.
37. Kawakami S, Numao N, Okubo Y, Koga F, Yamamoto S, Saito K, et al. Development, validation, and head-to-head comparison of logistic regression-based nomograms and artificial neural network models predicting prostate cancer on initial extended biopsy. Eur Urol 2008;54:601-11.
38. van Vugt HA, Roobol MJ, Kranse R, Maattanen L, Finne P, Hugosson J, et al. Prediction of prostate cancer in unscreened men: external validation of a risk calculator. Eur J Cancer 2011;47:903-9.
39. Lughezzani G, Budaus L, Isbarn H, Sun M, Perrotte P, Haese A, et al. Head-to-head comparison of the three most commonly used preoperative models for prediction of biochemical recurrence after radical prostatectomy. Eur Urol 2010;57:562-8.
40. Weinstein S, Obuchowski NA, Lieber ML. Clinical evaluation of diagnostic tests. AJR Am J Roentgenol 2005;184:14-9. (Pubitemid 40040897)
41. Steyerberg EW, Vickers AJ. Decision curve analysis: a discussion. Med Decis Making 2008; 28:146-9.
42. Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making 2006;26:565-74. (Pubitemid 44728772)
43. Kattan MW, Marasco J. What is a real nomogram? Semin Oncol 2010;37:23-6.