Multiple model evaluation absent the gold standard through model combination

Journal of the American Statistical Association

Volumn 103, Issue 483, 2008, Pages 897-909

  Iversen Jr , Edwin S ^a   Parmigiani, Giovanni ^b   Chen, Sining ^b  

^a Duke University   (United States)

^b JOHNS HOPKINS UNIVERSITY   (United States)

Author keywords

Bayesian analysis; Breast cancer susceptibility genes; Measurement error; Model combination; Model evaluation

Indexed keywords

EID: 54949093248     PISSN: 01621459     EISSN: None     Source Type: Journal    
DOI: 10.1198/016214507000001012     Document Type: Article

References (50)

1. Andrulis, I., Anton-Culver, H., Beck, J., Bove, B., Boyd, J., Buys, S., Godwin, A., Hopper, J., Li, F., Neuhausen, S., Ozcelik, H., Peel, D., Santella, R., Southey, M., van Orsouw, N., Venter, D., Vijg, J., and Whittemore, A. (2002), "Comparison of DNA- and RNA-Based Methods for Detection of Truncating BRCA1 Mutations," Human Mutations, 20, 65-73.
2. Aretini, P., D'Andrea, E., Pasini, B., Viel, A., Costantini, R., Cortesi, L., Ricevuto, E., Agata, S., Bisegna, R., Boiocchi, M., Caligo, M., Chieco-Bianchi, L., Cipollini, G., Crucianelli, R., D'Amico, C., Federico, M., Ghimenti, C., DeGiacomi, C., DeNicolo, A., Puppa, L., Ferrari, S., Ficorella, C., Iandolo, D., Manoukian, S., Marchetti, P., Marroni, F., Menin, C., Montagna, M., Ottini, L., Pensotti, V., Pierotti, M., Radice, P., Santarosa, M., Silingardi, V., Turchetti, D., Bevilacqua, G., and Presciuttini, S. (2003), "Different Expressivity of BRCA1 and BRCA2: Analysis of 179 Italian Pedigrees With Identified Mutation," Breast Cancer Research and Treatment, 81, 71-79.
3. Barcenas, C., Hosain, G. M. M., Arun, B., Zong, J., Zhou, X., Chen, J., Cortada, J. M., Mills, G. B., Tomlinson, G. E., Miller, A. R., Strong, L. C., and Amos, C. I. (2006), "Assessing BRCA Carrier Probabilities in Extended Families," Journal of Clinical Oncology, 24, 354-360.
4. Berry, D. A., Parmigiani, G., Sanchez, J., Schildkraut, J., and Winer, E. (1997), "Probability of Carrying a Mutation of Breast-Ovarian Cancer Gene BRCA1 Based on Family History," Journal of the National Cancer Institute, 89, 227-238.
5. Bordley, R. F. (1982), "A Multiplicative Formula for Aggregating Probability Assessments," Management Science, 28, 1137-1148.
6. Claus, E. B., Risch, N., and Thompson, W. D. (1990), "Age of Onset as an Indicator of Familial Risk of Breast Cancer," American Journal of Epidemiology, 131, 961-972.
7. _ (1991), "Genetic Analysis of Breast Cancer in the Cancer and Steroid Hormone Study," American Journal of Human Genetics, 48, 232-242.
8. Clyde, M., and George, E. (2004), "Model Uncertainty," Statistical Science, 19, 81-94.
9. Couch, F. J., DeShano, M. L., Blackwood, M. A., Calzone, K., Stopfer, J., Campeau, L., Ganguly, A., Rebbeck, T., Weber, B. L., Jablon, L., Cobleigh, M. A., Hoskins, K., and Garber, J. E. (1997), "BRCA1 Mutations in Women Attending Clinics That Evaluate the Risk of Breast Cancer," New England Journal Medicine, 336, 1409-1415.
10. Dawid, A., DeGroot, M., and Mortera, J. (1995), "Coherent Combinations of Experts' Opinions" (with discussion), Test, 4, 263-313.
11. DeGroot, M. H., and Fienberg, S. E. (1983), "The Comparison and Evaluation of Forecasters," The Statistician, 32, 12-22.
12. Edwards, S., Kote-Jarai, Z., Meitz, J., Hamoudi, R., Hope, Q., Osin, P., Jackson, R., Southgate, C., Singh, R., Falconer, A., Dearnaley, D., Ardern-Jones, A., Murkin, A., Dowe, A., Kelly, J., Williams, S., Oram, R., Stevens, M., Teare, D., Ponder, B., Gayther, S., Easton, D., and Eeles, R. (2003), "Two Percent of Men With Early-Onset Prostate Cancer Harbor Germline Mutations in the BRCA2 Gene," American Journal of Human Genetics, 72, 1-12.
13. Elston, R. C., and Stewart, J. (1971), "A General Model for the Genetic Analysis of Pedigree Data," Human Heredity, 21, 523-542.
14. Eng, C., Brody, L., Wagner, T., Devilee, P., Vijg, J., Szabo, C., Tavtigian, S., Nathanson, K., Ostrander, E., and Frank, T. (2001), "Interpreting Epidemiological Research: Blinded Comparison of Methods Used to Estimate the Prevalence of Inherited Mutations in BRCA1," Journal of Medical Genetics, 38, 824-833.
15. Euhus, D. M., Smith, K. C., Robinson, L., Stucky, A., Olopade, O. I., Cummings, S., Garber, J. E., Chittenden, A., Mills, G. B., Rieger, P., Esserman, L., Crawford, B., Hughes, K. S., Roche, C. A., Ganz, P. A., Seldon, J., Fabian, C. J., Klemp, J., and Tomlinson, G. (2002), "Pretest Prediction of BRCA1 or BRCA2 Mutation by Risk Counselors and the Computer Model BRCAPRO," Journal of the National Cancer Institute, 94, 844-851.
16. Fraedrich, K., and Smith, N. (1989), "Combining Predictive Schemes in Long-Range Forecasting," Journal of Climate, 2, 291-294.
17. Frank, T. S., Deffenbaugh, A. M., Reid, J. E., Hulick, M., Ward, B. E., Lingenfelter, B., Gumpper, K. L., Scholl, T., Tavtigian, S. V., Pruss, D. R., and Critchfield, G. C. (2002), "Clinical Characteristics of Individuals With Germline Mutations in BRCA1 and BRCA2: Analysis of 10,000 Individuals," Journal of Clinical Oncology, 20, 1480-1490.
18. Frank, T. S., Manley, S. A., Olopade, O. I., Cummings, S., Garber, J. E., Bernhardt, B., Antman, K., Russo, D., Wood, M. E., Mullineau, L., Isaacs, C., Peshkin, B., Buys, S., Venne, V., Rowley, P. T., Loader, S., Offit, K., Robson, M., Hampel, H., Brener, D., Winer, E. P., Clark, S., Weber, B., Strong, L. C., and Thomas, A. (1998), "Sequence Analysis of BRCA1 and BRCA2: Correlation of Mutations With Family History and Ovarian Cancer Risk," Journal of Clinical Oncology, 16, 2417-2425.
19. French, S. (1985), "Group Consensus Probability Distributions: A Critical Survey," in Bayesian Statistics 2, eds. J. M. Bernardo, M. H. DeGroot, D. V. Lindley, and A. F. M. Smith, Amsterdam: North-Holland, pp. 183-201.
20. _ (1995), Discussion of "Coherent Combinations of Experts' Opinions," by A. David, M. DeGroot, and J. Mortera, Test, 4, 294-296.
21. Geisler, J., Hatterman-Zogg, M., Rathe, J., Lallas, T., Kirby, P., and Buller, R. (2001), "Ovarian Cancer BRCA1 Mutation Detection: Protein Truncation Test (PTT) Outperforms Single-Strand Conformation Polymorphism Analysis (SSCP)," Human Mutations, 18, 337-344.
22. Gel, Y., Raftery, A., and Gneiting, T. (2004), "Calibrated Probabilistic Mesoscale Weather Field Forecasting: The Geostatistical Output Perturbation (GOP) Method" (with discussion), Journal of the American Statistical Association, 99, 575-590.
23. Genest, C., and McConway, K. (1990), "Allocating the Weights in the Linear Opinion Pool," Journal of Forecasting, 9, 53-73.
24. Genest, C., and Schervish, M. J. (1985), "Modeling Expert Judgements for Bayesian Updating," The Annals of Statistics, 13, 1198-1212.
25. Genest, C., and Zidek, J. V. (1986), "Combining Probability Distributions: A Critique and an Annotated Bibliography," Statistical Science, 1, 114-148.
26. Geweke, J. (1992), "Evaluating the Accuracy of Sampling-Based Approaches to Calculating Posterior Moments," in Bayesian Statistics 4, eds. J. O. Berger, J. M. Bernado, A. P. Dawid, and A. F. M. Smith, London: Oxford University Press, pp. 169-193.
27. Gilpin, C., Carson, N., and Hunter, A. (2000), "A Preliminary Validation of a Family History Assessment Form to Select Women at Risk for Breast or Ovarian Cancer for Referral to a Genetics Center," Clinical Genetics, 58, 299-308.
28. Gneiting, T., Raftery, A., Westveld, A., and Goldman, T. (2005), "Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation," Monthly Weather Review, 133, 1098-1118.
29. Harrell, F. Jr., Califf, R., Pryor, D., Lee, L., and Rosati, R. (1982), "Evaluating the Yield of Medical Tests," Journal of the American Medical Association, 247, 2543-2546.
30. Hartge, P., Struewing, J. P., Wacholder, S., Brody, L. C., and Tucker, M. A. (1999), "The Prevalence of Common BRCA1 and BRCA2 Mutations Among Ashkenazi Jews," American Journal of Human Genetics, 64, 963-970.
31. Heidelberger, P., and Welch, P. (1983), "Simulation Run Length Control in the Presence of an Initial Transient," Operations Research, 31, 1109-1144.
32. Hui, S. L., and Walter, S. D. (1980), "Estimating the Error Rates of Diagnostic Tests," Biometrics, 36, 167-171.
33. Iversen, E. S. Jr., Parmigiani, G., and Berry, D. (1999), "Validating Bayesian Prediction Models: A Case Study in Genetic Susceptibility to Breast Cancer," in Case Studies in Bayesian Statistics, Vol. IV, eds. C. Gatsonis, R. E. Kass, B. Carlin, A. Carriquiry, A. Gelman, I. Verdinelli, and M. West, New York: Springer, pp. 321-338.
34. Key, J., Pericchi, L., and Smith, A. (1999), "Bayesian Model Choice: What and Why?" in Bayesian Statistics 6, eds. J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, London: Oxford University Press, pp. 343-370.
35. Kharin, V., and Zwiers, F. (2002). "Climate Predictions With Multimodel Ensembles," Journal of Climate, 15, 793-799.
36. Kraft, P., and Thomas, D. C. (2000), "Bias and Efficiency in Family-Based Gene-Characterization Studies: Conditional, Prospective, Retrospective, and Joint Likelihoods," American Journal of Human Genetics, 66, 1119-1131.
37. Krishnamurti, T., Kishtawal, C., LaRow, T., Bachiochni, D., Zhang, Z., Willifor, C., Gadgil, S., and Surendran, S. (1999), "Improved Weather Forecasts and Seasonal Climate Forecasts From Multimodel Superensemble," Science, 285, 1548-1550.
38. Lacroix, M., Leclercq, G., and Breast Med Consortium (2005), "The Portrait of Hereditary Breast Cancer," Breast Cancer Research and Treatment, 89, 297-304.
39. Lindor, N., Greene, M., and the Mayo Familial Cancer Program (1998), "The Concise Handbook of Family Cancer Syndromes," Journal of the National Cancer Institute, 90, 1039-1071.
40. Makridakis, S., and Winkler, R. (1983), "Averages of Forecasts: Some Empirical Results," Management Science, 29, 987-996.
41. Myriad Genetics (2003), "BRACAnalysis Technical Specification," technical report, Myriad Genetics Laboratories.
42. Parmigiani, G., Berry, D. A., and Aguilar, O. (1998), "Determining Carrier Probabilities for Breast Cancer Susceptibility Genes BRCA1 and BRCA2," American Journal of Human Genetics, 62, 145-158.
43. Parmigiani, G., Chen, S., Iversen, E. Jr., Friebel, T., Finkelstein, D., Anton-Culver, H., Ziogas, A., Weber, B., Eisen, A., Malone, K., Daling, J. R., Hsu, L., Ostrander, E. A., Peterson, L., Schildkraut, J., Isaacs, C., Corio, C., Leondaridis, L., Tomlinson, G., Amos, C., Strong, L., Berry, D., Weitzel, J., Sand, S., Dutson, D., Kerber, R., Peshkin, B., and Euhus, D. (2007), "Validity of Models for Prediction of BRCA1 and BRCA2 Mutations," Annals of Internal Medicine, 147, 441-450.
44. Raftery, A. E., and Lewis, S. M. (1996), "Implementing MCMC," in Markov Chain Monte Carlo in Practice, eds. W. R. Gilks, S. Richardson, and D. J. Spiegelhalter, London: Chapman & Hall, pp. 115-127.
45. Risch, H. A., McLaughlin, J. R., Cole, D. E. C., Rosen, B., Bradley, L., Kwan, E., Jack, E., Vesprini, D. J., Kuperstein, G., Abrahamson, J. L. A., Fan, I., Wong, B., and Narod, S. A. (2001), "Prevalence and Penetrance of Germline BRCA1 and BRCA2 Mutations in a Population Series of 649 Women With Ovarian Cancer," American Journal of Human Genetics, 68, 700-710.
46. Roa, B. B., Boyd, A. A., Volcik, K., and Richards, C. S. (1996), "Ashkenazi Jewish Population Frequencies for Common Mutations in BRCA1 and BRCA2," Nature Genetics, 14, 185-187.
47. Thompson, P. (1977), "How to Improve Accuracy by Combining Independent Forecasts," Monthly Weather Review, 105, 228-229.
48. Tierney, L. (1994), "Markov Chains for Exploring Posterior Distributions," The Annals of Statistics, 22, 1701-1728.
49. Vahteristo, P., Eerola, H., Tamminen, A., Blomqvist, C., and Nevanlinna, H. (2001), "A Probability Model for Predicting BRCA1 and BRCA2 Mutations in Breast and Breast-Ovarian Cancer Families," British Journal of Cancer, 84, 704-708.
50. Walter, S. D., and Irwig, L. M. (1988), "Estimation of Test Error Rates, Disease Prevalence and Relative Risk From Misclassified Data: A Review," Journal of Clinical Epidemiology, 41, 923-937.