Metrics for evaluating performance and uncertainty of Bayesian network models

Ecological Modelling

Volumn 230, Issue , 2012, Pages 50-62

  Marcot, Bruce G ^a  

^a PACIFIC NORTHWEST RESEARCH STATION   (United States)

Author keywords

Bayesian network model; Error rates; Model performance; Model validation; Probability analysis; Sensitivity analysis; Uncertainty

Indexed keywords

BAYESIAN NETWORK MODELS; ERROR RATES; MODEL PERFORMANCE; MODEL VALIDATION; PROBABILITY ANALYSIS; UNCERTAINTY;

BAYESIAN NETWORKS; COMPUTER SIMULATION; PROBABILITY DISTRIBUTIONS; SENSITIVITY ANALYSIS;

UNCERTAINTY ANALYSIS;

BAYESIAN ANALYSIS; MODEL VALIDATION; POPULATION VIABILITY ANALYSIS; PROBABILITY; SENSITIVITY ANALYSIS; UNCERTAINTY ANALYSIS; WILDLIFE MANAGEMENT;

EID: 84857706006     PISSN: 03043800     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ecolmodel.2012.01.013     Document Type: Article

References (49)

1. Aalders I. Modeling land-use decision behavior with Bayesian belief networks. Ecology and Society 2008, 13(1):16. http://www.ecologyandsociety.org/vol13/iss1/art16/ [online].
2. Adkison M.D. Drawbacks of complex models in frequentist and Bayesian approaches to natural-resource management. Ecological Applications 2009, 19(1):198-205.
3. Akaike H. Information theory and an extension of the maximum likelihood principle. Second International Symposium on Information Theory 1973, 267-281. B.N. Petrov, F. Csaki (Eds.).
4. Allouche O., Tsoar A., Kadmon R. Assessing the accuracy of species distribution models: prevalence, kappa and the true skill statistic (TSS). Journal of Applied Ecology 2006, 43(6):1223-1232.
5. Amstrup S.C., Marcot B.G., Douglas D.C. A Bayesian network modeling approach to forecasting the 21st century worldwide status of polar bears. Arctic Sea Ice Decline: Observations, Projections, Mechanisms, and Implications. Geophysical Monograph 180 2008, 213-268. American Geophysical Union, Washington, DC. E.T. DeWeaver, C.M. Bitz, L.-B. Tremblay (Eds.).
6. Amstrup S.C., DeWeaver E.T., Douglas D.C., Marcot B.G., Dumer G.M., Bitz C.M., Bailey D.A. Greenhouse gas mitigation can reduce sea-ice loss and increase polar bear persistence. Nature 2010, 468(7326):955-958.
7. Ascough J.C., Maier H.R., Ravalico J.K., Strudley M.W. Future research challenges for incorporation of uncertainty in environmental and ecological decision-making. Ecological Modelling 2008, 219(3-4):383-399.
8. Atger F. Relative impact of model quality and ensemble deficiencies on the performance of ensemble based probabilisitic forecasts evaluated through the Brier score. Nonlinear Processes in Geophysics 2004, 11(3):399-409.
9. Atkinson A.B. On the measurement of inequality. Journal of Economic Theory 1970, 2:244-263.
10. Ayyub B.M., Klir G.J. Uncertainty Modeling and Analysis in Engineering and the Sciences 2006, Chapman & Hall/CRC, Boca Raton, FL, 378 pp.
11. Bolstad W.M. Introduction to Bayesian Statistics 2007, Wiley-Interscience, New York, 464 pp. 2nd ed.
12. Boyce M.S., Vernier P.R., Nielsen S.E., Schmiegelow F.K.A. Evaluating resource selection functions. Ecological Modelling 2002, 157:281-300.
13. Burnham K.P., Anderson D. Model Selection and Multi-model Inference: A Practical Information-Theoretic Approach 2010, Springer Verlag, New York, 514 pp.
14. Cain J.D., Batchelor C.H., Waughray D.K.N. Belief networks: a framework for the participatory development of natural resource management strategies. Environment, Development and Sustainability 1999, 1:123-133.
15. Castillo E., Gutierrez J.M., Hadi A.S. Modeling probabilistic networks of discrete and continuous variables. Journal of Multivariate Analysis 1998, 64:48-65.
16. Cortes C., Mohri M. Confidence intervals for the area under the ROC curve. Advances in Neural Information Processing Systems 17. Proceedings of the 2004 Conference 2005, 305-312. The MIT Press, Cambridge, Massachusetts. L.K. Saul, Y. Weiss, L. Bottou (Eds.).
17. Curran J.M. An introduction to Bayesian credible intervals for sampling error in DNA profiles. Law, Probability and Risk 2005, 4:115-126.
18. Dlamini W.M. A Bayesian belief network analysis of factors influencing wildfire occurrence in Swaziland. Environmental Modelling & Software 2010, 25(2):199-208.
19. Gutierrez B.T., Plant N.G., Thieler E.R. A Bayesian network to predict coastal vulnerability to sea level rise. Journal of Geophysical Research 2011, 116:F02009. 10.1029/2010JF001891.
20. Gutzwiller K.J., Flather C.H. Wetland features and landscape context predict the risk of wetland habitat loss. Ecological Applications 2011, 21:968-982.
21. Hand D. Construction and Assessment of Classification Rules 1997, John Wiley & Sons, New York, 232 pp.
22. Hill M.O. Diversity and evenness: a unifying notation and its consequences. Ecology 1973, 54:427-432.
23. Huang H.-C., Martinez F., Mateu J., Montes F. Model comparison and selection for stationary space-time models. Computational Statistics & Data Analysis 2007, 51(9):4577-4596.
24. Jay C.V., Marcot B.G., Douglas D.C. Projected status of the Pacific walrus (Odobenus rosmarus divergens) in the 21st century. Polar Biology 2011, 34(7):1065-1084.
25. Jensen F.V., Nielsen T.D. Bayesian Networks and Decision Graphs 2007, Springer, 448 pp. 2nd ed.
26. Kohavi R., Provost F. Glossary of terms. Journal of Machine Learning 1998, 30(2-3):271-274.
27. Lobo J.M., Jiménez-Valverde A., Real R. AUC: a misleading measure of the performance of predictive distribution models. Global Ecology and Biogeography 2008, 17:145-151.
28. Mantel N. The detection of disease clustering and a generalized regression approach. Cancer Research 1967, 27(2):209-220.
29. Marcot B.G. Characterizing species at risk I: modeling rare species under the Northwest Forest Plan. Ecology and Society 2006, 11(2):10. www.ecologyandsociety.org/vol11/iss2/art10/ [online].
30. Marcot B.G., Steventon J.D., Sutherland G.D., McCann R.K. Guidelines for developing and updating Bayesian belief networks applied to ecological modeling and conservation. Canadian Journal of Forest Research 2006, 36:3063-3074.
31. Mastrandrea, M.D., Field, C.B., Stocker, T.F., Edenhofer, O., Ebi, K.L., Frame, D.J., Held, H., Kriegler, E., Mach, K.J., Matschoss, P.R., Plattner, G.-K., Yohe, G.W., Zwiers, F. W., 2010. Guidance note for lead authors of the IPCC Fifth Assessment Report on Consistent Treatment of Uncertainties. IPCC Cross-Working Group Meeting on Consistent Treatment of Uncertainties, 6-7 July 2010. Available at Jasper Ridge, CA, 5 pp. http://www.ipcc.ch/.
32. McCann R., Marcot B.G., Ellis R. Bayesian belief networks: applications in natural resource management. Canadian Journal of Forest Research 2006, 36:3053-3062.
33. Monte L. Multi-model approach and evaluation of the uncertainty of model results. Rationale and applications to predict the behaviour of contaminants in the abiotic components of the fresh water environment. Ecological Modelling 2009, 220(12):1469-1480.
34. Mouton A.M., De Baets B., Goethals P.L.M. Ecological relevance of performance criteria for species distribution models. Ecological Modelling 2010, 221(16):1995-2002.
35. Neapolitan R.E. Learning Bayesian Networks 2003, Prentice Hall, Upper Saddle River, NJ, 674 pp.
36. Pauli J.N., Whiteman J.P., Marcot B.G., McClean T.M., Ben-David M. DNA-based approach to aging martens (Martes americana and M. caurina). Journal of Mammalogy 2011, 92(3):500-510.
37. Phillips S.J., Elith J. POC plots: calibrating species distribution models with presence-only data. Ecology 2010, 91(8):2476-2484.
38. Ponciano J.M., Taper M.L., Dennis B., Lele S.R. Hierarchical models in ecology: confidence intervals, hypothesis testing, and model selection using data cloning. Ecology 2009, 90(2):356-362.
39. Bayesian Belief Networks: A Practical Guide to Applications 2008, Wiley, West Sussex, England, 432 pp. O. Pourret, P. Naïm, B.G. Marcot (Eds.).
40. Ricotta C., Avena G.C. On the information-theoretical meaning of Hill's parametric evenness. Acta Biotheoretica 2002, 50:63-71.
41. Sadras V., Bongiovanni R. Use of Lorenz curves and Gini coefficients to assess yield inequality within paddocks. Field Crops Research 2004, 90(2-3):303-310.
42. Schwarz G. Estimating the dimension of a model. The Annals of Statistics 1978, 6(2):461-464.
43. Speigelhalter D.J., Best N.G., Carlin B.P., van der Linde A. Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society Series B 2002, 64:583-639.
44. Thogmartin W.E. Sensitivity analysis of North American bird population estimates. Ecological Modelling 2010, 221(2):173-177.
45. Uusitalo L. Advantages and challenges of Bayesian networks in environmental modelling. Ecological Modelling 2007, 203:312-318.
46. Van Allen T., Singh A., Greiner R., Hooper P. Quantifying the uncertainty of a belief net response: Bayesian error-bars for belief net inference. Artificial Intelligence 2008, 172(4-5):483-513.
47. Walters C. Adaptive Management of Renewable Resources 1986, MacMillan Pub. Co, New York.
48. Williams B.K. Resolving structural uncertainty in natural resources management using POMDP approaches. Ecological Modelling 2011, 222(5):1092-1102.
49. Zarnetske P.L., Edwards T.C., Moisen G.G. Habitat classification modeling with incomplete data: pushing the habitat envelope. Ecological Applications 2007, 17(6):1714-1726.