Validation and calibration of probabilistic predictions in ecology

Methods in Ecology and Evolution

Volumn 5, Issue 10, 2014, Pages 1023-1032

  Chivers, Corey ^a   Leung, Brian ^a,b   Yan, Norman D ^c  

^a MCGILL UNIVERSITY   (Canada)

^b MCGILL UNIVERSITY   (Canada)

^c YORK UNIVERSITY   (Canada)

Author keywords

AUC; Binary outcomes; Forecasting; Prediction; Probability; Validation

Indexed keywords

BYTHOTREPHES LONGIMANUS; CLADOCERA;

EID: 84939264416     PISSN: None     EISSN: 2041210X     Source Type: Journal    
DOI: 10.1111/2041-210X.12238     Document Type: Article

References (49)

1. Araújo, M., Pearson, R., Thuiller, W. & Erhard, M. (2005). Validation of species-climate impact models under climate change. Global Change Biology, 11, 1504-1513.
2. Bolker, B.M. (2008). Ecological Models and Data in R. Princeton University Press, Princeton, NJ.
3. Bradie, J., Chivers, C. & Leung, B. (2013). Importing risk: quantifying the propagule pressure-establishment relationship at the pathway level. Diversity and Distributions, 19, 1020-1030.
4. Broennimann, O., Fitzpatrick, M.C., Pearman, P.B., Petitpierre, B., Pellissier, L., Yoccoz, N.G. et al. (2012). Measuring ecological niche overlap from occurrence and spatial environmental data. Global Ecology and Biogeography, 21, 481-497.
5. Brook, B.W., O'Grady, J.J., Chapman, A.P., Burgman, M.A., Akçakaya, H.R. & Frankham, R. (2000). Predictive accuracy of population viability analysis in conservation biology. Nature, 404, 385-387.
6. Burgman, M.A. & Yemshanov, D. (2013). Risks, decisions and biological conservation. Diversity and Distributions, 19, 485-489.
7. Cairns, A. & Yan, N.D. (2011). New reports of Bythotrephes, summary of methodology and database design. Tech. Rep. January, Canadian Aquatic Invading Species Network, Toronto, ON.
8. Chen, I.C., Hill, J.K., Ohlemüller, R., Roy, D.B. & Thomas, C.D. (2011). Rapid range shifts of species associated with high levels of climate warming. Science, 333, 1024-1026.
9. Clark, J.S., Carpenter, S.R., Barber, M., Collins, S., Dobson, A., Foley, J.A. et al. (2001). Ecological forecasts: an emerging imperative. Science, 293, 657-660.
10. Cressie, N., Calder, C.A., Clark, J.S., Ver Hoef, J.M. & Wikle, C.K. (2009). Accounting for uncertainty in ecological analysis: the strengths and limitations of hierarchical statistical modeling. Ecological Applications: A publication of the Ecological Society of America, 19, 553-570.
11. Drake, J.M. & Lodge, D.M. (2006). Allee effects, propagule pressure and the probability of establishment: risk analysis for biological invasions. Biological Invasions, 8, 365-375.
12. Elith, J., Phillips, S.J., Hastie, T., Dudík, M., Chee, Y.E. & Yates, C.J. (2011). A statistical explanation of MaxEnt for ecologists. Diversity and Distributions, 17, 43-57.
13. Freeman, E.A. & Moisen, G.G. (2008). A comparison of the performance of threshold criteria for binary classification in terms of predicted prevalence and kappa. Ecological Modelling, 217, 48-58.
14. Gertzen, E.L. & Leung, B. (2011). Predicting the spread of invasive species in an uncertain world: accommodating multiple vectors and gaps in temporal and spatial data for Bythotrephes longimanus. Biological Invasions, 13, 2433-2444.
15. Gertzen, E.L., Leung, B. & Yan, N.D. (2011). Propagule pressure, Allee effects and the probability of establishment of an invasive species (Bythotrephes longimanus). Ecosphere, 2, 1-17.
16. Gneiting, T. & Katzfuss, M. (2014). Probabilistic forecasting. Annual Review of Statistics and its Application, 1, 125-151.
17. Guisan, A. & Thuiller, W. (2005). Predicting species distribution: offering more than simple habitat models. Ecology Letters, 8, 993-1009.
18. Guisan, A. & Zimmermann, N.E. (2000). Predictive habitat distribution models in ecology. Ecological Modelling, 135, 147-186.
19. Heard, G.W., McCarthy, M.A., Scroggie, M.P., Baumgartner, J.B. & Parris, K.M. (2013). A Bayesian model of metapopulation viability, with application to an endangered amphibian. Diversity and Distributions, 19, 555-566.
20. Higgins, S.I., Richardson, D.M. & Cowling, R.M. (2001). Validation of a spatial simulation model of a spreading alien plant population. Journal of Applied Ecology, 38, 571-584.
21. Hirsch, R. (1991). Validation samples. Biometrics, 47, 1193-1194.
22. Hosmer, D. & Lemeshow, S. (2000). Assessing the fit of the model. Applied Logistic Regression, 2nd edn. John Wiley & Sons, Inc., New York, USA.
23. Jolliffe, I.T. (2007). Uncertainty and inference for verification measures. Weather and Forecasting, 22, 637-650.
24. Kuk, A.Y.C., Li, J. & John Rush, A. (2014). Variable and threshold selection to control predictive accuracy in logistic regression. Journal of the Royal Statistical Society: Series C (Applied Statistics), 63, 657-672.
25. Lawson, C.R., Hodgson, J.A., Wilson, R.J. & Richards, S.A. (2014). Prevalence, thresholds and the performance of presence-absence models. Methods in Ecology and Evolution, 5, 54-64.
26. Lee, T.M. & Jetz, W. (2011). Unravelling the structure of species extinction risk for predictive conservation science. Proceedings Biological Sciences/ The Royal Society, 278, 1329-1338.
27. Leung, B. & Steele, R.J. (2013). The value of a datum - how little data do we need for a quantitative risk analysis? Diversity and Distributions, 19, 617-628.
28. MacIsaac, H.J., Borbely, J.V.M., Muirhead, J.R. & Graniero, P.A. (2004). Backcasting and forecasting biological invasions of inland lakes. Ecological Applications, 14, 773-783.
29. Manel, S., Williams, H.C. & Ormerod, S. (2002). Evaluating presence-absence models in ecology: the need to account for prevalence. Journal of Applied Ecology, 38, 921-931.
30. Manly, B.F. (2001). Randomization, Bootstrap and Monte Carlo Methods in Biology, 2nd edn. CRC Press, Chapman & Hall, Boca Raton, FL.
31. Márcia Barbosa, A., Real, R., Muñoz, A.R. & Brown, J.A. (2013). New measures for assessing model equilibrium and prediction mismatch in species distribution models. Diversity and Distributions, 19, 1333-1338.
32. Mason, S. (2008). Understanding forecast verification statistics. Meteorological Applications, 15, 31-40.
33. Meyer, J., Ingersoll, C., McDonald, L. & Boyce, M. (1986). Estimating uncertainty in population growth rates: jackknife vs. bootstrap techniques. Ecology, 67, 1156-1166.
34. Miller, M.E., Hui, S.L. & Tierney, W.M. (1991). Validation techniques for logistic regression models. Statistics in Medicine, 10, 1213-1226.
35. Murphy, A. (1993). What is a good forecast? An essay on the nature of goodness in weather forecasting. Weather and Forecasting, 8, 281-293.
36. Pearce, J. & Ferrier, S. (2000). Evaluating the predictive performance of habitat models developed using logistic regression. Ecological Modelling, 133, 225-245.
37. Peterson, A.T., Papes, M. & Soberón, J. (2008). Rethinking receiver operating characteristic analysis applications in ecological niche modeling. Ecological Modelling, 213, 63-72.
38. Phillips, S.J. & Elith, J. (2010). POC plots: calibrating species distribution models with presence-only data. Ecology, 91, 2476-2484.
39. R Core Team (2013). R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.
40. Randin, C.F., Dirnböck, T., Dullinger, S., Zimmermann, N.E., Zappa, M. & Guisan, A. (2006). Are niche-based species distribution models transferable in space? Journal of Biogeography, 33, 1689-1703.
41. Schmolke, A., Thorbek, P., DeAngelis, D.L. & Grimm, V. (2010). Ecological models supporting environmental decision making: a strategy for the future. Trends in Ecology & Evolution, 25, 479-486.
42. Seebens, H., Gastner, M.T. & Blasius, B. (2013). The risk of marine bioinvasion caused by global shipping. Ecology Letters, 16, 782-790.
43. Staples, D., Taper, M. & Dennis, B. (2004). Estimating population trend and process variation for PVA in the presence of sampling error. Ecology, 85, 923-929.
44. Suter, G.W. (2006). Ecological Risk Assessment, 2nd edn. CRC Press, Taylor & Francis Group, Boca Raton, FL.
45. Václavík, T. & Meentemeyer, R.K. (2012). Equilibrium or not? Modelling potential distribution of invasive species in different stages of invasion. Diversity and Distributions, 18, 73-83.
46. Ward, G., Hastie, T., Barry, S., Elith, J. & Leathwick, J.R. (2009). Presence-only data and the em algorithm. Biometrics, 65, 554-563.
47. Wenger, S.J., Som, N.A., Dauwalter, D.C., Isaak, D.J., Neville, H.M., Luce, C.H. et al. (2013). Probabilistic accounting of uncertainty in forecasts of species distributions under climate change. Global Change Biology, 19, 3343-3354.
48. Wilks, D. (2011). Statistical Methods in the Atmospheric Sciences, Vol. 100 of International Geophysics. Academic Press, Oxford.
49. Yan, N., Dunlop, W., Pawson, T. & Mackay, L. (1992). Bythotrephes cederstroemi (Schoedler) in Muskoka lakes: first records of the European invader in inland lakes in Canada. Canadian Journal of Fisheries and Aquatic Sciences, 49, 422-426.