Impacts of imperfect reference data on the apparent accuracy of species presence-absence models and their predictions

Global Ecology and Biogeography

Volumn 20, Issue 3, 2011, Pages 498-508

  Foody, Giles M ^a  

^a UNIVERSITY OF NOTTINGHAM   (United Kingdom)

Author keywords

Accuracy; Error; Gold standard; Model; Prevalence; Sample size

Indexed keywords

ACCURACY ASSESSMENT; DATA PROCESSING; ECOLOGICAL MODELING; ECOLOGICAL THEORY; ERROR ANALYSIS; PREDICTION; SENSITIVITY ANALYSIS;

EID: 79953782022     PISSN: 1466822X     EISSN: 14668238     Source Type: Journal    
DOI: 10.1111/j.1466-8238.2010.00605.x     Document Type: Article

References (58)

1. Albert, P.S. & Dodd, L.E. (2004) A cautionary note on the robustness of latent class models for estimating diagnostic error without a gold standard. Biometrics, 60, 427-435.
2. Albert, P.S., McShane, L.M. & Shih, J.H. (2001) Latent class modeling approaches for assessing diagnostic error without a gold standard: with applications to p53 immunohistochemical assays in bladder tumors. Biometrics, 57, 610-619.
3. Allouche, O., Tsoar, A. & Kadmon, R. (2006) Assessing the accuracy of species distribution models: prevalence, kappa and the true skill statistic (TSS). Journal of Applied Ecology, 43, 1223-1232.
4. Austin, M.P. (2007) Species distribution models and ecological theory: a critical assessment and some possible new approaches. Ecological Modelling, 200, 1-19.
5. Austin, M.P., Belbin, L., Meyers, J.A., Doherty, M.D. & Luoto, M. (2006) Evaluation of statistical models used for predicting species distributions: role of artificial data and theory. Ecological Modelling, 199, 197-216.
6. Choi, Y.-K., Johnson, W.O., Collins, M.T. & Gardner, I.A. (2006) Bayesian inferences for receiver operating characteristics curves in the absence of a gold standard. Journal of Agricultural, Biological and Environmental Statistics, 11, 210-229.
7. Conlisk, E., Conlisk, J., Enquist, B., Thompson, J. & Harte, J. (2009) Improved abundance prediction from presence-absence data. Global Ecology and Biogeography, 18, 1-10.
8. Connor, R.J. (1987) Sample size for testing differences in proportions for the paired-sample design. Biometrics, 43, 207-211.
9. Cronin, A.M. & Vickers, A.J. (2008) Statistical methods to correct for verification bias in diagnostic studies are inadequate when there are few false negatives: a simulation study. BMC Medical Research Methodology, 8, 75, doi:10.1186/1471-2288-8-75.
10. Enøe, C., Georgiadis, M.P. & Johnson, W.O. (2000) Estimation of sensitivity and specificity of diagnostic tests and disease prevalence when the true disease state is unknown. Preventive Veterinary Medicine, 45, 61-81.
11. Enøe, C., Anderson, S., Sørensen, V. & Willeberg, P. (2001) Estimation of sensitivity, specificity and predictive values of two serologic tests for the detection of antibodies against Actinobacillus pleuropneumoniae serotype 2 in the absence of a reference test (gold standard). Preventive Veterinary Medicine, 51, 227-243.
12. Espeland, M.A. & Handelman, S.L. (1989) Using latent class models to characterize and assess relative error in discrete measurements. Biometrics, 45, 587-599.
13. Farber, O. & Kadmon, R. (2003) Assessment of alternative approaches for bioclimatic modelling with special emphasis on the Mahalanobis distance. Ecological Modelling, 160, 115-130.
14. Fielding, A.H. & Bell, J.F. (1997) A review of methods for the assessment of prediction errors in conservation presence/absence models. Environmental Conservation, 24, 38-49.
15. Foody, G.M. (2008a) Harshness in image classification accuracy assessment. International Journal of Remote Sensing, 29, 3137-3158.
16. Foody, G.M. (2008b) Refining predictions of climate change impacts on plant species distribution through the use of local statistics. Ecological Informatics, 3, 228-236.
17. Foody, G.M. (2009a) Sample size determination for image classification accuracy assessment and comparison. International Journal of Remote Sensing, 30, 5273-5291.
18. Foody, G.M. (2009b) Classification accuracy comparison: hypothesis tests and the use of confidence intervals in evaluations of difference, equivalence and non-inferiority. Remote Sensing of Environment, 113, 1658-1663.
19. Foody, G.M. (2010) Assessing the accuracy of land cover change with imperfect ground reference data. Remote Sensing of Environment, 114, 2271-2285.
20. Franklin, J., Wejnert, K.E., Hathaway, S.A., Rochester, C.J. & Fisher, R.N. (2009) Effect of species rarity on the accuracy of species distribution models for reptiles and amphibians in southern California. Diversity and Distributions, 15, 167-177.
21. Freeman, E.A. & Moisen, G.G. (2008) A comparison of the performance of threshold criteria for binary classification in terms of predicted prevalence and kappa. Ecological Modelling, 217, 48-58.
22. Gart, J.J. & Buck, A.A. (1966) Comparison of a screening test and a reference test in epidemiologic studies: II a probabilistic model for the comparison of diagnostic tests. American Journal of Epidemiology, 83, 593-602.
23. Graham, C.H., Elith, J., Hijmans, R.J., Guisan, A., Peterson, A.T., Loiselle, B.A. & the NCEES Predicting Species Distributions Working Group (2008) The influence of spatial errors in species occurrence data used in distribution models. Journal of Applied Ecology, 45, 239-247.
24. Guisan, A., Lehmann, A., Ferrier, S., Austin, M., Overton, J.M.C.C., Aspinall, R. & Hastie, T. (2006) Making better biogeographical predictions of species' distributions. Journal of Applied Ecology, 43, 386-392.
25. Hartel, T., Öllerer, K., Farczády, L., Monga, C.I. & Bǎncilǎ, R. (2009) Using species detectability to infer distribution, habitat use and absence of a cryptic species: the smooth snake (Coronella autriaca) in Saxon Transylvania. Acta Scientiarum Transylvanica, 17, 61-76.
26. Hoehler, F.K. (2000) Bias and prevalence effects on kappa viewed in terms of sensitivity and specificity. Journal of Clinical Epidemiology, 53, 499-503.
27. Hoenig, J.M. & Heisey, D.M. (2001) The abuse of power: the pervasive fallacy of power calculations for data analysis. The American Statistician, 55, 1-6.
28. Hortal, J., Jiménez-Valverde, A., Gómez, J.F., Lobo, J.M. & Baselga, A. (2008) Historical bias in biodiversity inventories affects the observed environmental niche of the species. Oikos, 117, 847-858.
29. Jiménez-Valverde, A. & Lobo, J.M. (2007) Threshold criteria for conversion of probability of species presence to either-or presence-absence. Acta Oecologica, 31, 361-369.
30. Johnson, C.J. & Gillingham, M.P. (2008) Sensitivity of species-distribution models to error, bias and model design: an application to resource selection functions for woodland caribou. Ecological Modelling, 213, 143-155.
31. Joseph, L.N., Field, S.A., Wilcox, C. & Possingham, H.P. (2006) Presence-absence versus abundance data for monitoring threatened species. Conservation Biology, 20, 1679-1687.
32. Kazmierczak, S.C. (1999) Statistical techniques for evaluating the diagnostic utility of laboratory tests. Clinical Chemistry and Laboratory Medicine, 37, 1001-1009.
33. Liu, C., White, M. & Newell, G. (2009) Measuring the accuracy of species distribution models: a review. Proceedings of the 18th World IMACs/MODSIM Congress, pp. 4241-4247. Modelling and Simulation Society of Australia and New Zealand and International Association for Mathematics and Computers in Simulation, Cairns, Australia.
34. Lobo, J.M. (2008) More complex distribution models or more representative data? Biodiversity Informatics, 5, 14-19.
35. Lobo, J.M., Jiménez-Valverde, A. & Real, R. (2008) AUC: a misleading measure of the performance of predictive distribution models. Global Ecology and Biogeography, 17, 145-151.
36. MacKenzie, D.L. (2005) Was it there? Dealing with imperfect detection for species presence/absence data? Australian and New Zealand Journal of Statistics, 47, 65-74.
37. McPherson, J.M., Jetz, W. & Rogers, D.J. (2004) The effects of species' range sizes on the accuracy of distribution models: ecological phenomenon or statistical artefact? Journal of Applied Ecology, 41, 811-823.
38. Manel, S., Williams, H.C. & Ormerod, S.J. (2001) Evaluating presence-absence models in ecology: the need to account for prevalence. Journal of Applied Ecology, 38, 921-931.
39. Messam, L.L.M.V., Branscum, A.J., Collins, M.T. & Gardner, I.A. (2008) Frequentist and Bayesian approaches to prevalence estimation using examples from Johne's disease. Animal Health Research Reviews, 9, 1-23.
40. Miettinen, O.S. (1968) The matched pairs design in the case of all-or-none responses. Biometrics, 24, 339-352.
41. Osborne, P.E. & Leitão, P.J. (2009) Effects of species and habitat positional errors on the performance and interpretation of species distribution models. Diversity and Distributions, 15, 671-681.
42. Parviainen, M., Marmion, M., Luoto, M., Thuiller, W. & Heikkinen, R.K. (2009) Using summed individual species models and state-of-the-art modelling techniques to identify threatened plant species hotspots. Biological Conservation, 142, 2501-2509.
43. Pepe, M.S. & Janes, H. (2007) Insights into latent class analysis of diagnostic test performance. Biostatistics, 8, 474-484.
44. Qu, Y., Tan, M. & Kutner, M.H. (1996) Random effects models in latent class analysis for evaluating accuracy of diagnostic tests. Biometrics, 52, 797-810.
45. Rahme, E. & Joseph, L. (1998) Estimating the prevalence of a rare disease: adjusted maximum likelihood. The Statistician, 47, 149-158.
46. Real, R., Barbosa, A.M. & Vargas, J.M. (2006) Obtaining environmental favourability functions from logistic regression. Environmental and Ecological Statistics, 13, 237-245.
47. Ringvall, A., Peterson, H., Ståhl, G. & Lämås, T. (2005) Surveyor consistency in presence/absence sampling for monitoring vegetation in a boreal forest. Forest Ecology and Management, 212, 109-117.
48. Staquet, M., Rozencweig, M., Lee, Y.J. & Muggia, F.M. (1981) Methodology for the assessment of new dichotomous diagnostic tests. Journal of Chronic Diseases, 34, 599-610.
49. Stockwell, D.R.B. & Peterson, A.T. (2002) Effects of sample size on accuracy of species distribution models. Ecological Modelling, 148, 1-13.
50. Strayer, D.L. (1999) Statistical power of presence-absence data to detect population declines. Conservation Biology, 13, 1034-1038.
51. Trout, A.T., Kaufmann, T.J. & Kallmes, D.F. (2007) No significant difference...says who? American Journal of Neuroradiology, 28, 195-197.
52. Turner, W.R. (2006) Interactions among spatial scales constrain species distributions in fragmented urban landscapes. Ecology and Society, 11, 6. Available at: (accessed 8 September 2010).
53. Valenstein, P.N. (1990) Evaluating diagnostic tests with imperfect standards. American Journal of Clinical Pathology, 93, 252-258.
54. Vermunt, J.K. (1997) LEM: a general program for the analysis of categorical data. Department of Methodology and Statistics, Tilburg University, Tilburg, The Netherlands.
55. Williams, R.A.J. & Peterson, A.T. (2009) Ecology and geography of avian influenza (HPA1 H5N1) transmission in the Middle East and northeastern Africa. International Journal of Health Geographics, 8, 47, doi:10.1186/1476-072X-8-47.
56. Wisz, M.S., Hijmans, R.J., Peterson, A.T., Graham, C.H., Guisan, A. & NCEAS Predicting Species Distributions Working Group (2008) Effects of sample size on the performance of species distribution models. Diversity and Distributions, 14, 763-773.
57. Yang, I. & Becker, M.P. (1997) Latent variable modelling of diagnostic accuracy. Biometrics, 53, 948-958.
58. Zedaker, S.M., Gregoire, T.G. & Miller, J.H. (1994) Sample-size needs for forestry herbicide trials. Canadian Journal of Forest Research, 23, 2153-2158.