Statistical solutions for error and bias in global citizen science datasets

Biological Conservation

Volumn 173, Issue , 2014, Pages 144-154

  Bird, Tomas J ^a,b   Bates, Amanda E ^b   Lefcheck, Jonathan S ^c   Hill, Nicole A ^b   Thomson, Russell J ^b   Edgar, Graham J ^b   Stuart Smith, Rick D ^b   Wotherspoon, Simon ^b   Krkosek, Martin ^d   Stuart Smith, Jemina F ^b   Pecl, Gretta T ^b   Barrett, Neville ^b   Frusher, Stewart ^b  

^a UNIVERSITY OF MELBOURNE   (Australia)

^b UNIVERSITY OF TASMANIA   (Australia)

^c VIRGINIA INSTITUTE OF MARINE SCIENCE   (United States)

^d UNIVERSITY OF TORONTO   (Canada)

Author keywords

Additive models; Biodiversity; Experimental design; Linear models; Reef life survey; Species distribution models; Statistical analysis; Volunteer data

Indexed keywords

EID: 84899787240     PISSN: 00063207     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.biocon.2013.07.037     Document Type: Article

References (88)

1. Ahrends A., Fanning E., Rahbek C., Burgess N.D., Gereau R.E., Marchant R., Bulling M.T., Lovett J.C., Platts P.J., Wilkins Kindemba V., Owen N. Funding begets biodiversity. Divers. Distrib. 2011, 17:191-200.
2. Alroy J. Geographical, environmental and intrinsic biotic controls on Phanerozoic marine diversification. Palaeontology 2010, 53(6):1211-1235.
3. Amano T., Okamura H., Carrizo S.F., Sutherland W.J. Hierarchical models for smoothed population indices: the importance of considering variations in trends of count data among sites. Ecol. Ind. 2012, 13:243-252.
4. Arab A., Wildhaber M.L., Wikle C.K., Gentry C.N. Zero-inflated modeling of fish catch per unit area resulting from multiple gears: application to channel catfish and shovelnose sturgeon in the Missouri River. N. Am. J. Fish. Manage. 2008, 28(4):1044-1058.
5. Ashcroft M.B., Gollan J.R., Batley M. Combining citizen science, bioclimatic envelope models and observed habitat preferences to determine the distribution of an inconspicuous, recently detected introduced bee (Halictus smaragdulus Vachal Hymenoptera: Halictidae) in Australia. Biol. Invasions 2012, 14:515-527.
6. Barbet-Massin M., Jiguet F., Elbert C.H., Thuiller W. Selecting pseudo-absences for species distribution models: how, where and how many?. Methods Ecol. Evol. 2012, 3:327-338.
7. Boakes E.H., McGowan P.J.K., Fuller R.A., Chang-qing D., Clark N.E., O'Connor K., Mace G.M. Distorted views of biodiversity: spatial and temporal bias in species occurrence data. PLoS Biol. 2010, 8:e1000385.
8. Bolker B.M., Brooks M.E., Clark M., Connie J., Geange S.W., Poulsen J.R., Stevens H.H., White J.-S.S. Review: generalized linear mixed models: a practical guide for ecology and evolution. Trends Ecol. Evol. 2009, 24:127-135.
9. Bonney R., Cooper C.B., Dickinson J., Kelling S., Phillips T., Rosenberg K.V., Shirk J. Citizen science: a developing tool for expanding science knowledge and scientific literacy. Bioscience 2009, 59:977-984.
10. Bonter N.D., Cooper C.B. Data validation in citizen science: a case study from project FeederWatch. Front. Ecol. Environ. 2012, 10(6):305-307.
11. Bray B.S., Schramm J.L. Evaluation of a statewide volunteer angler diary program for use as a fishery assessment tool. N. Am. J. Fish. Manag. 2001, 21(3):606-615.
12. Brunsdon C., Comber L. Assessing the changing flowering date of the common lilac in North America: a random coefficient model approach. Geoinformatica 2012, 16:675-690.
13. Busby J.R. BIOCLIM - A bioclimate analysis and prediction system. Nature conservation: cost effective biological surveys and data analysis. Nature Conservation 1991, 64-68. CSIRO, Melbourne. C.R. Margules, M.P. Austin (Eds.).
14. Butt N., Slade E., Thompson J., Malhi Y., Riutta T. Quantifying the sampling error in tree census measurements by volunteers and its effect on carbon stock estimates. Ecol. Appl. 2013, 23(4):936-943.
15. Cadotte M.W., Davies J.T., Regetz J., Kembel S.W., Cleland E., Oakley T.H. Phylogenetic diversity metrics for ecological communities: integrating species richness, abundance and evolutionary history. Ecol. Lett. 2010, 13(1):96-105.
16. Chao A., Jost L. Coverage-based rarefaction and extrapolation: standardizing samples by completeness rather than size. Ecology 2012, 93:2533-2547.
17. Chao A., Gotelli N., Hsieh T.C., Sander E., Ma K.H., Colwell R.K., Ellison A.M. Rarefaction and extrapolation with Hill numbers: a framework for sampling and estimation in species diversity studies. Ecol. Monogr. 2013.
18. Comber A., See L., Fritz S., Van der Velde M., Perger C., Foody G. Using control data to determine the reliability of volunteered geographic information about land cover. Int. J. Appl. Earth Obs. Geoinf. 2013, 23:37-48.
19. Conn, P., McClintock, B.T., Cameron, M., Johnson, D.S., Moreland, E., Boveng, P.L., in press. Accomodating species identification errors in transect surveys. Ecology.
20. Cox T.E., Philippoff J., Baumgartner E., Smith C.M. Expert variability provides perspectives on the strengths and weaknesses of citizen-driven intertidal monitoring program. Ecol. Appl. 2012, 22(4):1201-1212.
21. Crall A.W., Newman G., Stohlgren T.J., Holfelder K.A., Graham J., Waller D.M. Assessing citizen science data quality: a case study. Conserv. Lett. 2011, 4:433-442.
22. Cutler A., Cutler D.R., Edwards T.C., Beard K.H., Hess K.T., Gibson J., Lawler J.J. Random forests for classification in ecology. Ecology 2007, 88:2783-2792.
23. De'ath G., Fabricius K.E. Classification and regression trees: a powerful yet simple technique for ecological data analysis. Ecology 2000, 81:3178-3192.
24. Delaney D., Sperling C., Adams C., Leung B. Marine invasive species: validation of citizen science and implications for national monitoring networks. Biol. Invasions 2008, 10:117-128.
25. deSolla S.R., Shirose L.J., Fernie K.J., Barrett G.C., Brousseau C.S., Bishop C.A. Effect of sampling effort and species detectability on volunteer based anuran monitoring programs. Biol. Conserv. 2005, 121:585-594.
26. Dickinson J.L., Zuckerberg B., Bonter D.N. Citizen science as an ecological research tool: challenges and benefits. An. Ecol., Evol. Syst. 2010, 41:149-172.
27. Dormann F.D., McPherson J.M., Araújo M.B., Bivand R., Bolliger J., Gudrun C., Davies R.G., Hirzel A., Jetz W., Kissling D., Kühn I., Ohlemü ller R., Peres-Neto P.R., Reineking B., Schröder B. Methods to account for spatial autocorrelation in the analysis of species distribution data: a review. Ecography 2007, 30:609-628.
28. Dudík M., Phillips S.J., Schapire R.E. Correcting sample selection bias in maximum entropy density estimation. Advances in Neural Information Processing Systems 2005, 323-330. MIT Press, Cambridge, Massachusetts, USA.
29. Edgar G.J., Stuart-Smith R.D. Ecological effects of marine protected areas on rocky reef communities; a continental-scale analysis. Mar. Ecol. Prog. Ser. 2009, 388:51-62.
30. Edgar G.J., Barrett N.S., Morton A.J. Biases associated with the use of underwater visual census techniques to quantify the density and size-structure of fish populations. J. Exp. Mar. Biol. Ecol. 2004, 308(2):269-290.
31. Elith J., Lehmann A., Li J., Lohmann L.G., Loiselle B.A., Manion G., Moritz C., Nakamura M., Nakazawa Y., Peterson A.T., Phillips S.J., Graham C.H., Richardson K., Scachetti-Pereira R., Schapire R.E., Soberón J., Williams S., Wisz M.S., Zimmermann N.E., Araujo M., Anderson R.P., Dudík M., Ferrier S., Guisan A., Hijmans R.J., Huettmann F., Leathwick J.R. Novel methods improve prediction of species' distributions from occurrence data. Ecography 2006, 29:129-151.
32. Elith J., Leathwick J.R., Hastie T. A working guide to boosted regression trees. J. Anim. Ecol. 2008, 802.
33. Farmer R.G., Leonard M.L., Horn A.G. Observer effects and avian-call-count survey quality: rare-species biases and overconfidence/effets des observateurs et qualite des inventaires par le denombrement des chants: biais sur les especes rares et exces de confiance. Auk 2012, 129:76.
34. Ferrier S., Guisan A. Spatial modelling of biodiversity at the community level. J. Appl. Ecol. 2006, 43:393-404.
35. Fewster R.M., Buckland S.T., Siriwardena G.M., Baillie S.R., Wilson J.D. Analysis of population trends for farmland birds using generalized additive models. Ecology 2000, 81:1970-1984.
36. Fink D., Hochachka W.M. Using data mining to discover biological patterns in citizen science observations. Citizen Science: Public Participation in Environmental Research 2012, Comstock Publishing Associates. J.L. Dickinson, R. Bonney (Eds.).
37. Fink D., Hochachka W.M., Zuckerberg B., Winkler D.W., Shaby B., Munson M.A., Hooker G., Riedewald M., Sheldon D., Kelling S. Spatiotemporal exploratory models for broad-scale survey data. Ecol. Appl. 2010, 20:2131-2147.
38. Franklin J. Mapping Species Distributions 2009, Cambridge University Press, New York.
39. Gelfand A.E., Schmidt A.M., Wu S., Silander J.A., Latimer A., Rebelo A.G. Modelling species diversity through species level hierarchical modelling. J. Roy. Stat. Soc.: Ser. C (Appl. Stat.) 2005, 54:1-20.
40. Gotelli N.J., Colwell R.J. Quantifying biodiversity: procedures and pitfalls in the measurement and comparison of species richness. Ecol. Lett. 2001, 4:379-391.
41. Hastie T., Tibshirani R.J. Generalized Additive Models 1990, Chapman and Hall, London.
42. Hedges L.V., Tipton E., Johnson M.C. Robust variance estimation in meta-regression with dependent effect size estimates. Res. Synth. Meth. 2010, 1:39-65.
43. Hochachka W.M., Fink D., Hutchinson R.A., Sheldon D., Wong W.-K., Kelling S. Data-intensive science applied to broad-scale citizen science. Trends Ecol. Evol. (Pers. Ed.) 2012, 27:130-137.
44. Holt B.G., Rioja-Nieto R., MacNeil M.A., Lupton J., Rahbek C. Comparing diversity data collected using a protocol designed for volunteers with results from a professional alternative. Methods Ecol. Evol. 2013, 4(4):383-392.
45. Karpievitch, Y.V., Hill, E.G., Leclerc, A.P., Dabney, A.R., Almeida, J.S., 2009. An introspective comparison of random forest-based classifiers for the analysis of cluster-correlated data by way of RF++. PLoS One. 2009 Sep 18, 4:9.
46. Kery M., Gardner B., Monnerat C. Predicting species distributions from checklist data using site-occupancy models. J. Biogeogr. 2010, 37:1851-1862.
47. Kery M., Royle J.A., Schmid H., Schaub M., Volet B., Hafliger G., Zbinden N. Site-occupancy distribution modeling to correct population-trend estimates derived from opportunistic observations. Conserv. Biol. 2010, 24:1388-1397.
48. Latimer A.M., Wu S., Gelfand A.E., Silander J.J.A. Building statistical models to analyze species distributions. Ecol. Appl. 2006, 16:33-50.
49. Lawler J.L., O'Connor R.R. How well do consistently monitoried breeding bird survey routes represent the environments of the conterminous United States?. Condor 2004, 106:801-814.
50. Legendre P., Fortin M.-J., Gurevitch J., Hohn M., Myers D. The consequences of spatial structure for the design and analysis of ecological field surveys. Ecography 2002, 25:601-615.
51. Lunn D.J., Thomas T., Best N., Spiegelhalter D. WinBUGS - a Bayesian modeling framework: concepts, structure, and extensibility. Stat. Comput. 2000, 10(4):325-337.
52. MacKenzie D.I. Occupancy Estimation and Modeling: Inferring Patterns and Dynamics of Species Occurrence 2006, Elsevier, Burlington, MA.
53. Mateo R.G., Felicisímo Á.M., Muñoz J. Effects of the number of presences on reliability and stability of MARS species distribution models: the importance of regional niche variation and ecological heterogeneity. J. Veg. Sci. 2010, 21:908-922.
54. Maunder M.N., Punt A.E. Standardizing catch and effort data: a review of recent approaches. Fish. Res. 2004, 70:141-159.
55. Miller D.A., Nichols J.D., McClintock B.T., Grant E.H.C., Bailey L.L., Weir L.A. Improving occupancy estimation when two types of observational error occur: non-detection and species misidentification. Ecology 2011, 92(7):1422-1428.
56. Monk J. How long should we ignore imperfect detection of species in the marine environment when modelling their distribution?. Fish Fisheries 2013, 1467-2979.
57. Olden Julian D., Lawler Joshua J., Poff N.L. Machine learning methods without tears: a primer for ecologists. Quart. Rev. Biol 2008, 171.
58. Parsons B., Short J., Roberts D.D. Using community observations to predict occurrence of malleefowl (Leipoa ocellata) in the Western Australian wheatbelt. Biol. Conserv. 2009, 142:364-374.
59. Pattengill-Semmens C.V., Semmens B.X. Conservation and management applications of the reef volunteer fish monitoring program. Environ. Monit. Assess. 2003, 81:43-50.
60. Pearce J.L., Boyce M.S. Modelling distribution and abundance with presence-only data. J. Appl. Ecol. 2006, 43(3):405-412.
61. Petchey O.L., Gaston K.J. Functional diversity: back to basics and looking forward. Ecol. Lett. 2006, 9:741-758.
62. Phillips S.J., Elith J. On estimating probability of presence from use-availability or presence-background data. Ecology 2013, 94(6):1409-1419.
63. Phillips S.J., Anderson R.P., Schapired R.E. Maximum entropy modeling of species geographic distributions. Ecol. Model. 2006, 190:231-259.
64. Phillips S.J., Dudík M., Elith J., Graham C.H., Lehmann A., Leathwick J., Ferrier S. Sample selection bias for presence-only distribution models: implications for background and pseudo-absence data. Ecol. Appl. 2009, 19(1):181-197.
65. R Core Team, 2013. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0. http://www.R-project.org/.
66. Royle J.A., Dorazio R.M. Hierarchical Modeling and Inference in Ecology: the Analysis of Data from Populations, Metapopulations and Communities 2008, Academic, Amsterdam, Boston. first ed.
67. Royle J.A., Kéry M., Gautier R., Schmid H. Hierarchical spatial models of abundance and occurrence from imperfect survey data. Ecol. Monogr. 2007, 77(3):465-481.
68. Royle J.A., Chandler R.B., Yackulic R.B., Nichols J.D. Likelihood analysis of species occurrence probability from presence-only data for modelling species distributions. Methods Ecol. Evol. 2012, 3(3):545-554.
69. Sarda-Palomera F., Brotons L., Villero D., Sierdsema H., Newson S.E., Jiguet F. Mapping from heterogeneous biodiversity monitoring data sources. Biodivers. Conserv. 2012, 21:2927-2948.
70. Segurado P., Araújo M.B., Kunin W.E. Consequences of spatial autocorrelation for niche-based models. J. Appl. Ecol. 2006, 43:433-444.
71. Sela S.J., Simnoff J.S. RE-EM trees: a data mining approach for longitudinal and clustered data. Mach. Learn. 2012, 86:169-207.
72. Seys J., Offringa H., Van Waeyenberge J., Meire P., Kuijken E. An evaluation of beached bird monitoring approaches. Mar. Pollut. Bull. 2002, 44:322-333.
73. Silvertown J. A new dawn for citizen science. Trends Ecol. Evol. (Pers. Ed.) 2009, 24:467-471.
74. Snäll T., Kindvall O., Nilsson J., Pärt T. Evaluating citizen-based presence data for bird monitoring. Biol. Conserv. 2011, 144:804-810.
75. Stuart-Smith, R.D., Bates, A.E., Lefcheck, J.S., Duffy, J.E., Baker, S.C., Thomson, R.J., Stuart-Smith, J.F., Hill, N.A., Kininmonth, S.J., Airoldi, L., Becerro, M.A., Campbell, S.J., Dawson, T.P., Navarrete, S.A., Soler, G., Strain, E.M.A., Willis, T.J., Edgar, G.J., in press. Integrating abundance and functional traits reveals new global hotspots of fish diversity. Nature.
76. Sullivan B.L., Wood C.L., Iliff M.J., Bonney R.E., Fink D., Kelling S. EBird: a citizen-based bird observation network in the biological sciences. Biol. Conserv. 2009, 142:2282-2292.
77. Sunde P., Jesson L. It counts who counts: an experimental evaluation of the importance of observer effects on spotlight count estimates. Eur. J. Wildl. Res. 2013, 1612-4642:1-9.
78. Tulloch A.I.T., Szabo J.T. A behavioral ecology approach to understand volunteer surveying for citizen science datasets. Emu 2012, 112:313-335.
79. Tulloch A.I.T., Possingham H.P., Joseph L.N., Szabo J., Martin T.G. Realizing the full potential of citizen science monitoring schemes. Biol. Conserv. 2013, 165:128-138.
80. Walker P.A., Cocks K.D. HABITAT: a procedure for modelling a disjoint environmental envelope for a plant or animal species. Global Ecol. Biogeogr. Lett. 1991, 1:108-118.
81. Welham S., Cullis B., Gogel B., Gilmour A., Thompson R. Prediction in linear models. Aust. NZ J. Stat. 2004, 46(3):325-347.
82. Wiggins, A., Crowston, K., 2011. From conservation to crowd sourcing: a typology of citizen science. In: System Sciences (HICSS), 2011 44th Hawaii International Conference on, pp. 1-10.
83. Wiggins, A., Newman, G., Stevenson, R.D., Crowston, K., 2011. Mechanisms for data quality and validation in citizen science. In: e-Science Workshops (eScienceW), 2011 IEEE Seventh International Conference on, pp. 14-19.
84. Wikle C.K. Hierarchical Bayesian models for predicting the spread of ecological processes. Ecology 2003, 84(6):1382-1394.
85. Yackulic C.B., Chandler R., Zipkin E.F., Royle J.A., Nichols J.D., Campbell Grant E.H., Veran S. Presence-only modelling using MAXENT: when can we trust the inferences?. Methods Ecol. Evol. 2013.
86. Zuur A.F. Mixed Effects Models and Extensions in Ecology with R 2009, Springer New York, New York.
87. Zuur A.F., Smith G.M., Ieno E.N. Analysing Ecological Data 2007, Springer, New York.
88. Zuur A.F., Elena N.I., Elphick C.S. A protocol for data exploration to avoid common statistical problems. Methods Ecol. Evol. 2010, 1(1):3-14.