Elicitation by design in ecology: Using expert opinion to inform priors for Bayesian statistical models

Ecology

Volumn 90, Issue 1, 2009, Pages 265-277

  Choy, Samantha Low ^a   O'Leary, Rebecca ^a   Mengersen, Kerrie ^a  

^a QUEENSLAND UNIVERSITY OF TECHNOLOGY   (Australia)

Author keywords

Design; Ecology; Expert elicitation; Framework; informative Bayesian analysis; Prior information; Protocol

Indexed keywords

BAYESIAN ANALYSIS; DATA ACQUISITION; ECOLOGICAL MODELING; LEARNING; NUMERICAL MODEL;

ANIMAL; ARTICLE; BAYES THEOREM; BIOLOGICAL MODEL; DEMOGRAPHY; ECOLOGY; ENVIRONMENTAL MONITORING; EXPERT WITNESS; MARSUPIAL; METHODOLOGY; PHYSIOLOGY; STATISTICAL MODEL;

ANIMALS; BAYES THEOREM; DEMOGRAPHY; ECOLOGY; ENVIRONMENTAL MONITORING; EXPERT TESTIMONY; MACROPODIDAE; MODELS, BIOLOGICAL; MODELS, STATISTICAL;

EID: 63849151270     PISSN: 00129658     EISSN: None     Source Type: Journal    
DOI: 10.1890/07-1886.1     Document Type: Article

References (98)

1. Accad, A., S. Low Choy, D. Pullar, and W. Rochester. 2005. Bioregional classification via model-based clustering. Pages 1326-1332 in A. Zerger and R. M. Argent, editors. MODSIM 2005 International Congress on Modelling and Simulation. Modelling and Simulation Society of Australia and New Zealand. Canberra, Australia.
2. Al-Awadhi, S. A., and P. H. Garthwaite. 2006. Quantifying expert opinion for modelling fauna habitat distributions. Computational Statistics 21:121-140.
3. Alho, J. M., J. Kangas, and O. Kolehmainen. 1996. Uncertainty in expert predictions of the ecological consequences of forest plans. Applied Statistics 45:1-14.
4. Anholt. B. R., E. Werner, and D. K. Skelly. 2000. Effect of food and predators on the activity of four larval ranid frogs. Ecology 81:3509-3521.
5. Bernardo, J. 2006. Objective Bayesian statistics: an introduction to Bayesian reference analysis. Postgraduate Tutorial Seminar. Valencia 8/1SBA 2006 World Meeting on Bayesian Statistics, Valencia, Spain.
6. Borsuk, M. E. 2004. Predictive assessment of fish health and fish kills in the Neuse river estuary using elicited expert judgment. Human and Ecological Risk Assessment 10(2): 415-434.
7. Bousquet, N. 2008. Diagnostics of prior-data agreement in applied Bayesian analysis. Journal of Applied Statistics 35(9): 1011-1029.
8. Box, G., and G. Tiao. 1973. Bayesian inference in statistical analysis. Wiley Publishing, New York, New York, USA.
9. Boyce, M. S., L. L. Irwin, and R. Barker. 2005. Demographic meta-analysis: synthesizing vital rates for Spotted Owls. Journal of Applied Ecology 42:38-49.
10. Burgman, M. 2004. Expert frailties in conservation risk assessment and listing decisions. Pages 20-29 in D. L. P. Hutchings and C. Dickman. editors. Threatened species legislation: is it just an Act? Royal Zoological Society of New South Wales. Mosman, NSW, Australia.
11. Chao, C. T. 2003. Markov chain Monte Carlo on optimal adaptive sampling selections. Environmental and Ecological Statistics 10(1):129 - 151.
12. Cheeseman, P., and J. Stutz. 1996. Bayesian classification (autoclass): theory and results. Pages 153-180 in U. M. Fayyad, G. Piatetsky-Shapiro, P. Smith, and R. Uthurusamy, editors. Advances in knowledge discovery and data mining. MIT Press, Cambridge, Massachusetts, USA.
13. Chen, M. H., J. G. Ibrahim, and C. Yiannoutsos. 1999. Prior elicitation, variable selection and Bayesian computation for logistic regression models. Journal of the Royal Statistical Society, Series B (Methodological) 61:223-242.
14. Cheng, K. F., and J. H. Chen. 2005. Bayesian models for population-based case-control studies when the population is in Hardy Weinberg equilibrium. Genetic Epidemiology 28: 183-192.
15. Clark, J. S., G. Ferraz, N. Oguge, H. Hays, and J. DiCostanzo. 2005. Hierarchical Bayes for structured, variable populations: from recapture data to life-history prediction. Ecology 86:2232-2244.
16. Clark, K. E. J. E. Applegate, L. J. Niles, and D. S. Dobkin. 2006. An objective means of species status assessment: adapting the Delphi technique. Wildlife Society Bulletin 34: 419-425.
17. Clemen, R. T., and T. Reilly. 2001. Making hard decisions with decision tools. Duxbury; Thompson Learning, Pacific Grove, California, USA.
18. Clemen, R. T., and R. L. Winkler. 1999. Combining probability distributions from experts in risk analysis. Risk Analysis 19(2):187-203.
19. Clyde, M. A. 1999. Comment on Bayesian model averaging: a tutorial by Höeting et al. Statistical Science 14:382-417.
20. Cox. D. R. 2000. Comments on paper by Lindley. Statistician 49(3):321-324.
21. Craig, P. S., M. Goldstein, A. H. Seheult, and J. A. Smith. 1998. Constructing partial prior specifications for models of complex physical systems. Statistician 47:37-53.
22. Crome, F. H. J., M. R. Thomas, and L. A. Moore. 1996. A novel Bayesian approach to assessing impacts of rain forest logging. Ecological Applications 6:1104-1123.
23. Cummings, C. L., H. M. Alexander, A. A. Snow, L. H. Rieseberg, M. J. Kim. and T. M. Culley. 2002. Fecundity selection in a sunflower crop-wild study: can ecological data predict crop allele changes? Ecological Applications 12:1661-1671.
24. Dawid, A. P. 1982. The well-calibrated Bayesian (with discussion). Journal of the American Statistical Association 77(379):605-610.
25. Dawid, A. P. 2004. Probability, causality and the empirical world: a Bayes-de Finetti-Popper-Borel synthesis. Statistical Science 19(1):44-57.
26. Denham, R., and K. Mengersen. 2007. Geographically assisted elicitation of expert opinion for regression models. Bayesian Analysis 2(1):99-136.
27. Dickey, J. M., and T. J. Jiang. 1998. Filtered-variate prior distributions for histogram smoothing. Journal of the American Statistical Association 93:651-662.
28. Diebolt, J., and C. P. Robert. 1994. Estimation of finite mixture distributions through Bayesian sampling. Journal of the Royal Statistical Society, Series B. Methodological 56(2):363-375.
29. Ellison, A. 1996. An introduction to Bayesian inference for ecological research and environmental decision making. Ecological Applications 6:1036-1046.
30. Ellison, A. M. 2004. Bayesian inference in ecology. Ecology-Letters 7:509-520.
31. Eyre, T., A. Kelly, and V. Neldner. 2006. Biocondition: a terrestrial vegetation condition assessment tool for biodiversity in Queensland, field assessment manual. Version 1.5. Technical report. Environmental Protection Agency, Brisbane, Australia.
32. Ferrier, S., G. Watson, J. Pearce, and M. Drielsma. 2002. Extended statistical approaches to modelling spatial pattern in biodiversity in northeast New South Wales. I. Species-level modelling. Biodiversity and Conservation 11(12):2275 - 2307.
33. Fleischman, S. J., and D. L. Burwen. 2003. Mixture models for the species apportionment of hydroacoustic data, with echo-envelope length as the discriminatory variable. ICES Journal of Marine Science 60:592-598.
34. Fleishman, E., R. M. Nally, J. P. Fay, and D. D. Murphy. 2001. Modeling and predicting species occurrence using broad-scale environmental variables: an example with butterflies of the great basin. Conservation Biology 15(6): 1674-1685.
35. Fraley, C., and A. Raftery. 1999. Mclust: software for model-based cluster analysis. Journal of Classification 16:297-306.
36. Fuentes, M., T. G. F. Kittel, and D. Nychka. 2006, Sensitivity of ecological models to their climate drivers: statistical ensembles for forcing. Ecological Applications 16:99-116.
37. Garthwaite, P. H., and J. M. Dickey. 1985. Double and single-bisection methods for subjective probability assessments in a location-scale family. Journal of Econometrics 29:149-163.
38. Garthwaite, P. H., and A. O'Hagan. 2000. Quantifying expert opinion in the UK water industry: an experimental study. Statistician 49:455-477.
39. Gavasakar. U. 1988. A comparison of two elicitation methods for a prior distribution for a binomial parameter. Management Science 34:784-790.
40. Gelman, A. 2006. Prior distributions for variance parameters in hierarchical models. Bavesian Analysis 1(3):515-533.
41. Gelman, A., J. B. Carlin, H. S. Sterm and D. B. Rubin. 2004. Bayesian data analysis. Second edition. Chapman and Hall/CRC, Boca Raton, Florida, USA.
42. Gneiting, T., and A. E. Raftery. 2007. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association 102(477):359-378.
43. Hahn, E. D. 2006. Re-examining informative prior elicitation through the lens of Markov chain Monte Carlo methods. Journal of the Royal Statistical Society, Series A 169:37-48.
44. Hall, M. J., A. W. Minns, and A. K. M. Ashrafuzzaman. 2002. The application of data mining techniques for the regionalisation of hydrological variables. Hydrology and Earth System Sciences 6(4):685-694.
45. Hamilton, G., C. Alston, T. Chiffings, E. Abal, B. Hart, and K. Mengersen. 2005. Integrating science through Bayesian belief networks: case study of Lyngbya in Moreton Bay. Pages 392-399 in A. Zerger and R. M. Argent, editors. MODSIM 2005 International Congress on Modelling and Simulation. Modelling and Simulation Society of Australia and New Zealand, Canberra, Australia.
46. Hobbs, N. T., and R. Hilborn. 2006. Alternatives to statistical hypothesis testing in ecology: a guide to self teaching. Ecological Applications 16:519.
47. Hobert, J. P., and G. Casella. 1996. The effect of improper priors on Gibbs sampling in hierarchical linear mixed models. Journal of the American Statistical Association 91(436): 1461-1473.
48. Hogarth, R. M. 1975. Cognitive processes and the assessment of subjective probability distributions. Journal of the American Statistical Association 70(350):271-289.
49. Jeffreys, H. 1961. Theory of probability. Third edition. Oxford, Clarendon Press, UK.
50. Jenkinson, D. 2005. The elicitation of probabilities: a review of the statistical literature. Technical report, University of Sheffield, Sheffield, UK.
51. Kadane, J. B. 1980. Predictive and structural methods for eliciting prior distributions. North-Holland, Amsterdam, The Netherlands.
52. Kadane, J. B. 1990. The 1988 Wald Memorial Lectures: the present position in Bayesian statistics: comment. Statistical Science 5(1):80-82.
53. Kadane, J. B., J. M. Dickey, R. L. Winkler, W. S. Smith, and S. C. Peters. 1980. Interactive elicitation of opinion for a normal linear model. Journal of the American Statistical Association 75:845-854.
54. Kadane, J. B., and L. J. Wolfson. 1998. Experiences in elicitation. Statistician 47:3-19.
55. Kahneman, D., and A. Tversky. 1972. Subjective probability: a judgment of representativeness. Cognitive Psychology 3: 430-454.
56. Kuhnert, P. M., T. G. Martin, K. Mengersen, and H. P. Possingham. 2005. Assessing the impacts of grazing levels on bird density in woodland habitat: a Bayesian approach using expert opinion. Environmetrics 16:717-747.
57. Kynn, M. 2006. Designing elicitor: software to graphically elicit expert priors for logistic regression models in ecology. (http://www.winbugs- development.org.uk/elicitor/files/designing.elicitor.pdf)
58. Kynn, M. 2008. The "heuristics and biases" bias in expert elicitation. Journal of the Royal Statistical Society, Scries A 171(1):239-264.
59. Lavine, M., and M. West. 1992. A Bayesian method for classification and discrimination. Canadian Journal of Statistics 20:451-461.
60. Léon, C. L., F. J. Vázquez-Polo, and R. L. González. 2003. Elicitation of expert opinion in benefit transfer of environmental goods. Environmental and Resource Economics 26: 199-210.
61. Lindley, D. V. 2000. Philosophy of statistics. Statistician 49(3): 293-337.
62. Link, W. A., and J. R. Sauer. 1996. Extremes in ecology: avoiding the misleading effects of sampling variation in summary analyses. Ecology 77:1633-1640.
63. Low Choy, S., K. Mengersen, and J. Rousseau. 2008. Encoding expert opinion on skewed non-negative distributions. Journal of Probability and Statistics 3(1): 1-21.
64. Low Choy, S., B. Stewart-Koster, A. Kelly, T. Eyre, and K. M. Mengersen. 2005. Identifying good condition in native vegetation: a Bayesian regression and decision theoretic approach. Pages 435-441 in A. Zerger and R. M. Argent, editors. MODSIM 2005 International Congress on Modelling and Simulation. Modelling and Simulation Society of Australia and New Zealand, Canberra, Australia.
65. Martin, T. G., P. M. Kuhnert, K. Mengersen, and H. P. Possingham. 2005. The power of expert opinion in ecological models: a Bayesian approach examining the impact of livestock grazing on birds. Ecological Applications 15: 266-280.
66. Mather, P. M., and J. C. Doornkamp. 1970. Clustering multivariate analysis in geography with particular reference to drainage-basin morphometry. Transactions of the Institute of British Geographers 51:163-187.
67. McCarthy, M. A., and P. Masters. 2005. Profiting from prior information in Bayesian analyses of ecological data. Journal of Applied Ecology 42:1012-1019.
68. Merritt, W. S., R. A. Letcher, and A. J. Jakeman. 2003. A review of erosion and sediment transport models. Environmental Modelling and Software 18:761-799.
69. Neidner, J. 2006. Why is vegetation condition important to government? A case study from Queensland. Ecological Management and Restoration 7:S5-S7.
70. Neldner, V. J., B, A. Wilson, E. J. Thompson, and H. A. Dillewaard. 2004. Methodology for survey and mapping of Regional Ecosystems and vegetation communities in Queensland. Version 3.0. Technical report. Queensland Herbarium, Environmental Protection Agency, Brisbane, Australia.
71. O'Hagan, A. 1998. Eliciting expert beliefs in substantial practical applications. Statistician 47:21-35.
72. O'Hagan, A., C. E. Buck, A. Daneshkhah, R. Eiser, P. Garthwaite, D. Jenkinson, J. Oakley, and T. Rakow. 2006. Uncertain judgements: eliciting experts' probabilities. John Wiley and Sons, New York, New York, USA.
73. O'Leary, R. A., S. J. Low Choy, K. Mengersen, M. Kynn, P. M. Kuhnert, R. Denham, T. J. Martin, J. Murray, and P. J. Jarman. 2008a. Comparison of expert elicitation methods for logistic regression for presence of endangered brush-tailed rock-wallaby Petrogale penicillata. Environmetrics. [doi: 10.1002/env.935]
74. O'Leary, R., J. Murray, S. Low Choy, and K. Mengersen. 20086. Expert elicitation for bayesian classification trees. Journal of Applied Probability and Statistics 3(1):95-106.
75. Pascual, M. A., and P. Kareiva. 1996. Predicting the outcome of competition using experimental data: maximum likelihood and Bayesian approaches. Ecology 77:337-349.
76. Pearce, J. L., K. Cherry, M. Drielsma, S. Ferricr, and G. Whish. 2001. Incorporating expert opinion and fine-scale vegetation mapping into statistical models of faunal distribution. Journal of Applied Ecology 38:412-424.
77. Phillips, L. D., and S. J. Wisbey. 1993. The elicitation of judgmental probability distributions from groups of experts: a description of the methodology and records of seven formal elicitation sessions held in 1991 and 1992. Report nss/r282. Nirex UK, Didcot, UK.
78. Prato, T. 2005. Bayesian adaptive management of ecosystems. Ecological Modelling 183:147-156.
79. Pullar, D., S. Low Choy, and W. Rochester. 2004. Ecoregion classification using a Bayesian approach and model-based clustering: expert elicitation using a spatial and graphical tool. Pages 927-932 in C. Pahl-Wostl, S. Schmidt, A. E. Rizzoli, and A. J. Jakeman, editors. Transactions of the Second Biennial Meeting of the International Environmental Modelling and Software Society, Volume 2. International Environmental Modelling and Software Society (iEMSs), Manno, Switzerland.
80. Reckhow, K. H. 1990. Bayesian inference in non-replicated ecological studies. Ecology 71:2053-2059.
81. Renooij, S. 2001. Probability elicitation for belief networks: issues to consider. Knowledge Engineering Reviews 16: 255-269.
82. Rochester, W., A. Accad, S. Low Choy, V. Neldner, D. Pullar, and K, Williams. 2004. Final report UQ-EPA subregion classification project. Technical report. The University of Queensland, Brisbane, Australia.
83. Roloff, G. J., and B. J. Kernohan. 1999. Evaluating reliability of habitat suitability index models. Wildlife Society Bulletin 27:973-985.
84. Shepherd, G. G., and C. W. Kirkwood. 1994. Managing the judgmental probability elicitation process: a case study of analyst/manager interaction. IEEE Transactions on Engineering Management 41:414-425.
85. Siu, N., and D. Kelly. 1998. Bayesian parameter estimation in probabilistic risk assessment. Reliability Engineering and System Safety 62:89-116.
86. Smith, C. S., A. L. Howes, B. Price, and C. A. McAlpine. 2007. Using a Bayesian belief network to predict suitable habitat of an endangered mammal the Julia Creek dunnart (Sminthopsis donglasi). Biological Conservation 139:333-347.
87. Spetzler, C. S., and C.-A. S. Staël von Holstein. 1975. Probability encoding in decision analysis. Management Science 22(3): 340-358.
88. Stepinski, T., and R. Vilalta. 2005. Digital topography models for Martian surfaces, IEEE GeoScience and Remote Sensing Letters 2(3):260-264.
89. Taylor, B. L., P. R. Wade, R. A. Stehn, and J. F. Cochrane. 1996. A Bayesian approach to classification criteria for Spectacled Eiders. Ecological Applications 6:1077-1089.
90. ter Braak, C. J. F., and R. S. Etienne. 2003. Improved Bayesian analysis of metapopulation data with an application to a tree frog metapopulation. Ecology 84:231-241.
91. Tversky, A., and D. Kahneman. 1973. Availability: a heuristic for judging frequency and probability. Cognitive Psychology 5:207-2327
92. Tversky, A., and D. Kahneman. 1974. Judgment under uncertainty: heuristics and biases. Science 185:1124-1131.
93. Tversky, A., and D. Kahneman. 1981. The framing of decisions and the psychology of choice. Science 211:453-458.
94. Wade, P. R. 2001. Bayesian methods in conservation biology. Ecological Applications 6:1036-1046.
95. Walls, L., and J. Quigley. 2001, Building prior distributions to support Bayesian reliability growth modelling using expert judgment. Reliability Engineering and System Safety 74(2): 117-128.
96. Weber, M., and K. Borcherding. 1993. Behavioral influences on weight judgments in multiattribute decision making. European Journal of Operational Research 67:1-12.
97. Winkler, R. L. 1967. The assessment of prior distributions in Bayesian analysis. Journal of the American Statistical Association 62:776-800.
98. Wintle, B. A., M. A. McCarthy, C. T. Volinsky, and R. P. Kavanagh. 2003. The use of Bayesian model averaging to better represent uncertainty in ecological models. Conservation Biology 17(6):1579-1590.