Experiences in elicitation

Journal of the Royal Statistical Society Series D: The Statistician

Volumn 47, Issue 1, 1998, Pages 3-19

  Kadane, Joseph B ^a   Wolfson, Lara J ^b  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b Carnegie Mellon University ^*

Author keywords

Demography; Expert opinion; Prior probability; Probability assessment; Time series

Indexed keywords

EID: 0002645879     PISSN: 00390526     EISSN: None     Source Type: Journal    
DOI: 10.1111/1467-9884.00113     Document Type: Article

References (69)

1. Alpert, M. and Raiffa, H. (1982) A progress report on the training of probability assessors. In Judgment under Uncertainty: Heuristics and Biases (eds D. Kahneman, P. Slovic and A. Tversky), pp. 294-305. Cambridge: Cambridge University Press.
2. Arriaga and Associates (1993) Population Analysis with Microcomputers, vol. 1, Presentation of Techniques. Washington DC: US Bureau of the Census.
3. Barnett, V. (1982) Comparative Statistical Inference. New York: Wiley.
4. Berry, D. A., Wolff, M. C. and Sack, D. (1992) Public health decision making: a sequential vaccine trial. In Bayesian Statistics 4 (eds J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith), pp. 79-96. Oxford: Oxford University Press.
5. Black, P. and Laskey, K. (1989) Models for elicitation in Bayesian ANOVA: implementation and application. Proc. Statist. Comput. Sect. Am. Statist. Ass., 247-252.
6. Blanck, T. J. J., Conahan, T. J., Merin, R. G., Prager, R. L. and Richter, J. J. (1996) Being an expert. In Bayesian Methods and Ethics in a Clinical Trial Design (ed. J. B. Kadane), pp. 159-162. New York: Wiley.
7. Chaloner, K. (1996) The elicitation of prior distributions. In Case Studies in Bayesian Biostatistics (eds D. Berry and D. Stangl), pp. 141-156. New York: Dekker.
8. Chaloner, K., Church, T., Louis, T. A. and Matts, J. P. (1993) Graphical elicitation of a prior distribution for a clinical trial. Statistician, 42, 341-353.
9. Chaloner, K. M. and Duncan, G. T. (1983) Assessment of a beta prior distribution: PM elicitation. Statistician, 32, 174-180.
10. _(1987) Some properties of the Dirichlet-multinomial distribution and its use in prior elicitation. Communs Statist. Theory Meth., 16, 511-523.
11. Cooke, R. M. (1991) Experts in Uncertainty: Opinion and Subjective Probability in Science. New York: Oxford University Press.
12. Daponte, B. O., Kadane, J. B. and Wolfson, L. J. (1997) Bayesian demography: projecting the Iraqi Kurdish population 1977-1990. J. Am. Statist. Ass., to be published.
13. Dawid, A. P. (1982) The well-calibrated Bayesian (with discussion). J. Am. Statist. Ass., 77, 605-613.
14. DeGroot, M. H. and Fienberg, S. E. (1982) Assessing probability assessors: calibration and refinement. In Statistical Decision Theory and Related Topics III, vol. 1, pp. 291-314. New York: Academic Press.
15. Dickey, J. M. (1980) Beliefs about beliefs: a theory of stochastic assessments of subjective probabilities. In Bayesian Statistics (eds J. M. Bernardo, M. H. DeGroot and A. F. M. Smith). Valencia: Valencia University Press.
16. Dickey, J. M., Dawid, A. P. and Kadane, J. B. (1986) Subjective-probability assessment methods for multivariate-t and matrix-t models. In Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti (eds P. Goel and A. Zellner), pp. 177-195. Amsterdam: North-Holland.
17. DuMouchel, W. (1988) A Bayesian model and a graphical elicitation procedure for multiple comparisons. In Bayesian Statistics 3 (eds J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith), pp. 127-145. Oxford: Oxford University Press.
18. Flournoy, N. (1994) A clinical experiment in bone marrow transplantation: estimating a percentage point of a quantal response curve. In Case Studies in Bayesian Statistics (eds C. Gatsonis, J. Hodges, R. Kass and N. Singpurwalla), pp. 324-336. New York: Springer.
19. Freedman, L. S. and Spiegelhalter, D. J. (1983) The assessment of subjective clinical opinion and its use in relation to stopping rules for clinical trials. Statistician, 32, 153-180.
20. _(1992) Application of Bayesian statistics to decision making during a clinical trial. Statist. Med., 11, 23-35.
21. French, S. (1980) Updating of belief in the light of someone else's opinion. J. R. Statist. Soc. A, 143, 43-48.
22. _(1986) Calibration and the expert problem. Mangmnt Sci., 32, 315-321.
23. Garthwaite, P. (1994) Assessments of prior distributions for regression models: an experimental study. Communs Statist. Simuln, 23, 871-895.
24. Garthwaite, P. H. and Dickey, J. M. (1988) Quantifying expert opinion in linear regression problems. J. R. Statist. Soc. B, 50, 462-474.
25. _(1992) Elicitation of prior distributions for variable-selection problems in regression. Ann. Statist., 20, 1697-1719.
26. Gavasakar, U. (1988) A comparison of two elicitation methods for a prior distribution for a binomial parameter. Mangmnt Sci., 34, 784-790.
27. Gokhale, D. V. and Press, S. J. (1982) Assessment of a prior distribution for the correlation coefficient in a bivariate normal distribution. J. R. Statist. Soc. A, 145, 237-249.
28. Goldman, L., Cook, E. F., Brand, D. A., Lee, T. H., Rouan, G. W., Weisberg, M. C., Acampora, D., Stasiulewicz, C., Walshon, J., Terranova, G., Gottlieb, L., Kobernick, M., Goldstein-Wayne, B., Cohen, D., Daley, K., Brandt, A. A., Jones, D., Mellors, J. and Jakubowski, R. (1988) A computer protocol to predict myocardial infarction in emergency department patients with chest pain. New Engl. J. Med., 318, 797-803.
29. Hogarth, R. M. (1975) Cognitive processes and the assessment of subjective probability distributions (with discussion). J. Am. Statist. Ass., 70, 271-294.
30. Kadane, J. B. (1980) Predictive and structural methods for eliciting prior distributions. In Bayesian Analysis in Econometrics and Statistics (ed. A. Zellner). Amsterdam: North-Holland.
31. _(1994) An application of robust Bayesian analysis to a medical experiment. J. Statist. Planng Inf., 40, 221-232.
32. _(ed.) (1996) Bayesian Methods and Ethics in a Clinical Trial Design. New York: Wiley.
33. Kadane, J. B., Chan, N. H. and Wolfson, L. J. (1996) Priors for unit root models. J. Econometr., 75, 99-111.
34. Kadane, J. B., Dickey, J. M., Winkler, R. L., Smith, W. S. and Peters, S. C. (1980) Interactive elicitation of opinion for a normal linear model. J. Am. Statist. Ass., 75, 845-854.
35. Kadane, J. B. and Hastorf, C. A. (1988) Bayesian paleoethnobotany. In Bayesian Statistics 3 (eds J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith), pp. 243-259. Oxford: Oxford University Press.
36. Kadane, J. B. and Lichtenstein, S. (1982) A subjectivist view of calibration. Technical Report 233. Department of Statistics, Carnegie Mellon University, Pittsburgh.
37. Kadane, J. B. and Schum, D. A. (1992) Opinions in dispute: the Sacco-Vanzetti case. In Bayesian Statistics 4 (eds J. M. Bernardo, J. O. Berger, A. P. Dawid and A. F. M. Smith), pp. 79-96. Oxford: Oxford University Press.
38. _(1996) A Probabilistic Analysis of the Sacco and Vanzetti Evidence. New York: Wiley.
39. Kadane, J. B. and Wolfson, L. J. (1996) Priors for the design and analysis of clinical trials. In Case Studies in Bayesian Biostatistics (eds D. Berry and D. Stangl), pp. 157-184. New York: Dekker.
40. Kirnbauer, R., Schnatter, S. and Gutknecht, D. (1987) Bayesian estimation of design floods under regional and subjective prior information. In Probability and Bayesian Statistics (ed. R. Viertl), pp. 285-294. New York: Plenum,.
41. Laskey, K. B. and Black, P. (1989) Models for elicitation in Bayesian analysis of variance. In Computer Science and Statistics: Proc. 21st Symp. Interface, pp. 242-247. Alexandria: American Statistical Association.
42. Lindley, D. V., Tversky, A. and Brown, R. V. (1979) On the reconciliation of probability assessments (with discussion). J. R. Statist. Soc. A, 142, 146-180.
43. Logan, D. M. (1985) The value of probability assessment. PhD Thesis. Department of Engineering-Economic Systems, Stanford University, Stanford.
44. Meyer, M. and Booker, J. (1991) Knowledge-based Expert Systems, vol. 5, Eliciting and Analyzing Expert Judgment: a Practical Guide. New York: Academic Press.
45. Morgan, M. G. and Henrion, M. (1990) Uncertainty: a Guide to Dealing with Uncertainty in Quantitative Risk and Policy Analysis. New York: Cambridge University Press.
46. Mullin, T. M. (1986) Understanding and supporting the process of probabilistic estimation. PhD Thesis. Department of Engineering and Public Policy, Carnegic Mellon University, Pittsburgh.
47. Murphy, A. H. and Winkler, R. L. (1977) Reliability of subjective probability forecasts of precipitation and temperature. Appl. Statist., 26, 41-47.
48. _(1984) Probability forecasting in meteorology. J. Am. Statist. Ass., 79, 489-500.
49. Nelson, C. R. and Plosser, C. I. (1982) Trends and random walks in macroeconomic time series. J. Monet. Econ., 10, 139-162.
50. Raftery, A. E., Givens, G. H. and Zeh, J. E. (1995) Inference from a deterministic population dynamics model for bowhead whales (with discussion). J. Am. Statist. Ass., 90, 402-430.
51. Savage, L. J. (1954) The Foundations of Statistics. New York: Wiley.
52. Seidenfeld, T. (1985) Calibration, coherence and scoring rules. Phil. Sci., 52, 274-294.
53. Singpurwalla, N. D. and Song, M. S. (1987) The analysis of weibull lifetime data incorporating expert opinion. In Probability and Bayesian Statistics (ed. R. Viertl), pp. 431-442. New York: Plenum.
54. Spiegelhalter, D. J. and Freedom, L. S. (1986) A predictive approach to selecting the size of a clinical trial, based on subjective clinical opinion. Statist. Med., 5, 1-13.
55. _(1988) Bayesian approaches to clinical trials. In Bayesian Statistics 3 (eds J. M. Bernardo, M. H. DeGroot, D. V. Lindley and A. F. M. Smith), pp. 453-477. Oxford: Oxford University Press.
56. Speigelhalter, D. J., Freedman, L. S. and Parmar, M. K. (1993, Applying Bayesian ideas in drug development and clinical trials. Statist. Med., 12, 1501-1511.
57. _ (1994) Bayesian approaches to randomized trials (with discussion). J. R. Statist. Soc. A, 157, 357-416.
58. Tversky, A. and Kahneman, D. (1974) Judgment under uncertainty: heuristics and biases. Science, 185, 1124-113.
59. Vardeman, S. and Meeden, G. (1983) Calibration, sufficiency and domination considerations for Bayesian probability assessors. J. Am. Statist. Ass., 78, 808-816.
60. Wallsten, T. S. and Budescu, D. V. (1983) Encoding subjective probabilities: a psychology, and psychometric review. Mangmnt Sci., 29, 151-173.
61. Wilson, A. G. (1994) Cognitive factors affecting subjective probability assessment. Discussion Paper 94-02. Institute of Statistics and Decision Sciences, Duke University, Durham.
62. Winkler, R. L. (1967a) The assessment of prior distributions in Bayesian analysis. J. Am. Statist. Ass., 62, 776-800.
63. _(1967b) The quantification of judgment: some methodological suggestions. J. Am. Statist. Ass., 62, 1105-1120.
64. _(1980) Prior information, predictive distributions, and Bayesian model-building. In Bayesian Analysis in Econometrics and Statistics (ed. A. Zellner). Amsterdam: North-Holland.
65. _(1986) On "good probability appraisers". In Bayesian Inference and Decision Techniques (eds P. Goel and A. Zellner), pp. 265-278. Amsterdam: Elsevier Science.
66. Winkler, R. L. and Murphy, A. H. (1968) "Good" probability appraisers. J. Appl. Meteorol., 7, 751-758.
67. Wolfson, L. J. (1995) Elicitation of priors and utilities for Bayesian analysis. PhD Thesis. Department of Statistics, Carnegie Mellon University, Pittsburgh.
68. Wolpert, R. L. (1989) Eliciting and combining subjective judgments about uncertainty. Int. J. Technol. Assmnt Hlth, Care, 5, 537-557.
69. Wright, G. and Ayton, P. (1994) Subjective Probability. New York: Wiley.