Experts and foresight: review and experience

International Journal of Foresight and Innovation Policy

Volumn 1, Issue 1-2, 2004, Pages 33-69

  Loveridge, Denis ^a  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

Author keywords

experts; foresight; human judgement; practical experience; preference; science in court; uncertainty

Indexed keywords

EID: 34248538977     PISSN: 17402816     EISSN: 17402824     Source Type: Journal    
DOI: 10.1504/ijfip.2004.004651     Document Type: Article

References (228)

1. Alpert, M. and Raiffa, H. (1969) ‘A progress report on the training of probability assessors’, Unpublished Manuscript, Harvard University.
2. Anon. (1996) A Vital Knowledge System: Dutch Research With a View to the Future, Foresight Steering Committee, June.
3. Anon. (2001) ‘Messages from the second round of foresight’, The Steering Group Report, OST, August.
4. Anon. (2002) Foresight: Proposals for the Future Programme, OST, June.
5. Barclay, S. and Peterson, C.R. (1973) ‘Two methods for assessing probability distributions’, Decisions and Designs Inc., Technical Report, Vol. 73, 1 August.
6. Barker, A. and Peters, B.G. (1993) The Politics of Expert Advice: Creating, Using and Manipulating Scientific Knowledge for Public Policy, Edinburgh University Press.
7. Basak, I. (1998) ‘Probabilistic judgements specified partly in the analytical hierarchy process’, European Journal of Operational Research, Vol. 108, pp.153–164.
8. Beach, L.R., Christensen-Szalanski, J. and Barnes, V. (1987) ‘Assessing human judgement: has it been done, can it be done, should it be done?’ in G. Wright and P. Ayton (Eds.) Judgmental Forecasting, John Wiley & Sons Ltd.
9. Bell, D.E., Keeney, R.L. and Raiffa, H. (1977) ‘Conflicting objectives in decisions’, International Series on Applied Systems Analysis, John Wiley: IIASA.
10. Beyth-Marom, R. (1982) ‘How probable is probable? A numerical translation of verbal probability expressions’, Journal of Forecasting, Vol. 1, pp.257–269.
11. Budescu, D.V. and Rantilla, A.K. (2000) ‘Confidence in aggregation of expert opinions’, Acta Psychologica, Vol. 104, pp.371–398.
12. Bunn, D.W. (1979a) ‘A perspective on subjective probability for prediction and decision’, Tech. For. & Soc. Change, Vol. 14, pp.39–45.
13. Bunn, D.W. (1979b) ‘Estimation of subjective probability distributions in forecasting and decision making’, Tech. For. & Soc. Change, Vol. 14, pp.205–216.
14. Bunn, D.W. and Salo, A.A. (1993) ‘Forecasting with scenarios’, European Journal of Operational Research, Vol. 68, pp.291–303.
15. Casti, J.L. (1992) Searching for Certainty: What Science Can Know About the Future, Scribiners.
16. Chakraborty, D. (2001) ‘Structural quantization of vagueness in linguistic expert opinions in an evaluation programme’, Fuzzy Sets and Systems, Vol. 119, pp.171–186.
17. Cooke, R.M. (1991) Experts in Uncertainty: Opinion and Subjective Probability in Science, OUP.
18. Delgado, M., Herrera, F., Herrera-Viedma, E. and Martinez, L. (1998) ‘Combining numerical and linguistic information in group decision making’, Journal of Information Sciences, Vol. 107, pp.177–194.
19. DeWispelare, A.R., Herren, L.T. and Clemen, R.T. (1995) ‘The use of probability elicitation in the high level nuclear waste regulation program’, International Journal of Forecasting, Vol. 11, pp.5–24.
20. Dransfield, H., Pemberton, J. and Jacobs, G. (2000) ‘Quantifying weighted expert opinion: the future of interactive television and retailing’, Tech. For. & Soc. Change, Vol. 63, pp.81–90.
21. Finetti, B. de (1985) ‘Cambridge probability theorists’, Manchester School of Economic and Social Studies, Vol. 4, December, pp.347–346.
22. Foster, K.R. and Huber, P.W. (1999) Judging Science: Scientific Knowledge and the Federal Courts, The MIT Press.
23. Foster, K.R., Bernstein, D.E. and Huber, P.W. (1993) Phantom Risk: Scientific Inference and the Law, The MIT Press.
24. Georghiou, L.G, Giusti, W.L, Cameron, H.M. and Gibbons, M. (1988) ‘The use of co-nomination analysis in the evaluation of collaborative research’, in A.F.J van Raan (Ed.) Handbook of Quantitative Studies of Science and Technology, North-Holland.
25. Green, P.E, Wind, Y. and Jain, A.K. (1972) ‘Preference measurement of item collections’, Journal of Marketing Research, Vol. IX, November, pp.371–377.
26. Hammond, K.R and Adelman, L. (1976) ‘Science, value and human judgment’, Science, Vol. 194, 22 October, pp.389–396.
27. Harries, C. (1999) ‘Judgmental inputs to the forecasting process: research and practice’, Tech. For. & Soc. Change, Vol. 61, pp.75–82.
28. Hogarth, R. (1987) Judgement and Choice: The Psychology of Decision, 2nd Edition, John Wiley.
29. Hogarth, R. (1988) Judgement and Choice, Wiley-Interscience, Second Edition.
30. Holling, C.S. (1977) ‘The curious behaviour of complex systems: lessons from ecology’, in H. Linstone and W.H.C. Simmonds (Eds.) Futures Research: New Directions, Addison-Wesley.
31. Huber, G.P. (1974) ‘Methods for quantifying probabilities and multi-variate utilities’, Decision Sciences, Vol. 5, pp.430–458.
32. Hull, J.C. (1976) ‘Review of the literature on the assessment of subjective probability distributions’, Cranfield Institute of Technology, School of Management, PhD Thesis (part of).
33. Lichtenstein, S. and Slovic, P. (1971) ‘Reversals of preference between bids and choices in gambling decisions’, Journal of Experimental Psychology, Vol. 89, pp.46–55.
34. Lichtenstein, S. and Slovic, P. (1973) ‘Response induced reversals of preference in gambling’, Journal of Experimental Psychology, Vol. 101, pp.16–20.
35. Lipinski, A.J. and Loveridge, D.J. (1982) ‘How we forecast: Institute for the Future’s study of the UK: 1978–1995’, Futures, Vol. 14, No. 3, pp.205–239.
36. Lipinski, A.J, Lipinski, H.M. and Randolph, R.H. (1973) ‘Computer-assisted expert interrogation: a report on current methods development’, Tech. For. & Soc. Ch., Vol. 5, pp.3–18.
37. Loveridge, D. (1981) ‘Systems and ecology: a study of the application of systems thinking and some simple ecological principles to new business development’, M. Phil Thesis, University of Sussex.
38. Loveridge, D., Georghiou, L. and Nedeva, M. (1995) ‘UK technology foresight programme Delphi survey’, A Report to the Office of Science and Technology, September.
39. Mayo, D.G. and Hollander, R.D. (1991) Acceptable Evidence: Science and Values in Risk Management, Oxford University Press. Here I wish to acknowledge the work of my one time colleague Alison Dale whose review of this book I have drawn on here.
40. Medsker, L., Tan, M. and Turban, E. (1995) ‘Knowledge acquisition from multiple experts: problems and issues’, Expert Systems With Applications, Vol. 9, No. 1, pp.35–40.
41. Morris, P.A. (1979) ‘Bayesian expert resolution’, PhD Dissertation, Stanford University, May.
42. Morrison, D.G. (1967) ‘Critique of ranking procedures and subjective probability distributions’, Management Science, Vol. 14, B, 4, pp.253–254.
43. Murphy, A.H. and Winkler, R.L. (1973) ‘Subjective forecasting of temperature: some experimental results’, in Proceedings of the Third Conference on Probability and Statistics in Atmospheric Science, American Meteorological Society, Boulder, Colorado, June.
44. Nedeva, M., Georghiou, L., Loveridge, D. and Cameron, H. (1996) ‘The use of co-nomination to identify expert participants for technology foresight’, R&D Management, Vol. 26, No. 2, April.®
45. Peterson, C.R., Snapper, K.J. and Murphy, A.H. (1972) ‘Credible interval temperature forecasts’, Bull. American Meteorological Society, Vol. 53, No. 10, pp.966–970.
46. Saaty, T.L. (1980) The Analytical Hierarchy Process, McGraw-Hill.
47. Savage, L.J. (1954) The Foundations of Statistics, Dover Publications, and (1972) second revised edition.
48. Shafer, G. (1976) A Mathematical Theory of Evidence, Princeton University Press.
49. Simon, H. (1957) Models of Man, New York: Wiley.
50. Stanek, D.M. and Mokhtarian, P.L. (1998) ‘Developing models of preference for home-based and center-based telecommuting: findings and forecasts’, Technological Forecasting and Social Change, Vol. 57, pp.53–74.
51. Stirling, A. and Meyer, S. (1999) ‘Rethinking risk: a pilot multi-criteria mapping of a genetically modified crop in agricultural systems in the UK’, Report on research supported by the ESRC’s Global Environmental Change Programme, August.
52. The Royal Society (1981) ‘The assessment and perception of risk’, Proc. Roy. Soc. London, Series A, Vol. 376, p.1764.
53. The Royal Society (1992) Risk: Analysis, Perception, Management, London: The Royal Society.
54. The Royal Society (1997) Science, Policy and Risk, London: The Royal Society, London.
55. Vickers, Sir G. (1963) ‘Appreciative behaviour’, Acta Psychologica, Vol. 21, pp.274–293.
56. Vickers, Sir G. (1981) ‘Systems analysis: a tool subject or judgment demystified?’, Policy Sciences, Vol. 14, pp.23–29.
57. Watkins, P.R. (1984) ‘Preference mapping of perceived information structure: implications for decision support systems design’, Decision Sciences, Vol. 15, pp.92–106.
58. Weinberg, A.M. (1972) ‘Science and trans-science’, Minerva, Vol. 10, pp.209–222.
59. Williams, P.M. (1978) ‘On a new theory of epistemic probability’, Brit. Jnl. Phil. Sci., Vol. 29, pp.375–387.
60. Winkler, R.L. (1967) ‘The assessment of prior distributions in Bayesian analysis’, Journal American Statistical Society, Vol. 62, pp.1103–1120.
61. Xu, Z. (2000) ‘On consistency of the weighted geometric mean complex judgment matrix in AHP’, European Journal of Operational Research, Vol. 126, pp.683–687.
62. Zio, E. (1996) ‘On the use of the analytical hierarchy process in the aggregation of expert judgements’, Reliability Engineering and System Safety, Vol. 53, pp.127–138.
63. Abbott, A. (1996) ‘Gene tests: who benefits from risk?’, Nature, Vol. 379, 1 February, pp.389–392.
64. Ackerman, W. (1981) ‘Cultural values and social choice of technology’, Int. Soc. Sci. J., Vol. 33, No. 3, pp.445–465.
65. Adelman, L. and Mumpower, J. (1979) ‘The analysis of expert judgement’, Tech. For. & Soc. Ch, Vol. 15, pp.191–204.
66. Aggleton, P. et al. (1994) ‘Risking everything? Risk behaviour, behaviour change, and AIDS’, Science, Vol. 265, 1 July, pp.341–344.
67. Amara, R.C. and Lipinski, A.J. (1972) ‘Some views on the use of expert judgement’, Tech. For. & Soc. Ch, Vol. 3, pp.279–289.
68. Amara, R.C. and Lipinski, A.J. (1983) Business Planning for an Uncertain Future, Pergamon Press.
69. Anon. (1982) ‘At risk – a survey of international insurance’, The Economist.
70. Anon. (1981) ‘Business International’s country risk assessment service’, BI/CAS, April.
71. Apostolakis, G. (1990) ‘The concept of probability in safety assessments of technological systems’, Science, Vol. 250, 7 December, pp.1359–1364.
72. Asch, S. (1958) ‘Effects of group pressure upon the modification and distortion of judgements’, Readings in Social Psychology, 3rd. Ed., London: Holt Rinehardt & Winston.
73. Ashby, Lord (1981) ‘The assessment and perception of risk: introductory remarks’, Proc. Roy. Soc. Lond. A, Vol. 376, pp.3–4.
74. Ashworth, J. (1997) ‘Science, policy and risk’, Discussion Meeting held at the Royal Society, Tuesday 18 March.
75. Bartholomew, D.J. (1975) ‘Probability and social science’, Int. Soc. Sci. Jnl., Vol. 27, No. 3, pp.421–436.
76. Benforado, J. et al. (1987) ‘Comparative ecological risk – a report of the ecological risk workgroup’, US EPA, Feb., pERA-OTIS 87/6768.
77. Blackman Jr., A.W. (1971) ‘The use of Bayesian techniques in Delphi forecasts’, Tech. For. & Soc. Ch.. Vol. 2, pp.261–268.
78. Blokker, E.F. et al. (1981) ‘Risk analysis in the chemical industry’, Angewandte Systemanalyse, Vol. 2, No. 4, pp.167–182.
79. Bowen, K.C. and Stratton, A. (1981) ‘The information content of data in defined choice situations’, Jnl. OR, Vol. 32, pp.79–98.
80. Bowman, E.H. (1980) ‘A risk/return paradox for strategic management’, M.I.T.-ILP, WP, pp.1107–1108.
81. Brainerd, C.J. (1981) ‘Working memory and the developmental analysis of probability judgement’, Psychological Review, Vol. 88, No. 6, Nov., pp.463–502.
82. Brehmer, B. (1976) ‘Social judgement theory and the analysis of interpersonal conflict’, Psychological Bull., Vol. 83, No. 6, pp.985–1003.
83. Brockhaus, W.L. (1975) ‘A quantitative analytical methodology for judgemental and policy decisions’, Tech. For. & Soc. Ch., Vol. 7, pp.127–137.
84. Brown, B. (1968) ‘The Delphi process: a methodology used for elicitation of opinions of experts’, The RAND Corp., P-3925.
85. Cane, V.R. (1977) ‘Models of subjective probability’, SSRC Res. Rept., HR3609.
86. Christie, B. (1975) ‘Travel or telecommunicate? Some factors affecting the choice’, Comm. Studies Group, London: Joint Unit for Planning Research, University College London & London School Economics.
87. Christie, B. and Kingan, S. (1975) ‘Type of experience as a determinant of judgements of telecommunication: an arousal theory interpretation’, Communications Studies Group, Joint Unit for Planning Research, University College London & London School Economics.
88. Collen, R. ‘A study of Sherlock Holmes based on Kostick’s perception and preference inventory’.
89. Dale, A.J. (1998) ‘Acceptable evidence: science and values in risk management’, 1993 Book review of book by D.G. Mayo and R.D. Hollander, Futures Delgado, M. et al. Combining Numerical and Linguistic Information in Group Decision Making, Journal of Information Sciences, Vol. 107, pp.177–194.
90. Dickey, J.M. (1979) ‘Expert uncertainty and the use of subjective probability models’, Private Communication.
91. Dickey, J.M. (1979) ‘Beliefs about beliefs: a theory for stochastic assessment of subjective probabilities’, Private Communication.
92. Dommelen, A. van (1996) (Ed.) Coping with Deliberate Release: The Limits of Risk Assessment, Tilburg/Buenos Aires: International Centre for Human and Public Affairs, Series B: Social Studies of Science and Technology 1.
93. Dressier, F.R.S. (1972) ‘Subjective methodology in forecasting’, Tech. For. & Soc. Ch., Vol. 3, pp.427–439.
94. Eliashberg, J. and Winkler, R.L. (1981) ‘Risk sharing and group decision making’, Mangt. Sci, Vol. 27, No. 11, Nov., pp.1221–1235.
95. Farrell, C. (1986) ‘The insurance crisis: now everybody is in a risky business’, Business Week, March 10, pp.62–66.
96. Finetti, B. de (1964) ‘Foresight: its logical laws, its subjective sources’, in Kyburg Jnr. and Smokler Studies in Subjective Probability, N.Y.: John Wiley & Sons, pp.93–158.
97. Fishburn, P.C. (1983) ‘Research in decision theory: a personal perspective’, Math. Soc. Sci., Vol. 5, pp.129–148.
98. Fishoff, B. et al. (1978) ‘How safe is safe enough? A psychometric study of attitudes towards technological risks and benefits’, Policy Sci., in press.
99. Foerster, H. von (1972) ‘Perception of the future and the future of perception’, Inst. Sci., Vol. 1, No. 1, pp.31–43.
100. Funtowicz, S.O. and Ravetz, J.R. (1990) ‘Uncertainty and quality in science for policy’, Theory and Decision Library, Series A: Philosophy and Methodology of the Social Sciences Kluwer Academic Publishers.
101. George, J.J. (1988) ‘Expanding the R&D pipeline with reduced risk’, SRI-BIP File No. D88–1222.
102. Gershuny, J. (1976) ‘The choice of scenarios’, Futures, Vol. 8, No. 6, pp.496–508.
103. Gillies, D. (1991) ‘Intersubiective probability and confirmation theory’, Brit. J. Phil. Sci., Vol. 42, pp.513–533.
104. Giusti, W.I. and Georghiou, L. (1988) ‘The use of co-nomination analysis in real-time evaluation of an R&D programme’, Scientometrics, Vol. 14, Nos. 3–4, pp.265–281.
105. Goupy, V. de M. et al. (1978) ‘A study of political risk: Latin America, Asia, and Africa’, Hudson Res. Europe, prepared for World 1995, Paris, Nov.
106. Green, M. and Harrison, P.J. (1973) ‘Fashion forecasting for a mail order company using a Bayesian approach’, ORQ, Vol. 24, pp.193–205.
107. Green, P.E. and Frank, R.E. ‘Bayesian statistics and marketing research’, Applied Statistics, pp.172–190.
108. Gregory, R.L. (1974) Concepts and Mechanisms of Perception, Duckworth Press.
109. Grether, D.M. and Wilde, L.L. (1983) ‘Consumer choice and information’, Info. Econ. & Policy, Vol. 1, pp.115–144.
110. Grove, J.W. (1985) ‘Rationality at risk: science against pseudoscience’, Minerva, Vol. 23, No. 2, pp.216–240.
111. Hammond, J.S. (1967) ‘Better decisions with preference theory’, Harvard Bus. Rev., Dec., pp.123–141.
112. Hammond, K.R. (1974) ‘Human judgement and social policy’, Program of Research on Human Judgement and Social Interaction.
113. Harris, C.C. (1993) ‘p53: at the crossroads of molecular carcinogenesis and risk assessment’, Science, Vol. 262, 24 December, pp.1980–1982.
114. Heilig, K. (1978) ‘Carnap and de Finetti on bets and the probability of singular events: the Dutch book argument reconsidered’, Brit. J. Phil. Sci., Vol. 29, pp.325–346.
115. Hirota, T. (1995) ‘Innovation risk dilemma’, Private Communication.
116. Horwich, P. (1982) Probability and Evidence, Cambridge Univ. Press.
117. Howard, R.A. et al. (1977) Readings in Decision Analysis, Stanford Research Institute.
118. Howson, C. and Urbach, P. (1991) ‘Bayesian reasoning in science’, Nature, Vol. 350, 4 April, pp.371–374.
119. Hunsaker, P.L. (1975) ‘Incongruity adaptation capability and risk preference in turbulent decision-making environments’, Org. Behav. & Human Performance, Vol. 14, pp.173–185.
120. Jones, M. (1976) ‘Time, our lost dimension: toward a new theory of perception, attention and memory’, Psychological Rev., Vol. 83, No. 5, pp.323–355.
121. Kadane, J.B. et al. (1978) ‘Interactive elicitation of opinion for a normal linear model’, Carnegie-Mellon University, Tech. Report No. 150, Dept. Stats.
122. Kahn, P. (1996) ‘Coming to grips with genes and risk’, Science, Vol. 274, 25 October, pp.496–498.
123. Kahneman. D. et al. (1982) Judgment under Uncertainty: Heuristics and Biases, Cambridge: UP.
124. Kanizsa, G. ‘Subjective contours’, Unknown.
125. Kaplan, M.F. and Schwartz, S. (1975) Human Judgement and Decision Processes, NY, SF and London: Academic Press.
126. Kerr, R.A. (1996) ‘A new way to ask the experts: rating radioactive waste risks’, Science, Vol. 274, No. 8 November, pp.913–914.
127. Kmietowicz, Z.W. and Pearman, A.D. (1981) ‘Decision theory, strict ranking of probabilities and statistical dominance’, Sch. Econ. Studies Discussion Paper 108.
128. Knief, et al. (1991) (Eds.) Risk Management, London: Taylor & Francis Ltd.
129. Kruger, L. et al. (1987) The Probabilistic Revolution – Volume 1 – Ideas in History, The MIT Press.
130. Kruger, L. et al. (1987) The Probabilistic Revolution – Volume 2 – Ideas in Science, The MIT Press.
131. Kyburg Jnr., H.E. and Smokler, H.E. (1964) (Eds.) Studies in Subjective Probability, N.Y.: John Wiley & Sons.
132. Lee, T.R. (1981) ‘The public’s perception of risk and the question of irrationality’, Proc. Roy. Soc. Lond. A, Vol. 376, pp.5–16.
133. Levi, I. (1979) ‘Support and surprise: L.J. Cohen’s view of inductive probability’, Brit. Jnl. Phil. Sci, Vol. 30, pp.279–292.
134. Levi, I. (1983) The Enterprise of Knowledge: An Essay on Knowledge, Credal Probability and Chance, The M.I.T. Press.
135. Lindley, D.V. (1982) ‘The subjectivist view of decision-making’, European Jnl. Op. Res., Vol. 9, pp.213–222.
136. Long, J. (1983) ‘Risk and benefit’, ‘A report and reflections on a series of consultations St. George’s House, Windsor Castle’, St. George’s House Paper No. 5.
137. Lonsdale, A.J. (1978) ‘Judgement research in policy analysis’, Futures, Vol. 10, No. 3, pp.213–226.
138. Loveridge, D. (1984) ‘Seeing things differently …’, Lecture at Inst. of Welding.
139. Madigan, Sir R.T. (1985) ‘The anatomy of risks’, SRI-BIP D83–820.
140. Makridakis, S. (1988) ‘Metaforecasting: ways of improving forecasting accuracy and usefulness’, Int. Jnl. Forecasting, Vol. 4, pp.467–491.
141. Malakoff, D. (1999) ‘Bayes offers a ‘‘new way’’ to make sense of numbers’, Science, Vol. 286, 19 November, pp.1460–1464.
142. Malsch, I. (1997) ‘Nanotechnology in Europe: expert’s perceptions and scientific relations between sub-areas’, IPTS Technical Report Series, eUr 17710 EN, October.
143. Margolis, H. (1988) Patterns, Thinking and Cognition: A Theory of Judgement, Univ. Chicago Press.
144. Matthews, R. (1993) ‘Linguistics on trial’, New Sci., 21 August, pp.12–13.
145. Milburn, M.A. (1978) ‘Sources of bias in the prediction of future events’, Org. Behav. & Human Perf., Vol. 21, pp.17–26.
146. Miller, H.I. and Gunary, D. (1993) ‘Serious flaws in the horizontal approach to biotechnology risk’, Science, Vol. 262, 3 December, pp.1500–1501.
147. Missan, D. (1980) ‘Linguistic innovation’, Private Communication.
148. Mitchell, R.B. (1980) ‘Subjective conditional probabilities – a new approach’, Tech. For. & Soc. Ch, Vol. 16, pp.343–349.
149. Moskowitz, H. and Sarin, R.K. (1980) ‘Improving conditional probability assessments for forecasting and decision making’, Inst. Res. in the Behaviour, Economic and Mangt. Sci., Krannert Grad. Sch. Mangt., Perdue Univ., Paper No. 734.
150. Mumford, E. and Spackman, H. (1975) (Eds.) Human Choice and Computers.
151. North, D.W. (1968) ‘A tutorial introduction to decision analysis’, IEEE Trans. Sys. Sci. & Cyb, SSC-4, Sept., pp.200–210.
152. Osgood, C.E. (1974) ‘Probing subjective culture. Pt.1: cross-linguistic tool-making’, Jnl. Communication, Winter, pp.21–35.
153. Otway, H.J. and Pahner, P.D. (1976) ‘Risk assessment’, Futures, Vol. 8, No. 2, pp.122–134.
154. Otway, H.J. et al. (1975) ‘Revealed preferences: comments on the Starr benefit-risk relationships’, IIASA, RM-75–5, Laxenburg.
155. Pendse, S.G. (1978) ‘Category perception, language and brain hemispheres: and information transmission approach’, Behavioural Science, Vol. 23, pp.421–428.
156. Platz, W.E. and Oberlaender, F.A. (1995) ‘On problems of expert opinion on holocaust survivors submitted to the compensation authorities in Germany’, International Journal of Law and Psychiatry, Vol. 18, No. 3, pp.30–-321.
157. Plous, S. (1993) The Psychology of Judgment and Decision Making, McGraw-Hill.
158. Polya, G. et al. ‘Mathematics, logic, probability and statistics’, in I.J. Good The Scientist Speculates – An Anthology of Partly Baked Ideas.
159. Ponder, B. (1997) ‘Genetic testing for cancer risk’, Science, Vol. 278, 7 November, pp.1050–1054.
160. Raiffa, H. (1968) Look Up, Addison-Wesley.
161. Raiffa, H. (1970) Decision Analysis, Introductory Lectures on Choices Under Uncertainty, Reading, Mass: Addison Wesley.
162. Roberts, L. (1991) ‘Dioxin risks revisited’, Science, Vol. 251, 8 February, pp.624–626.
163. Roberts, P. (1989) ‘Study project – New challenges and choices: strategic vision and the management of the UK land resource’, Strategic Planning Society, April.
164. Rosemund, C. et al. (1993) ‘Nonuniform probability of glutamate release at a hippocampal synapse’, Science, Vol. 262, 29 October, pp.754–759.
165. Rothschild, Lord (1978) ‘Risk’, The Listener, 30 November, The Dimbleby Lecture.
166. Rowley, C.K. (1979) ‘Liberalism and collective choice’, Nat. West. Quart. Rev., May.
167. Ruspini, E.H. (1990) ‘Possibility as similarity: the semantics of fuzzy logic’, SRI-BIP D90–1478, October.
168. Sackman, H. (1974) ‘In defense of Delphi: a review of Delphi assessment, expert opinion, forecasting and group processes’, RAND Report R-1283-PR; Reviewed by Coates, J.F. (1975) Tech. For. & Soc. Ch., Vol. 7, pp.193–194.
169. Sackman, H. (1974) ‘Scientific inquiry or political process? Remarks on Delphi assessment, expert opinion, forecasting and group processes’, RAND Report R-1283-PR: Reviewed by Goldschmidt, P.G. (1975) Tech. For. & Soc. Ch., Vol. 7, pp.195–213.
170. Sahal, D. and Yee, K. (1975) ‘Delphi: an investigation from a Bayesian viewpoint’, Tech. For. & Soc. Ch., Vol. 7, pp.165–178.
171. Salo, A.A. and Bunn, D.W. (1995) ‘Decomposition in the assessment of judgmental probability forecasts’, Tech. For. & Soc. Ch., Vol. 49, pp.13–25.
172. Sampson, G. (1975) ‘Theory choice in a two-level science’, Brit. Jnl. Phil. Sci., Vol. 26, pp.303–318.
173. Sarin, R.K. (1978) ‘Elicitation of subjective probabilities in the context of decision making’, Decision Sci., Vol. 9, No. 1, pp.37–48.
174. Schick, F. (1979) ‘Self-knowledge, uncertainty and choice’, Brit. Jnl. Phil. Sci., Vol. 30, pp.235–252.
175. Schmeidler, D. ‘Subjective probability without additivity’, The Foerder inst. Econ. Res. Tel-Aviv University.
176. Schomberg, R. von (1995) ‘Contested technology: ethics, risk and public debate’, International Centre for Human and Public Affairs, Series B: Social Studies of Science and Technology, 1 Tilburg/Buenos Aires.
177. Simon, J.D. (1985) ‘Political risk forecasting’, Futures, Vol. 17, No. 2, April, pp.132–148.
178. Slovic, P. et al. (1991) ‘Perceived risk, trust, and the politics of nuclear waste’, Science, Vol. 254, 13 December, pp.1603–1607.
179. Slovic, P., Fischoff, B. and Lichtenstein, S. (1981) ‘Perceived risk: psychological factors and social implications’, Proc. Roy. Soc. Lond. A, Vol. 376, pp.17–33.
180. Spetzler, C.S. and Holstein, von C.A.S.S. (1975) ‘Probability encoding in decision analysis’, Mangt. Sci., Vol. 22, p.3.
181. Staelin, R. and Turner, R.E. (1973) ‘Error in judgemental sales forecasts: theory and results’, Jnl. Marketing Res., Vol. 10, pp.10–16.
182. Starr, C. (1969) ‘Social benefits versus technological risk’, Science, Vol. 165, pp.1232–1238.
183. Stevens, C.F. and Harrison, P.J. (1973) Bayesian Forecasting Pt. 1: The Dynamic Linear Model Pt. 2, University of Warwick.
184. Stirling, A. and Mayer, S. (1999) ‘Rethinking risk’, SPRU Report.
185. Stirling, A. et al. (1999) ‘On science and pre-caution in the management of technological risk’, Report to EC Forward Studies Unit, May.
186. Swalm, R.O. (1966) ‘Utility theory – insights into risk taking 1966’, Harvard Bus. Rev., Nov/Dec., pp.123–136.
187. Thomas, K. (1981) ‘Comparative risk perception: how the public perceives the risks and benefits of energy systems’, Proc. Roy. Soc. Lond. A, Vol. 376, pp.35–50.
188. Tversky, A. and Kahneman, D. (1974) ‘Judgement under uncertainty: heuristics and biases’, Science, Vol. 185, pp.1124–1131.
189. Tversky, A. and Kahneman, D. (1983) ‘Extensional versus intuitive reasoning: the conjunctional fallacy in probability judgement’, Psychological Rev., Vol. 90, No. 4, pp.293–315.
190. Velimirovic, H. (1975) ‘An anthropological view of risk phenomena’, IIASA, RM-75–55.
191. Vickers, Sir G. (1965) The Art of Judgement, London: Tavistock Publications.
192. Vickers, Sir G. (1966) ‘The multivalued choice’, Proc. 2nd. Int. Symp. on Communication Theory and Research, Excelsior Springs, Missouri, pp.259–278.
193. Vines, G. (2000) ‘Rethinking risk’, SPRU Report.
194. Warner, Sir F. (1992) ‘Risk, analysis, perception and management’, Report of the Royal Society Study Group.
195. Warner, Sir F. and Slater, D.H. (1981) ‘The assessment and perception of risk’, Royal Society’s Study Group on Risk, Proc. Royal Society of London, Series A, No. 1764, 30 April.
196. Weisman, J. and Holzman, A.G. (1972) ‘Engineering design optimization under risk’, Mangt. Sci, Vol. 19, pp.235–249.
197. Weizenbaum, J. (1978) Computer Power and Human Reason: From Judgement to Calculation, W.H. Freeman & Company.
198. Wilde, L.L. (1980) ‘A model of “satisfactory” choice: optimal and non-optimal satisficing’, California Inst. Tech., Div. Humanities & Soc. Sci., Working Paper 363, Nov.
199. Winkler, R.L. (1982) ‘Research directions in decision making under uncertainty’, Decision Sci., Vol. 13, No. 4, pp.517–533.
200. Wright, G. and Ayton, P. (1987) Judgmental Forecasting, John Wiley.
201. Wright, G. et al. (1996) ‘The role and validity of judgment in forecasting’, International Journal of Forecasting, Vol. 12, pp.1–8.
202. Wright, M. (2000) ‘Risk roulette’, Green Futures, November/December.
203. Wriston, W.B. (1986) Risk and other Four-Letter Words, N.Y.: Harper & Row (book excerpt from Security National Pacific Bank).
204. Xu, Z. (2000) ‘On consistency of the weighted geometric mean complex judgment matrix in AHP’, European Journal of Operational Research, Vol. 126, pp.683–687.
205. Yager, R.R. (1979) ‘An eigenvalue method of obtaining subjective probabilities’, Behavioural Science, Vol. 24, pp.382–387.
206. Zeckhauser, R.J. and Viscusi, W.K. (1990) ‘Risk within reason’, Science, Vol. 248, 4 May, pp.559–564.
207. Complex here has the connotation of complexity and not simply complication as described by Charles Perrow (1984) in Normal Accidents, Basic Books.
208. It is necessary to treat this term carefully; here it implies information that has been operated on by an individual’s value/norm set and involves the notion of selectivity in relation to the expert’s perception of the scope of the inquiry and (i) the selection of what is perceived to be the relevant information and (ii) the bounds of the individual’s information set.
209. This point will be returned to later in discussing the use of survey methods in foresight programmes.
210. See for example Perrow in Note 1 of this paper and the report into the BSE inquiry, http://www.bse.org.uk/
211. This is described later.
212. Cooke also devotes time to this concern.
213. There is a voluminous literature on probability encoding related to utility, but that is not necessarily related to scenario building.
214. #x2018;Climate Change to the Year 2000: a survey of expert opinion’, Conducted by the Research Directorate of the National Defense University jointly with the US Department of Agriculture, Defense Advanced Research Projects Agency, National Oceanic and Atmospheric Administration and the Institute for the Future, DARPA Contract MDA903–77-C-0914, February, 1978, and the Institute for the Futures 1978 Study of the Future of the UK (see Lipinski and Loveridge, 1982).
215. Bendectin was claimed to have caused congenital limb defects to a child due to her mother taking the drug during the early stages of pregnancy.
216. Due to Abraham Wald; see Savage (1954).
217. Note that this had been referred to as inconsistency by Raiffa in 1957 and also by Granberg in 1966.
218. My additions are in parentheses.
219. I pointed to the first, but in a different way, in an internal company paper in 1977, later published as Loveridge, D. (1983) ‘Computers and you’, Futures, December, pp.498–503.
220. In what follows I have made some personal additions but for the sake of ease of reading these are not indicated.
221. See Popper on ‘holism’ and Simon on ‘bounded rationality’.
222. I have described a process for scenario building in (1992) The Challenge of the External Environment, Supplementary Reading Book 1, Open Business School Course B885, The Open University.
223. These stem from empirical experience gained by the Institute for the Future which classified experts under three categories: generalists, persons of thought and persons of present and future action. The first have a spread of interests, perception and a high level of awareness, the second have deep knowledge in a special field and the last are likely decision makers.
224. It would be very easy to quote absurd limits in order to score ten apparently ‘correct’ results, but that amounts to ill considered ‘hand waving’ and is exactly the kind of response decision makers do not want from expert advisers.
225. Such people have been found in practical experience.
226. Whilst this part of the interview had the appearance of a party game its intent was serious enough and the outcome often left a marked impression on the respondent.
227. In the study of the future of the UK this was important, for a single variable only the historical reference material, sent in advance, was provided.
228. Author’s note.