-
5
-
-
0014114056
-
Assets and liabilities in group problem solving: The need for an integrative function
-
(1967)
Psychol. Rev.
, vol.74
, pp. 239-249
-
-
Maier1
-
6
-
-
0001536179
-
An experimental application of the Delphi method to the use of experts
-
Also RAND RM-727-PR
-
(1963)
Management Science
, vol.9
, pp. 458-467
-
-
Dalkey1
Helmer2
-
18
-
-
0000805064
-
Alternative approaches to the theory of choice in risk-taking situations
-
(1951)
Econometrica
, vol.19
, pp. 404-437
-
-
Arrow1
-
19
-
-
0001115512
-
Consumption theory in terms of revealed preference
-
(1948)
Economica
, vol.15
, pp. 243-253
-
-
Samuelson1
-
22
-
-
0000424516
-
An internal consistency check for scale values determined by the method of successive intervals
-
(1952)
Psychometrika
, vol.17
, pp. 169-180
-
-
Edwards1
Thurstone2
-
32
-
-
84911563685
-
The education innovation study
-
(See P-3499.), Most of the contents of this paper are discussed also in RAND P-3499. It is a description of the process and output of a seminar group which studied innovation in education and came up with some recommendations and proposed priorities. The Delphi technique was used, but the emphasis in the discussions is on the results rather than the methodology. There is an interesting work flow chart which would be useful for any studies of this type.
-
(1967)
Am. Behavioral Scientist
, vol.10
, pp. 21-27
-
-
Adelson1
Alkin2
Carey3
Helmer4
-
33
-
-
56049102939
-
-
P-3925, The RAND Corporation, There is an introduction, a description of the technique and a discussion of the applications. Very repetitive of other RAND reports on the same subject.
-
(1968)
Delphi Process
-
-
Brown1
-
35
-
-
0344587425
-
-
P-2986, The RAND Corporation, This is a report on an experiment involving Delphi with the added computation of a concensus based on self-appraised competence ratings. The authors provide their arguments for the importance of such studies in Operations Research.The experiment consisted of 20 questions submitted to 23 respondents consisting of RAND research staff. The questions were of the almanac type, and the results are given in the report. Their conclusions were that convergence of the medians to the true value occurred in the majority of cases and that the use of self-appraised competence ratings in forming a concensus appeared to be a powerful tool for increasing the reliability of group estimates.
-
(1964)
Improving the Reliability of Estimates Obtained from a Concensus of Experts
-
-
Brown1
Helmer2
-
37
-
-
0347609909
-
The Delphi technique: Implementation in the corporate environment
-
This article investigates the possibilities of using Delphi in the formal corporate environment, and points out some of the technique in this context. The method is explained in the more or less standard manner, and the importance of panel selection and the need for some structure to evalute and use the results are stressed. It use is proposed for the development of new products. The article is brief and general.
-
(1968)
Mgmt Ser.
, vol.5
, pp. 37-42
-
-
Campbell1
-
38
-
-
0004168307
-
-
P-3704, The RAND Corporation, A general introduction and a report of the state of the research at the time. RAND Report RM-588-PR is a much more comprehensive treatment of the subject. Most of the results of comparing techniques for use of group expertise proved statistically insignificant, so no strong conclusions could be drawn.
-
(1967)
Delphi
-
-
Dalkey1
-
39
-
-
84916873916
-
-
P-3805, The RAND Corporation, He accepts “quality of life” as a vague slogan, and tries to come up with some means of defining it. Accepts the restriction of social programs as aimed at doing something for the individual, and to description, which is difficult enough, rather than prescription. Possible methods can be divided into “armchair analysis” and public surveys. Gives various lists of general factors required for the well-being of humans. These items are difficult to relate to human behavior and to policy. He conducted a Delphi exercise with some RAND staffers to weight these factors.“Whatever QOL is, it is determined mainly by some very general features of the individual and his environment, and not by specifics.”“A somewhat more empirical approach is furnished by the cross-sectional survey.” And he discusses a few.A research proposal: to prepare a comprehensive set [[Truncated]]
-
(1968)
Quality of Life
-
-
Dalkey1
-
41
-
-
0345047580
-
-
P-3948, The RAND Corporation, “Opinion plays a basic role in long range forecasting of technological and social development. The traditional ways of dealing with opinion have significant drawbacks. Our experiments have shown that it is possible to design techniques for putting the opinions of a group of individuals together that avoid some of these drawbacks. However, the improvement we obtain is small.”The experiment discussed is the use of the Delphi technique on almanac data with RAND personnel, as discussed in P-2986 and P-2973.
-
(1968)
Predicting the Future
-
-
Dalkey1
-
42
-
-
0004093456
-
-
RM-5888-PR, The RAND Corporation, This is one of the latest reports on the Delphi technique. It describes some experiments using the Delphi technique to output almanac-type answers from groups. It describes Delphi, examines its benefits. There is a comparison of face-to-face and anonymous interaction, with neither one showing any statistically significant advantage over the other. It also describes the mechanism of improvement with iteration, showing that the “swingers”, those not sure of their opinions, generally cause most of the improvement in the median. The whole experiment seems very preliminary, and the conclusions are not firm.
-
(1969)
The Delphi Method: An Experimental Study of Group Opinion
-
-
Dalkey1
-
43
-
-
0001536179
-
An experimental application of the Delphi method to the use of experts
-
Also the RAND Corporation, RM-727-PR (1962), This is the first unclassified report on the Delphi technique as such.“The present paper gives an account of an experiment conducted about ten years ago [presumably 1953]. The content of the paper has, for security reasons, only now been released for open publication.”“The experiment was designed to apply expert opinion to the selection from the view-point of a Soviet strategic planner, of an optimal U.S. industrial target system and to the estimation of the number of A-bombs required to reduce the munitions output by prescribed amount.”The experiment, using seven experts, is described. The questionnaires are shown, along with the resulting estimates. The final results were corrected by the authors by bringing in extra information and combining results, using their own judgment. There is a critique of the process, and some references.
-
(1963)
Management Science
, vol.9
, pp. 458-467
-
-
Dalkey1
Helmer2
-
44
-
-
84916840801
-
Analysis of the Exploratory Development Project Evaluation Experiment
-
Department of Operations Research, Case Western Reserve University, This report describes a study done for the Army Material Command using a modified Delphi procedure to evaluate research and development projects, whose value is of course often of a very subjective nature. Data was collected, using a modified Delphi, from a 12 member panel and linear regression models of project value were constructed as a function of eight critical factors. There is a description of the procedures used and conclusions drawn. Delphi has its usual attendant advantages, disadvantages and questions; the addition of regression analysis aids in the evaluation of results.
-
(1969)
Technical Memorandum No. 165
-
-
Dean1
Mathis2
-
45
-
-
84916865071
-
Forecasters turn to group guesswork
-
This is an excellent survey article of the Delphi technique, giving many examples of industrial application. It tends to be somewhat optimistic, and is of course very general.
-
(1970)
Busin. Week
-
-
-
46
-
-
0003571012
-
-
P-2982, The RAND Corporation, This is a report on an experiment using about 150 experts around the world to come up with some concensus predictions about the probable timing of various social and technological innovations. The Delphi technique was used, and the results are given as well as an explanation of the method. This paper generated a great deal of interest, apparently, in the technique.
-
(1964)
Report on a Long Range Forecasting Study
-
-
Gordon1
Helmer2
-
47
-
-
0003655203
-
-
P-3571, The RAND Corporation, A review of the various committees and bodies of experts making predictions about the more or less distant future, including Dalkey's group and many others. Includes many of the predictions and timetables.“The demonstration that no possible combination of known substances, known forms of machinery and known forms of force, can be united in a practical machine by which man shall fly long distances through the air seems to the writer as complete as it is possible for the demonstration of any physical fact to be.” Simon Newcomb, American astronomer ca. 1900.“The popular mind pictures gigantic flying machines speeding across the Atlantic and carrying innumerable passengers in a way analogous to our modern steamships.… It seems safe to say that such ideas must be wholly visionary, and even if a machine could get across with one or two passengers the expense would be prohibitive. [[Truncated]]
-
(1967)
The Year 2000
-
-
Haydon1
-
49
-
-
0342900530
-
-
P-2973, The RAND Corporation, “We are by now all familiar with the cautionary warning that the reliability of a study is by no means assured, and often not even enhanced, by the mere fact that an elaborate computer program had been set up and many machine hours of computations have gone into the production of its findings.”Although there are shortcomings to the use of intuitive judgment, it often is the only approach available and improving it is a worthwhile objective. The problem of deciding to observe experts working on real problems versus controlled experiments. Describes use of Delphi for a long-range (25–50 yr) forecasting study with T. Gordon. There is of course no way of testing the validity of the forecasts. Suggests the desirability of “some objective means of discriminating among respondents regarding their competence in answering a particular question.” The almanac experiment [[Truncated]]
-
(1964)
Convergence of Expert Concensus Through Feedback
-
-
Helmer1
-
50
-
-
0004242231
-
-
Basic Books, New York, This book argues that the social sciences should use the techniques of technology rather than the hard sciences, to try to cope with today's problems despite the fact that we don't have a complete theory. There is a discussion of the Delphi technique and description of an experiment.
-
(1966)
Social Technology
-
-
Helmes1
-
51
-
-
84864546185
-
-
P-3443, The RAND Corporation, An exercise at the University of Pittsburgh. Helmer outlines the rules of the exercise, which takes about 5 hr and 30 people. The various forms used in the game are also included. This report is essentially a detailed preparation for a stimulation game to study the future.
-
(1966)
A Use of Stimulation for the Study of Future Values
-
-
Helmer1
-
53
-
-
0003925219
-
-
P-3558, The RAND Corporation, There has been a change of attitude in viewing the future: philosophically, pragmatically and methodologically. The so-called soft sciences are on the verge of a revolution. In dealing with experts: (1) select your experts wisely; (2) create the proper conditions under which they can perform most ably; (3) if you have several experts on a particular issue available, use considerable caution in deriving from their various opinions a single combined position.Describes the Delphi technique, points out advantages and disadvantages and outlines areas of further research.
-
(1967)
Analysis of the Future: The Delphi Method
-
-
Helmer1
-
55
-
-
84916819681
-
-
also in, Essentially contains the same information, in summary form, as P-3643 and P-3611.“A first major application of the Delphi Technique to long-range forecasting in a number of different areas was carried out by Ted Gordon and myself three years ago under the auspices of the RAND Corporation.” p. 7.
-
(1967)
Vital Speeches
, vol.33
, pp. 497-499
-
-
Helmer1
-
56
-
-
84875764071
-
-
P-3607, The RAND Corporation, The size of the Scientific Establishment will at least double by the year 2000. The new attitude toward the future—a systematic analysis. Rising social consciousness among scientists. Changes in method—decay of fatalism. Lack of policy-orientation in the social sciences is being overcome. “In view of the absence of a proper theoretical foundation and the consequent inevitability of having, to some extent, to rely on intuitive judgment, we are faced with two options: we can either wait indefinitely until we have an adequate theory enabling us to deal with socio-metric and political problems as confidently as we do with problems in physics and chemistry, or we can make the most of an admittedly unsatisfactory situation and try to obtain the relevant intuitive insights of experts and then use their judgments as systematically as possible. The use of the Delphi approach represents an [[Truncated]]
-
(1967)
The Future of Science
-
-
Helmer1
-
57
-
-
84916840797
-
-
P-3611, The RAND Corporation, The social sciences must accept operations research techniques, adopt the systems approach as a basic principle, pay more than lip service to interdisciplinary collaboration and orient toward the future. Mathematics, conceptual analysis, cybernetics, model building, simulation, organization theory, analysis of the future.
-
(1967)
Methodology of Societal Studies
-
-
Helmer1
-
58
-
-
84916840796
-
-
P-3643, The RAND Corporation, Discusses the role that the forecasting of technological developments plays in shaping the future of our society. Gives some predictions for the year 2000. Forecasting is an aid to decision-makers, and can effect the future. Analysis of the future requires: (1) a survey of alternatives; (2) an analysis of preferences; (3) constructive policy research. He feels the prospects of socio-technological progress during the next third of this century are very high, due to increased scientific manpower, the computer and a reorientation toward policy-related research. Advent of social technology. Maybe a comprehensive theory of organizations.“… we may well look forward to the emergence of a new breed of modern-day constructive Utopians.” O.R.?
-
(1967)
Prospects of Technological Progress
-
-
Helmer1
-
59
-
-
0344587422
-
-
P-3721, The RAND Corporation, The scope of Delphi is greater than just trying to forecast the future. Its properties: feedback, anonymous debate, statistical analysis of the results. Can get convergence or two poles. All this is very much in the experimental stage. He gives ideas for further investigation: differential weights to opinions of different experts, use of on-line computer consoles. Problems of round-table discussions. Proposes a hierarchical structure of panels. There is a good summary at the end of the paper.
-
(1967)
Systematic Use of Expert Opinions
-
-
Helmer1
-
60
-
-
84898043948
-
-
P-2718, The RAND Corporation, This is one of the earlier papers. It “… considers the possible use of operational gaming, or simulation using human players, to examine an economy as a whole.” “[Operational gaming] undoubtedly is most useful when applied with a clear objective in mind to well-structured problems based on abundant data.” However, there is more chance in a developing economy of applying operations research to “strategic” problems. The model is seen essentially as a communications medium. Provides a good argument for use of simulation before one has an adequate theory. Discusses computer simulation, and the objections to it, then advantages of gaming, where use of computers is minimal, reliance is on the intuitive expertise of specialists, and emphasis is on clearer problem formulation. For gaming, break down economy into sectors. Include intangibles, also get feedback. [[Truncated]]
-
(1963)
An Approach to the Study of a Developing Economy by Operational Gaming
-
-
Helmer1
Quade2
-
62
-
-
0002234307
-
On the Epistemology of the Inexact Sciences
-
also in, This is a fairly long and complex paper that tries to establish a clearer definition for the “soft” sciences. Epistemology “is concerned with the role of evidence in the attainment of scientific laws and with the scientific procedures implied by that role”. Abstract—This is a new epistemological approach to the inexact sciences. The purpose of all science is to explain and predict in an adjective manner. While in the exact sciences explanation and prediction have the same logical structure, this is not so in the inexact sciences. This permits various methodological innovations in the exact sciences, e.g. expert judgment and simulation.It is a fiction of long standing that there are two classes of science, the exact and the inexact, and that the social sciences by and large are members of the second class … This widely prevalent attitude seems to us fundamentally mistaken, for it finds a [[Truncated]]
-
(1959)
Management Science
, vol.6
, pp. 25-52
-
-
Helmer1
Rescher2
-
63
-
-
0344957029
-
The prediction of social and technological events
-
also the RAND Corporation, P-93 (1949)., Abstract—A group of individuals with a generally high education level was asked to make predictions concerning a large number of future events in order to investigate certain aspects of the use of expert opinion in policy making. In particular, the following questions were explored: how good are expert predictions in areas germane to policy; how can such predictions be improved; and how can the reliability of a given opinion be appraised beforehand? While the design of the study makes projection of its results on other situaations questionable, it throws considerable light on the problems involved. Among other results, it was found that confidence in prediction does not necessarily show a correlation with success in prediction, that predictions made by groups of people are more likely to be right than predictions made by the same individuals working alone, and that the reliability of predictions can [[Truncated]]
-
(1950)
Public Opinion Quarterly
, vol.14
, pp. 13-110
-
-
Kaplan1
Skogstad2
Girshick3
-
64
-
-
0014114056
-
Assets and liabilities in group problem solving: The need for an integrative function
-
Abstract—Research on group problem solving reveals that the group has both advantages and disadvantages over individual problem solving. If the potentials for group problem solving can be exploited and if its deficiencies can be avoided, it follows that group problem solving can attain a level of proficiency not ordinarily achieved. The requirement for achieving this level of group performance seems to hinge on developing a style of discussion leadership which maximizes the group's assets and minimizes its habilities. Since members possess the essential ingredients for the solutions, the deficiencies that appear in group solutions reside in the processes by which group solutions develop. These processes can determine whether the group functions effectively or ineffectively. The critical factor in a group's potential is organization and integration. With training, a leader can supply these functions and serve as the group's central nervous system, thus [[Truncated]]
-
(1967)
Psychol. Rev.
, vol.74
, pp. 239-249
-
-
Maier1
-
66
-
-
84916820667
-
-
Translated from, “The present of today is 10 or 15 yr in advance, and the future begins after that.”“Most of the ills from which we suffer at present—the lack of school buildings and teachers, housing shortages, traffic problems and air pollution—seem insoluble because we have not taken advantage of some twenty years of forecasters capable simultaneously by predicting the features of the world to come, of alerting us to these matters and of making the necessary recommendations.”Discusses Futuribles, directed by Bertrand de Jouvenel, and the Prospective journal and institute, founded in 1957 by Gaston Berger for the “study of technical, scientific, economic and social causes which are speeding up the evolution of the modern world, and for the forecasting of situations which could arise from their combined influences”.According to Helmer, Futuribles influenced American intellectuals toward [[Truncated]]
-
(1966)
Realites
, Issue.245
, pp. 50-58
-
-
Neiswender1
-
68
-
-
77956897082
-
-
P-3529, The RAND Corporation, New approaches and techniques of cost-effectiveness analysis are being proposed constantly; the field is changing. He is concerned with three trends: computers, mathematics and the direct use of expertise. Concentration by the author is on the latter. He sees great advances in computers and their use, concentration on game theory in mathematics. Talks about the opposition to cost/effectiveness. Discusses Delphi.“The first experiment with the Delphi process (about 1948) used several racing forms in an attempt to improve the prediction of horse race outcomes. Although the experiment showed promise, criticism of its subject matter and some obvious defects set the efforts back about ten years or so.”
-
(1967)
Cost-Effectiveness: Some Trends in Analysis
-
-
Quade1
-
69
-
-
84916840802
-
Evaluation and Budgeting Model for a System of Social Agencies
-
Department of Operations Research, Case Western Reserve University, This report describes a relatively large-scale experiment on scaling group opinion using a procedure based essentially on Delphi, in that it used controlled feedback and anonymity. The main difference was that an overt attempt was made to achieve concensus, rather than stopping at a prescribed number of rounds.The objective of the study was to estimate the relative values of services performed by the agencies of the Jewish Community Federation of Cleveland, as perceived by the members of that community. There was a test run using members of the research staff who were themselves members of the community, then a procedure using the J.C.F. staff, and finally a panel of community lay leaders. There was a ranking of 250 “client-service packages”. Correlation was surprisingly good among the three groups. The experiment demonstrated the feasibility of obtaining a set [[Truncated]]
-
(1969)
Technical Memorandum No. 167
-
-
Reisman1
Mantel2
Dean3
Eisenberg4
-
70
-
-
84916840800
-
On a Computer-Aided Systems Approach to Personnel Administration
-
Department of Operations Research, University, Abstract—A systematic evaluation methodology has been developed which integrates some major concepts from value, utility, decision, subjective probability theories and the Delphi method for obtaining a consensus of opinions. These theories are applied to the process of evaluation of personnel for recruitment, promotions, merit raises, transfer, salary administration, training and development. The model requires and utilizes as inputs explicitly stated sets of long range goals, short-range objectives, resource needs, evaluative criteria, weighting and utility functions, as well as the subjective judgments of appropriate evaluators. The processing of this information may be implemented by manual calculations, batch processing on an IBM 1620 computer, or by direct simulation on a large time-sharing computer system. By utilizing standard statistical procedures and the decision rule to maximize [[Truncated]]
-
(1968)
Technical Memorandum No. 147
-
-
Reisman1
Taft2
-
71
-
-
84916840799
-
Computer Time-Saving Applications in Economic Analysis: An Integrated Approach
-
Department of Operations Research, Case Western Reserve University, Abstract—A generalized evaluation program known as EVAL, and a general “present worth” program known as CERBS have been developed in the FORTRAN IV Computer language and are currently operational on a time-sharing or batch basis. The programs lead an individual or a group of people through a systematic step-by-step procedure for evaluating the relative value (utility including present worth) of each of a given set of alternatives with respect to explicitly stated goals and objectives. Applications in such areas as equipment design, selection, optimization and replacement; personnel administration; and allocation of resources are presented.
-
(1969)
Technical Memorandum No. 151
-
-
Reisman1
Taft2
-
72
-
-
84916840794
-
-
P-3593, The RAND Corporation, Limits “the perspective to two sectors of particular interest alike to makers of public policy and to reflective citizens in the modern world: science and technology on the one hand, and our human and social environment upon the other”.It is now in fashion to speculate about the future. The Futures Industry. The problem of predictive methodology. Some major difficulties for prediction: feedback, chance occurrence, fashions, values, the problem of data.This is an excellent summary of the future as an object of research.
-
(1967)
The Future as an Object of Research
-
-
Rescher1
-
73
-
-
84916840793
-
-
P-3735, The RAND Corporation, This paper attempts to analyze and clarify the possible interaction of prediction and event. (1) Is the interaction of prediction and event predictable? (2) Can the interaction of prediction and event be controlled ? (3) How and to what extent should people making predictions try to take into account the possible effects of their predictions? (4) Is the entire phenomenon of self-affecting predictions of any practical use?He uses flow charts, a probabilistic model, a cybernetic model and discusses various types of prediction. In general, this is a good overview of the problem, without any real theory of answers.
-
(1967)
Some Comments on the Problem of Self-Affecting Predictions
-
-
Rochberg1
-
74
-
-
84915449647
-
The Design of a Policy Delphi
-
National Resource Analysis Center, Systems Evaluation Division, Executive Office of the President, Office of Emergency Preparedness, This paper is a comprehensive and definitive examination of the use of the Delphi method in policy issues. It develops the thesis that the Delphi could be useful as an adjunct to the committee approach in policy formulation; it is argued that the two are complimentary, rather than one being superior.The paper opens with a review of Delphi, argues for its use in policy decisions while pointing out possible dangers, and recommends some procedural guidelines.There is a highly complete and up-to-date bibliography included.
-
(1970)
Technical Memorandum T.M.-123
-
-
Turoff1
|
