Expert Judgment Quantification

Green Energy and Technology

Volumn 60, Issue , 2013, Pages 31-65

  Amer, Muhammad ^a   Daim, Tugrul ^a  

^a PORTLAND STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ENERGY MANAGEMENT; ENERGY RESOURCES;

DECISION MAKERS; EXPERT JUDGMENT; EXPERT JUDGMENT QUANTIFICATIONS;

DECISION MAKING;

EID: 84883163879     PISSN: 18653529     EISSN: 18653537     Source Type: Book Series    
DOI: 10.1007/978-1-4471-5097-8_3     Document Type: Article

References (172)

1. Rohrbaugh J (1979) Improving the quality of group judgment: social judgment analysis and the Delphi technique. Organ Behav Hum Perform 24:73-92
2. Meyer MA, Booker JM (1991) Eliciting and analyzing expert judgment: a practical guide. Academic Press Limited, London
3. Keeney RL, von Winterfeldt D (1989) On the uses of expert judgment on complex technical problems. IEEE Trans Eng Manage 36:83-86
4. Durance P, Godet M (2010) Scenario building: uses and abuses. Technol Forecast Soc Chang 77:1488-1492
5. Bradfield R, Wright G, Burt G, Cairns G, Van Der Heijden K (2005) The origins and evolution of scenario techniques in long range business planning. Futures 37:795-812
6. Cooke RM (1991) Experts in uncertainty: opinion and subjective probability in science. Oxford University Press, New York
7. Simola K, Mengolini A, Bolado-Lavin R (2005) Formal expert judgment: an overview. European Commission, Directorate General Joint Research Centre (DG JRC), Institute for Energy. http://ie.jrc.ec.europa.eu/publications/scientific_publications/2005/EUR21772EN. pdf. Accessed 18 March 2011
8. Börjeson L, Höjer M, Dreborg K-H, Ekvall T, Finnveden G (2006) Scenario types and techniques: towards a user's guide. Futures 38:723-739
9. Amer M, Daim TU (2010) Application of technology roadmaps for renewable energy sector. Technol Forecast Soc Chang 77:1355-1370
10. Gerdsri N (2005) An analytical approach on building a technology development envelope (TDE) for roadmapping of emerging technologies. Ph.D. Dissertation, Systems Science: Engineering Management, Portland State University, Portland
11. Gerdsri P (2009) A systematic approach to developing national technology policy and strategy for emerging technologies. Ph.D. Dissertation, Engineering and Technology Management (ETM) Department, Portland State University, Portland
12. Schwartz P (1996) The Art of the Long View: Planning for the Future in an Uncertain World. Currency Doubleday, New York
13. Chermack TJ, Lynham SA, Ruona WEA (2001) A review of scenario planning literature. Futures Res Q 17:7-31
14. Kahn H, Wiener AJ (1967) The year 2000: a framework for speculation on the next thirtythree years. The Macmillan, New York
15. Varum CA, Melo C (2007) Strategic planning in an uncertain business environment: the diffusion of scenario planning. In: Conference Factores de Competitividade, Competitiveness Factors: A Portuguese Perspective, Aveiro
16. Mietzner D, Reger G (2005) Advantages and disadvantages of scenario approaches for strategic foresight. Int J Technol Intell Planning 1:220-239
17. Varum CA, Melo C (2010) Directions in scenario planning literature: a review of the past decades. Futures 42:355-369
18. Van Der Heijden K (1996) Scenarios: the art of strategic conversation. Wiley, Chichester
19. Rigby D, Bilodeau B (2007) Selecting management tools wisely. Harv Bus Rev 85:20-22
20. Porter AL, Roper AT, Mason TW, Rossini FA, Banks J (1991) Forecasting and management of technology, 1st edn. Wiley, New York
21. Saritas O, Oner MA (2004) Systemic analysis of UK foresight results: joint application of integrated management model and roadmapping. Technol Forecast Soc Chang 71:27-65
22. Chermack TJ, Lynham SA, van der Merwe L (2006) Exploring the relationship between scenario planning and perceptions of learning organization characteristics. Futures 38:767-777
23. Schoemaker PJH, van der Heijden CAJM (1992) Integrating scenarios into strategic planning at royal Dutch/Shell. Strategy Leadersh 20:41-46
24. Joseph CF (2000) Scenario planning. Technol Forecast Soc Chang 65:115-123
25. Hiltunen E (2009) Scenarios: process and outcome. J Futures Stud 13:151-152
26. Linneman RE, Klein HE (1979) The use of multiple scenarios by U.S. industrial companies. Long Range Plan 12:83-90
27. Linneman RE, Klein HE (1983) The use of multiple scenarios by U.S. industrial companies: a comparison study, 1977-1981. Long Range Plan 16:94-101
28. Amer M, Daim TU, Jetter A (2013) A review of scenario planning. Futures 46:23-40
29. Blomgren H, Jonsson P, Lagergren F (2011) Getting back to scenario planning: strategic action in the future of energy Europe. In: 8th international conference on the European energy market (EEM), Zagreb, pp 792-801
30. Silberglitt R, Hove A, Shulman P (2003) Analysis of the US energy scenarios: metascenarios, pathways, and policy implications. Technol Forecast Soc Chang 70:297-315
31. Czaplicka-Kolarz K, Stanczyk K, Kapusta K (2009) Technology foresight for a vision of energy sector development in Poland till 2030. Delphi survey as an element of technology foresighting. Technol Forecast Soc Chang 76:327-338
32. Di W, Rui N, Hai-ying S (2011) Scenario analysis of China's primary energy demand and CO2 emissions based on IPAT model. Energy Procedia 5:365-369
33. Keles D, Möst D, Fichtner W (2011) The development of the German energy market until 2030-a critical survey of selected scenarios. Energy Policy 39:812-825
34. Zhou N, Lin J (2008) The reality and future scenarios of commercial building energy consumption in China. Energy Build 40:2121-2127
35. Winebrake JJ, Creswick BP (2003) The future of hydrogen fueling systems for transportation: an application of perspective-based scenario analysis using the analytic hierarchy process. Technol Forecast Soc Chang 70:359-384
36. Sørensen B, Hauge Petersen A, Juhl C, Ravn H, Søndergren C, Simonsen P, Jørgensen K, Henrik Nielsen L, Larsen HV, Erik Morthorst P, Schleisner L, Sørensen F, Engberg Pedersen T (2004) Hydrogen as an energy carrier: scenarios for future use of hydrogen in the Danish energy system. Int J Hydrogen Energy 29:23-32
37. Wietschel M, Hasenauer U, de Groot A (2006) Development of European hydrogen infrastructure scenarios-CO2 reduction potential and infrastructure investment. Energy Policy 34:1284-1298
38. Contaldi M, Gracceva F, Mattucci A (2008) Hydrogen perspectives in Italy: analysis of possible deployment scenarios. Int J Hydrogen Energy 33:1630-1642
39. Antoine B, Goran K, Neven D (2008) Energy scenarios for Malta. Int J Hydrogen Energy 33:4235-4246
40. Weisser D (2004) Costing electricity supply scenarios: a case study of promoting renewable energy technologies on Rodriguez, Mauritius. Renewable Energy 29:1319-1347
41. Chen T-Y, Yu OS, Hsu GJY, Hsu F-M, Sung W-N (2009) Renewable energy technology portfolio planning with scenario analysis: a case study for Taiwan. Energy Policy 37:2900-2906
42. Madlener R, Kowalski K, Stagl S (2007) New ways for the integrated appraisal of national energy scenarios: the case of renewable energy use in Austria. Energy Policy 35:6060-6074
43. Varho V, Tapio P (2005) Wind power in Finland up to the year 2025-'soft' scenarios based on expert views. Energy Policy 33:1930-1947
44. Morales JM, Mínguez R, Conejo AJ (2010) A methodology to generate statistically dependent wind speed scenarios. Appl Energy 87:843-855
45. Rowe G, Wright G (1999) The Delphi technique as a forecasting tool: issues and analysis. Int J Forecast 15:353-375
46. Raubitschek R (1988) Multiple scenario analysis and business planning. In: Lamb R, Shrivastava P (eds) Advances in strategic management, vol 5. JAI Press Inc., London
47. Linstone HA, Turoff M (1975) The Delphi method: techniques and applications. Addison-Wesley, Reading
48. Martino JP (1983) Technological forecasting for decision making, 2nd edn. North-Holland, New York
49. Rowe G, Wright G, Bolger F (1991) Delphi: a reevaluation of research and theory. Technol Forecast Soc Chang 39:235-251
50. Erffmeyer RC, Erffmeyer ES, Lane IM (1986) The Delphi technique: an empirical evaluation of the optimal number of rounds. Group and Organ Manage 11:120-128
51. Lindeman CA (1975) Delphi survey of priorities in clinical nursing research. Nurs Res 24:434-441
52. Torrance EP (1957) Group decision-making and disagreement. Soc Forces 35:314-318
53. Powell C (2003) The Delphi technique: myths and realities. J Adv Nurs 41:376-382
54. Woudenberg F (1991) An evaluation of Delphi. Technol Forecast Soc Chang 40:131-150
55. Jairath N, Weinstein J (1994) The Delphi methodology: a useful administrative approach. Can J Nurs Adm 7:29-42
56. Linstone HA, Turoff M (2011) Delphi: a brief look backward and forward. Technol Forecast Soc Chang 78:1712-1719
57. Chaffin WW, Talley WK (1980) Individual stability in Delphi studies. Technol Forecast Soc Chang 16:67-73
58. Gordon T, Pease A (2006) RT Delphi: an efficient, "round-less" almost real time Delphi method. Technol Forecast Soc Chang 73:321-333
59. Celiktas MS, Kocar G (2010) From potential forecast to foresight of Turkey's renewable energy with Delphi approach. Energy 35:1973-1980
60. Steinert M (2009) A dissensus based online Delphi approach: an explorative research tool. Technol Forecast Soc Chang 76:291-300
61. Geist MR (2010) Using the Delphi method to engage stakeholders: a comparison of two studies. Eval Program Plann 33:147-154
62. Rauch W (1979) The decision Delphi. Technol Forecast Soc Chang 15:159-169
63. Turoff M (1970) The design of a policy Delphi. Technol Forecast Soc Chang 2:149-171
64. Delbecq AL, Van de Ven AH, Gustafson DH (1975) Group techniques for program planning. A guide to nominal group and Delphi processes. Scott, Foresman and Company, Glenview
65. Murphy M, Black N, Lamping D, McKee CM, Sanderson CFB, Askham J (1998) Consensus development methods, and their use in clinical guideline development. Health Technol Assess 2:1-65
66. Pill J (1971) The Delphi method: substance, context, a critique and an annotated bibliography. Socio-Econ Plan Sci 5:57-71
67. Graham B, Regehr G, Wright JG (2003) Delphi as a method to establish consensus for diagnostic criteria. J Clin Epidemiol 56:1150-1156
68. Brockhoff K (1975) The performance of forecasting groups in computer dialogue and face to face discussions. In: Linstone HA, Turoff M (eds) The Delphi method: techniques and applications. Addison-Wesley, Reading
69. Riggs WE (1983) The Delphi technique: an experimental evaluation. Technol Forecast Soc Chang 23:89-94
70. Larreche JC, Moinpour R (1983) Managerial judgment in marketing: the concept of expertise. J Mark Res 20:110-121
71. Coates JF (1975) In defense of Delphi: a review of Delphi assessment, expert opinion, forecasting, and group process by H. Sackman. Technol Forecast Soc Chang 7:193-194
72. Sackman H (1974) Delphi critique: expert opinion, forecasting, and group process. Lexington Books, Lexington
73. Martino JP (2003) A review of selected recent advances in technological forecasting. Technol Forecast Soc Chang 70:719-733
74. Breiner S, Cuhls K, Grupp H (1994) Technology foresight using a Delphi approach: a Japanese German study. R&D Manage 24:141-153
75. Biloslavo R, Dolinek S (2008) Scenario planning for climate strategies development by integrating group Delphi, AHP and dynamic fuzzy cognitive maps. In: Portland International Center for Management of Engineering and Technology (PICMET), Cape Town, South Africa, pp 1103-1111
76. Huss WR, Honton EJ (1987) Scenario planning-what style should you use? Long Range Plan 20:21-29
77. Bengisu M, Nekhili R (2006) Forecasting emerging technologies with the aid of science and technology databases. Technol Forecast Soc Chang 73:835-844
78. Gupta UG, Clarke RE (1996) Theory and applications of the Delphi technique: a bibliography (1975-1994). Technol Forecast Soc Chang 53:185-211
79. Iniyan S, Sumathy K (2000) An optimal renewable energy model for various end-uses. Energy 25:563-575
80. Iniyan S, Sumathy K (2003) The application of a Delphi technique in the linear programming optimization of future renewable energy options for India. Biomass Bioenergy 24:39-50
81. Pätäri S (2010) Industry-and company-level factors influencing the development of the forest energy business-insights from a Delphi study. Technol Forecast Soc Chang 77:94-109
82. Rikkonen P, Tapio P (2009) Future prospects of alternative agro-based bioenergy use in Finland-Constructing scenarios with quantitative and qualitative Delphi data. Technol Forecast Soc Chang 76:978-990
83. Ronde P (2003) Delphi analysis of national specificities in selected innovative areas in Germany and France. Technol Forecast Soc Chang 70:419-448
84. Sharma DP, Nair PSC, Balasubramanian R (2003) Analytical search of problems and prospects of power sector through Delphi study: case study of Kerala state, India. Energy Policy 31:1245-1255
85. Suganthi L, Williams A (2000) Renewable energy in India-a modelling study for 2020-2021. Energy Policy 28:1095-1109
86. Utgikar VP, Scott JP (2006) Energy forecasting: predictions, reality and analysis of causes of error. Energy Policy 34:3087-3092
87. Blind K, Cuhls K, Grupp H (2001) Personal attitudes in the assessment of the future of science and technology: a factor analysis approach. Technol Forecast Soc Chang 68:131-149
88. Cuhls K, Kuwahara T (1994) Outlook of Japanese and German future technology: comparing technology forecast surveys. Phisica-Verlag, Heidelberg
89. Amer M, Daim T (2012) Technology and science policies in transitional economy: a case of Turkey. Sci Technol Soc 17:297-321
90. Gerdsri P Kocaoglu D (2009) A systematic approach to developing national technology policy and strategy for emerging technologies: a case study of nanotechnology for Thailand's agriculture industry. In: Portland International Center for Management of Engineering and Technology (PICMET), Portland, pp 447-461
91. Chakravarti AK, Vasanta B, Krishnan ASA, Dubash RK (1998) Modified Delphi methodology for technology forecasting case study of electronics and information technology in India. Technol Forecast Soc Chang 58:155-165
92. Shin T (1998) Using Delphi for a long-range technology forecasting, and assessing directions of future R&D activities the Korean exercise. Technol Forecast Soc Chang 58:125-154
93. Tichy G (2004) The over-optimism among experts in assessment and foresight. Technol Forecast Soc Chang 71:341-363
94. Daim T, Yates D, Peng Y, Jimenez B (2009) Technology assessment for clean energy technologies: the case of the Pacific northwest. Technol Soc 31:232-243
95. Saaty TL (2003) Decision-making with the AHP: why is the principal eigenvector necessary. Eur J Oper Res 145:85-91
96. Wang J-J, Jing Y-Y, Zhang C-F, Zhao J-H (2009) Review on multi-criteria decision analysis aid in sustainable energy decision-making. Renew Sustain Energy Rev 13:2263-2278
97. Salo A, Gustafsson T, Ramanathan R (2003) Multicriteria methods for technology foresight. J Forecast 22:235-255
98. Elkarmi F, Mustafa I (1993) Increasing the utilization of solar energy technologies (SET) in Jordan analytic hierarchy process. Energy Policy 21:978-982
99. Forman EH, Gass SI (2001) The analytic hierarchy process-an exposition. Operations Research 49:469-486
100. Saaty TL (1980) The analytic hierarchy process: planning, priority setting, resource allocation. McGraw Hill International, New York
101. Gerdsri N, Kocaoglu DF (2007) Applying the analytic hierarchy process (AHP) to build a strategic framework for technology roadmapping. Math Comput Model 46:1071-1080
102. Kocaoglu DF (1983) A participative approach to program evaluation. IEEE Trans Eng Manage 30:112-118
103. Kablan MM (2004) Decision support for energy conservation promotion: an analytic hierarchy process approach. Energy Policy 32:1151-1158
104. Ahsan MK, Bartlema J (2004) Monitoring healthcare performance by analytic hierarchy process: a developing-country perspective. Int Trans Oper Res 11:465-478
105. Yates D, Jimenez BT, Peng Y (2007) Portland general electric (PGE): clean power generation wind project in biglow canyon boardman coal plant. In: PICMET, Portland, pp 2530-2549
106. Akash BA, Mamlook R, Mohsen MS (1999) Multi-criteria selection of electric power plants using analytical hierarchy process. Electric Power Syst Res 52:29-35
107. Lee SK, Yoon YJ, Kim JW (2007) A study on making a long-term improvement in the national energy efficiency and GHG control plans by the AHP approach. Energy Policy 35:2862-2868
108. Lee SK, Mogi G, Kim JW (2008) The competitiveness of Korea as a developer of hydrogen energy technology: the AHP approach. Energy Policy 36:1284-1291
109. Lee SK, Mogi G, Kim JW, Gim BJ (2008) A fuzzy analytic hierarchy process approach for assessing national competitiveness in the hydrogen technology sector. Int J Hydrogen Energy 33:6840-6848
110. Lee AHI, Chen HH, Kang H-Y (2008) Multi-criteria decision making on strategic selection of wind farms. Renewable Energy 34:120-126
111. Aras H, Erdogmus S, Koc E (2004) Multi-criteria selection for a wind observation station location using analytic hierarchy process. Renewable Energy 29:1383-1392
112. Jaber JO, Jaber QM, Sawalha SA, Mohsen MS (2008) Evaluation of conventional and renewable energy sources for space heating in the household sector. Renew Sustain Energy Rev 12:278-289
113. Lee TL, Lin HM, Jeng DS, Hsu TW (2008) Application of fuzzy analytic hierarchy process to assess the potential of offshore wind energy in Taiwan. In: Proceedings of the international offshore and polar engineering conference. Vancouver, pp 461-465
114. Pilavachi PA, Chatzipanagi AI, Spyropoulou AI (2009) Evaluation of hydrogen production methods using the analytic hierarchy process. Int J Hydrogen Energy 34:5294-5303
115. Tsoutsos T, Drandaki M, Frantzeskaki N, Iosifidis E, Kiosses I (2009) Sustainable energy planning by using multi-criteria analysis application in the island of Crete. Energy Policy 37:1587-1600
116. Wijayatunga PDC, Siriwardena K, Fernando WJLS, Shrestha RM, Attalage RA (2006) Strategies to overcome barriers for cleaner generation technologies in small developing power systems: Sri Lanka case study. Energy Convers Manage 47:1179-1191
117. Chatzimouratidis AI, Pilavachi PA (2008) Multicriteria evaluation of power plants impact on the living standard using the analytic hierarchy process. Energy Policy 36:1074-1089
118. Chatzimouratidis AI, Pilavachi PA (2009) Technological, economic and sustainability evaluation of power plants using the analytic hierarchy process. Energy Policy 37:778-787
119. Clarke RR (1997) Validation and legitimation of an analytic hierarchy approach to integrated resource planning for electric utilities. In: Energy conversion engineering conference (IECEC-97), Honolulu, pp 2197-2201
120. Afgan NH, Carvalho MG (2002) Multi-criteria assessment of new and renewable energy power plants. Energy 27:739-755
121. Amer M, Daim TU (2011) Selection of renewable energy technologies for a developing country: a case study of Pakistan. Energy Sustain Dev 15:420-435
122. Millett SM, Honton EJ (1991) A manager's guide to technology forecasting and strategy analysis methods. Battelle Press, Columbus
123. Rohrbaugh J (1981) Improving the quality of group judgment: social judgment analysis and the nominal group technique. Organ Behav Hum Perform 28:272-288
124. Herbert TT, Yost EB (1979) A comparison of decision quality under nominal and interacting consensus group formats: the case of the structured problem. Decis Sci 10:358-370
125. Islam R (2001) Modification of the nominal group technique by using the analytic hierarchy process. In: Multiple Criteria Decision Making in the New Millennium, vol 507, pp 294-303
126. Henderson NR (2009) Managing moderator stress: take a deep breath. You can do this! Mark Res 21:28-29
127. Sutton RI, Hargadon A (1996) Brainstorming groups in context: effectiveness in a product design firm. Adm Sci Q 41:685-718
128. Miles I, Keenan M (2002) Practical guide to regional foresight in the United Kingdom. Directorate-General for Research, European Commission. http://cordis.europa.eu/foresight/ cgrf.htm. Accessed 14 March 2011
129. Keeney RL, von Winterfeldt D (1991) Eliciting probabilities from experts in complex technical problems. IEEE Trans Eng Manage 38:191-201
130. Cojazzi G, Fogli D, Grassini G, De Gelder P, Gryffroy D, Bolado R, Hofer E, Virolainen R, Coe IM, Bassanelli A, Puga J, Papazoglou I, Zuchuat O, Cazzoli E, Eyink J, Guida G, Pinola L, Pulkkinen U, Simola K, von Winterfeldt D, Valeri A (2001) Benchmark exercise on expert judgment techniques in PSA Level 2. Nucl Eng Des 209:211-221
131. Cojazzi G, Fogli D (2000) Benchmark exercise on expert judgment techniques in PSA level 2, extended final report. http://ie.jrc.ec.europa.eu/publications/scientific_publications/2005/ EUR21772EN.pdf
132. Meyer MA, Booker JM (1991) Selecting and motivating the experts. In: Eliciting and analyzing expert judgment: a practical guide, pp 85-98, 1st edn. Academic Press Limited, London
133. Camerer CF, Johnson EJ (1991) The process-performance paradox in expert judgment. In: Ericsson KA, Smith J (eds) Toward a general theory of expertise: prospects and limits, 1st edn. Cambridge University Press, Cambridge, pp 195-217
134. McGraw KL, Harbison-Briggs K (1989) Knowledge acquisition: principles and guidelines. Prentice-Hall International, New Jersey
135. Landeta J (2006) Current validity of the Delphi method in social sciences. Technol Forecast Soc Chang 73:467-482
136. Jetter AJ (2006) Elicitation-extracting knowledge from experts. In: Jetter AJ, Kraaijenbrink J, Schroder H-H, Wijnhoven F (eds) Knowledge integration: the practice of knowledge management in small and medium enterprises, pp 65-76. Physica-Verlag, Heidelberg
137. Schaller RR (2004) Technology innovation in the semiconductor industry: a case study of the international technology roadmap for semiconductors (ITRS) Ph.D. Dissertation, School of Public Policy, George Mason University, Fairfax
138. Shanteau J (1992) The psychology of experts: an alternative view. In: Wright, G. Bolger F (eds) Expertise and decision support, Plenum Press, pp 11-23. New York
139. Ascher W (1978) Forecasting: An Appraisal for Policymakers and Planners. Johns Hopkins University Press, Baltimore
140. Mitchell VW (1991) The Delphi technique: an exposition and application. Technol Anal Strat Manage 3:333-358
141. Okoli C, Pawlowsk SD (2004) The Delphi method as a research tool: an example, design considerations and applications. Inf Manage 42:15-29
142. Daim TU (1998) Technology evaluation and acquisition strategies and their implications in the U.S. electronics manufacturing industry. Ph.D. Dissertation, System Science: Engineering and Technology Management, Portland State University, Portland
143. Hill KQ, Fowles J (1975) The methodological worth of the Delphi forecasting technique. Technol Forecast Soc Chang 7:179-192
144. Cooke RM, Goossens LHJ (2000) Procedures guide for structured expert judgement. http:// ie.jrc.ec.europa.eu/publications/scientific_publications/2005/EUR21772EN.pdf. Accessed 12 May 2011
145. Dajani JS, Sincoff MZ, Talley WK (1979) Stability and agreement criteria for the termination of Delphi studies. Technol Forecast Soc Chang 13:83-90
146. Kastein MR, Jacobs M, van der Hell RH, Luttik K, Touw-Otten FWMM (1993) Delphi, the issue of reliability: a qualitative Delphi study in primary health care in the Netherlands. Technol Forecast Soc Chang 44:315-323
147. Goodman LA, Kruskal WH (1959) Measures of association for cross classifications. II: further discussion and references. J Am Stat Assoc 54:123-163
148. Mumpower JL, Stewart TR (1996) Expert judgement and expert disagreement. Thinking Reasoning 2:191-211
149. Bland JM, Altman DG (1997) Statistics notes: Cronbach's alpha. Br Med J 314:572
150. Terrados J, Almonacid G, Pérez-Higueras P (2009) Proposal for a combined methodology for renewable energy planning. Application to a Spanish region. Renew Sustain Energy Rev 13:2022-2030
151. Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420-428
152. Wuensch KL (2007) Inter-rater agreement. http://www.ecu.edu/psyc/
153. Coates JF (1975) In defense of Delphi: a review of Delphi assessment: expert opinion, forecasting, and group process by H. Sackman. Technol Forecast Soc Chang 7:193-194
154. Kandasamy WBV, Smarandache F (2003) Fuzzy cognitive maps and neutrosophic cognitive maps. Indian institute of technology, Chennai
155. Taber R (1991) Knowledge processing with fuzzy cognitive maps. Expert Syst Appl 2:83-87
156. Taber W, Siegel M (1987) Estimation of experts' weights using fuzzy cognitive maps. In: IEEE International Conference on Neural Networks, pp 319-326
157. Taber R, Yager RR, Helgason CM (2007) Quantization effects on the equilibrium behavior of combined fuzzy cognitive maps. Int J Intell Syst 22:181
158. Hans-Horst S, Jetter AJM (2003) Integrating market and technological knowledge in the fuzzy front end: an FCM-based action support system. Int J Technol Manage 26:517-539
159. Jetter AJM (2003) Educating the guess: strategies, concepts, and tools for the fuzzy front end of product development. In: Portland International Center for Management of Engineering and Technology (PICMET), Portland, pp 261-273
160. Amer M, Jetter AJ, Daim TU (2011) Development of fuzzy cognitive map (FCM) based scenarios for wind energy. Int J Energy Sect Manage 5:564-584
161. Babbie ER (2007) The practice of social research, 12th edn. Wadsworth, Belmont
162. Ono R, Wedemeyer DJ (1994) Assessing the validity of the Delphi technique. Futures 26:289-304
163. Martino JP (1970) The consistency of Delphi forecasts. Futurist 4:63-64
164. Keeney S, Hasson F, McKenna HP (2001) A critical review of the Delphi technique as a research methodology for nursing. Int J Nurs Stud 38:195-200
165. Engels TCE, Powell Kennedy H (2007) Enhancing a Delphi study on family-focused prevention. Technol Forecast Soc Chang 74:433-451
166. van Zolingen SJ, Klaassen CA (2003) Selection processes in a Delphi study about key qualifications in senior secondary vocational education. Technol Forecast Soc Chang 70:317-340
167. Jolson MA, Rossow GL (1971) The Delphi process in marketing decision making. J Mark Res 8:443-448
168. Brown BB, Helmer O (1964) Improving the reliability of estimates obtained from a consensus of experts, pp 1-13. http://www.rand.org/pubs/papers/P2986
169. Dalkey NC, Brown BB, Cochran SW (1970) The Delphi Method IV: effect of percentile feedback and feed-in of relevant facts. http://www.rand.org/pubs/research_memoranda/ RM6118
170. Landeta J, Matey J, Ruíz V, Galter J (2008) Results of a Delphi survey in drawing up the input-output tables for Catalonia. Technol Forecast Soc Chang 75:32-56
171. Chambers JC, Mullick SK, Smith DD (1971) How to choose the right forecasting technique. Harv Bus Rev 49:45-74
172. Levary RR, Han D (1995) Choosing a technological forecasting method, Indus Manag 37:14-18