Measuring societal effects of transdisciplinary research projects: Design and application of an evaluation method

Evaluation and Program Planning

Volumn 30, Issue 4, 2007, Pages 325-338

  Walter, Alexander I ^a   Helgenberger, Sebastian ^a   Wiek, Arnim ^a   Scholz, Roland W ^a  

^a ETH ZURICH   (Switzerland)

Author keywords

Evaluation model; Impact evaluation; Societal effects; Societal impact; Stakeholder participation; Transdisciplinary research

Indexed keywords

DECISION MAKING; LOCAL PARTICIPATION; SOCIAL IMPACT; SOCIAL NETWORK; STAKEHOLDER;

ADULT; ARTICLE; CONSUMER; DECISION SUPPORT SYSTEM; FEMALE; HEALTH CARE QUALITY; HUMAN; INTERDISCIPLINARY COMMUNICATION; MALE; METHODOLOGY; MIDDLE AGED; PERSONNEL; QUESTIONNAIRE; REGRESSION ANALYSIS; RESEARCH; SCIENCE; SOCIAL BEHAVIOR; SOCIAL SUPPORT; STANDARD;

ADULT; CONSUMER PARTICIPATION; DECISION SUPPORT TECHNIQUES; FEMALE; HUMANS; INTERDISCIPLINARY COMMUNICATION; MALE; MIDDLE AGED; PROGRAM EVALUATION; QUESTIONNAIRES; REGRESSION ANALYSIS; RESEARCH; RESEARCH PERSONNEL; SCIENCE; SOCIAL RESPONSIBILITY; SOCIAL SUPPORT;

EID: 35348879715     PISSN: 01497189     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.evalprogplan.2007.08.002     Document Type: Article

References (80)

1. Bamberger M., Rugh J., Church M., and Fort L. Shoestring evaluation: Designing impact evaluations under budget, time and data constraints. American Journal of Evaluation 25 1 (2004) 5-37
2. Bammer G. Integration and implementation sciences: Building a new specialization. Ecology and Society 10 2 (2005) Art. 6
3. Bantilan M.C.S., Chandra S., Mehta P.K., and Keatinge J.D.H. Dealing with diversity in scientific outputs: Implications for international research evaluation. Research Evaluation 13 2 (2004) 87-93
4. Baron R.M., and Kenny D.A. The moderator mediator variable distinction in social psychological-research-conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology 51 6 (1986) 1173-1182
5. Beierle T.C. Using social goals to evaluate public participation in environmental decisions. Policy Studies Review 16 3/4 (1999) 75-103
6. Bergmann, M., Brohmann, B., Hoffmann, E., Loibl, M. C., Rehaag, R., Schramm, E., et al. (2005). Quality criteria of transdisciplinary research. A guide for the formative evaluation of research projects. Frankfurt: Institute for social-ecologic research.
7. Bockstaller C., and Girardin P. How to validate environmental indicators. Agricultural Systems 76 2 (2003) 639-653
8. Brown R.L. Assessing specific mediational effects in complex theoretical models. Structural Equation Modeling 4 2 (1997) 142-156
9. Burdge R.J., Fricke P., Finsterbusch K., Freudenburg W.R., Gramling R., Holden A., et al. Guidelines and principles for social impact assessment-Interorganizational committee on guidelines and principles for social impact assessment. Environmental Impact Assessment Review 15 1 (1995) 11-43
10. Carnes S.A., Schweitzer M., Peelle E.B., Wolfe A.K., and Munro J.F. Measuring the success of public participation on environmental restoration and waste management activities in the US Department of Energy. Technology in Society 20 4 (1998) 385-406
11. Clark W.C., and Dickson N.M. Sustainability science: The emerging research program. Proceedings of the National Academy of Sciences 100 14 (2003) 8059-8061
12. Cloquell-Ballester V.A., Monterde-Diaz R., and Santamarina-Siurana M.C. Indicators validation for the improvement of environmental and social impact quantitative assessment. Environmental Impact Assessment Review 26 1 (2006) 79-105
13. Cozzens S.E. The knowledge pool: Measurement challenges in evaluating fundamental research programs. Evaluation and Program Planning 20 1 (1997) 77-89
14. Currall S.C., and Judge T.A. Measuring trust between organizational boundary role persons. Organizational Behavior and Human Decision Processes 64 2 (1995) 151-170
15. Currie M., King G., Rosenbaum P., Law M., Kertoy M., and Specht J. A model of impacts of research partnerships in health and social services. Evaluation and Program Planning 28 4 (2005) 400-412
16. Davenport S., Grimes C., and Davies J. Collaboration and organizational learning: A study of a New Zealand collaborative research program. International Journal of Technology Management 18 3/4 (1999) 173-187
17. Donaldson S.I. Mediator and moderator analysis in program development. In: Sussman S. (Ed). Handbook of program development for health behavior research (2001), Sage, Newbury Park, CA 470-496
18. Earle T.C., and Siegrist M. Morality information, performance information, and the distinction between trust and confidence. Journal of Applied Social Psychology 36 2 (2006) 383-416
19. Economic and Social Research Council UK (ESRC). (1996). Evaluation guide one: Evaluating research programmes. Swindon: ESRC.
20. Friedmann J. Toward a non-euclidean mode of planning. Journal of the American Planning Association 59 4 (1993) 482-485
21. Funtowicz S.O., and Ravetz J.R. Science for the post-normal age. Futures 25 (1993) 739-755
22. Gallopin G., Funtowicz S., O'Connor M., and Ravetz J. Science for the twenty-first century: From social contract to the scientific core. International Social Science Journal 53 2 (2001) 219-243
23. Garrett-Jones S., and Aylward D. Some recent developments in the evaluation of university research outcomes in the United Kingdom. Research Evaluation 9 1 (2000) 69-75
24. Georghiou L. Issues in the evaluation of innovation and technology policy. Evaluation 4 1 (1998) 37-51
25. Gibbons M., Limoges C., Nowotny H., Schwartzmann S., Scott P., and Trow M. The new production of knowledge. The dynamics of science and research in contemporary societies (1994), Sage, London
26. Gilovich T., and Medvech V.H. The experience of regret-what, when, and why. Psychological Review 102 2 (1995) 379-395
27. Grunwald A. Strategic knowledge for sustainable development: The need for reflexivity and learning at the interface between science and society. International Journal of Foresight and Innovation Policy 1 1/2 (2004) 150-167
28. Hillier J. Fighting over the forests: Environmental conflict and decision-making capacity in forest planning processes. Australian Geographical Studies 41 3 (2003) 251-269
29. Hirsch Hadorn, G. (Ed.), (2002). Unity of knowledge in transdisciplinary research for sustainability. Oxford, UK: EOLSS Publishers 〈http://www.eolss.net〉.
30. Hirsch Hadorn G., Bradley D., Pohl C., Rist S., and Wiesmann U. Implications of transdisciplinarity for sustainability research. Ecological Economics 60 1 (2006) 119-128
31. Hirsch Hadorn G., Pohl C., and Scheringer M. Methodology of transdisciplinary research. In: Hirsch Hadorn G. (Ed). Unity of knowledge in transdisciplinary research for sustainability (2002), EOLSS Publishers, Oxford, UK. http://www.eolss.net 〈www.eolss.net〉
32. Hoffman-Riem, H., Hirsch Hadorn, G., & Pohl, C. (in press). Introduction. In Hirsch Hadorn, Gertrude; Hoffmann-Riem, Holger; Biber-Klemm, Susette; Grossenbacher-Mansuy, Walter; Joye, Dominique; Pohl, Christian; Wiesmann, Urs; Zemp, Elisabeth (eds.). 2007. Handbook of Transdisciplinary Research. A Proposition by the Swiss Academies of Arts and Sciences. Heidelberg: Springer (in press).
33. Judd C.M., Smith E.R., and Kidder L. Research methods in social relations (1991), Rinehart and Winston, Fort Worth
34. Konisky D.M., and Beierle T.C. Innovations in public participation and environmental decision making: Examples from the great lakes region. Society and Natural Resources 14 9 (2001) 815-826
35. Krott M. Evaluation of transdisciplinary research (2002), EOLSS Publishers, Oxford, UK. http://www.eolss.net 〈http://www.eolss.net〉
36. Lillis D.A. Towards a new science evaluation system for New Zealand. Research Evaluation 8 2 (2000) 145-150
37. Loukopoulos P., and Scholz R.W. Future urban sustainable mobility: Using "area development negotiations" for scenario assessment and for participatory strategic planning. Environment and Planning A 36 12 (2004) 2203-2226
38. Luukkonen T. Research evaluation in Europe: State of the art. Research Evaluation 11 2 (2002) 81-84
39. MacKinnon D.P., Lockwood C.M., Hoffman J.M., West S.G., and Sheets V. A comparison of methods to test mediation and other intervening variable effects. Psychological Methods 7 1 (2002) 83-104
40. Margerum R.D. Collaborative planning: Building consensus and building a distinct model for practice. Journal of Planning Education and Research 21 3 (2002) 237-253
41. Meppem T., and Gill R. Planning for sustainability as a learning concept. Ecological Economics 26 2 (1998) 121-137
42. Mieg H.A. System experts and decision making experts in transdisciplinary projects. International Journal of Sustainability in Higher Education 7 3 (2006) 341-351
43. Mogalle M. Management transdisziplinärer Forschungsprozesse [Management of transdisciplinary research projects] (2001), Birkhäuser, Basel, CH
44. Müller D.B., Tjallingii S.P., and Canters K.J. A transdisciplinary learning approach to foster convergence of design, science and deliberation in urban and regional planning. Systems Research and Behavioral Science 22 3 (2005) 193-208
45. Nowotny H. The place of people in our knowledge. European Review 7 2 (1999) 247-262
46. Nowotny H., Scott P., and M. Gibbons M. Re-thinking science-knowledge and the public in an age of uncertainty (2001), Polity, London
47. O'Connor M. Dialogue and debate in a post-normal practice of science: A reflexion. Futures 31 7 (1999) 671-687
48. Olsen O.E., and Lindoe P. Trailing research based evaluation; phases and roles. Evaluation and Program Planning 27 4 (2004) 371-380
49. Petrosino A. Mediators and moderators in the evaluation of programs for children: Current practice and agenda for improvement. Evaluation Review 24 1 (2000) 47-72
50. Pohl C. Transdisciplinary collaboration in environmental research. Futures 37 10 (2005) 1159-1178
51. Preacher J.K., and Hayes A.F. SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, Instruments, and Computers 36 4 (2004) 717-731
52. Preacher, J. K., & Hayes, A. F. (under review). Asymptotic and resampling strategies for assessing and comparing indirect effects in simple and multiple mediator models. Organizational Research Methods.
53. Ramadier T. Transdisciplinarity and its challenges: The case of urban studies. Futures 36 4 (2004) 423-439
54. Randolph J., and Bauer M. Improving environmental decision-making through collaborative methods. Policy Studies Review 16 3/4 (1999) 168-192
55. Ravetz J. Integrated assessment for sustainability appraisal in cities and regions. Environmental Impact Assessment Review 20 (2000) 31-64
56. Robinson J., and Tansey J. Co-production, emergent properties and strong interactive social research: The Georgia Basin futures project. Science and Public Policy 33 2 (2006) 151-1260
57. Rossi P.H., Lipsey M.W., and Freeman H.E. Evaluation a systematic approach. 7th ed. (2004), Sage Publications, Thousand Oaks
58. Rowe G., and Frewer L.J. Evaluating public-participation exercises: A research agenda. Science, Technology, and Human Values 29 4 (2004) 512-557
59. Sanna L.J., and Schwarz N. Debiasing the hindsight bias: The role of accessibility experiences and (mis)attributions. Journal of Experimental Social Psychology 39 3 (2002) 287-295
60. Scholz R.W. What research has found out on the cooperation of teachers and the effect of team teaching (ERIC Document Reproduction Service No. ED 173154) (1978), Education Resources Information Center, Lanham, MD
61. Scholz R.W., Lang D., Walter A.I., Wiek A., and Stauffacher M. Transdisciplinary case studies as a means of sustainability learning: Historical framework and theory. International Journal of Sustainability in Higher Education 7 3 (2006) 226-251
62. Scholz R.W., and Marks D. Learning about transdisciplinarity: Where are we? Where have we been? Where should we go?. In: Thomson Klein J., Grossenbacher-Mansuy W., Häberli R., Bill A., Scholz R.W., and Welti M. (Eds). Transdisciplinarity: Joint problem solving among science, technology, and society (2001), Birkhäuser Verlag AG, Basel 236-252
63. Scholz R.W., Mieg H.A., and Oswald J.E. Transdisciplinarity in groundwater management-towards mutual learning of science and society. Water Air and Soil Pollution 123 1-4 (2000) 477-487
64. Scholz, R. W., & Stauffacher, M. (in press). Managing transition in clusters: Area development negotiations as a tool for sustaining traditional industries in a Swiss pre-alpine region. Environment and Planning A.
65. In: Scholz R.W., Stauffacher M., Bösch S., and Krütli P. (Eds). Appenzell Ausserrhoden-Umwelt Wirtschaft Region. ETH-UNS Fallstudie 2002 [Environment Economy Region. ETH-UNS Case Study 2002] (2003), Rüegger, Zurich
66. In: Scholz R.W., Stauffacher M., Bösch S., and Wiek A. (Eds). Landschaftsnutzung für die Zukunft: der Fall Appenzell Ausserrhoden. ETH-UNS Fallstudie 2001 [Utilisation of landscape for the future: The case of Appenzell Ausserrhoden. ETH-UNS Case Study 2001] (2002), Rüegger, Zurich
67. Scholz R.W., and Tietje O. Embedded case study methods: Integrating quantitative and qualitative knowledge (2002), Sage, Thousand Oaks
68. Schwarz N., and Oyserman D. Asking questions about behavior: Cognition, communication, and questionnaire construction. American Journal of Evaluation 22 (2001) 127-160
69. Stauffacher M., Walter A.I., Lang D., Wiek A., and Scholz R.W. Learning to research environmental problems from a functional socio-cultural constructivism perspective: The transdisciplinary case study approach. International Journal of Sustainability in Higher Education 7 3 (2006) 252-275
70. Stokols D., Harvey R., Gress J., Fuqua J., and Phillips K. In vivo studies of transdisciplinary scientific collaboration: Lessons learned and implications for active living research. American Journal of Preventive Medicine 28 2 (2005) 202-213
71. In: Thompson Klein J.T., Grossenbacher-Mansuy W., Haeberli R., Bill A., Scholz R.W., and Welti M. (Eds). Transdisciplinarity: Joint problem solving among science, technology, and society (2001), Birkhäuser, Basel
72. Tonn B., English M., and Travis C. A framework for understanding and improving environmental decision making. Journal of Environmental Planning and Management 23 2 (2000) 163-183
73. Tress B., Tress G., and Fry G. Integrative studies on rural landscapes: Policy expectations and research practice. Landscape and Urban Planning 70 1/2 (2005) 177-191
74. Tress G., Tress B., and Fry G. Clarifying integrative research concepts in landscape ecology. Landscape Ecology 20 4 (2005) 479-493
75. van Schooten M., Vanclay F., and Slootweg R. Conceptualizing social change processes and social impacts. In: Becker H.A., and Vanclay F. (Eds). The international handbook of social impact assessment: Conceptual and methodological advances Vol. 74-91 (2003), Edward Elgar, Cheltenham
76. Vanclay F. Conceptualising social impacts. Environmental Impact Assessment Review 22 3 (2002) 183-211
77. Walter A.I., and Scholz R.W. Critical success conditions of collaborative methods: A comparative evaluation of transport planning projects. Transportation 34 2 (2007) 195-212
78. Walter, A. I., Wiek, A., & Scholz, R. W. (in press). Implementing regional sustainable development: The transdisciplinary case study research approach. In: H. Hoffmann-Riem, G. Hirsch Hadorn, D. Joye, U. Wiesmann, E. Zemp, C. Pohl, W. Grossenbacher & S. Biber-Klemm (Eds.), Handbook of transdisciplinary research. New York: Springer.
79. Wiek A. Challenges of transdisciplinary research as interactive knowledge generation-Experiences from transdisciplinary case study research. GAIA-Ecological Perspectives for Science and Society 16 1 (2007) 52-57
80. Wiek, A., & Walter, A. I. (under review). A transdisciplinary approach for formalized integrated planning and decision-making in complex systems. European Journal of Operational Research.