Toward a methodological scheme for capturing societal effects of participatory sustainability research

Research Evaluation

Volumn 23, Issue 2, 2014, Pages 117-132

  Wiek, Arnim ^a   Talwar, Sonia ^b   O'Shea, Meg ^c   Robinson, John ^c  

^a ARIZONA STATE UNIVERSITY   (United States)

^b NATURAL RESOURCES CANADA   (Canada)

^c UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Evaluation framework; impact assessment; real world impacts; stakeholder engagement

Indexed keywords

EID: 84898845918     PISSN: 09582029     EISSN: 14715449     Source Type: Journal    
DOI: 10.1093/reseval/rvt031     Document Type: Article

References (64)

1. Bantilan, M. C. S. et al. (2004) ' Dealing with Diversity in Scientific Outputs: Implications for International Research Evaluation' , Research Evaluation, 13: 87-93.
2. Barron, S. et al. (2012) ' A Climate Change Adaptation Planning Process for Low-Lying Communities Vulnerable to Sea Level Rise' , Sustainability, 4: 2176-208.
3. Bell, S., Shaw, B. and Boaz, A. (2011) ' Real-World Approaches to Assessing the Impact of Environmental Research on Policy' , Research Evaluation, 20: 227-37.
4. Bergmann, M. et al. (2005) Quality Criteria of Transdisciplinary Research: A Guide for the Formative Evaluation of Research Projects. Frankfurt am Main, Germany: Institute for Social-Ecological Research.
5. Blackstock, K. L., Kelly, G. J. and Horsey, B. L. (2007) 'Developing and Ap plying a Framework to Evaluate Participatory Research for Sustainability' , Ecological Economics, 60: 726-42.
6. Brisolara, S. (1998) ' The History of Participatory Evaluation and Current Debates in the Field' , New Directions for Evaluation, 80: 25-41.
7. Cumming G. Guffey S. J. Norwood C. 2008 Opportunities and Challenges in Community Capacity-Building: Lessons from Participatory Research in Macon County, North Carolina '. In Wilmsen, C. et al. (eds) Partnerships for Empowerment: Participatory Research for Community-Based Natural Resource Management, pp. 127-46. London, UK: Earthscan.
8. Currie, M. et al. (2005) 'A M odel of Impacts of Research Partnerships in Health and Social Services' , Evaluation and Program Planning, 28: 400-12.
9. Daniell, K. A., Mazri, C. and Tsoukia' s, A. (2010) ' Real World Decision-Aiding: A Case of Participatory Water Management' . In: Rios Insua, D. and French, S. (eds) e-Democracy: A Group Decision and Negotiation Perspective, pp. 125-50. Dordrecht: Springer.
10. Davenport, S., Grimes, C. and Davies, J. (1998) ' Research Collaboration and Behavioural Additionality: A New Zealand Case Study' , Technology Analysis & Strategic Management, 10/1: 55-68.
11. Davenport S Grimes C. Davies, J.. (1999 a) 'Collaborative Research Programmes: Building Trust from Difference ', Technovation, 19: 31-40.
12. Davenport S. Grimes C. Davies, J. (1999 b) 'Collaboration and Organizational Learning: A Study of a New Zealand Collaborative Research Program ', International Journal of Technology Management, 18: 17 3-87.
13. DeFries, R. S. et al. (2012) ' Planetary Opportunities-A Social Contract for Global Change Science to Contribute to a Sustainable Future' , BioScience, 62/6: 603-6.
14. Demeritt, D. (2010) ' Harnessing Science and Securing Societal Impacts from Publicly Funded Research: Reflections on UK Science Policy' , Environment and Planning A, 42: 515-23.
15. Donovan, C. (2008) 'The Australian Research Quality Framewo rk: A Live Experiment in Capturing the Social, Economic, Environmental, and Cultural Returns of Publicly Funded Research ', New Directions for Evaluation, 118: 47-60.
16. Donovan, C. and Hanney, S. (2011) 'The 'Payback Framework ' Explained', Research Evaluation, 20: 181-3.
17. Doppelt, B. (2003) Leading Change toward Sustainability-A Change-Management Guide for Business, Government and Civil Society. Sheffield: Greenleaf Publishing.
18. Ekboir, J. (2003) ' Why Impact Analysis Should Not be Used for Research Evaluation and What the Alternatives Are' , Agricultural Systems, 78: 166-84.
19. Flyvbjerg, B. (2002) Making Social Science Matter: Why Social Inquiry Fails and How It Can Succeed Again. Cambridge: Cambridge University Press.
20. Funtowicz, S. and Ravetz, J. (1993) 'Science for the Post Normal Age ', Futures, 25: 739-55.
21. Gibbons, M. (1999) 'Science's New Social Contract with Society ', Nature, 402: C81-4.
22. Gibbons, M. et al. (1994) The New Production of Knowledge: The Dynamics of Science and Research in Contemporary Societies. London: SAGE Publications.
23. Gray, D. O. (2008) ' Making Team Science Better: Applying Improvement-Oriented Evaluation Principles to Evaluation of Cooperative Research Centers' , New Directions for Evaluation, 118: 73-87.
24. Guston, D. H. (2008) 'Innovation Policy: Not Just a Jumbo Shrimp ', Nature, 454: 940-1.
25. Hessels, L. K., Van Lente, H. and Smits, R. (2009) 'In Search o f Relevance: The Changing Contract between Science and Society' , Science and Public Policy, 36/5: 387-401.
26. Hirsch Hardon, G. et al. (2006) ' Implications of Transdisciplinarity for Sustainable Research' , Ecological Economics, 60: 119-28.
27. Innes, J. E. and Booher, D. E. (1999) 'Consensus Building and Complex Adaptive Systems ', Journal of the American Planning Association, 65: 412-23.
28. Kru?tli P. et al. 2010 Functional-Dynamic Public Participation in Technological Decision-Making: Site Selection Processes of Nuclear Waste Repositories Journal of Risk Research, 13/7 861-75.
29. Lang, D. J. et al. (2012 ) ' Transdisciplinary Research in Sustainability Science-Practice, Principles and Challenges' , Sustainability Science, 7(Suppl. 1): 25-43.
30. Lillis, D. A. (2000) 'Towards a New Science Evaluation System for New Z ealand' , Research Evaluation, 8: 145-50.
31. Lubchenco, J. (1998) ' Entering the Century of the Environment: A New Social Contract for Science' , Science, 279: 491-7.
32. Maasen, S. and Lieven, O. (2006) 'Transdisciplinarity: A New Mode of Governing Science ' , Science and Public Policy, 33: 399-410.
33. Meagher, L., Lyall, C. and Nutley, S. (2011) 'Flows of Knowledge, Expertise and In fluence: A Method for Assessing Policy and Practice Impacts from Social Science Research' , Research Evaluation, 17: 163-73.
34. Melles, G., De Vere, I. and Misic, V. (2011) 'Socially Responsible Design: Thinking be yond the Triple Bottom Line to Socially Responsive and Sustainable Product Design' , CoDesign, 7/3-4: 143-54.
35. Miller, T. R. et al. (2014) ' The Future of Sustainability Science: A Solutions-Oriented Research Agenda' , Sustainability Science, 9/2: 239-46.
36. Molas-Gallart, J., Tang, P. and Morrow, S. (2000) ' Assessing the Non-Academic Impact of Grant-Funded Socio-Economic Research: Results from a Pilot Study' , Research Evaluation, 9/3: 171-82.
37. Nidumolu, R., Prahalad, C. K. and Rangaswami, M. R. (2009) 'Why Sustainability is Now the Key Driver of Innovation ' , Harvard Business Review, 87: 57-64.
38. Norman, C. D. et al. (2011) ' Evaluating the Science of Discovery in Complex Health Systems' , American Journal of Evaluation, 32: 70-84.
39. OECD. (1997) The Evaluation of Scientific Research: Selected Experiences. Paris: OECD.
40. Phillipson, J. et al. (2012) ' Stakeholder Engagement and Knowledge Exchange in Environmental Research' , Journal of Environmental Management, 95: 56-65.
41. Quinlan, K. M., Kane, M. and Trochim, W. M. K. (2008) ' Evaluation of Large Research Initiatives: Outcomes, Challenges, and Methodological Considerations' , New Directions for Evaluation, 118: 61-72.
42. Robinson, J. (2008) ' Being Undisciplined-Transgressions and Intersections in Academia and Beyond' , Futures, 40: 70-86.
43. Robinson, J. and Tansey, J. (2006) 'Co-production, Emergent Properties and Strong Interactive Social Research: T he Georgia Basin Futures Project' , Science and Public Policy, 33: 151-60.
44. Robinson, J. et al. (2006) ' Sustainability as a Problem of Design: Interactive Science in the Georgia Basin' , The Integrated Assessment Journal, 6/4: 165-92.
45. Sarewitz, D. et al. (2012) 'The Sustainability Solutions Agenda ', New Solutions, 22/2: 139-51.
46. Schroth, O. et al. (2011) 'Multiple-Case Study of Landscape Visua lizations as a Tool in Transdisciplinary Planning Workshops' , Landscape Journal, 30/1: 53-71.
47. Shirk, J. L. et al. (2012) ' Public Participation in Scientific Research: A Framework for Deliberate Design' , Ecology and Society, 17/2: 29.
48. Spaapen, J., Dijstelbloem, H. and Wamelink, F. (2007) Evaluating Research in Context: A Method for Comprehensive Assessment, 2nd edn. Hague, the Netherlands: Consultative Committee of Sector Co uncils for Research and Development (COS).
49. Spaapen, J. and Van Drooge, L. (2011) 'Introducing 'Productive Interactions ' in Social Impact Assessment', Research Evaluation, 20: 211-8.
50. Spa?th, P. 2008 Learning Ex-Post: Towards a Simple Method and Set of Questions for the Self-Evaluation of Transdisciplinary Research GAIA-Ecological Perspectives for Science and Society 17 224-32.
51. Talwar, S., Wiek, A. and Robinson, J. (2011) 'User Engagement in Sustainability Research ' , Science and Public Policy, 38: 379-90.
52. Taut, S. and Brauns, D. (2003) 'Resistance to Evaluation: A Psychological Perspective ', Evaluation, 9: 247-69.
53. U.S. National Research Council (2012) ' Climate and Social Stress: Implications for Security Analysis' . In: Steinbruner, J. D., Stern, P. C. and Husbands, J. L. (eds) Committee on Assessing the Impacts of Climate Change on Social and Political Stresses, Board on Environmental Change and Society, Division on Behavioral and Social Sciences and Education. Wa shington, DC: The National Academies Press.
54. Van Der Leeuw, S. et al. (2012) ' How Much Time Do We Have Urgency and Rhetoric in Sustainability Science' , Sustainability Science, 7(Suppl. 1): 115-20.
55. Van Kerkhoff, L. and Lebel, L. (2006) ' Linking Knowledge and Action for Sustainable Development' , Annual Review of Environment and Resources, 31: 445-77.
56. Walter, A. et al. (2007) ' Measuring Societal Effects of Transdisciplinary Research Projects-Design and Application of an Evaluation Method' , Evaluation and Program Planning, 30: 325-38.
57. Wells, G. L., Memon, A. and Penrod, S. D. (2006) 'Eyewitness Evidence: Improving Its Probative Value ' , Psychological Science in the Public Interest, 7: 45-75.
58. Wiek, A. (2007) ' Challenges of Transdisciplinary Research as Interactive Knowledge Generation-Experiences from Transdisciplinary Case Study Research ', GAIA-Ecological Perspectives for Science and Society, 16/1: 52-7.
59. Wiek A. (2013) ' Exploring Climate Change Futures-A Formative Study on Participatory Scenario Construction' , Working Paper. Sustainability Transition and Intervention Research Lab. School of Sustainability, Arizona State University.
60. Wiek, A. et al. (2007) ' Sustainable Governance of Emerging Technologies-Critical Constellations in the Agent Network of Nanotechnology' , Technology in Society, 29: 388-406.
61. Wiek A. (2011) ' A Comparative Evaluation of Participatory Scenario Studies on Regional Land Use Futures' , Project Report. School of Sustainability, Arizona State University.
62. Wiek A. (2012) ' From Complex Systems Analysis to Transformational Change: A Comparative Appraisal of Sustainability Science Projects' , Sustainability Science, 7(Suppl. 1): 5-24.
63. Woodin g, S. et al. (2005) ' Payback Arising from Research Funding: Evaluation of the Arthritis Research Campaign' , Rheumatology, 44: 1145-56.
64. Wynne, B. et al. (2007) 'Taking European Knowledge Society Ser iously' , Report of the Expert Group on Science and Governance to the Science, Economy and Society Directorate, Directorate-General for Research. European Commission, Brussels.