Developing and applying a framework to evaluate participatory research for sustainability

Ecological Economics

Volumn 60, Issue 4, 2007, Pages 726-742

  Blackstock, K L ^a   Kelly, G J ^b   Horsey, B L ^b  

^a SCOTTISH CROP RESEARCH INSTITUTE   (United Kingdom)

^b CSIRO   (Australia)

Author keywords

Evaluation; Methodology; Participatory research; Sustainability Science

Indexed keywords

ASSESSMENT METHOD; CONCEPTUAL FRAMEWORK; PARTICIPATORY APPROACH; RESEARCH WORK; SUSTAINABILITY;

AUSTRALASIA; AUSTRALIA;

EID: 33846673887     PISSN: 09218009     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ecolecon.2006.05.014     Document Type: Article

References (91)

1. Abelson J., Forest P., Eyles J., Smith P., Martin E., and Gauvin F. Deliberations about deliberative methods: issues in the design and evaluation of public participation processes. Social Science & Medicine 57 (2003) 239-251
2. Alston J.M., Norton G.W., and Pardey P.G. Science Under Scarcity: Principles and Practice for Agricultural Research Evaluation and Priority Setting (1995), Cornell University Press, Ithaca
3. Arnstein A. A ladder of citizen participation. Journal of the American Institute of Planners 26 4 (1969) 216-233
4. Asthana S., Richardson S., and Halliday J. Partnership working in public policy provision: a framework for evaluation. Social Policy and Administration 36 (2002) 780-795
5. BeckerJ, 2004. Making Sustainable Development Evaluations Work in Sust. Dev. 12 200-211
6. Beierle T., and Konisky D. What are we gaining from stakeholder involvement? Observations from Environmental Planning in the Great Lakes. Environment and Planning C, Government and Policy 19 (2001) 515-527
7. Bellamy J.A. Moving beyond "talk the talk": lessons from community-based research partnerships for coastal zone management. Paper Presented at: Coastal Zone Asia Pacific 2004 Conference (CZAP 2004) 'Improving the Quality of Life in Coastal Areas', 5-9 September 2004 (2004)
8. Bellamy J.A., and McDonald G.T. Through multi-scaled lenses: a systems approach to evaluating natural resource management. Regional Natural Resource Management Planning: the challenges of evaluation as seen through different lenses. CIRM Social Dimensions of Natural Resource Management Working Group Symposium, 15th October, 2004 (2005) 11-18
9. Bellamy J.A., Walker D.H., McDonald G.T., and Syme G.J. A systems approach to the evaluation of natural resource management initiatives. Journal of Environmental Management 63 (2001) 407-423
10. Biggs S. Resource-Poor Farmer Participation in Research: a Synthesis of Experiences From Nine National Agricultural Research Systems. OFCOR Comparative Study Paper vol. 3 (1989), International Service for National Agricultural Research, The Hague
11. Blackstock K.L. A critical look at community based tourism. Community Development Journal 40 (2005) 39-49
12. Blackstock, K.L., 2005b. Evaluating the impact of participatory research: literature review. Unpublished discussion document*.
13. Bloomfield D.K., Collins C.F., and Munton R. Deliberation and inclusion: vehicles for increasing trust in UK public governance. Environment and Planning. C, Government and Policy 19 (2001) 501-513
14. Bossel H. Finding a comprehensive set of indicators of sustainable development by application of orientation theory. In: Moldan B., and Billharz S. (Eds). Sustainability Indicators: Report of the project on indicators of sustainable development (1997), John Wiley and Sons, Chichester 101-109
15. Botcheva L., Roller White C., and Huffman L.C. Learning culture and outcomes measurement practices in community agencies. American Journal of Evaluation 23 (2002) 421-434
16. Brinkerhoff J.M. Assessing and improving partnership relationships and outcomes: a proposed framework. Evaluation and Program Planning 25 (2002) 215-231
17. Cash D.W., Clark W.C., Alcock F., Dickson N.M., Eckley N., Guston D.H., Jager J., and Mitchell R.B. Knowledge systems for sustainable development. PNAS 100 14 (2003) 8086-8091
18. Chelimsky E. The coming transformation in evaluation. In: Chelimsky E.S., and Standish W.R. (Eds). Evaluation for the 21st Century: a Handbook (1997), Sage Publications, California
19. Coenen F.J.H., Huitema D., and O'Toole L.J. Participation and environment. In: Coenen F.J.H., Huitema D., and O'Toole L.J. (Eds). Participation and the Quality of Environmental Decision Making (1998), Kluwer Academic Press, Netherlands 1-22
20. Davies G., and Burgess J. Challenging the 'view from nowhere': citizen reflections on specialist expertise in a deliberative process. Health and Place 10 (2004) 349-361
21. Davies, B.D., Blackstock, K.L., Brown, K.M. and Shannon, P. 2004. Challenges in creating local agri-environmental cooperation action amongst farmers and other stakeholders: final report Scottish Executive Environment and Rural Affairs Department Flexible Fund: MLU/927/03*.
22. De Marchi B., and Ravetz J. Participatory Approaches to Environmental Policy. EVE Policy Research Brief vol. 10 (2001), Cambridge Research for the Environment, Cambridge
23. Denzin N. The research Act: A Theoretical Introduction to Sociological Methods. 3rd edition (1989), Prentice Hall, Englewood Cliffs, NJ
24. Dore J., and Woodhill J. Sustainable Rural Development: Final Report (1999), Greening Australia, Canberra
25. Dovers S. Environment and Sustainability Policy: Creation, Implementation and Evaluation (2004), The Federation Press, Canberra
26. Dryzek J. Deliberative Democracy and Beyond: Liberals, Critics, Contestations (2000), Oxford University Press, Oxford
27. Ekboir J. Why impact analysis should not be used for research evaluation and what the alternatives are. Agricultural Systems 78 (2003) 166-184
28. Fischer F. Citizens, Experts and the Environment: the Politics of Local Knowledge (2000), Duke University Press, Durham, NC
29. Funtowicz S., Guimaraes-Pereira A., Lonza-Ricci L., and Wolf O. Recommendations for Sustainability-Oriented European Research Programs, Deliverable 6, AIRP-SD Project, EC-STRATA Program (2003). http://www.seri.at/airp-sd/start/_docs/Executive%20Summary%20D6.pdf [accessed 2.2.06]http://www.seri.at/airp-sd/start/_docs/Executive%20Summary%20D6.p df
30. Gibbon M., Labonte R., and Laverack G. Evaluating community capacity. Health and Social Care in the Community 10 (2002) 485-491
31. Grant, A., Curtis, A. 2004. Refining Evaluation Criteria for Public Participation Using Stakeholder Perspectives of Process and Outcomes. Rural Society, 14: 142-162.
32. Greenwood D.J., Whyte W.F., and Harkavy I. Participatory action research as a process and as a goal. Human Relations 46 (1993) 175-192
33. In: Gunderson L.H., and Holling C.S. (Eds). Panarchy: Understanding Transformations in Human and Natural Systems (2002), Island Press, Washington, DC 507 pp.
34. Hinterberger F., Bosch G., and Giljum S. AIRP-SD Final Report: Executive Summary (2003). http://www.seri.at/airp-sd/start/_docs/AIRP-SD%20Final%20Report_Executiv e%20summary.pdf [accessed 2.2.06] http://www.seri.at/airp-sd/start/_docs/AIRP-SD%20Final%20Report_Executiv e%20summary.pdf
35. Holloway J. Understanding Evaluation: Part 18 of Managing the Project (2001), Open University Press, Milton Keynes
36. Horsey B. An Evaluation of the Douglas Shire Sustainable Futures Projects: Document Review (2005), CSIRO, Canberra*
37. Hoverman S. The value of evaluation through the local implementation lens. In: Bellamy J. (Ed). Regional Natural Resource Management Planning: the Challenges of Evaluation as Seen Through Different Lenses: Papers From an Occasional Symposium held 15th October 2004 (2005), CSIRO Sustainable Ecosystems, Brisbane 35-42
38. Johnson N., Lilja N., Ashby J., and Garcia J. The practice of participatory research and gender analysis in natural resource management. Natural Resources Forum 28 (2004) 189-200
39. Kasemir B., Jaeger C., and Jaeger J. Citizen Participation in Sustainability Assessments. In: Kasemir B., Jaeger J., Jaeger C., and Gardner M. (Eds). Public Participation in Sustainability Science (2003), Cambridge University Press, Cambridge
40. Kates R.W., Clark W.C., Corell R., Hall J.M., Jaeger C.C., Lowe I., McCarthy J.J., Schellnhuber H.J., Bolin B., Dickson N.M., Faucheux S., Gallopin G.C., Grübler A., Huntley A., Jäger J., Jodha N.S., Kasperson R.E., Mabogunje A., Matson P., Mooney H., Moore III B., and O'Riordan T. Sustainability Science. Science 292 5517 (2001) 641-642
41. Kelly, G.J., Haslam-McKenzie, F., submitted for publication. Housing affordability in a sea change community. Housing Studies.
42. Kelly G.J. Measham T., 2005. Participation in Participatory Research, unpublished working document, Canberra: CSIRO*.
43. Kelly G.J., Measham T., Horsey B., Leitch A., and Smith T. Reflections on the Stream: a Review of the Community Capacity to Manage Systems Research Stream (2005), CSIRO Sustainable Ecosystems, Canberra*
44. Kelly, G.J., Blackstock, K., and Horsey, B., in progress. Limits to learning for developing a sustainable region: lessons from northeast Queensland. Under review by Society and Natural Resources.
45. Kemmis S., and McTaggart R. Participatory Action Research. In: Denzin N.K., and Lincoln Y. (Eds). Handbook of Qualitative Research. 2nd ed. (2000), Sage Publications, Thousand Oaks
46. Kenyon W. A Proposed Framework for Evaluating Trust in Participatory Processes, Paper Given at the European Society for Ecological Economics Conference, Lisbon, Portugal, June 2005 (2005)
47. Laverack G. An identification and interpretation of the organisational aspects of community empowerment. Community Development Journal 36 (2001) 134-145
48. Maarleveld M., and Dangbégnon C. Managing natural resources: a social learning perspective. Agriculture and Human Values 16 (1999) 267-280
49. MacNaughten P., and Jacobs M. Public identification with sustainable development - investigating cultural barriers to participation. Global Environmental Change: Human and Policy Dimensions 7 (1997) 5-24
50. MacNeil C. Evaluator as steward of citizen deliberation. American Journal of Evaluation 23 (2002) 45-54
51. Martin V. Completing and Evaluating the Project: Part 16 of Managing the Project (2001), Open University Press, Milton Keynes
52. Martin A., and Sherington J. Participatory research methods - implementation, effectiveness and institutional context. Agricultural Systems 55 2 (1997) 195-216
53. Mayumi K., and Glampietro M. The epistemological challenge of self modifying systems: governance and sustainability in the post-normal science era. Ecological Economics 57 3 (2006) 382-399
54. Muller A. A flower in full blossom? Ecological Economics at the cross-roads between normal and post-normal science. Ecological Economics 45 (2003) 19-27
55. OECD., 2004. Stakeholder Involvement Techniques: Short Guide and Annotated Bibliography, A report from Nuclear Energy Agency Forum for Stakeholder Confidence. Available from http://www.nea.fr/html/rwm/reports/2004/nea5418-stakeholder.pdf [Accessed 3rd February 2006].
56. Okali C., Sumberg J., and Farrington J. Farmer Participatory Research (1994), Intermediate Technology Publications, London
57. O'Meara P., Chesters J., and Han G. Outside - looking in: evaluating a community capacity building project. Rural Society 14 2 (2004) 126-141
58. O'Neill J. Representing people, representing nature, representing the world. Environment and Planning. C, Government & Policy 19 (2001) 483-500
59. O'Riordan T. Globalism, localism and identity. Fresh Perspectives on the Transition to Sustainability (2001), Earthscan Publications Ltd, London
60. Osorio-Peters S. AIRP-SD Work Package Six: Executive Summary (2003). http://www.seri.at/airp-sd/start/_docs/Executive%20Summary%20_final_%20p df [accessed 2.2.06] http://www.seri.at/airp-sd/start/_docs/Executive%20Summary%20_final_.pdf
61. Pain R. Social geography: participatory research. Progress in Human Geography 28 (2004) 652-663
62. Patton M.Q. How to Use Qualitative Methods in Evaluation (1987), Sage, London
63. Patton M.Q. Toward distinguishing empowerment evaluation and placing it in a larger context. Evaluation Practice 18 (1997) 147-163
64. Patton M.Q. Utilization-Focussed Evaluation. 3rd edn (1998), Sage, Thousand Oaks CA
65. Pellizzoni L. The myth of best argument: power, deliberation and reason. British Journal of Sociology 52 (2001) 59-86
66. Pellizzoni L. Uncertainty and participatory democracy. Environmental Values 12 (2003) 195-224
67. Pini B. Focus groups, feminist research and farm women: opportunities for empowerment in rural social research. Journal of Rural Studies 18 (2002) 339-351
68. Pini B. On being a nice country girl and an academic feminist: using reflexivity in rural social research. Journal of Rural Studies 20 (2004) 169-179
69. Reed, M., Fraser, D.G., Morse, S., and Dougill, A.J. (2005). Integrating methods for developing sustainability indicators to facilitate learning and action in ecology and society 10 (1) r3 [online] URL: http://www.ecologyandsocietyorg/vol10/iss1/resp3.
70. Richards C., Sherlock K., and Carter C. Practical Approaches to Participation. SERP Policy Brief vol. 1 (2004), Macaulay Institute, Aberdeen*
71. Rowe G., and Frewer L. Public participation methods: a framework for evaluation in science. Technology and Human Values 25 (2000) 3-29
72. Rowe G., Marsh R., and Frewer L.J. Evaluation of a deliberative conference in science. Technology and Human Values 29 (2004) 88-121
73. Rural Industries Research and Development Corporation. RIRDC Corporate Plan 2003-2008 (2003). http://www.rirdc.gov.au/corporateplan/
74. Schulz A.J., Israel B.A., and Lantz P. Instrument for evaluating dimensions of group dynamics within community based participatory research partnerships. Evaluation and Program Planning 26 (2003) 249-262
75. Scott A.J. The contribution of forums to rural sustainable development: a preliminary evaluation. Journal of Environmental Management 54 (1998) 291-303
76. Sherlock (2002). Community matters: reflections from the field in Sociological Research Online Vol 7 (2).
77. SLIM. Developing Conducive and Enabling Institutions for Concerted Action: SLIM Policy Briefing No. 3, May 2004 (2004). http://slim.open.ac.uk http://slim.open.ac.uk [accessed 18th October 2004]
78. Smith G. Taking deliberation seriously: institutional design and green politics. Environmental Politics 10 (2001) 72-93
79. Steelman T., and Ascher W. Public involvement methods in natural resource policy making: advantages, disadvantages and tradeoffs. Policy Sciences 30 2 (1997) 71-90
80. Stirling A. Opening up or closing down: analysis, participation and power in the social appraisal of technology. In: Leach M., Scoones I., and Wynne B. (Eds). Science, Citizenship and Globalisation (2004), Zed, London
81. Strager M.P., and Rosenberger R.S. Incorporating stakeholder preferences for land conservation: weights and measures in spatial MCA. Ecological Economics 58 1 (2006) 79-92
82. Thurston W., MacKean G., Vollman A., Casebeer A., Weber M., Maloff B., and Bader J. Public participation in regional health policy: a theoretical framework. Health Policy 73 3 (2005) 237-252
83. Van den Hove S. Participatory approaches to environmental policy making: the European Commission Climate Policy Process as a case study. Ecological Economics 33 (2000) 457-472
84. Walker D.H., Vela K.J., and Kotzman M. Regional Planning and the Sugar Industry (2004), CSIRO Sustainable Ecosystems, Canberra
85. Wallerstein N. Power between the evaluator and the community: research relationships within New Mexico's healthier communities. Social Science & Medicine 49 (1999) 39-53
86. Wandersman A. A framework of participation in community organisations. Journal of Applied Behavioural Science 17 (1981) 27-58
87. Weaver, P., no date. A methodological framework for evaluating sustainability science, paper for deliverable 3 [accessed 2.2.06]http://www.seri.at/airp-sd/start/_docs/AIRP-SD_Del3_ExecutiveSumm ary.pdf.
88. Webler T. Right discourse and citizen participation: an evaluative yardstick. In: Renn O., Webler T., and Wiedemann P. (Eds). Fairness and Competence in Citizen Participation: Evaluating Models for Environmental Discourse (1995), Kluwer 35-86
89. Webler T., Tuler S., and Krueger R. What is a good public participation process?. Environmental Management 27 (2001) 435-450
90. Wicker A.W. Substantive theorizing. American Journal of Community Psychology 17 (1989) 531-547
91. Wilcox D. The Guide to Effective Participation (2000). http://www.partnerships.org.uk/guide/index.htm [accessed 10th November, 2003]http://www.partnerships.org.uk/guide/index.htm