Enhancing the reflexivity of system innovation projects with system analyses

American Journal of Evaluation

Volumn 31, Issue 2, 2010, Pages 143-161

  van Mierlo, Barbara ^a   Arkesteijn, Marlèn ^a   Leeuwis, Cees ^a  

^a WAGENINGEN UNIVERSITY   (Netherlands)

Author keywords

Innovation networks; Methodology development; Monitoring and evaluation; Reflexivity; System innovation; System learning

Indexed keywords

EID: 77952874867     PISSN: 10982140     EISSN: 15570878     Source Type: Journal    
DOI: 10.1177/1098214010366046     Document Type: Article

References (50)

1. Abma, T.A., & Widdershoven, G.A.M. (2005). Sharing stories. Narrative and dialogue in responsive nursing evaluation. Evaluation & Health Professions, 28, 90-109.
2. Argyris, C., & Schön, D.A. (1996). Organizational learning II: Theory, method, and practice. Reading, MA: Addison-Wesley.
3. Arkesteijn, M., van Mierlo, B., & Potters, J. (2007). Methoden voor monitoring en evaluatie van innovatieprojecten [Methods for monitoring and evaluation of innovation projects]. Lelystad, the Netherlands: Praktijkonderzoek Plant en Omgeving BV.
4. Beck, U. (1997). The reinvention of politics: Rethinking modernity in the global social order. Cambridge, UK: Polity Press.
5. Bijker, W.E., Hughes, T.P., & Pinch, T.J. (1987). The social construction of technological systems: New directions in the sociology and history of technology. Cambridge, MA: MIT Press.
6. Christie, A.C., & Alkin, M.C. (2008). Evaluation theory tree re-examined. Studies in Educational Evaluation, 34, 131-135.
7. Davies, R., & Dart, J. (2005). The 'most significant change' (MSC) technique: A guide to its use. Retrieved April 20, 2005, from http://www.mande.co.uk/docs/MSCGuide.htm.
8. de Bruijn, J.A., & ten Heuvelhof, E.F. (1995). Netwerkmanagement. Strategieën, instrumenten en normen [Network management: Strategies, instruments and norms]. Utrecht, the Netherlands: Lemma.
9. Dörner, D. (1996). The logic of failure: Recognizing and avoiding error in complex situations. Reading, MA: Perseus Books.
10. European Centre for Development Policy Management. (2006). Capacity.org, Issue September 2006. Retrieved October 2, 2007, from http://www.capacity.org/en/journal/archives/monitoring_and_evaluation.
11. Engel, P.G.H., & Carlsson, C. (2002). Enhancing learning through evaluation: Approaches, dilemmas and some possible ways forward. Paper presented at the 2002 EES Conference, Seville.
12. Estrella, M. (Ed.). (2000). Learning from change. Issues and experiences in participatory monitoring and evaluation. London, UK: Intermediate Technology Publications Ltd.
13. Fisher, F., Miller, G.J., & Sidney, M.S. (2007). Handbook of public policy analysis: Theory, politics, and methods. Boca Raton, FL: CRC Press.
14. Geels, F.W. (2004). From sectoral systems of innovation to socio-technical systems: Insights about dynamics and change from sociology and institutional theory. Research Policy, 33, 897-920.
15. Giddens, A. (1998). The third way. The renewal of social democracy. Cambridge, UK: Polity.
16. Grin, J. (2006). Reflexive modernization as a governance issue. In J. P. Voß, D. Bauknecht, & R. Kemp (Eds.), Reflexive governance for sustainable development (pp. 57-82). Cheltenham, UK: Edward Elgar.
17. Grin, J., Felix, F., Bos, B., & Spoelstra, S. (2004). Practices for reflexive design: Lessons from a Dutch programme on sustainable agriculture. International Journal on Foresight and Innovation Policy, 1, 126-149.
18. Grin, J., & van Staveren, A. (2007). Werken aan systeeminnovaties. Lessen uit de praktijk van InnovatieNetwerk [Working on system innovations. Lessons from practice of the Innovation Network]. Assen, the Netherlands: Van Gorcum.
19. Guba, E.G., & Lincoln, Y.S. (1989). Fourth generation evaluation. Newbury Park, CA: SAGE.
20. Holling, C.S. (1995). What barriers? What bridges? In L. H. Gunderson, C. S. Holling, & S. S. Light (Eds.), Barriers and bridges to the renewal of ecosystems and institutions (pp. 3-37). New York, NY: Colombia University Press.
21. Jacobs, M. (1999). Sustainable development as a contested concept. In A. Dobson (Ed.), Fairness and futurity: Essays on environmental sustainability and social justice (pp. 21-45). UK: Oxford University Press.
22. Jickling, B. (1994). Why I don't want my children to be educated for sustainable development: Sustainable belief. Trumpeter, 11. Retrieved August 21, 2009, from http://www.icaap.org/iuicode?6.11.3.3.
23. Klein Woolthuis, R.J.A., Gilsing, V., & Lankhuizen, M. (2005). A system failure framework for innovation policy design. Technovation, 25, 609-619.
24. Kleiner, A., & Roth, G. (1997). Learning histories: A new tool for turning organizational experience into action. Retrieved November 7, 2007, from http://ccs.mit.edu/lh/21CWP002.html and http://ccs.mit.edu/lh/.
25. Lay, M., & Papadopoulos, I. (2007). An exploration of fourth generation evaluation in practice. Evaluation: The International Journal of Theory, Research and Practice, 13, 495-504.
26. Loeber, A., van Mierlo, B., Grin, J., & Leeuwis, C. (2007). The practical value of theory: Conceptualizing learning in the pursuit of a sustainable development. In A. Wals (Ed.), Social learning towards a more sustainable world (pp. 83-98). Wageningen, the Netherlands: Wageningen Academic Publishers.
27. Mark, M.M., Greene, J.C., & Shaw, I.F. (2006). The evaluation of policies, programs, and practices. In I. Shaw, J. C. Greene, & M. M. Mark (Eds.), The Sage handbook of evaluation (pp. 1-30). London, UK: SAGE.
28. Mebratu, D. (1998). Sustainability and sustainable development: Historical and conceptual review. Environmental Impact Assessment Review, 18, 493-520.
29. Mertens, D.M. (1999). Inclusive evaluation: Implications of transformative theory for evaluation. American Journal of Evaluation, 20, 1-14.
30. Oesp. (1997). Who are the question makers? A participatory evaluation handbook. New York, NY: Office of the Evaluation and Strategic Planning, UNDP.
31. Osborne, S. P. (Ed.). (2000). Public-private partnerships. Theory and practice in international perspective. London, UK: Routledge.
32. Patton, Q.M. (2001). Qualitative research & evaluation methods. Newbury Park, CA: SAGE.
33. Pitman, G., Feinstein, O.N., & Ingram, G.K. (2005). Evaluating development effectiveness. World Bank series on evaluation and development. (Vol. 7). New Brunswick, NJ: Transaction Publishers.
34. Prigogine, I., & Stengers, I. (1990). Orde uit chaos: een nieuwe dialoog tussen de mens en de natuur [From chaos to order: A new dialogue between mankind and nature]. Amsterdam, the Netherlands: Uitgeverij Bert Bakker.
35. Regeer, B., & Bunders, J. (2003). The epistemology of transdisciplinary research. From knowledge integration to communities of practice. Interdisciplinary Environmental Review, V, 98-118.
36. Ringsing, B., & Leeuwis, C. (2008). Learning about advocacy: A case-study of challenges, everyday practices and tensions. Evaluation, 14, 413-436.
37. Rip, A., & Kemp, R. (1998). Technological change. In S. Rayner & E. L. Malcolm (Eds.), Human choice and climate change. An international assessment (Vol. 2, pp. 327-399). Washington, DC: Batell Press.
38. RIZA, The National Institute for Water Management and Waste Water Management. (2005). Emissies van gewasbeschermingsmiddelen in de glastuinbouw. [Emission of crop protection chemicals in greenhouse horticulture]. Lelystad, the Netherlands: RIZA.
39. Rogers, P.J. (2008). Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation, 14, 29-48.
40. Rotmans, J., Loorbach, D., & van de Brugge, R. (2005). Transitiemanagement en duurzame ontwikkeling; Coevolutionaire sturing in het licht van complexiteit [Transition management and sustainable development; Co-evolutionary steering in the light of complexity]. Beleidswetenschap, 19, 3-23.
41. Schön, D.A. (1983). The reflective practitioner. How professionals think in action. New York, NY: Basic Books Inc.
42. Stake, R.E. (1983). Program evaluation, particularly responsive evaluation. In G. Madaus, M. Scriven, & D. Stufflebeam (Eds.), Evaluation models (pp. 291-304). Norwell, MA: Kluwer Academic Publishers.
43. Thiele, G., Devaux, A., Velasco, C., & Horton, D. (2007). Horizontal evaluation. Fostering knowledge sharing and program improvement within a network. American Journal of Evaluation, 28, 493-508.
44. Uphoff, N., & Combs, J. (2001). Some things can't be true but are: Rice, rickets and what else? Ithaca, NY: Cornell International Institute for Food, Agriculture and Development.
45. van Mierlo, B., Arkesteijn, M., & Leeuwis, C. (2007). Reflexieve procesmonitoring & Telen met Toekomst. De eerste fase [Reflexive process monitoring: The first phase]. Wageningen, the Netherlands: PPO/ WUR CIS.
46. van Mierlo, B., Arkesteijn, M., & Leeuwis, C. (2009). Collective analyses of barriers and opportunities for a sustainable development with the innovation system framework. In K. J. Poppe, C. J. A. M. Termeer, & M. J. Slingerland (Eds.), Transitions towards sustainable agriculture and food chains in peri-urban areas (pp. 139-161). Wageningen, the Netherlands: Wageningen Academic Publishers.
47. van Mierlo, B., Leeuwis, C., Smits, R., & Klein Woolthuis, R. (2010). Learning towards system innovation. Evaluating a systemic instrument. Technological Forecasting and Social Change, 77, 318-334.
48. Voß, J. P., Bauknecht, D., & Kemp, R. (Eds.). (2006). Reflexive governance for sustainable development. Cheltenham, UK: Edward Elgar.
49. Voß, J.P., & Kemp, R. (2006). Sustainability and reflexive governance: Introduction. In J. P. Voß, D. Bauknecht, & R. Kemp (Eds.), Reflexive governance for sustainable development (pp. 3-28). Cheltenham, UK: Edward Elgar.
50. Wiskerke, J.S.C., & van der Ploeg, J.D. (2004). Seeds of transition. Essays on novelty production, niches and regimes in agriculture. Series: European Perspectives on Rural Development. Assen, the Netherlands: Royal Van Gorcum.