Challenges of evaluating democracy assistance: Perspectives from the donor side

Democratization

Volumn 14, Issue 1, 2007, Pages 151-165

  Green, Andrew T ^a   Kohl, Richard D ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

AID; DEMOCRACY; DEMOCRATIZATION; FOREIGN POLICY; POLICY MAKING;

EID: 34347206804     PISSN: 13510347     EISSN: 1743890X     Source Type: Journal    
DOI: 10.1080/13510340601024363     Document Type: Article

References (30)

1. A second workshop was held in Spring 2006 at International IDEA (Institute for Democracy and Electoral Assistance) in Stockholm, supported by the Swedish International Development Cooperation Agency (SIDA), that focused more specifically on evaluation practice.
2. In this report we adopt the nomenclature of distinguishing between specific projects which occur in one country and address one specific sub-sector and programmes which cover all projects in a particular DG sector or sub-sector, whether in one or many countries. The discussion in the workshop covered both projects and programmes.
3. This stems largely from a report on the 'Measuring the Impact of Democracy and Governance Assistance' workshop, held in March 2005 at the Clingendael Institute, funded by USAID and organized by Clingendael, The Hague, the Netherlands.
4. Peter Burnell, 'The Domestic Political Impact of Foreign Aid: Recalibrating the Research Agenda', The European Journal of Development Research, Vol. 16, No. 2 (2004), pp.396-416.
5. Even when evaluations are conducted, teams typically spend only two to three weeks in the field, which is increasingly recognized as insufficient time to conduct quality research.
6. An extreme example is the case of political party assistance referred to above. Because this is often dominated by party institutes, they are the main (if not sole) repository of experience, leaving donor agencies without expertise to evaluate party assistance programmes.
7. See Gordon Crawford, 'Promoting Democracy From Without - Learning from Within (Part II)', Democratization, Vol. 10, No. 2 (2003), pp.1-20
8. Burnell (note 4), pp.396-416.
9. There is a financial disincentive for external firms to include local experts on teams, also. The cost structure of an agreement with a donor agency usually includes an overhead percentage that acts to discourage the use of less-expensive local experts and encourage the use of more-expensive donorcountry- based consultants.
10. Timing can also be a problem in terms of administrative procedures. By the time a Request for Proposal (RFP) is issued, contracts signed, and the research completed, the work is too often irrelevant, overtaken by events, or rendered moot by changes in programming and priorities.
11. DG analyses normally do not take into account the impact of other donors' work in the same or related issue area.
12. This can be seen within the European Union where member countries are often not aware of each other's DG programmes.
13. USAID has published the NGO Sustainability Index since 1998 for post-communist states. The Index aggregates indicators of seven aspects: legal environment, organization capacity, financial viability, advocacy, public image, service provision, and sector infrastructure; see USAID's website, available at http://www.usaid.gov/locations/europe_eurasia/ dem_gov/ngoindex/. Andrew Green has developed the Non-Profit Sector (NPS) indicator for post-communist states since 1991. NPS is based on four legal features: Allowed organization types, registration procedures, tax and other economic concessions, and space for political activity; see Andrew T. Green, 'Comparative Development of Post-Communist Civil Societies', Europe-Asia Studies, Vol. 54, No. 3 (2002), pp.455-71.
14. http://www.oecd.org/dac/stats/statlinks
15. This also assumes that the data was accurate within the donor agency, a difficult assumption to make when some agencies are introducing new management systems for tracking budgets, procurement, and programme results.
16. In the Kenya example cited above, a majority of civil society funding was captured in the 'Strengthening Civil Society' subcategory within the 'Government and Civil Society' category, but significant amounts were also found in 'Other Social Infrastructure and Services', 'Women in Development', 'Support to NGOs', and some multi-sector categories; other ostensibly DG-relevant subcategories actually capture economic reform activities and even landmine clearance, but one could not simply exclude certain subcategories in toto because some agencies reclassify and place actual DG activities in them.
17. Some agencies have adopted the United Nations Millennium Development Goals (MDGs) as performance indicators. Unfortunately for democracy and governance programming, there is no DG in the MDGs.
18. Experts thought that 55 per cent of the population had been the victim of corruption, the survey showed 11 per cent; experts thought over 30 per cent view bribery as acceptable, the survey showed less than 5 per cent; experts also generally underestimated population support for economic reform issues, which in some cases exceeded that among experts themselves. See Matthew Sudders and Joachim Nahem, Governance Indicators: A Users' Guide (New York: UNDP, 2004), p.13;
19. or Javier Herrera, Mireille Razafindrakoto and Francois Roubaud, 'Measuring Poverty, Participatory Democracy and Governance Issues through Official Household Surveys in Francophone Africa and the Andean Countries', Paper presented at the OECDs Metagora Forum, Paris, May 2005, p.10.
20. USAID's Office of Democracy and Governance developed its DG Assessment in the late 1990s, and it has been used in over 60 strategic assessments, see http://www.usaid.gov/our_work/ democracy_and_governance/publications/pdfs/pnach305.pdf. USAID and other agencies have also created assessment tools for 'fragile' or post-conflict countries.
21. See http://www.grc-exchange.org/g_themes/politicalsystems_drivers.html
22. This is similar to USAID's approach of Strategic Objectives, Intermediate Objectives, and so on.
23. 'Governance Questionnaire - An Instrument for Analysing Political Environments', in draft form as of March 2005.
24. Developed by GTZ in conjunction with the consulting firm 'nextpractice', available at http://www/gtz.de/en/leistungsangebote/6339.htm
25. See the surveys carried out by LAPOP, available at http://www.lapopsurveys.org
26. See the Afraobarometer survey, available at http://www.afrobarometer.org
27. Jeroen de Zeeuw and Krishna Kumar (eds), Promoting Democracy in Postconflict Societies (Boulder, CO: Lynne Rienner, 2006).
28. See also J. de Zeeuw, 'Projects Do not Create Institutions: The Record of Democracy Assistance in Post-Conflict Societies', Democratization Vol. 12, No. 4 (2005), pp.48-527.
29. Luc van de Goor and Jeroen de Zeeuw, Workshop on 'Measuring the Impact of Democracy and Governance Assistance', Clingendael Institute, The Hague, Netherlands, 11 March 2005.
30. The data-set was compiled by Andrew Green. The analysis conducted by Mitchell Seligson, Steven Finkel, and Anibal Perez-Linan used advanced quantitative techniques and found that USAID democracy assistance had a significant positive impact on democratic development; see http://www.pitt.edu/~politics/democracy/democracy.html