Assessing the impact of research on policy: A literature review

Science and Public Policy

Volumn 36, Issue 4, 2009, Pages 255-270

  Boaz, Annette ^a   Fitzpatrick, Siobhan ^a   Shaw, Ben ^b  

^a KING'S COLLEGE LONDON   (United Kingdom)

^b POLICY STUDIES INSTITUTE   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ENVIRONMENTAL POLICY; POLICY APPROACH; POLICY STRATEGY; POLLUTION POLICY; RESEARCH WORK; STRATEGIC APPROACH;

EID: 69949134127     PISSN: 03023427     EISSN: None     Source Type: Journal    
DOI: 10.3152/030234209X436545     Document Type: Article

References (81)

1. Acosta, A and B Douthwaite 2005. Appreciative inquiry: an approach for learning and change based on our own best practices, ILAC Brief 6, Rome: Institutional Learning and Change. , last accessed 7 April 2009.
2. AHRC, Arts and Humanities Research Council 2007. AHRC Impact Strategy. Bristol: AHRC. , last accessed 7 April 2009.
3. Almeida, C and E Bascálo 2006. Use of research results in policy decision-making, formulation, and implementation: a review of the literature. Cadernos de Saúde Pública, 22(Suppl.), S7-S19. , last accessed 7 April 2009.
4. Anderson, R 2006. The policy impact of population health surveys: an illustration of the measurement challenges using the NSW Health Survey. Evidence and Policy, 2(2), 167-183.
5. Arnold, E, A Muscio and R Zaman 2005. Reviews Reviewed: Lessons from the First Six International Panel Reviews 1999- 2004. Brighton: Technopolis, for the Engineering and Physical Sciences Research Council. .
6. BBSRC, Biotechnology and Biological Sciences Research Council 2005. BBSRC Strategy for Evaluating Research Programmes. SB 44/2005. Swindon: BBSRC. , last accessed 7 April 2009.
7. Beacham, B, L Kalucy and E McIntyre 2005. Understanding and measuring research impact. FOCUS on ... (2), 12pp. , last accessed 7 April 2009.
8. Big Lottery Fund 2005. Increasing the Impact of Research, Evaluation and Learning. London: Big Lottery Fund. , last accessed 7 April 2009.
9. Boaz, A, D Ashby and K Young 2001. Systematic reviews: what have they got to offer evidence based policy and practice? Working Paper No 2, ESRC UK Centre for Evidence Based Policy and Practice: , last accessed 7 April 2009.
10. Boaz, A, S Fitzpatrick and B Shaw 2008. Assessing the impact of research on policy: a review of the literature for a project on bridging research and policy through outcome evaluation. , last accessed 7 April 2009.
11. Brofoss, K E 1998. The Research Council of Norway's use of research evaluation: an assessment of research evaluation as a strategic tool. Research Evaluation, 7(3), December, 134-140.
12. Bumgarner, R, D Hattaway, J McGann and H Wise 2006. Center for Global Development: Evaluation of Impact. Washington, DC: Arabella Philanthropic Investment Advisors. , last accessed 7 April 2009.
13. Butler, L and I McAllister 2007. Metrics or Peer Review? Evaluating the 2001 UK Research Assessment Exercise in Political Science. Canberra: Australian National University, Research School of Social Sciences. , last accessed 7 April 2009.
14. Buxton, M, S Hanney, T Packwood, S Roberts and P Youll 2000. Assessing benefits from Department of Health and National Health Service research and development. Public Money and Management, 20(4), 29-34.
15. Carden, F 2004. Issues in assessing the policy influence of research. International Social Science Journal, 56(179), 135-151.
16. Carden, F 2005. Making the most of research: the influence of IDRC-supported research on policy processes. Paper for the international conference, African Economic Research Institutions and Policy Development: Opportunities and Challenges, Dakar, 28-29 January 2005. , last accessed 7 April 2009.
17. Carden, F (Forthcoming) Using comparative data: a systems approach to a multiple case study. (chapter for a book on case methods).
18. CGIAR Science Council 2006. Impact Assessment of Policy- oriented Research in the CGIAR: a Scoping Study Report. Rome: CGIAR Science Council Secretariat.
19. CHASS, Council for the Humanities, Arts and Social Sciences 2005. Measures of quality and impact of publicly funded research in the humanities, arts and social sciences. CHASS Occasional Paper 2. Canberra: CHASS. , last accessed 7 April 2009.
20. Christiansen, K with I Hovland 2003. The PRSP initiative: multilateral policy change and the role of research, Working Paper 216. London, Overseas Development Institute. , last accessed 7 April 2009.
21. Council for Medical Sciences 2002. The Societal Impact of Applied Health Research: towards a Quality Assessment System. Amsterdam: Royal Netherlands Academy for Arts and Sciences. , last accessed 7 April 2009.
22. Cranfield, S and H Ward 2002. 'Managing Change in the NHS' (2001) Publications: Report of a Study on the Uptake and Use of Two SDO Publications. London: National Co-ordinating Centre for the NIHR Service Delivery and Organisation Programme (NCCSDO). , last accessed 7 April 2009.
23. Cunningham, P, M Boden, S Glynn and P Hills 2001. Measuring and Ensuring Excellence in Government Science and Technology: International Practices. France, Germany, Sweden and the United Kingdom. Manchester: University of Manchester PREST (Policy Research in Engineering, Science and Technology)., last accessed 7 April 2009.
24. Davies, R and J Dart 2005. The 'Most Significant Change' (MSC) Technique: a Guide to Its Use. London: CARE International et al. , last accessed 7 April 2009.
25. Davies, H, S Nutley and I Walter 2005. Approaches to Assessing the Non-academic Impact of Social Science Research: Report of the ESRC Symposium on Assessing the Non-academic Impact of Research. St Andrews: University of St Andrews, Research Unit for Research Utilisation. , last accessed 7 April 2009.
26. Delanghe, H and U Muldur 2007. Ex-ante impact assessment of research programmes: the experience of the European Union's 7th Framework Programme. Science and Public Policy, 34(3), 169-183.
27. Earl, S and F Carden 2002. Learning from complexity: the International Development Research Centre's experience with outcome mapping. Development in Practice, 12(3/4), 518-524.
28. EC, European Commission 2005. Assessing the Impact of Energy Research, EUR 21354. Luxembourg: Office for Official Publications of the European Communities. , last accessed 7 April 2009.
29. Elliott, H and J Popay 2000. How are policy makers using evidence? Models of research utilisation and local NHS policy making. Journal of Epidemiology and Community Health, 54(6), 461-468.
30. Fahrenkrog, G, W Polt, J Rojo, A Tübke and K Zinöcker, eds. 2002. RTD Evaluation Toolbox: Assessing the Socio-economic Impact of RTD-policies. IPTS Technical Report Series, EUR 20382. Seville: European Commission, Joint Research Centre, Institute for Prospective Technological Studies. , last accessed 7 April 2009.
31. Fisher, E and J D Holland 2003. Social development as knowledge building: research as a sphere of policy influence. Journal of International Development, 15(7), 911-924.
32. Flint, M E S 1998. Natural Resources Research Evaluation Synthesis, Evaluation Report EV:604. London: Department for International Development. , last accessed 7 April 2009.
33. Gabbay, J, A le May, H Jefferson, D Webb, R Lovelock, J Powell and J Lathlean 2003. A case study of knowledge management in multi-agency consumer-informed 'communities of practice': implications for evidence-based policy development in health and social services. Health, 7(3), 283-310.
34. Garfinkel, M S, D Sarewitz and A L Porter 2006. A societal outcomes map for health research and policy. American Journal of Public Health, 96(3), 441-446.
35. Georghiou, L and E Davis eds. 1988. Evaluation of R&D: a Poli cymaker's Perspective: Proceedings of the International Work shop on Assessment and Evaluation, Queen Elizabeth II Conference Centre London. London: HMSO.
36. Godin, B and C Doré 2003. Measuring the Impacts of Science: Beyond the Economic Dimension. Montreal: Canadian Science and Innovation Indicators Consortium. , last accessed 7 April 2009.
37. Guy, K, E Amanatidou and F Psarra 2005. Framework Programme 5 (FP5) impact assessment: a survey conducted as part of the five-year assessment of European Union research activities (1999-2003). Science and Public Policy, 32(5), October, 349-366.
38. Hanney, S, A Davies and M Buxton 1999. Assessing benefits from health research projects: can we use questionnaires instead of case studies? Research Evaluation, 8(3), December, 189-199.
39. Hanney, S R, J Grant, S Wooding and M J Buxton 2004. Proposed methods for reviewing the outcomes of health research: the impact of funding by the UK's 'Arthritis Research Campaign'. Health Research Policy and Systems, 2(4). , last accessed 7 April 2009.
40. Hanney, S, M Buxton, C Green, D Coulson and J Raftery 2007. An assessment of the impact of NHS Health Technology Assessment Programme. NHS Health Technology Assessment Programme, 11(53). , last accessed 7 April 2009.
41. Hilderbrand, M and J Simon 2000. The role of research in child health policy and programs in Pakistan. In: Lessons in Research to Action: Case Studies from Seven Countries, pp. 77-86. Geneva: Council on Health Research for Development.
42. HMT, Her Majesty's Treasury, Department for Education and Skills, Office of Science and Technology and Department of Trade and Industry 2002. Cross-cutting Review of Science and Research: Final Report. London, HM Treasury. , last accessed 7 April 2009.
43. Hooton, N, D Lee-Smith, G Nasinyama and D Romney 2006. Learning Lessons from the Kampala Urban Agriculture Policy Process, Process and Partnership for Pro-Poor Policy Change Project report. International Livestock Research Institute, and London: Overseas Development Institute. , last accessed 7 April 2009.
44. Hovland, I 2007. Making a Difference: M&E of Policy Research, Working Paper 281. London, Overseas Development Institute. , last accessed 7 April 2009.
45. Jones, S A, B Fischhoff and D Lach 1999. Evaluating the science-policy interface for climate change research. Climatic Change, 43(3), 581-599.
46. Kilpatrick, H E 1998. Some Useful Methods for Measuring the Benefits of Social Science Research, Impact Assessment Discussion Paper No. 5. Washington, DC: International Food Policy Research Institute. , last accessed 7 April 2009.
47. Kivimaa, P, H Mela and E Furman 2007. Approaches and Practices in Mid-term and Ex-post Evaluation of Environmental Research Programmes. Scientific Knowledge for Environmental Protection (SKEP) Work Package 3, 28 June 2007. , last accessed 7 April 2009.
48. Kostoff, R N 1994. Assessing research impact: US government retrospective and quantitative approaches. Science and Public Policy, 21(1), February, 13-22.
49. Kostoff R N 1995. Federal research impact assessment: axioms, approaches, applications. Scientometrics, 34(2), 163-206.
50. Kuruvilla, S, N Mays and G Walt 2007. Describing the impact of health services and policy research, Journal of Health Services Research and Policy, 12(Suppl. 1), 23-31.
51. Leksmono, C, J Young, N Hooton, H Muriuki and D Romney 2006. Informal traders lock horns with the formal milk industry: the role of research in pro-poor dairy policy shift in Kenya, Working Paper 266. London: Overseas Development Institute. , last accessed 7 April 2009.
52. Lewis, J M 2005. A network approach for researching partnerships in health. Australia and New Zealand Health Policy, 2(22). , last accessed 7 April 2009.
53. Luukkonen-Gronow, T 1987. Scientific research evaluation: a review of methods and various contexts of their application. R&D Management, 17(3), 207-221.
54. Merkx, F, I van der Weijden, A M Oostveen, P van den Besselaar and J Spaapen 2007. Evaluation of Research in Context: Quick Scan of an Emerging Field. The Hague: Rathenau Instituut and EriC (Evaluating Research in Context). , last accessed 7 April 2009.
55. Michelson, E S 2006. Approaches to research and development performance assessment in the United States: an analysis of recent evaluation trends. Science and Public Policy, 33(8), October, 546-560.
56. Molas-Gallart, J and P Tang 2007. Policy and Practice Impacts of ESRC Funded Research: Case Study of the ESRC Centre for Business Research. Swindon: Economic and Social Research Council.
57. NAO, National Audit Office 2003. An international review on governments' research procurement strategies. A paper in support of Getting the Evidence: Using Research in Policy Making, HC 586-II, session 2002-2003. London: Stationery Office. , last accessed 7 April 2009.
58. Nederhof, A 2006. Bibliometric monitoring of research performance in the social sciences and the humanities: a review. Scientometrics, 66(1), 81-100.
59. NHS, National Health Service SDO 2007. The Impact of the NHS Service Delivery and Organisation Research and Development Programme 2001-2006. London: National Co-ordinating Centre for the NIHR Service Delivery and Organisation Programme (NCCSDO). , last accessed 7 April 2009.
60. Norton, G W and J Alwang 1998. Policy for plenty: measuring the benefits of policy-oriented social science research, Impact Assessment Discussion Paper No. 6. Washington, DC: International Food Policy Research Institute. , last accessed 7 April 2009.
61. Nutley, S, J Percy-Smith and W Solesbury 2003. Models of research impact: a cross sector review of literature and practice, Building Effective Research 4. London: Learning and Skills Research Centre. , last accessed 7 April 2009.
62. Nutley, S, I Walter and H Davies 2007. Using Evidence: How Research Can Inform Public Services. Bristol: Policy Press.
63. OSI, Office of Science and Innovation 2006. Science Review of the Department for Environment, Food and Rural Affairs. London: Office of Science and Innovation. , last accessed 7 April 2009.
64. Peckham, S, M Willmott, P Allen, S Anderson and N Goodwin forthcoming 2008. Assessing the impact of the NHS Service Delivery and Organisation Research Programme. Submitted to Evidence and Policy.
65. Petticrew, M and H Roberts 2005. Systematic Reviews in the Social Sciences: a Practical Guide. Oxford: Blackwell.
66. Pielke, R A 1995. Usable information for policy: an appraisal of the US Global Change Research Program. Policy Sciences, 28(1), 39-77.
67. Research Councils UK 2007. Increasing the Economic Impact of the Research Councils. Swindon: Research Councils UK. , last accessed 7 April 2009.
68. Rider Smith, D with A Sutherland 2002. Institutionalising IMPACT Orientation: Building a Performance Approach that Enhances the Impact Orientation of Research Organisations. London: University of Greenwich, Natural Resources Institute. , last accessed 7 April 2009.
69. Ryan, J G 2002. Synthesis report of workshop on assessing the impact of policy-oriented social science research in Scheveningen, the Netherlands, November 12-13, 2001, Impact Assessment Discussion Paper No. 15. Washington, DC: International Food Policy Research Institute. , last accessed 7 April 2009.
70. Sayce, K and P Norrish 2006. Perceptions and Practice: an Anthology of Impact Assessment Experiences. Wageningen: Technical Centre for Agricultural and Rural Cooperation. , last accessed 7 April 2009.
71. Shadish, W, T Cook and C Leviton 1991. Foundations of Programme Evaluation: Theories of Practice. Thousand Oaks, CA: Sage.
72. Smith, V H 1998. Measuring the benefits of social science research, Impact Assessment Discussion Paper No. 2. Washington, DC: International Food Policy Research Institute. , last accessed 7 April 2009.
73. Thiele, G, A Devaux, C Velasco and K Manrique 2006. Horizontal Evaluation: Stimulating Social Learning Among Peers. ILAC Brief 13, Rome: Institutional Learning and Change. , last accessed 7 April 2009.
74. Walter, I, S Nutley and H Davies 2003. Research Impact: a Cross Sector Review - Literature Review. St Andrews: St Andrews University, Research Unit for Research Utilisation. , last accessed 7 April 2009.
75. Walter, I, S Nutley and H Davies 2004. Assessing Research Impact: Report of Seminar, 15-16 January 2004. St Andrews: University of St Andrews, Research Unit for Research Utilisation. , last accessed 7 April 2009.
76. Weiss, C 1998. Evaluation: Methods for Studying Programs and Policies, 2nd edn. London: Prentice Hall International.
77. Williams, D and A D Rank 1998. Measuring the economic benefits of research and development: the current state of the art. Research Evaluation, 7(1), April, 17-30.
78. Wooding, S, S Hanney, M Buxton and J Grant 2004. The returns from arthritis research: a report prepared by RAND Europe for the Arthritis Research Campaign. Volume 1: approach, analysis and recommendations. Cambridge: RAND Europe. , last accessed 7 April 2009.
79. Wooding, S, E Nason, L Klautzer, J Rubin, S Hanney and J Grant 2007. Policy and Practice Impacts of Research Funded by the Economic and Social Research Council: a Case Study of the Future of Work Programme, Approach and Analysis. Cambridge: RAND Europe. , last accessed 7 April 2009.
80. World Bank 2004. Monitoring and Evaluation: Some Tools, Methods and Approaches. Washington, DC, The World Bank. , last accessed 7 April 2009.
81. Zilberman, D and A Heiman 1999. The value of economic research, Impact Assessment Discussion Paper No. 7. Washington, DC: International Food Policy Research Institute. , last accessed 7 April 2009.