Assessment, evaluations, and definitions of research impact: A review

Research Evaluation

Volumn 23, Issue 1, 2014, Pages 21-32

  Penfield, Teresa ^a   Baker, Matthew J ^a   Scoble, Rosa ^b   Wykes, Michael C ^a  

^a UNIVERSITY OF EXETER   (United Kingdom)

^b BRUNEL UNIVERSITY   (United Kingdom)

Author keywords

assessment; evidence; impact; research evaluation

Indexed keywords

EID: 84892738894     PISSN: 09582029     EISSN: 14715449     Source Type: Journal    
DOI: 10.1093/reseval/rvt021     Document Type: Review

References (50)

1. Australian Research Council. (2008) ERA Indicator Principles (Pubd online) accessed 6 Oct 2012.
2. Berg, L.O. and Ma? nsson, C. (2011) A White Paper on Charity Impact Measurement (Pubd online) accessed 6 Aug 2012.
3. Bernstein, A. et al. (2006) A Framework to Measure the Impact of Investments in Health Research (Pubd online) accessed 26 Oct 2012.
4. Bornmann, L. and Marx, W. (2013) 'How Good is Research Really?', European Molecular Biology Organization (EMBO) Reports, 14: 226-30.
5. Buxton, M., Hanney, S. and Jones, T. (2004) 'Estimating the Economic Value to Societies of the Impact of Health Research: A Critical Review', Bulletin of the World Health Organization, 82: 733-9.
6. Canadian Academy of Health Sciences Panel on Return on Investment in Health Research. (2009) Making an Impact. A Preferred Framework and Indicators to Measure Returns on Investment in Health Research (Pubd online) accessed 26 Oct 2012.
7. Cooke, J. and Nadim, T. (2011) Measuring Impact Under CERIF at Goldsmiths (Pubd online) accessed 26 Oct 2012.
8. Corbyn, Z. (2009) Anti-Impact Campaign's 'Poster Boy' Sticks up for the Ivory Tower. Times Higher Education (Pubd online) accessed 26 Oct 2012.
9. Davies, H., Nutley, S. and Walter, I. (2005) Assessing the Impact of Social Science Research: Conceptual, Methodological and Practical Issues (Pubd online) accessed 26 Oct 2012.
10. Decker, R. et al. (2007) A Profile of Federal-Grant Administrative Burden Among Federal Demonstration Partnership Faculty (Pubd online) accessed 26 Oct 2012.
11. Department for Business, Innovation and Skills. (2012) Guide to BIS 2012-2013 (Pubd online) accessed 24 Oct 2012.
12. Donovan, C. (2008) 'The Australian Research Quality Framework: A live experiment in capturing the social, economic, environmental and cultural returns of publicly funded research'. In: Coryn, C. L. S. and Scriven, M. (eds) Reforming the Evaluation of Research. New Directions for Evaluation, 118, pp. 47-60.
13. -. (2011) 'Impact is a Strong Weapon for Making an Evidence-Based Case Study for Enhanced Research Support but a State-of-the-Art Approach to Measurement is Needed', LSE Blog, (Pubd online) accessed 26 Oct 2012.
14. Donovan, C. and Hanney, S. (2011) 'The "Payback Framework" explained', Research Evaluation, 20: 181-3.
15. Duryea, M., Hochman, M. and Parfitt, A. (2007) Measuring the Impact of Research, Research Global, 27, 8-9 (Pubd online) http://www.atn.edu.au/docs/ Research%20Global%20-%20Measuring%20the%20impact%20of%20research.pdf accessed 26 Oct 2012.
16. Ebrahim, A. and Rangan, K. (2010) 'The Limits of Nonprofit Impact: A Contingency Framework for Measuring Social Performance', Working Paper. Harvard Business School. (Pubd online) accessed 26 Oct 2012.
17. European Science Foundation. (2009) Evaluation in National Research Funding Agencies: Approaches, Experiences and Case Studies (Pubd online)〈http://www.esf.org/index.php?eID=tx-ccdamdl-file&p[file]= 25668&p[dl]=1&p[pid]=6767&p[site]= European%20Science%20Foundation&p[t]=1351858982&hash= 93e987c5832f10aeee3911bac23b4e0f&l=en〉 accessed 26 Oct 2012.
18. Jones, M.M. and Grant, J. (2013) 'Methodologies for Assessing and Evidencing Research Impact. RAND Europe'. In: Dean. et al. (eds) 7 Essays on Impact. DESCRIBE Project Report for JISC. University of Exeter.
19. Grant, J. (2006) Measuring the Benefits from Research, RAND Europe (Pubd online) accessed 26 Oct 2012.
20. Grant, J. et al. (2009) Capturing Research Impacts. A Review of International Practice (Pubd online) accessed 26 Oct 2012.
21. HM Treasury, Department for Education and Skills, Department of Trade and Industry. (2004) 'Science and Innovation Framework 2004-2014'. London: HM Treasury.
22. Hanney, S. and Gonza?lez-Block, M. A. (2011) 'Yes, Research can Inform Health Policy; But can we Bridge the 'Do-Knowing it's been Done' Gap?', Health Research Policy and Systems, 9: 23.
23. Hughes, A. and Martin, B. (2012) 'Council for Industry and Higher Education, UK Innovation Research Centre. Enhancing Impact. The Value of Public Sector R&D', Summary report, (Pubd online) 〈http://ukirc.ac.uk/object/ report/8025/doc/CIHE-0612ImpactReport-summary.pdf〉 accessed 26 Oct 2012.
24. Husbands-Fealing, K. (2013) 'Assessing impacts of higher education systems', University of Minnesota, in Dean et al. (eds) 7 Essays on Impact. DESCRIBE project report for JISC. University of Exeter.
25. Johnston, R. (1995) 'Research Impact Quantification', Scientometrics, 34: 415-26.
26. Jump, P. (2011) 'HEFCE Reduces Points of Impact in REF', Times Higher Education, (Pubd online) 〈http://www.timeshighereducation.co.uk/story.asp? storyCode=415340& sectioncode=26〉 accessed 26 Oct 2012.
27. Kelly, U. and McNicoll, I. (2011) 'National Co-ordinating Centre for Public Engagement', Through a Glass, Darkly: Measuring the Social Value of Universities (Pubd online) 〈http://www.publicengagement.ac.uk/sites/ default/files/8009 6%20NCCPE%20Social%20Value%20Report.pdf〉 accessed 26 Oct 2012.
28. Kuruvilla, S. et al. (2006) 'Describing the Impact of Health Research: A Research Impact Framework', BMC Health Services Research, 6: 134.
29. LSE Public Policy Group. (2011) 'Maximising the Impacts of Your Research: A Handbook for Social Scientists' (Pubd online)〈http://www2.lse.ac.uk/ government/research/resgroups/LSE PublicPolicy/Docs/LSE-Impact-Handbook-April- 2011.pdf〉 accessed 26 Oct 2012.
30. Lane, J. (2010) 'Let's Make Science Metrics More Scientific', Nature, 464: 488-9.
31. MICE Project. (2011) Measuring Impact Under CERIF (MICE) Project Blog (Pubd online)〈http://mice.cerch.kcl.ac.uk/〉 accessed 26 Oct 2012.
32. Mugabushaka, A. and Papazoglou, T. (2012) 'Information systems of research funding agencies in the "era of the Big Data". The case study of the Research Information System of the European Research Council'. In: Jeffery, K. and Dvor?a?k, J. (eds) E-Infrastructures for Research and Innovation: Linking Information Systems to Improve Scientific Knowledge, Proceedings of the 11th International Conference on Current Research Information Systems (June 6-9, 2012), pp. 103-12. Prague, Czech Republic.
33. Nason, E. et al. (2008) 'RAND Europe', Health Research-Making an Impact. The Economic and Social Benefits of HRB-funded Research. Dublin: Health Research Board.
34. Reeves, M. (2002) 'Arts Council England', Measuring the Economic and Social Impact of the Arts: A Review, (Pubd online) 〈http://www.artscouncil. org.uk/media/uploads/docu ments/publications/340.pdf〉 accessed 26 Oct 2012.
35. REF 2014. (2010) Research Excellence Framework Impact Pilot Exercise: Findings of the Expert Panels, (Pubd online) 〈http://www.ref.ac.uk/media/ ref/content/pub/researchexcelle nceframeworkimpactpilotexercisefindingsoftheexpertpanels/re01-10.pdf〉 accessed 26 Oct 2012.
36. -. (2011a) Assessment Framework and Guidance on Submissions, (Pubd online) 〈http://www.ref.ac.uk/media/ref/content/pub/ assessmentframeworkandguidanceonsubmissions/02-11.pdf〉 accessed 26 Oct 2012.
37. -. (2011b) Decisions on Assessing Research Impact, (Pubd online) 〈http://www.ref.ac.uk/media/ref/content/pub/assessm entframeworkandguidanceonsubmissions/GOS%20including %20addendum.pdf〉 accessed 19 Dec 2012.
38. -. (2012) Panel Criteria and Working Methods, (Pubd online) 〈http://www.ref.ac.uk/media/ref/content/pub/panelcriteriaan dworkingmethods/01-12.pdf〉 accessed 26 Oct 2012.
39. Russell Group. (2009) Response to REF Consultation, (Pubd online) 〈http://www.russellgroup.ac.uk/uploads/REF-consult ation-response-FINAL- Dec09.pdf〉 accessed 26 Oct 2012.
40. Scoble, R. et al. (2009) Research Impact Evaluation, a Wider Context. Findings from a Research Impact Pilot, (Pubd online) 〈http://www.libsearch. com/visit/1953719〉 accessed 26 Oct 2012.
41. -. (2010) 'Institutional Strategies for Capturing Socio-Economic Impact of Research', Journal of Higher Education Policy and Management, 32: 499-511.
42. SIAMPI Website (2011) (Pubd online)〈http://www.siampi.eu/Pages/SIA/ 12/625.bGFuZz1FTkc.html〉 accessed 26 Oct 2012.
43. Spaapen, J. et al. (2011) SIAMPI Final Report (Pubd online)〈http:// www.siampi.eu/Content/SIAMPI/SIAMPI-Final%20 report.pdf〉 accessed 26 Oct 2012.
44. Spaapen, J. and Drooge, L. (2011) 'Introducing 'Productive Interactions' in Social Impact Assessment', Research Evaluation, 20: 211-18.
45. The Allen Consulting Group. (2005) Measuring the Impact of Publicly Funded Research. Canberra: Department of Education, Science and Training.
46. The SROI Network. (2012) A Guide to Social Return on Investment (Pubd online)〈http://www.thesroinetwork.org/publications/doc-details/ 241-a-guide-to-social-return-on-inve stment-2012〉 accessed 26 Oct 2012.
47. University and College Union. (2011) Statement on the Research Excellence Framework Proposals, (Pubd online) 〈http://www.ucu.org.uk/media/pdf/n/q/ ucu-REFstatement-finalsignatures.pdf〉 accessed 26 Oct 2012.
48. Vonortas, N. and Link, A. (2013) Handbook on the Theory and Practice of Program Evaluation. Cheltenham, UK: Edward Elgar Publishing Limited.
49. Whitehead, A. (1929) 'Universities and their function', The Aims of Education and other Essays. New York: Macmillan Company.
50. Wooding, S. et al. (2007) 'RAND Europe', Policy and Practice Impacts of Research Funded by the Economic Social Research Council, (Pubd online)〈http://www.esrc.ac.uk/-images/Case-Study-of-the-Future-of-Work- Programme-Volume-2-tcm 8-4563.pdf〉 accessed 26 Oct 2012.