The research excellence framework and the 'impact agenda': Are we creating a Frankenstein monster?

Research Evaluation

Volumn 20, Issue 3, 2011, Pages 247-254

  Martin, Ben R ^a  

^a UNIVERSITY OF SUSSEX   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 80053585978     PISSN: 09582029     EISSN: None     Source Type: Journal    
DOI: 10.3152/095820211X13118583635693     Document Type: Article

References (58)

1. ABRC, Advisory Board for the Research Councils 1987. A Strategy for the Science Base: a Discussion Document, produced by the Advisory Board for the Research Councils. London: HMSO.
2. Adair, John G 1984. The Hawthorne effect: a reconsideration of the methodological effect. Journal of Applied Psychology, 69(2), 334-345.
3. Bekhradnia, Bahram 2009. Proposals for the Research Excellence Framework: a Critique. Oxford: Higher Education Policy Institute.
4. Bernal, John D 1939. The Social Function of Science. London: Routledge.
5. Brewer, John 2011a. Viewpoint: from public impact to public value. Methodological Innovations Online, 6(1), 9-12.
6. Brewer, John 2011b. The impact of impact. Research Evaluation, 20(3), September, 255-256.
7. Chrystal K Alec and Paul D Mizen 2001. Goodhart's Law: its origins, meaning and implications for monetary policy. Prepared for the Festschrift in honour of Charles Goodhart held on 15-16 November 2001 at the Bank of England.
8. Collins, Harry 2011. Measures, markets and information. Methodological Innovations Online, 6(1), 3-6.
9. Davis, Caroline 2002. Analysis; UK's spin-offs reap £3 billion reward. Times Higher Education, 25 October 2002. http://www.timeshighereducation.co.uk/story.asp?storyCode=172516§ioncode=26, last accessed 12 July 2011.
10. DEST, Department of Education, Science and Training 2006. Research Quality Framework: Assessing the Quality and Impact of Research in Australia: the Recommended RQF. Canberra: DEST.
11. Donovan, Claire 2007. The qualitative future of research evaluation. Science and Public Policy, 34(8), October, 585-597.
12. Donovan, Claire 2008. The Australian Research Quality Framework: a live experiment in capturing the social, economic, environmental, and cultural returns of publicly funded research. In Reforming the Evaluation of Research: New Directions for Evaluation, eds C L S Coryn and M Scriven, 118, 47-60.
13. Donovan, Claire 2011. State of the art in assessing research impact: introduction to a special issue: Research Evaluation, 20(3), September, 175-179.
14. Geuna, Aldo and Ben R Martin 2003. University research evaluation and funding: an international comparison. Minerva, 41, 277-304.
15. Goodhart, Charles A E 1975. Monetary relationships: a view from Threadneedle Street. In Papers in Monetary Economics, Vol. 1. Reserve Bank of Australia.
16. Grant, Jonathan, Philipp-Bastian Brutscher, Susan Ella Kirk, Linda Butler and Steven Wooding 2009. Capturing Research Impacts: a Review of International Practice. Report prepared for the Higher Education Funding Council for England. Cambridge: RAND Europe.
17. Hambrick, Donald C and Richard A D'Aveni 1988. Large corporate failures as downward spirals. Administrative Science Quarterly, 33, 1-23.
18. HEFCE, Higher Education Funding Council for England 2011a. Decisions on Assessing Research Impact. Bristol: HEFCE.
19. HEFCE, Higher Education Funding Council for England 2011b. Assessment Framework and Guidance on Submissions. Bristol: HEFCE.
20. Heisenberg, Werner 1930. The Physical Principles of the Quantum Theory. Chicago: University of Chicago Press.
21. Hicks, Diana 2011. Performance-based University Research Funding Systems. Atlanta: Georgia Institute of Technology.
22. HM Treasury 2006. Science and Innovation Investment Framework 2004-2014: Next Steps. Discussion paper prepared by HM Treasury, Department of Trade and Industry, Department for Education and Skills, and Department of Health. London: HMSO.
23. Holmwood, John 2011. The impact of 'impact' on UK social science. Methodological Innovations Online, 6(1), 13-17.
24. Irvine, John and Ben R Martin 1983. Assessing basic research: the case of the Isaac Newton Telescope. Social Studies of Science, 13, 49-86.
25. Irvine, John and Ben R Martin 1984. What direction for basic scientific research? In Science and Technology Policy in the 1990s and Beyond, eds M Gibbons, P Gummett and B M Udgaonkar, pp. 67-98. Harlow: Longman.
26. Klautzer, Lisa, Stephen Hanney, Eddy Nason, Jennifer Rubin, Jonathan Grant and Steven Wooding 2011. Assessing policy and practice impacts of social science research: the application of the Payback Framework to assess the Future of Work programme. Research Evaluation, 20(3), September, 201-209.
27. Martin, Ben R 1997. Evaluating science and scientists: factors affecting the acceptance of evaluation results. In Evaluating Science and Scientists: an East-West Dialogue on Research Evaluation in Post-Communist Europe, eds M S Frankel and J Cave, pp. 28-45. Budapest: Central European University Press.
28. Martin, Ben R 2007. Assessing the impact of basic research on society and the economy. Invited presentation at the FWFESF International Conference on 'Science Impact: Rethinking the Impact of Basic Research on Society and the Economy', Vienna, 11 May 2007.
29. Martin, Ben R 2012. The evolution of science policy and innovation studies. Research Policy (forthcoming).
30. Martin, Ben R and John Irvine 1981. Internal criteria for scientific choice: an evaluation of the research performance of electron high-energy physics accelerators. Minerva, XIX, 408-432.
31. Martin, Ben R and John Irvine 1983. Assessing basic research: some partial indicators of scientific progress in radio astronomy. Research Policy, 12, 61-90.
32. Martin, Ben R and Puay Tang 2006. The Economic and Social Benefits of Publicly Funded Basic Research. Report to the Office of Science and Innovation, Department of Trade and Industry. Brighton: SPRU.
33. Martin, Ben R and Richard Whitley 2010. The UK Research Assessment Exercise: a case of regulatory capture? In Reconfiguring Knowledge Production: Changing Authority Relationships in the Sciences and their Consequences for Intellectual Innovation, eds R Whitley, J Gläser and L Engwall, pp. 51-80. Oxford: Oxford University Press.
34. Martin, Ben R, Ammon Salter, Diana Hicks, Keith Pavitt, Jacqueline Senker, Margaret Sharp and Nick von Tunzelmann 1996. The Relationship between Publicly Funded Basic Research and Economic Performance: a SPRU Review. London: HM Treasury.
35. Moed, H F, W J M Burger, J G Frankfort and A F J van Raan 1985. The use of bibliometric data for the measurement of university research. Research Policy, 14(3), 131-149.
36. Molas-Gallart, Jordi and Puay Tang 2011. Tracing 'productive interactions' to identify social impacts: an example from the social sciences. Research Evaluation, 20(3), September, 219-226.
37. Molas-Gallart, Jordi, Ammon Salter, Pari Patel, Alister Scott and Xavier Duran 2002.Measuring Third StreamActivities. Brighton, UK: SPRU, University of Sussex.
38. Morgan, Oliver 2002. Master of science fiction. The Observer, 20 October 2002. http://www.guardian.co.uk/business/2002/oct/20/interviews. politics on 12 July 2011, last accessed 12 July 2011.
39. Narin, Francis 1976. Evaluative Bibliometrics. Cherry Hill, NJ: Computer Horizons Inc.
40. Nason, Eddy, Brendan Curran, Stephen Hanney, Barbara Janta, Gillian Hastings, Mairead O'Driscoll and Steven Wooding 2011. Assessing the impacts of the Health Research Board of Ireland's funding activities. Research Evaluation, 20(3), September, 193-200.
41. Nelkin, Dorothy 1979. The social responsibility of scientists. Annals of the New York Academy of Sciences, 334, 176-182.
42. Nightingale, Paul and Alister Scott 2007. Peer review and the relevance gap: ten suggestions for policy-makers. Science and Public Policy, 34(8), October, 543-553.
43. OST, Office of Science and Technology 1993. Realising Our Potential: a Strategy for Science, Engineering and Technology. White Paper prepared by the Office of Science and Technology (Cmnd. 2250). London: HMSO.
44. Phillimore A John 1989. University research performance indicators in practice: the University Grants Committee's evaluation of British universities, 1985-86. Research Policy, 18, 255-271.
45. Rhodes, R A W 1994. The hollowing out of the state: the changing nature of the public service in Britain. Political Quarterly, 65(2), 138-151.
46. Rip, Arie and Egbert Boeker 1975. Scientists and social responsibility in the Netherlands. Social Studies of Science, 5(4), 457-484.
47. Rose, Hilary and Steven Rose 1970. Science and Society. Harmondsworth: Penguin.
48. Salter, Ammon J and Ben R Martin 2001. The economic benefits of publicly funded basic research: a critical review. Research Policy, 30, 509-532.
49. Sastry, Tom and Bahram Bekhradnia 2006. Using Metrics to Allocate Research Funds. Oxford: Higher Education Policy Institute.
50. Scott, Jack E, Margaret Blasinsky, Mary Dufour, Rachel Mandal and Stephane Philogene 2011. Assessing the impact of an NIH program using the Research Payback Framework. Research Evaluation, 20(3), September, 185-192.
51. Sennett, Richard 2008. TheCraftsman. NewHaven: Yale University Press.
52. Smith, Simon, Vicky Ward and Allan House 2011. 'Impact' in the proposals for the UK's Research Excellence Framework: shifting the boundaries of academic autonomy. Research Policy (in press).
53. Spaapen, Jack and Leonie van Drooge 2011. The role of productive interactions in the assessment of social impact of research. Research Evaluation, 20(3), September, 211-218.
54. Tichy, Noel Mand Mary Anne Devanna 1986. The Transformational Leader. New York: Wiley.
55. Wakefield, Andrew J et al 1998. Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. The Lancet, 351(9103), 637-641 (subsequently retracted).
56. Weiss, Carol H 1980. Knowledge creep and decision accretion. Knowledge: Creation, Diffusion, Utilization, 1(3), 381-404.
57. Werskey P Gary 1971. British scientists and 'outsider' politics, 1931-1945. Science Studies, 1(1), 67-83.
58. Wickström, G and T Bendix 2000. The Hawthorne effect: what did the original Hawthorne studies actually show? Scandinavian Journal of Work, Environment and Health, 26(4), 363-367.