From Procrustes to Proteus: Trends and practices in the assessment of education research

International Journal of Research and Method in Education

Volumn 30, Issue 3, 2007, Pages 243-269

  Oancea, Alis ^a  

^a UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 35148818798     PISSN: 1743727X     EISSN: 17437288     Source Type: Journal    
DOI: 10.1080/17437270701614766     Document Type: Article

References (87)

1. Adam, D. (2002) Citation analysis: The counting house. Nature, 415, pp. 726-729.
2. Aristotle (1915) Ethica Nicomachea. The works of Aristotle, Oxford University Press, Oxford
3. AUT (2003) The risk to research in higher education in England, Association of University Teachers, London
4. Ball, P. (2005) Index aims for fair ranking of scientists. Nature, 436, p. 900.
5. Blunkett, D. (2000) Influence or irrelevance: Can social science improve government? Speech made by David Blunkett, Secretary of State for Education and Employment, to a meeting convened by the Economic and Social Research Council on 2 February 2000. Research Intelligence, 71, pp. 12-21.
6. Botte, A. (2004) Achievement or performance: Observation of productivity of educational research by bibliometric tools. A state-of-the-art-report. paper presented at the European Conference on Educational Research University of Crete, University of Crete
7. Braun, T., Glänzel, W. and Grupp, H. (1995) The scientometric weight of 50 nations in 27 science areas, 1989-1993: Part I. All fields combined, mathematics, engineering, chemistry and physics. Scientometrics, 33:3, pp. 263-293.
8. British Academy (2006) Response to the DfES consultation on the reform of higher education research assessment and funding
9. British Academy (2007) Peer review: The challenges for the humanities and social sciences, British Academy, London
10. Buxton, M., Hanney, S. and Jones, T. (2004) Estimating the economic value to societies of the impact of health research: A critical review. Bulletin of the World Health Organisation, 82, pp. 733-739.
11. Capron, H. (1992) Economic quantitative methods for the evaluation of the impact if R&D programmes, EC, Brussels
12. Cerroni, A. (2003) Socio-cognitive perverse effects in peer review. Reflections and proposals. Journal of Science Communication, 2:3
13. Chubin, D. E. (1994) Grants peer review in theory and practice. Evaluation Review, 18:1, pp. 20-30.
14. Clarke, C. (1998) Resurrecting educational research to raise standards: statement from the new minister responsible for research. Research Intelligence, 66, pp. 8-9.
15. CNER-Comité National D'évaluation de la Recherche (2003) Évaluation de la recherche publique dans les établissements publics français, France, Paris
16. Council of the Royal Society (1995) Peer review-an assessment of recent developments, Royal Society, London
17. Dadds, M. and Kynch, C. (2003) The impact of the RAE 3b rating on educational research in teacher education departments. Research Intelligence
18. Dearing Report (1997) Sir Ron Dearing. National committee of inquiry into higher education
19. DfES (2006) Reform of higher education research assessment and funding. A consultation
20. DfES (2006) DfES consultation on the reform of higher education research assessment and funding: Summary of responses
21. Dogan, M. and Pahre, R. (1990) Creative marginality. innovation at the intersections of social sciences, Westview Press, Oxford
22. Dunne, J. (1993) Back to the rough ground: 'Phronesis' and 'Techne' in modern philosophy and in Aristotle, University of Notre Dame Press, London
23. ESRC (2006) DfES consultation on reform of higher education research assessment and funding. ESRC questionnaire
24. (1997) Evaluating science and scientists: An east-west dialogue on research evaluation in post-communist Europe, CEU Press, Budapesta
25. Frederiksen, L. F., Hansson, F. and Wenneberg, S. B. (2003) The Agora and the role of research evaluation. Evaluation, 9:2, pp. 149-172.
26. Gauthier, E. (1998) Bibliometric analysis of scientific and technological research: A user's guide to the methodology, Observatoire des Sciences et des Technologies, Statistics Canada, Canada
27. Geuna, A. and Martin, B. R. (2003) University research evaluation and funding: An international comparison. Minerva, 41:4, pp. 277-304.
28. Gibbons, M. and Gheorghiou, L. (1987) Evaluation of research: A selection of current practices, OCDE, Paris
29. Glänzel, W. (1996) A bibliometric approach to social sciences: national research performances in 6 selected social science areas, 1990-1992. Scientometrics, 35:3, pp. 291-307.
30. Gogolin, I., Smeyers, P., Dujo, A. Garcia Del and Rusch-Feja, D. (2003) European social science citation index: A chance for promoting European research? Roundtable. European Educational Research Journal, 2:4, pp. 547-593.
31. Goodhart, C. A. E. (1975) Money, information and uncertainty, Macmillan, London
32. Hackett, E. J. and Chubin, D. E. (2003) Peer review for the 21st century: Applications to education research, paper for a. National Research Council (SUA) workshop, Washington, DC
33. Hammersley, M. (1997) Educational research and teaching: A response to David Hargreaves' TTA lecture. British Educational Research Journal, 23, pp. 141-162.
34. Hansson, F. (2002) Best practice in research evaluation? How to evaluate and select new scientific knowledge by introducing the social dimension in the evaluation of research quality. paper for the European Evaluation Society conference Seville, Spain
35. Hargreaves, D. H. (1996) Teaching as a research-based profession: possibilities and prospects. (Teacher Training Agency Annual Lecture) Teacher Training Agency, Teacher Training Agency, London
36. HEFCE/AHRC (2006) Use of research metrics in the arts and humanities
37. Hillage, J., Pearson, R., Anderson, A. and Tamkin, P. (1998) Excellence in research on schools, DfEE, London
38. HM Treasury (2006) Chancellor of the exchequer's budget statement
39. HM Treasury (2006) Science and innovation investment framework 2004-2014: Next steps
40. House of Commons Science and Technology Committee (2004) Research assessment exercise: A re-assessment, TSO, Norwich
41. Humphrey, C., Moizer, P. and Owen, D. (1996) Questioning the value of the research selectivity process in British university accounting. Accounting, Auditing and Accountability Journal, 8:3, pp. 144-164.
42. Johnes, G. (1994) Research performance measurement: What can international comparisons teach us?. Comparative Education, 30:3, pp. 205-216.
43. Johnstone, D. B. (1998) The financing and management of higher education: A status report of worldwide reforms, World Bank, Washington, DC
44. Kostoff, R. N. (1997) The handbook of research impact assessment, Office of Naval Research, Arlington
45. Kostoff, R. N. (1998) The use and misuse of citation analysis in research evaluation - comments on theories of citation?. Scientometrics, 43:1, pp. 27-43.
46. Leisyte, L., de Boer, H. and Enders, J. (2006) England - the prototype of the evaluative state. Reforming university governance. Changing conditions for research in four European countries, pp. 17-52. Lemmens, Bonn
47. Lindsey, D. (1978) The corrected quality ratio: A composite index of scientific contribution to knowledge. Social Studies of Science, 8:3, pp. 349-354.
48. MacRoberts, M. H. and MacRoberts, B. R. (1989) Problems of citation analysis: A critical review. Journal of the American Society for Information Science, 40:5, pp. 342-349.
49. Mansfield, E. (1980) Basic research and productivity increase in manufacturing. American Economic Review, 70, pp. 863-873.
50. Marx, W., Wanitschek, M. and Schier, H. (1999) Scientometric on fullerenes and nanotubes. Condensed Matters News, 7:4, pp. 3-7.
51. McIntyre, M. E. (1999) Audit, education and Goodhart's law or taking rigidity seriously. TV programme. BBC
52. Moed, H. F., Burger, W. J. M., Frankfort, J. G. and van Raan, A. F. J. (1985) A comparative study of bibliometric past performance analysis and peer judgement. Scientometrics, 8:3, pp. 149-159.
53. Narin, F. (1994) Patent bibliometrics. Scientometrics, 30:1, pp. 147-155.
54. Oancea, A. (2004) The distribution of educational research expertise in the UK-finding from the analysis of RAE 2001 submissions (I-II). Research Intelligence, 87, p. 88.
55. Oancea, A. (2005) Criticisms of educational research: Key topics and levels of analysis. British Educational Research Journal, 31:2, pp. 157-183.
56. Oancea, A. (2006) Procrustes or Proteus? Towards a philosophical dimension of research assessment. paper to the British Educational Research Association annual conference Warwick
57. Oancea, A. and Furlong, J. (2007) Expressions of excellence and the assessment of applied and practice-based research. Research Papers in Education, 22:2, pp. 119-137.
58. OSI-DTI (2007) PSA target metrics for the UK research base, Evidence Ltd., Leeds
59. Patsopoulos, N. A., Analatos, A. A. and Ioannidis, J. P. A. (2005) Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association, 293, pp. 2362-2366.
60. Piric, A. and Reeve, N. (1997) Evaluation of public investment in R&D-towards a contingency analysis. OECD Conference on Policy Evaluation in Innovation and Technology Report Parispp. 49-64.
61. Puxty, A. G., Sikka, P. and Wilmott, H. C. (1994) Systems of surveillance and the silencing of UK academic accounting. British Accounting Review, 26:2, pp. 137-171.
62. Rammel, B. (2006) Speech to the HEPI conference 21 June on research funding and assessment beyond 2008
63. Ranson, S. (2003) Public accountability in the age of neo-liberal governance. Journal of Education Policy, 18:5, pp. 459-480.
64. RCUK (2006) DfES consultation on the reform of higher education research assessment and funding. Submission from Research Councils UK
65. RCUK (2006) Efficiency and effectiveness of peer review project consultation
66. RCUK (2006) Report of the research councils UK efficiency and effectiveness of peer review project
67. Rey-Rocha, J., Martin-Sempere, M. J., Martinez-Frias, J. and Lopez-Vera, F. (2001) Some misuses of journal impact factor in research evaluation. Cortex, 37:4, pp. 595-597.
68. Roberts, G. (2003) Review of research assessment
69. Royal Academy of Engineering (2006) Reform of higher education research assessment and funding. Response to the department for education and skills
70. Royal Society (2003) Supporting basic research in science and engineering: A call for a radical review of university research and funding in the UK, Royal Society, London
71. Royal Society (2006) Response to the DfES consultation on the reform of higher education research assessment and funding
72. Sastry, T. (2006) A dangerous economy: The wider implications of the proposed reforms to the UK research councils' peer review system
73. Sastry, T. and Bekhradnia, B. (2006) Using metrics to allocate research funds. A short evaluation of alternatives to the research assessment exercise, Higher Education Policy Institute, Oxford
74. Seglen, P. O. (1997) Why the impact factor of journals should not be used for evaluating research. British Medical Journal, 314, pp. 498-502.
75. Slowey, M. (1995) Reflections on change: Academics in leadership roles. Implementing change from within universities and colleges, Kogan Page, London
76. Stake, R. E. (2001) Evaluation of testing and criterial thinking in education. paper to the American Psychological Association meeting San Francisco, CA
77. St. Clair, R. (2006) Replacing the RAE: The story so far. Research Intelligence, 97, pp. 21-22.
78. St. Clair, R. (2007) BERA response to RCUK consultation on efficiency and effectiveness of peer reviewing. Research Intelligence, 98, p. 12.
79. Thackray, A. (1977) Reflections on the measurement of science. Newsletter on Science, Technology, & Human Values, 19, pp. 20-29.
80. Thompson, K. J. (2004) Interdisciplinarity and complexity: An evolving relationship. E:CO, 61, pp. 2-10.
81. Tooley, J. and Darby, D. (1998) Educational research: A critique. A survey of published educational research, Office for Standards in Education, London
82. UK Evaluation Forum (2006) Medical research: Assessing the benefits to society, Academy of Medical Sciences, Medical Research Council and Wellcome Trust, London
83. UK Funding Bodies (2005) RAE 2008. Guidance to panels
84. United States of America Congress (1937) The National Cancer Institute Act
85. Universities UK (2006) Universities UK's response to the DfES consultation on the reform of higher education research assessment and funding
86. van Raan, A. F. (1997) Presentation to the NATO. Evaluation of performance and trends in basic and applied research by advanced bibliometric methods. A science policy instrument for nations with an economy in transition Prague
87. Wellcome Trust (2006) Response to the consultation on the reform of higher education research assessment and funding