Comparison of a citation-based indicator and peer review for absolute and specific measures of research-group excellence

Scientometrics

Volumn 97, Issue 3, 2013, Pages 767-777

  Mryglod, O ^a   Kenna, R ^b   Holovatch, Yu ^a   Berche, B ^c  

^a INSTITUTE FOR CONDENSED MATTER PHYSICS   (Ukraine)

^b Institute for Clean Growth and Future Mobility   (United Kingdom)

^c UNIVERSITÉ DE LORRAINE   (France)

Author keywords

Citations; Peer review; Research assessment exercise (RAE); Research excellence framework (REF)

Indexed keywords

EID: 84888002502     PISSN: 01389130     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11192-013-1058-9     Document Type: Article

References (36)

1. Billaut, J. -C., Bouyssou, D., & Vincke, P. (2010). Should you believe in the Shanghai ranking? Scientometrics 84, 237-263.
2. Bornmann, L. (2012). The Hawthorne effect in journal peer review, Scientometrics 91, 857-862.
3. Bornmann, L., Wallon, G., & Ledin, A. (2008). Is the h index related to (standard) bibliometric measures and to the assessments by peers? An investigation of the h index by using molecular life sciences data, Research Evaluation 17, 149-156.
4. Butler, D. (2010). University rankings smarten up, Nature, 464, 16-17.
5. Derrick, G. E., Haynes, A., Chapman, S., & Hall, W. D. (2001). The association between four citation metrics and peer rankings of research influence of Australian researchers in six fields of public health, PLoS One, 6, e18521.
6. Donovan, C. (2007). Future pathways for science policy and research assessment: Metrics vs peer review, quality vs impact. Science and Public Policy 34, 538-542.
7. Egghe, L. (2006). Theory and practise of the g-index, Scientometrics, 69(1), 131-152.
8. Evidence. (2010). Evidence (a Thomson Reuters business) report. The future of the UK university research base, July 2010.
9. Evidence. (2011). Funding research excellence: Research group size, critical mass and performance. A University Alliance report, July 2011.
10. Evidence. (2012). Bibliometric evaluation and international benchmarking of the UK's physics research, Summary report prepared for the Institute of Physics by Evidence, Thomson Reuters.
11. Florian, R. V. (2007). Irreproducibility of the results of the Shanghai academic ranking of world universities Scientometrics 72, 25-32.
12. Garfield, E. (1955). Citation indexes for science: A new dimension in documentation through association of Ideas. Science, 122(3159), 108-111.
13. Garfield, E. (1973). Citation frequency as a measure of research activity and performance in essays of an information scientist, Current Contents, 1, 406-408.
14. Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. PNAS, 102(46), 16569-16572.
15. Ioannidis, J. P. A. et al. (2007). International ranking systems for universities and institutions: A critical appraisal BMC Medicine 5, 30.
16. Kenna, R., & Berche, B. (2010). Critical mass and the dependency of research quality on group size. Scientometrics, 86(2), 527-540.
17. Kenna, R., & Berche, B. (2011). Critical masses for academic research groups and consequences for higher education research policy and management. Higher Education Management and Policy, 23(3), 1-21.
18. Macilwain, C. (2010). Wild goose chase. Nature, 463, 291.
19. Mryglod O., Kenna R., Holovatch Y., Berche B. (2012). Absolute and specific measures of research group excellence. Scientometrics doi: 10. 1007/s11192-012-0874-7.
20. Moed, H. F. (2005). Citation analysis in research evaluation. Dordrecht: Springer.
21. Nature. (2010). Editorial, Metrics Special. 465, p. 845. Retrieved April 2012, from http://www. nature. com/metrics.
22. Norris, M., & Oppenheim, C. (2003). Citation counts and the Research Assessment Exercise. V archaeology and the 2001 RAE, Journal of Documentation, 59(6), 709-730.
23. Oppenheim C., & Summers M. A. C. (2008). Citation counts and the Research Assessment Exercise, part VI. Unit of assessment 67 (music), Information Research, 13(2).
24. RAE. (2008). The panel criteria and working methods. Panel E. (2006). Retrieved October 19, 2012, from http://www. rae. ac. uk/pubs/2006/01/docs/eall. pdf.
25. Schubert, A., & Braun, T. (1996). Cross-field normalization of scientometric indicators. Scientometrics 36(3), 311-324.
26. Stauffer D. (2012). A biased review of sociophysics. Journal of Statistical Physics doi: 10. 1007/s10955-012-0604-9.
27. The official web-page of the RAE. (2008). Retrieved October 18, 2012, from http://www. rae. ac. uk/.
28. The official web-page of the Higher Education Funding Council for England. Funding for universities and colleges in 2009-10 (2009). Electronic Publication 01/2009 in the ADMIN-HEFCE Archives. Retrieved October 19, 2012.
29. The official web-page of Academic Ranking of World Universities (ARWU). (2012). Retrieved October 19, 2012, from http://www. shanghairanking. com.
30. The official web-page of Evidence Thomson Reuters. (2012). Retrieved October 18, 2012, from http://www. evidence. co. uk.
31. The official web-page of the REF. (2012). Retrieved October 19, 2012, from http://www. ref. ac. uk/.
32. van Raan, A. F. J. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods, Scientometrics, 62(1), 133-143.
33. Vinkler, P. (2001). An attempt for defining some basic categories of scientometrics and classifying the indicators of evaluative scientometrics. Scientometrics, 50(3), 539-544.
34. Vinkler, P. (2003). Relations of relative scientometric indicators. Scientometrics, 58(3), 687-694.
35. Warner, J. (2003). Citation Analysis and Research Assessment in the United Kingdom, American Society for Information Science and Technology, 30(1), 26-27.
36. Williams R., de Rassenfosse G., Jensen P., & Marginson S. (2012). U21 Ranking of National Higher Education Systems, Report of the project sponsored by Universitas 21, University of Melbourne.