The heterogeneous landscape of bibliometric indicators: Evaluating models for allocating resources at Swedish universities

Research Evaluation

Volumn 25, Issue 3, 2016, Pages 292-305

  Hammarfelt, Björn ^a,b   Nelhans, Gustaf ^a   Eklund, Pieta ^a   Åström, Fredrik ^c  

^a UNIVERSITY OF BORÅS   (Sweden)

^b LEIDEN UNIVERSITY   (Netherlands)

^c LUND UNIVERSITY   (Sweden)

Author keywords

Bibliometrics; Performance indicators; Research evaluation; Sweden; University governance

Indexed keywords

EID: 85042404635     PISSN: 09582029     EISSN: 14715449     Source Type: Journal    
DOI: 10.1093/reseval/rvv040     Document Type: Article

References (35)

1. Hammarfelt, B. and de Rijcke, S. (2015) 'Accountability in Context: Effects of Research Evaluation Systems on Publication Practices, Disciplinary Norms and Individual Working Routines in the Faculty of Arts at Uppsala University'. Research Evaluation, 24: 63-77.
2. Hicks, D. (2012) 'Performance-based University Research Funding Systems', Research Policy, 41/2: 251-61.
3. Hicks, D., Wouters, P., Waltman, L., de Rijcke, S. and Rafols, I. (2015). 'The Leiden Manifesto for Research Metrics', Nature, 520, 429-31.
4. Karlsson, S., Fogelberg, K., Kettis, Å., Lindgren, S., Sandoff, M. and Geschwind, L. (2014). 'Not Just Another Evaluation: a Comparative Study of Four Educational Quality Projects at Swedish Universities', Tertiary Education and Management, 20/3: 239-51.
5. Lorenz, C. (2012) 'If You're so Smart, Why are You Under Surveillance? Universities, Neoliberalism, and New Public Management', Critical Inquiry, 38/3: 599-629.
6. Nederhof, A. J. (2006) 'Bibliometric Monitoring of Research Performance in the Social Sciences and the Humanities: A Review', Scientometrics, 66/1: 81-100.
7. Nelhans, G. (2013) Citeringens praktiker. Diss. Göteborg: Göteborgs universitet. accessed 18 Feb 2015.
8. Aagaard, K. (2015). 'How Incentives Trickles Down: Local use of a National Bibliometric Indicator system', Science and Public Policy, 1-13. doi:10.1093/scipol/scu087
9. Ossenblok, T., Engels, T. and Sivertsen G. (2012) 'The representation of the social sciences and humanities in the Web of Sciencea comparison of publication patterns and incentive structures in Flanders and Norway (2005-9).' Research Evaluation, 21/4: 280-290.
10. Power, M. (1997) The Audit Society: Rituals of Verification. Oxford: Oxford University Press.
11. Alberts, B. (2013) 'Impact Factor Distortions', Science, 340/6134: 787.
12. Ahlgren, P., Colliander, C. and Persson, O. (2012) 'Field Normalized Citation rates, Field Normalized Journal Impact and Norwegian Weights for Allocation of University Research Funds', Scientometrics, 92/3: 767-80.
13. Åström, F. and Hansson, J. (2013) 'How Implementation of Bibliometric Practice Affects the Role of Academic Libraries', Journal of Librarianship and information Science, 45/4: 316-22.
14. Sandström, U. and Sandström E. (2009) 'The Field Factor: Towards a Metric for Academic Institutions', Research Evaluation, 18/3: 243-50.
15. Schneider, J. W. (2009) 'An Outline of the Bibliometric Indicator Used for Performance-Based Funding of Research Institutions in Norway', European Political Science, 8/3: 364-78.
16. Seglen, P. O. (1997) 'Why the Impact Factor of Journals Should not be Used for Evaluating Research', BMJ, 314/7079: 498-502.
17. Burrows, R. (2012) 'Living with the h-index? Metric Assemblages in the ContemporaryAcademy', The Sociological Review, 60/2: 355-72.
18. Butler, L. (2003) 'Explaining Australia's Increased Share of ISI Publications-the Effects of a Funding Formula Based on Publication Counts', Research Policy, 32/1: 143-55.
19. Sivertsen, G. (2008) 'Experiences with a Bibliometric Model for Performance Based Funding of Research Institutions', In Goriaz, J. and Schiebel, E. (eds) Book of Abstracts, 10th International Science and Technology Indicators Conference, pp. 126-128. 17-20 September 2008. Austria: University of Vienna.
20. Butler, L. (2010) 'Impacts of Performance-based Research Funding Systems: a Review of the Concerns and the Evidence', In: Chapter 4 in OECD, Performance-based Funding for Public Research in Tertiary Education Institutions: Workshop Proceedings. Paris: OECD Publishing.
21. Strathern, M. (2000) 'The Tyranny of Transparency', British Educational Research Journal, 26/3: 309-21.
22. Van Noorden, R. (2010) 'Metrics: A Profusion of Measures', Nature, 465/ 7300: 864-6.
23. Dahler-Larsen, P. (2012a) The Evaluation Society. Stanford: Stanford University Press.
24. Dahler-Larsen, P. (2012b) 'Evaluation as a Situational or a Universal good? Why Evaluability Assessment for Evaluation systems is a Good Idea, What it Might Look Like in Practice, and Why it is not Fashionable', Scandinavian Journal of Public Administration, 16/3: 29-46.
25. De Rijcke, S., Wouters, P., Rushforth, A., Franssen, T. and Hammarfelt, B. (2015) 'Evaluation Practices and Effects of Indicator Use - A Literature Review'. Research Evaluation, doi: 10.1093/reseval/rvv038, in press.
26. DiMaggio, P. J. and Powell, W. W. (1983) 'The Iron Cage Revisited: Institutional Isomorphism and Collective Rationality in Organizational Fields', American Sociological Review, 48/2: 147-60.
27. Furner, J. (2014) 'The Ethics of Evaluative Bibliometrics'. In Cronin, B. and Sugimoto, C. (eds), Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact. Cambridge, MA: MIT Press.
28. García-Pérez, M. (2011) 'Strange Attractors in the Web of Science Database', Journal of Informetrics, 5/1: 214-18.
29. Gingras, Y. (2014) 'Criteria for Evaluating Indicators'. In Cronin, B and Sugimoto, C. (eds), Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Impact. Cambridge, MA: MIT Press.
30. Verleysen, F., Ghesquière, P. and Engels, T. (2014) 'The Objectives, Design and Selection Process of the Flemish Academic Bibliographic Database for the Social Sciences and Humanities (VABB-SHW)'. In: Blockmans, W., Engwall, L. and Weaire, D. (eds.) Bibliometrics: Use and Abuse in the Review of Research Performance. London: Portland Press.
31. Weingart, P. (2005) 'Impact of Bibliometrics upon the Science System: Inadvertent Consequences?', Scientometrics, 62/1: 117-31.
32. Whitley, R. (2007) 'Changing Governance of the Public Sciences' In: Whitley, R. and Gläser, J. (eds) Sociology of the Sciences Yearbook, pp. 3-27. Dordrecht: Springer.
33. Wildgaard, L., Schneider, J. W. and Larsen, B. (2014) 'A Review of the Characteristics of 108 Author-level Bibliometric Indicators' Scientometrics, 101/1: 125-58.
34. Woelert, P. and Yates, L. (2014) 'Too Little and Too Much Trust: Performance Measurement in Australian Higher Education', Critical Studies in Higher Education, 56/2: 175-189.
35. Wouters, P. (2014) 'The Citation: From Culture to Infrastructure'. In: Cronin, B. and Sugimoto, C. R. (eds.) Beyond Bibliometrics: Harnessing Multidimensional Indicators of Scholarly Performance, Vol. 22, pp. 47-66. Cambridge, MA: MIT Press.