Evaluating university research: Same performance indicator, different rankings

Journal of Informetrics

Volumn 9, Issue 3, 2015, Pages 514-525

  Abramo, Giovanni ^a   D'Angelo, Ciriaco Andrea ^b  

^a CNR   (Italy)

^b UNIVERSITY OF ROME TOR VERGATA   (Italy)

Author keywords

Bibliometrics; Italy; Productivity; Research evaluation

Indexed keywords

BIBLIOMETRICS; ITALY; PERFORMANCE INDICATORS; PERFORMANCE MEASURE; RESEARCH EVALUATION; RESEARCH PERFORMANCE; SCIENTIFIC FIELDS; UNIVERSITY RESEARCH;

PRODUCTIVITY;

EID: 84930634738     PISSN: 17511577     EISSN: 18755879     Source Type: Journal    
DOI: 10.1016/j.joi.2015.04.002     Document Type: Article

References (35)

1. Abramo G., D'Angelo C.A. How do you define and measure research productivity?. Scientometrics 2014, 101(2):1129-1144.
2. Abramo G., D'Angelo C.A., Rosati F. The importance of accounting for the number of co-authors and their order when assessing research performance at the individual level in the life sciences. Journal of Informetrics 2013, 7(1):198-208.
3. Abramo G., D'Angelo C.A., Rosati F. Measuring institutional research productivity for the life sciences: The importance of accounting for the order of authors in the byline. Scientometrics 2013, 97(3):779-795.
4. Abramo G., Cicero T., D'Angelo C.A. The dispersion of research performance within and between universities as a potential indicator of the competitive intensity in higher education systems. Journal of Informetrics 2012, 6(2):155-168.
5. Abramo G., Cicero T., D'Angelo C.A. Revisiting the scaling of citations for research assessment. Journal of Informetrics 2012, 6(4):470-479.
6. Abramo G., Cicero T., D'Angelo C.A. The dangers of performance-based research funding in non-competitive higher education systems. Scientometrics 2011, 87(3):641-654.
7. Abramo G., Cicero T., D'Angelo C.A. The impact of non-productive and top scientists on overall university research performance. Journal of Informetrics 2013, 7(1):166-175.
8. Abramo G., D'Angelo C.A., Solazzi M. National research assessment exercises: A measure of the distortion of performance rankings when labor input is treated as uniform. Scientometrics 2010, 84(3):605-619.
9. Abramo G., D'Angelo C.A., Di Costa F. Assessment of sectoral aggregation distortion in research productivity measurements. Research Evaluation 2008, 17(2):111-121.
10. Ash M.G. Mythos Humboldt: Vergangenheit und Zukunft der deutschen Universitäten 1999, Böhlau, Vienna.
11. Auranen O., Nieminen M. University research funding and publication performance-An international comparison. Research Policy 2010, 39(6):822-834.
12. Billaut J.-C., Bouyssou D., Vincke P. Should you believe in the Shanghai ranking?: An MCDM view. Scientometrics 2010, 84(1):237-263.
13. Butler L. Assessing university research: A plea for a balanced approach. Science and Public Policy 2007, 34(8):565-574.
14. Commission of the European Communities Developing foresight for the development of higher education/research relations in the perspective of the European research area (ERA) 2004, European Commission, Directorate General for Research, Brussels, Retrieved from ftp.cordis.europa.eu/pub/foresight/docs/hleg_final25102002_en.pdf last (accessed 17/02/2015). Strata-Etan Expert Group (E. Bourgeois, Rapporteur) (Ed.).
15. CWTS CWTS Leiden university rankings 2014, Retrieved from http://www.leidenranking.com/ranking/2014 (last accessed 17/02/2015).
16. D'Angelo C.A., Giuffrida C., Abramo G. A heuristic approach to author name disambiguation in large-scale bibliometric databases. Journal of the American Society for Information Science and Technology 2011, 62(2):257-269.
17. Dehon C., McCathie A., Verardi V. Uncovering excellence in academic rankings: A closer look at the Shanghai ranking. Scientometrics 2010, 83(2):515-524.
18. Garfield E. Is citation analysis a legitimate evaluation tool?. Scientometrics 1979, 1(4):359-375.
19. Hicks D. Evolving regimes of multi-university research evaluation. Higher Education 2009, 57(4):393-404.
20. Hicks D. Performance-based university research funding systems. Research Policy 2012, 41(2):251-261.
21. Himanen L., Auranen O., Puuska H.M., Nieminen M. Influence of research funding and science policy on university research performance: A comparison of five countries. Science and Public Policy 2009, 36(6):419-430.
22. Liu N.C., Cheng Y. Academic ranking of world universities: Methodologies and problems. Higher Education in Europe 2005, 30(2):127-136.
23. Moed H.F. Citation Analysis in Research Evaluation 2005, Springer, The Netherlands, ISBN: 978-1-4020-3713-9.
24. Moed H.F., Burger W.J.M., Frankfort J.G., Van Raan A.F.J. The application of bibliometric indicators: Important field- and time-dependent factors to be considered. Scientometrics 1985, 8(3-4):177-203.
25. OECD Performance-based funding for public research in tertiary education institutions. OECD-Norway workshop, June 2010 2010, OECD, Paris.
26. QS-Quacquarelli Symonds World university rankings 2014 2014, Retrieved from http://www.topuniversities.com/qs-world-university-rankings (last accessed 17/02/2015).
27. Sauder M., Espeland W.N. The discipline of ranking: Tight coupling and organizational change. American Sociological Review 2009, 74(1):63-82.
28. SCIMAGO Scimago institution ranking 2014, Retrieved from http://www.scimagoir.com/research.php (last accessed 17/02/2015).
29. SJTU-Shanghai Jiao Tong University Academic ranking of world universities 2014 2014, Retrieved from: http://www.shanghairanking.com/ARWU2014.html (last accessed 17/02/2015).
30. Smart W. The impact of performance-based research fund on the research productivity of New Zealand Universities. Social Policy Journal of New Zealand 2009, 34:136-151.
31. THE-Times Higher Education World university rankings 2014-2015 2014, Retrieved from http://www.timeshighereducation.co.uk/world-university-rankings/2014-15/world-ranking (last accessed 17/02/2015).
32. Thompson B. GRE percentile ranks cannot be added or averaged: A position paper exploring the scaling characteristics of percentile ranks, and the ethical and legal culpabilities created by adding percentile ranks in making high-stakes admission decisions. Paper presented at the annual meeting of the mid-south educational research association 1993.
33. Vanecek J. The effect of performance-based research funding on output of R&D results in the Czech Republic. Scientometrics 2014, 98(1):657-681.
34. van Raan A.F.J. Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods?. Scientometrics 2005, 62(1):133-143.
35. Waltman L., Van Eck N.J., Van Leeuwen T.N., Visser M.S., Van Raan A.F.J. Towards a new crown indicator: Some theoretical considerations. Journal of Informetrics 2011, 5(1):37-47.