Ranking and mapping of universities and research-focused institutions worldwide based on highly-cited papers: A visualisation of results from multi-level models

Online Information Review

Volumn 38, Issue 1, 2014, Pages 43-58

  Bornmann, Lutz ^a   Stefaner, Moritz ^b   Anegón, Felix De Moya ^c   Mutz, Rüdiger ^d  

^a MAX PLANCK INSTITUTE   (Germany)

^b Consejo Superior de Investigaciones Científicas ^*

^c Institute of Public Goods and Policies   (Spain)

^d ETH ZURICH   (Switzerland)

Author keywords

Geography of science; Google Maps; Highly cited papers; Scientific excellence; Spatial scientometrics; University ranking

Indexed keywords

GEOGRAPHY OF SCIENCE; GOOGLE MAPS; HIGHLY-CITED PAPERS; SCIENTIFIC EXCELLENCE; SCIENTOMETRICS; UNIVERSITY RANKING;

APPLICATIONS; WORLD WIDE WEB;

SOCIETIES AND INSTITUTIONS;

EID: 84891131006     PISSN: 14684527     EISSN: None     Source Type: Journal    
DOI: 10.1108/OIR-12-2012-0214     Document Type: Article

References (49)

1. Aksnes, D.W. (2003), "A macro study of self-citation", Scientometrics, Vol. 56 No. 2, pp. 235-246. (Pubitemid 39583162)
2. Bornmann, L. (2012), "Measuring the societal impact of research", EMBO Reports, Vol. 13 No. 8, pp. 673-676.
3. Bornmann, L. (2013), "What is societal impact of research and how can it be assessed? A literature survey", Journal of the American Society of Information Science and Technology, Vol. 64 No. 2, pp. 217-233.
4. Bornmann, L. and Daniel, H.-D. (2008), "What do citation counts measure? A review of studies on citing behavior", Journal of Documentation, Vol. 64 No. 1, pp. 45-80.
5. Bornmann, L. and Leydesdorff, L. (2011), "Which cities produce more excellent papers than can be expected? A new mapping approach - using Google Maps - based on statistical significance testing", Journal of the American Society of Information Science and Technology, Vol. 62 No. 10, pp. 1954-1962.
6. Bornmann, L. and Waltman, L. (2011), "The detection of 'hot regions' in the geography of science: a visualization approach by using density maps", Journal of Informetrics, Vol. 5 No. 4, pp. 547-553.
7. Bornmann, L., De Moya Anegón, F. and Leydesdorff, L. (2012), "The new Excellence Indicator in the World Report of the SCImago Institutions Rankings 2011", Journal of Informetrics, Vol. 6 No. 2, pp. 333-335.
8. Bornmann, L., De Moya Anegón, F. and Mutz, R. (n.d.), "Do universities or research institutions with a specific subject profile have an advantage or a disadvantage in institutional rankings? A latent class analysis with data from the SCImago ranking", Journal of the American Society for Information Science and Technology (in press-b).
9. Bornmann, L., Leydesdorff, L. and Mutz, R. (2013a), "The use of percentiles and percentile rank classes in the analysis of bibliometric data: opportunities and limits", Journal of Informetrics, Vol. 7 No. 1, pp. 158-165.
10. Bornmann, L., Mutz, R. and Daniel, H.-D. (2013b), "A multilevel-statistical reformulation of citation-based university rankings: the Leiden Ranking 2011/2012", Journal of the American Society for Information Science and Technology, Vol. 64 No. 8, pp. 1649-1658.
11. Bornmann, L., Leydesdorff, L., Walch-Solimena, C. and Ettl, C. (2011a), "Mapping excellence in the geography of science: an approach based on Scopus data", Journal of Informetrics, Vol. 5 No. 4, pp. 537-546.
12. Bornmann, L., Mutz, R., Neuhaus, C. and Daniel, H.-D. (2008), "Use of citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results", Ethics in Science and Environmental Politics, Vol. 8 No. 1, pp. 93-102. (Pubitemid 351836496)
13. Bornmann, L., Schier, H., Marx, W. and Daniel, H.-D. (2011c), "Is interactive open access publishing able to identify high-impact submissions? A study on the predictive validity of Atmospheric Chemistry and Physics by using percentile rank classes", Journal of the American Society for Information Science and Technology, Vol. 62 No. 1, pp. 61-71.
14. Bornmann, L., Mutz, R., Marx, W., Schier, H. and Daniel, H.-D. (2011b), "A multilevel modelling approach to investigating the predictive validity of editorial decisions: do the editors of a high-profile journal select manuscripts that are highly cited after publication?", Journal of the Royal Statistical Society - Series A (Statistics in Society), Vol. 174 No. 4, pp. 857-879.
15. Bornmann, L., Bowman, B.F., Bauer, J., Marx, W., Schier, H. and Palzenberger, M. (n.d.), "Standards for using bibliometrics in the evaluation of research institutes", in Cronin, B. and Sugimoto, C. (Eds), Next Generation Metrics, MIT Press, Cambridge, MA (in press-a).
16. Buela-Casal, G., Gutiérrez-Martínez, O., Bermúdez-Sánchez, M. and Vadillo-Muñoz, O. (2007), "Comparative study of international academic rankings of universities", Scientometrics, Vol. 71 No. 3, pp. 349-365.
17. Calero Valdez, A., Schaar, A., Ziefle, M., Holzinger, A., Jeschke, S. and Brecher, C. (2012), "Using mixed node publication network graphs for analyzing success in interdisciplinary teams", in Huang, R., Ghorbani, A., Pasi, G., Yamaguchi, T., Yen, N. and Jin, B. (Eds), Active Media Technology, Springer, Berlin, pp. 606-617.
18. Frenken, K., Hardeman, S. and Hoekman, J. (2009), "Spatial scientometrics: towards a cumulative research program", Journal of Informetrics, Vol. 3 No. 3, pp. 222-232.
19. Glänzel, W., Debackere, K., Thijs, B. and Schubert, A. (2006), "A concise review on the role of author self-citations in information science, bibliometrics and science policy", Scientometrics, Vol. 67 No. 2, pp. 263-277. (Pubitemid 44091079)
20. Goldstein, H. and Healy, M.J.R. (1994), "The graphical representation of a collection of means", Journal of the Royal Statistical Society Series a-Statistics in Society, Vol. 158 No. 1, pp. 175-177.
21. Guerrero-Bote, V.P. and De Moya-Anegon, F. (2012), "A further step forward in measuring journals' scientific prestige: the SJR2 indicator", available at: http://arxiv.org/abs/1201.4639 (accessed 3 July 2012).
22. Hazelkorn, E. (2011), Rankings and the Reshaping of Higher Education. The Battle for World-class Excellence, Palgrave Macmillan, New York, NY.
23. Hennemann, S. (2012), "Evaluating the performance of geographical locations within scientific networks using an aggregation-randomization-re- sampling approach (ARR)", Journal of the American Society for Information Science and Technology, Vol. 63 No. 12, pp. 2393-2404.
24. Hox, J.J. (2010), Multilevel Analysis: Techniques and Applications, Routledge, New York, NY.
25. Kroth, A. and Daniel, H.D. (2008), "International university rankings - a critical review of the methodology", Zeitschrift für Erziehungswissenschaft, Vol. 11 No. 4, pp. 542-558.
26. Levine, M. and Ensom, M.H.H. (2001), "Post hoc power analysis: an idea whose time has passed?", Pharmacotherapy, Vol. 21 No. 4, pp. 405-409. (Pubitemid 32275267)
27. Leydesdorff, L. and Bornmann, L. (2012a), "Mapping (USPTO) patent data using overlays to Google maps", Journal of the American Society of Information Science and Technology, Vol. 63 No. 7, pp. 1442-1458.
28. Leydesdorff, L. and Bornmann, L. (2012b), "Testing differences statistically with the Leiden ranking", Scientometrics, Vol. 92 No. 3, pp. 781-783.
29. Leydesdorff, L. and Persson, O. (2010), "Mapping the geography of science: distribution patterns and networks of relations among cities and institutes", Journal of the American Society for Information Science and Technology, Vol. 61 No. 8, pp. 1622-1634.
30. Leydesdorff, L. and Rafols, I. (2009), "A global map of science based on the ISI subject categories", Journal of the American Society for Information Science and Technology, Vol. 60 No. 2, pp. 348-362.
31. Leydesdorff, L., Bornmann, L., Mutz, R. and Opthof, T. (2011), "Turning the tables in citation analysis one more time: principles for comparing sets of documents", Journal of the American Society for Information Science and Technology, Vol. 62 No. 7, pp. 1370-1381.
32. Mazloumian, A., Helbing, D., Lozano, S., Light, R.P. and Börner, K. (2013), "Global multi-level analysis of the 'Scientific Food Web'", Scientific Reports, Vol. 3, p. 1167.
33. Moineddin, R., Matheson, F.I. and Glazier, R.H. (2007), "A simulation study of sample size for multilevel logistic regression models", BMC Medical Research Methodology, Vol. 7 No. 1.
34. Rousseau, R. (2012), "Basic properties of both percentile rank scores and the I3 indicator", Journal of the American Society for Information Science and Technology, Vol. 63 No. 2, pp. 416-420.
35. SAS Institute Inc. (2008), SAS/STATw 9.2 User's Guide, SAS Institute, Cary, NC.
36. Scimago Research Group (2011), SIR World Report 2011, University of Granada, Granada.
37. Scimago Research Group (2012), SIR World Report 2012, University of Granada, Granada.
38. Sheskin, D. (2007), Handbook of Parametric and Nonparametric Statistical Procedures, Chapman & Hall/CRC, Boca Raton, FL.
39. Shin, J.C., Toutkoushian, R.K. and Teichler, U. (2011), University Rankings: Theoretical Basis, Methodology and Impacts on Global Higher Education, Springer, Dordrecht.
40. Snijders, T.A.B. and Bosker, R.J. (2004), Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling, Sage Publications, London.
41. Snyder, H. and Bonzi, S. (1998), "Patterns of self-citation across disciplines (1980-1989)", Journal of Information Science, Vol. 24 No. 6, pp. 431-435. (Pubitemid 128589603)
42. Spiegelhalter, D., Pearson, M. and Short, I. (2011), "Visualizing uncertainty about the future", Science, Vol. 333 No. 6048, pp. 1393-1400.
43. Sun, S.Y., Pan, W. and Wang, L.L. (2011), "Rethinking observed power concept, practice, and implications", Methodology - European Journal of Research Methods for the Behavioral and Social Sciences, Vol. 7 No. 3, pp. 81-87.
44. Tijssen, R. and Van Leeuwen, T. (2006), "Centres of research excellence and science indicators. Can 'excellence' be captured in numbers?", in Glänzel, W. (Ed.), Ninth International Conference on Science and Technology Indicators, Katholieke Universiteit Leuven, Leuven, pp. 146-147.
45. Tijssen, R., Visser, M. and Van Leeuwen, T. (2002), "Benchmarking international scientific excellence: are highly cited research papers an appropriate frame of reference?", Scientometrics, Vol. 54 No. 3, pp. 381-397.
46. Vinkler, P. (2010), The Evaluation of Research by Scientometric Indicators, Chandos Publishing, Oxford.
47. Waltman, L. and Schreiber, M. (2013), "On the calculation of percentile-based bibliometric indicators", Journal of the American Society for Information Science and Technology, Vol. 64 No. 2, pp. 372-379.
48. Waltman, L., Calero-Medina, C., Kosten, J., Noyons, E.C.M., Tijssen, R.J.W., Van Eck, N.J., Van Leeuwen, T.N., Van Raan, A.F.J., Visser, M.S. and Wouters, P. (2012), "The Leiden Ranking 2011/2012: data collection, indicators, and interpretation", Journal of the American Society for Information Science and Technology, Vol. 63 No. 12, pp. 2419-2432.
49. Yuan, K.H. and Maxwell, S. (2005), "On the post hoc power in testing mean differences", Journal of Educational and Behavioral Statistics, Vol. 30 No. 2, pp. 141-167. (Pubitemid 40882648)