Multilevel-statistical reformulation of citation-based university rankings: The Leiden ranking 2011/2012

Journal of the American Society for Information Science and Technology

Volumn 64, Issue 8, 2013, Pages 1649-1658

  Bornmann, Lutz ^a   Mutz, Rüdiger ^b   Daniel, Hans Dieter ^b,c  

^a MAX PLANCK INSTITUTE   (Germany)

^b ETH ZURICH   (Switzerland)

^c UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

social sciences

Indexed keywords

CONFIDENCE INTERVAL; INTERNATIONAL HIGHER EDUCATION; QUANTITATIVE COMPARISON; RANK DIFFERENCES; REGRESSION MODEL; SCIENTIFIC PERFORMANCE; STRONG GROWTH; TOTAL VARIATION;

GOLD; REGRESSION ANALYSIS; SOCIAL SCIENCES; SOCIETIES AND INSTITUTIONS;

EDUCATION;

EID: 84880300904     PISSN: 15322882     EISSN: 15322890     Source Type: Journal    
DOI: 10.1002/asi.22857     Document Type: Article

References (31)

1. Abramo, G., D'Angelo, C.A., &, Costa, F.D., (2010). Testing the trade-off between productivity and quality in research activities. Journal of the American Society for Information Science and Technology, 61 (1), 132-140.
2. Adler, R., Ewing, J., Taylor, P., &, Hall, P.G., (2009). A report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS). Statistical Science, 24 (1), 1-28. doi: 10.1214/09-sts285
3. Austrian Science Fund. (2007). Rethinking the impact of basic research on society and the economy. Vienna, Austria: Austrian Science Fund.
4. Ballou, D., Sanders, W., &, Wright, P., (2004). Controlling for student background in value-added assessment of teachers. Journal of Educational and Behavioral Statistics, 29 (1), 37-65.
5. Bauer, D.J., (2009). A note on comparing the estimates of models for cluster-correlated or longitudinal data with binary or ordinal outcomes. Psychometrika, 74 (1), 97-105. doi: 10.1007/S11336-008-9080-1
6. Bornmann, L., (in press). On the function of university rankings. Journal of the American Society of Information Science and Technology.
7. Bornmann, L., & de Moya, Anegõn, F., (2011). Some interesting insights from aggregated data published in the World Report SIR 2010. Journal of Informetrics, 5 (3), 486-488. doi: 10.1016/j.joi.2011.03.005
8. Bornmann, L., Mutz, R., Marx, W., Schier, H., &, Daniel, H.-D., (2011). A multilevel modelling approach to investigating the predictive validity of editorial decisions: do the editors of a high-profile journal select manuscripts that are highly cited after publication? Journal of the Royal Statistical Society Series A (Statistics in Society), 174 (4), 857-879. doi: 10.1111/j.1467-985X.2011.00689.x
9. Bornmann, L., de Moya Anegõn, F., &, Leydesdorff, L., (2012). The new Excellence Indicator in the World Report of the SCImago Institutions Rankings 2011. Journal of Informetrics, 6 (2), 333-335. doi: 10.1016/j.joi.2011.11.006
10. Bornmann, L., de Moya Anegõn, F., &, Mutz, R., (in press). Do universities or research institutions with a specific subject profile have an advantage or a disadvantage in institutional rankings? A latent class analysis with data from the SCImago ranking. Journal of the American Society of Information Science and Technology.
11. Buela-Casal, G., Gutiérrez-Martínez, O., Bermúdez-Sánchez, M., &, Vadillo-Muñoz, O., (2007). Comparative study of international academic rankings of universities. Scientometrics, 71 (3), 349-365.
12. Chen, K.-H., &, Liao, P.-Y., (2012). A comparative study on world university rankings: A bibliometric survey. Scientometrics, 92 (1), 89-103. doi: 10.1007/s11192-012-0724-7
13. Daniel, H.-D., &, Fisch, R., (1990). Research performance evaluation in the German university sector. Scientometrics, 19 (5-6), 349-361.
14. Goldstein, H., &, Spiegelhalter, D.J., (1996). League tables and their limitations: Statistical issues in comparisons of institutional performance. Journal of the Royal Statistical Society Series A (Statistics in Society), 159, 385-409.
15. Goldstein, H., Browne, W., &, Rasbash, J., (2002). Partitioning variation in generalized linear multilevel models. Understanding Statistics, 1, 223-232.
16. Hazelkorn, E., (2011). Rankings and the reshaping of higher education. The battle for world-class excellence. New York: Palgrave Macmillan.
17. Hemlin, S., (1996). Research on research evaluations. Social Epistemology, 10 (2), 209-250.
18. Hox, J.J., (2010). Multilevel analysis: techniques and applications (2nd ed.). New York, NY: Routledge.
19. Jovanovic, M., Jeremic, V., Savic, G., Bulajic, M., &, Martic, M., (2012). How does the normalization of data affect the ARWU ranking? Scientometrics, 93 (2), 319-327. doi: 10.1007/s11192-012-0674-0
20. Leydesdorff, L., Bornmann, L., Mutz, R., &, Opthof, T., (2011). Turning the tables in citation analysis one more time: Principles for comparing sets of documents. Journal of the American Society for Information Science and Technology, 62 (7), 1370-1381.
21. Miranda, L.C.M., &, Lima, C.A.S., (2010). On trends and rhythms in scientific and technological knowledge evolution: A quantitative analysis. International Journal of Technology Intelligence and Planning, 6 (1), 76-109.
22. Mutz, R., &, Daniel, H.D., (2007). Development of a ranking procedure by mixed Rasch model and multilevel analysis-Psychology as an example. Diagnostica, 53 (1), 3-16. doi: 10.1026/0012-1924.53.1.3
23. Mutz, R., &, Daniel, H.-D., (2012). Skewed citation distributions and bias factors: Solutions to two core problems with the journal impact factor. Journal of Informetrics, 6 (2), 169-176. doi: 10.1016/j.joi.2011.12.006
24. Raudenbush, S.W., (2004). What are value-added models estimating and what does this imply for statistical practice? Journal of Educational and Behavioral Statistics, 29 (1), 121-129.
25. SCImago Reseach, Group. (2012). SIR World Report 2012. Granada, Spain: University of Granada.
26. Shin, J.C., &, Toutkoushian, R.K., (2011). The past, present, and future of university rankings. In, J.C. Shin, R.K. Toutkoushian, &, U. Teichler, (Eds.), University rankings: Theoretical basis, methodology and impacts on global higher education (vol. 3, pp. 1-16). Dordrecht, Netherlands: Springer.
27. Shin, J.C., Toutkoushian, R.K., &, Teichler, U., (Eds.). (2011). University rankings: Theoretical basis, methodology and impacts on global higher education. Dordrecht, Netherlands: Springer.
28. van Raan, A.F.J., (2005). Fatal attraction: conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62 (1), 133-143.
29. van, Vught, F.A., &, Ziegele, F., (Eds.). (2012). Multidimensional ranking: The design and development of U-Multirank. Dordrecht, Netherlands: Springer.
30. Waltman, L., Calero-Medina, C., Kosten, J., Noyons, E.C.M., Tijssen, R.J.W., van, Eck, N.J., Wouters, P., et al. (2012). The Leiden Ranking 2011/2012: Data collection, indicators, and interpretation. Journal of the American Society for Information Science and Technology 63 (12), 2419-2432. doi: 10.1002/asi.22708
31. Williams, R., de Rassenfosse, G., Jensen, P., &, Marginson, S., (2012). U21 ranking of national higher education systems. Melbourne, Australia: University of Melbourne.