Beyond university rankings? Generating new indicators on universities by linking data in open platforms

Journal of the Association for Information Science and Technology

Volumn 68, Issue 2, 2017, Pages 508-529

  Daraio, Cinzia ^a   Bonaccorsi, Andrea ^b  

^a SAPIENZA UNIVERSITY OF ROME   (Italy)

^b UNIVERSITY OF PISA   (Italy)

Author keywords

[No Author keywords available]

Indexed keywords

BUDGET CONTROL; DECISION MAKING; EDUCATION;

BUDGET CONSTRAINT; DECISION MAKERS; DESIGN INDICATORS; EUROPEAN UNIVERSITIES; HETEROGENEOUS SOURCES; INTELLIGENT INTEGRATION; TRADITIONAL APPROACHES; USER REQUIREMENTS;

SOCIETIES AND INSTITUTIONS;

DATA BASE; INVESTMENT; UNIVERSITY;

EID: 84962251130     PISSN: 23301635     EISSN: 23301643     Source Type: Journal    
DOI: 10.1002/asi.23679     Document Type: Article

References (94)

1. Aguillo, I.F., Bar-Ilan, J., Levene, M., & Ortega, J.L. (2010). Comparing university rankings. Scientometrics, 85(1), 243–256.
2. Amsler, S., & Bolsmann, C. (2012). University rankings as social exclusion. British Journal of Sociology of Education, 33(2), 283–301.
3. Bonaccorsi, A. (Ed.). (2014). Knowledge, diversity and performance in European higher education. Cheltenham, UK: Edward Elgar.
4. Bonaccorsi, A., & Daraio, C. (2004). Econometric approaches to the analysis of productivity of R&D systems. Production functions and production frontiers. In H.F. Moed, W. Glanzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research (pp. 51–74). Dordrecht: Kluwer Academic Publishers.
5. Bonaccorsi, A., & Daraio, C. (Eds.). (2007). Universities and strategic knowledge creation. Specialization and performance in Europe. Cheltenham, UK: Edward Elgar Publisher.
6. Bonaccorsi, A., & Daraio, C. (2008). The differentiation of the strategic profile of higher education institutions. New positioning indicators based on microdata. Scientometrics, 74(1), 15–37.
7. Bornmann, L. (2014). On the function of university rankings. Journal of the American Society for Information Science and Technology, 65(2), 428–429.
8. Bornmann, L., Lutz, R., & Daniel, H.D. (2013a). Multilevel-statistical reformulation of citation-based university rankings: The Leiden ranking 2011/2012. Journal of the American Society for Information Science and Technology, 64(8), 1649–1658.
9. Bornmann, L., Moya Anegón, F., & Mutz, R. (2013b). Do universities or research institutions with a specific subject profile have an advantage or a disadvantage in institutional rankings? Journal of the American Society for Information Science and Technology, 64(11), 2310–2316.
10. Boschma, R.A. (2005). Proximity and innovation: A critical assessment. Regional Studies, 39(1), 61–74.
11. Boschma, R.A., & Frenken, K. (2010). The spatial evolution of innovation networks. A proximity perspective. In R.A. Boschma, R. Martin (Eds.), The handbook of evolutionary economic geography (pp. 120–135). Cheltenham, UK: Edward Elgar Publisher.
12. Bottazzi, L., & Peri, G. (2003). Innovation and spillovers in regions: Evidence from European patent data. European Economic Review, 47(4), 687–710.
13. Brooks, R.L. (2005). Measuring university quality. Review of Higher Education, 29(1), 1–22.
14. Buela-Casal, G., Gutiérrez-Martínez, O., Bermúdez-Sánchez, M., & Vadillo-Muñoz, O (2007). Comparative study of international academic rankings of universities. Scientometrics, 71(3), 349–365.
15. Calvanese, D., De Giacomo, G., Lembo, D., Lenzerini, M., Rosati, R., & Ruzzi, M. (2010). Using OWL in data integration. In Semantic web information management. Berlin Heidelberg: Springer-Verlag.
16. Campbell, D., Lefebvre, C., Picard-Aitken, M., Côté, G., Ventimiglia, A., Roberge, G., … Archambault E. (2013). Country and regional scientific production profiles. Directorate-General for Research and Innovation. Luxembourg: Publications Office of the European Union.
17. Cantwell, B., & Taylor, B.J. (2013). Global status, intra-institutional stratification and organizational segmentation: A time-dynamic tobit analysis of ARWU position among U.S. universities. Minerva 51(2), 195–223.
18. Charon, A., & Wauters, J.P. (2008). University ranking: A new tool for the evaluation of higher education in Europe. Nephrology Dialysis Transplantation, 23(1), 62–64.
19. Chen, K., & Liao, P. (2012). A comparative study on world university rankings: A bibliometric survey. Scientometrics, 92, 89–103.
20. Cremonini, L., Westerheijden, D.F., & Enders, J. (2008). Disseminating the right information to the right audience: Cultural determinants in the use (and misuse) of rankings. Higher Education, 55(3), 373–385.
21. Cronin, B. (2013). Thinking about data. Journal of the American Society for Information Science and Technology, 64(3), 435–436.
22. Daraio, C., Bonaccorsi, A., Geuna, A., Lepori, B., Bach, L., Bogetoft, P., … & Eeckaut, P.V. (2011). The European university landscape: A micro characterization based on evidence from the Aquameth project. Research Policy, 40(1), 148–164.
23. Daraio, C., Bonaccorsi, A., & Simar, L. (2015a). Rankings and university performance: A conditional multidimensional approach. European Journal of Operational Research, 244, 918–930.
24. Daraio, C., Bonaccorsi, A., & Simar, L. (2015b). Efficiency and economies of scale and specialization in European universities. A directional distance approach. Journal of Informetrics, 9, 430–448.
25. Daraio, C., Lenzerini, M., Leporelli, C., Moed, H.F., Naggar, P., Bonaccorsi, A., … Bartolucci, A. (2015c). Sapientia the ontology of multi-dimensional research assessment. In A.A. Salah, Y. Tonta, A.A. Akdag Salah, C. Sugimoto, & U. Al (Eds.), Proceedings of ISSI 2015 Istanbul: 15th International Society of Scientometrics and Informetrics Conference, Istanbul, Turkey, 29 June to 3 July, 2015 (pp. 965–977). Bogaziçi University Printhouse.
26. Deem, R., & Brehony, K.J. (2005). Management as ideology: The case of ''new management'' in higher education. Oxford Review of Education, 31(2), 217–235.
27. Dehon, C., Jacobs, D., & Vermandele, C. (2009). Ranking universities. Brussels: Editions de l'Université de Bruxelles.
28. Desrosières, A. (2008). Gouverner par les nombres. Paris: Presses de l'école des mines.
29. Docampo, D. (2012). Universities from Spain in the 2011 edition of the Shanghai ranking. Submitted to Revista de Educación. Working paper retrieved from http://www.gts.tsc.uvigo.es/ddocampo/ONLINE/docamporevistaeducacion.pdf.
30. Edwards, P.N. (2010). A vast machine: Computer models, climate data, and the politics of global warming. Cambridge, MA: Mit Press.
31. Edwards, P., Mayernik, M.S., Batcheller, A., Bowker, G., & Borgman, C. (2011). Science friction: Data, metadata, and collaboration. Social Studies of Science, 41, 667–690. 0306312711413314.
32. Egghe, L. (2005). Power laws in the information production process: Lotkaian informetrics. Oxford: Elsevier.
33. Ergas, H. (1987). Does technology policy matter? Technology and global industry: Companies and nations in the world economy (pp. 191–245). Washington, DC: National Academy Press.
34. Erkkilä, T. (Ed). (2013). Global university rankings. Challenges for European higher education. Basingstoke, UK: Palgrave Macmillan.
35. Estermann, T., & Nokkala, T. (2009). University autonomy in Europe I. Exploratory study. Brussels: European University Association.
36. Estermann, T., Nokkala, T., & Steinel, M. (2011). University autonomy in Europe II. The Scorecard. Brussels: European University Association.
37. Espeland, W.N., & Sander, M. (2007). Ranking and reactivity: How public measures recreate social worlds. American Journal of Sociology, 13(1), 1–40.
38. European Commission. (2010). Assessing Europe's university based research. Expert Group on Assessment of University-Based Research (AUBR). Retrieved from http://ec.europa.eu/research/science-society/document_library/pdf_06/assessing-europe-university-based-research_en.pdf.
39. European Parliamentary Research Service. (2014). Measuring scientific performance for improved policy making, ISBN 978-92-823-5532-9. Retrieved from http://www.europarl.europa.eu/RegData/etudes/etudes/join/2014/527383/IPOL-JOIN_ET(2014)527383(SUM01)_EN.pdf.
40. European University Association. (2012). Public funding observatory. Retrieved from http://www.eua.be/eua-work-and-policy-area/governance-autonomy-andfunding/public-funding-observatory.aspx.
41. Ferlie, E., Musselin, C., & Andresani, G. (2008). The steering of higher education systems: A public management perspective. Higher Education, 56(6), 325–348.
42. Florian, R.V. (2007). Irreproducibility of the results of the Shanghai academic ranking of world universities. Scientometrics, 72(1), 25–32.
43. Fumasoli, T., & Huisman, J. (2013). Strategic agency and system diversity: Conceptualizing institutional positioning in higher education. Minerva, 51(2), 155–169.
44. Goldstein, H., & Spigehalter, D.J. (1996). League tables and their limitations. Statistical issues in comparisons of institutional performance. Journal of the Royal Statistical Society. Series A (Statistics in Society), 159(3), 385–443.
45. Griffith, A., & Rask, K. (2007). The influence of the U.S. News and World Report collegiate rankings on the matriculation decisions of high-ability students: 1995-2004. Economics of Education Review, 26(2), 244–255.
46. Harvey, E.L. (2008). Rankings of higher education institutions: A critical review. Quality in Higher Education, 14 (3), 187–207.
47. Hazelkorn, E. (2007). The impact of league tables and ranking systems on higher education decision making. Higher Education Management and Policy, 19(2), 89–112.
48. Hazelkorn, E. (2009). Rankings and the battle for world-class excellence: Institutional strategies and policy choices. Higher Education Management and Policy, 21(1), 1–22.
49. Hazelkorn, E. (2011). Rankings and the reshaping of higher education. The battle for world-class excellence. New York: Palgrave Macmillan.
50. Institute for Higher Education Policy. (2007). College and university ranking systems: Global perspective and American challenges. Washington, DC: Institute for Higher Education Policy.
51. Institute for Higher Education Policy. (2009). Impact of college rankings on institutional decision making: Four country case studies. Washington, DC: Institute for Higher Education Policy.
52. Ioannidis, J.P., Patsopoulos, N.A., Kavvoura, F.K., Tatsioni, A., Evangelou, E., Kouri, I., … & Liberopoulos, G. (2007). International ranking systems for universities and institutions: A critical appraisal. BMC Medicine, 5(1), 30.
53. Katz, J.S. (2000). Scale-independent indicators and research assessment. Science and Public Policy, 27(1), 23–36.
54. Katz, J.S. (2005). Scale-independent bibliometric indicators. Measurement, 3(1), 24–28.
55. Kehm, B.M., & Bjørn, S. (Eds.). (2009). University rankings, diversity, and the new landscape of higher education. Rotterdam, Netherlands: Sense Publishers.
56. Kroth, A., & Daniel, H.D. (2008). International university rankings-A critical review of the methodology. Zeitschrift fur Erziehungswissenschaft, 11(4), 542–558.
57. Larivière, V., & Gingras, Y. (2011). Averages of ratios vs. ratios of averages: An empirical analysis of four levels of aggregation. Journal of Informetrics, 5(3), 392–399.
58. Lane, J. (2010). Let's make science metrics more scientific. Nature, 464, 488–489.
59. Lenzerini, M. (2011). Ontology-based data management. In Proceedings of the 20th ACM International Conference on Information Knowledge Management (pp. 5–6). ACM Information and Knowledge Management, CIKM.
60. Li, X., & Johnson, J.D. (2002). Evaluate IT investment opportunities using real options theory. Information Resources Management Journal, 15(3), 32–47.
61. Lukman, R., Krajinc, D., & Glavič, P. (2010). University ranking using research, educational and environmental indicators. Journal of Cleaner Production, 18(7), 619–628.
62. Lundvall, B.Å. (Ed.). (2010). National systems of innovation: Toward a theory of innovation and interactive learning (Vol. 2). London: Anthem Press.
63. Maassen, P., & Potman, H. (1990). Strategic decision-making in higher education. Higher Education, 20(4), 393–410.
64. Marope, P.T.M., Wells, P.J., & Hazelkorn, E. (Eds.). (2013). Rankings and accountability in higher education—uses and misuses. Paris: UNESCO Publishing.
65. Molinari, J.F., & Molinari, A. (2008). A new methodology for ranking scientific institutions. Scientometrics, 75(1), 163–174.
66. National Research Council. (2014). Science of science and innovation policy. Principal investigators' Conference Summary. Washington, DC: The National Academies Press.
67. Nelson, R. (Ed.). (1993). National innovation systems. Oxford: Oxford University Press.
68. OECD. (2008). Handbook on constructing composite indicators. Methodology and user guide. Paris: OECD.
69. OECD. (2011a). Regions and innovation policy. OECD Reviews of Regional Innovation. Paris: OECD.
70. OECD. (2011b). Issue brief: Public sector research funding. OECD Innovation policy platform. Retrieved from http://www.oecd.org/innovation/policyplatform/48136600.pdf.
71. Poggi, A., Lembo, D., Calvanese, D., De Giacomo, G., Lenzerini, M., & Rosati, R. (2008). Linking data to ontologies. Journal of Data Semantics, 10, 133–173.
72. Porter, T. (1995). Trust in numbers: The pursuit of objectivity in science and public life. Princeton: Princeton University Press.
73. Provan, D., & Abercromby, K. (2000). University league tables and rankings. Research in Higher Education, 45(5), 443–461.
74. Rauhvargers, A. (2011). Global university rankings and their impact. Leadership for WorldClass Universities. European University Association. Retrieved from http://www.eua.be/pubs/global_university_rankings_and_their_impact.pdf.
75. Rauhvargers, A. (2013). Global university rankings and their impact: Report II. European University Association. Retrieved from http://www.eua.be/Libraries/Publications_homepage_list/EUA_Global_University_Rankings_and_Their_Impact_-_Report_II.sflb.ashx.
76. Rawlings, C.M., & Bourgeois, M.D. (2004). The complexity of institutional niches: Credentials and organizational differentiation in a field of American higher education. Poetics, 32(6), 411–446.
77. Rhoades, G., & Sporn, B. (2002). Quality assurance in Europe and the U.S.: Professional and political framing of higher education policy. Higher Education, 43(3), 355–390.
78. Safón, V. (2013). What do global university rankings really measure? The search for the X factor and the X entity. Scientometrics, 97(2), 223–244.
79. Saisana, M., D'Hombres, B., & Saltelli, A. (2011). Rickety numbers: Volatility of university rankings and policy implications. Research Policy, 40(1), 165–177.
80. Salmi, J., & Saroyan, A. (2007). League tables as policy instruments: Uses and misuses. Higher Education Management and Policy, 19(2), 31.
81. Salmi, J. (2009). The challenge of establishing world-class universities. Washington, DC: World Bank Publications.
82. Shin, J.C., Toutkoushian, R.K., & Teichler, U. (Eds.). (2011). University rankings: Theoretical basis, methodology and impacts on global higher education. Dordrecht (the Netherlands): Springer.
83. Stake, J.E. (2006). The interplay between law school rankings, reputations, and resource allocations: Ways rankings mislead. Indiana Law Journal, 81, 229–270.
84. Stewart, J. (1995). Models of priority-setting for public sector research. Research Policy, 24(1), 115–126.
85. Taylor, P., & Braddock, R. (2007). International university ranking systems and the idea of university excellence. Journal of Higher Education Policy and Management, 29(3), 245–260.
86. Tierney, W.G. (2009). Globalization, international rankings and the American model: A reassessment. Higher Education Forum, 6, 1–18.
87. UOE. (2006). UOE data collection on education systems. Manual. Concepts, definitions, classifications. Montreal, Paris, Luxembourg: UNESCO, OECD, Eurostat.
88. Usher, A., & Savino, M. (2006). A world of difference: A global survey of university league tables. Toronto, ON: Educational Policy Institute.
89. Valderrama-Zurián, J.C., Aguilar-Moya, R., Melero-Fuentes, D., & Aleixandre-Benavent, R. (2015). A systematic analysis of duplicate records in Scopus. Journal of Informetrics, 9(3), 570–576.
90. Van Raan, A.F.J. (2006). Statistical properties of bibliometric indicators: Research group indicator distributions and correlations. Journal of the American Society for Information Science and Technology, 57(3), 408–430.
91. Van Raan, A.F.J. (2004). Measuring science: Capita selecta of current main issues. In H.F. Moed, W. Glänzel, & U. Schmoch (Eds.), Handbook of quantitative science and technology research (pp. 19–50). Dordrecht: Kluwer Academic.
92. Varga, A., & Horvath, M. (2014). Institutional and regional factors behind university patenting in Europe: An exploratory spatial analysis using EUMIDA data. In A. Bonaccorsi (Ed.), Knowledge, diversity and performance in European higher education (pp. 167–204). Cheltenham, UK: Edward Elgar.
93. Vertesy, D., Annoni, P., & Nardo, M. (2013). University systems: Beyond league tables. Engines of growth or ivory towers? Higher Education Evaluation and Development, 7(1), 21–43.
94. Williams, R., & Van Dyke, N. (2006). Rating major disciplines in Australian universities: Perceptions and reality. Melbourne: Institute of Applied Economic and Social Research.