The determinants of research performance in European universities: a large scale multilevel analysis

Scientometrics

Volumn 112, Issue 3, 2017, Pages 1147-1178

  Bonaccorsi, Andrea ^a   Secondi, Luca ^b  

^a Universita degli Studi di Pisa and IRVAPP FBK   (Italy)

^b UNIVERSITY OF TUSCIA   (Italy)

Author keywords

Citation analysis; European universities; Multilevel modelling; Research performance

Indexed keywords

EID: 85021161170     PISSN: 01389130     EISSN: 15882861     Source Type: Journal    
DOI: 10.1007/s11192-017-2442-7     Document Type: Article

References (139)

1. Agresti, A. (2002). Categorical data analysis. Hoboken: Wiley.
2. Aksnes, D. W., Rørstad, K., Piro, F. N., & Sivertsen, G. (2013). Are mobile researchers more productive and cited than non-mobile researchers? A large scale study of Norwegian scientists. Research Evaluation, 22, 215–223.
3. Allison, P. D., & Long, J. S. (1990). Departmental effects on scientific productivity. American Sociological Review, 55(4), 469–478.
4. Allison, P. D., & Stewart, J. A. (1974). Productivity differences among scientists: Evidence for accumulative advantage. American Sociological Review, 39, 596–606.
5. Alshare, K. A., Wenger, J., & Miller, D. (2007). The role of teaching, scholarly activities, and service on tenure, promotion, and merit pay decisions: Deans’ perspectives. Academy of Educational Leadership Journal, 11(1), 53–68.
6. Azoulay, P., Graff Zivin, J. F., & Monso, G. (2011). Incentives and creativity. Evidence from the academic life sciences. Rand Journal of Economics, 42(3), 527–554.
7. Azoulay, P., Graff Zivin, J. F., & Wang, J. (2010). Superstar extinction. Quarterly Journal of Economics, 125(2), 549–589.
8. Babu, A. R., & Singh, Y. P. (1998). Determinants of research productivity. Scientometrics, 43(3), 309–329.
9. Baird, L. L. (1986). What characterizes a productive research department? Research in Higher Education, 25(3), 211–225.
10. Bak, H. J., & Kim, D. H. (2015). Too much emphasis on research? An empirical examination of the relationship between research and teaching in multitasking environments. Research in Higher Education, 56, 84–860.
11. Baltagi, B. (2001). Econometric analysis of panel data. New York: Wiley.
12. Bazeley, P. (1998). Peer review and panel decisions in the assessment of Australian Research Council Project Grant applicants: What counts in a highly competitive context? Higher Education, 35(4), 435–452.
13. Beaver, D. B., & Rosen, R. (1979). Scientific co-authorship, research productivity and visibility in the French scientific élite, 1799-1830. Scientometrics, 1(2), 133–149.
14. Bechtel, W. (2006). Discovering cell mechanisms. The creation of modern cell biology. Cambridge: Cambridge University Press.
15. Becker, W. E., & Kennedy, P. E. (2006). The influence of teaching on research in economics. Southern Economic Journal, 72(3), 747–759.
16. Bland, C. J., Center, B. A., Finstad, D. A., Risbey, K. R., & Satples, J. (2006). The impact of appointment type on the productivity and commitment of full-time faculty in research and doctoral instititutions. The Journal of Higher Education, 77(1), 89–123.
17. Bland, C. J., & Ruffin, M. T. (1992). Characteristics of a productive research environment: Literature review. Academic Medicine, 67(6), 385–397.
18. Bloch, C., Graversen, E. K., & Pedersen, H. S. (2014). Competitive research grants and their impact on career performance. Minerva, 52, 77–96.
19. Bok, D. (2013). Higher education in America. Princeton: Princeton University Press.
20. Bolli, T., & Schläpfer, J. (2015). Job mobility, peer effects, and research productivity in economics. Scientometrics, 104, 629–650.
21. Bolli, T., & Somogyi, F. (2011). Do competitively acquired funds induce universities to increase productivity? Research Policy, 40, 136–147.
22. Bonaccorsi, A. (2009). Division of academic labor is limited by the size of the market. Differentiation and performance in postgraduate education. In M. McKelvey & M. Holmen (Eds.), Learning to compete in European universities. Cheltenham: Edward Elgar.
23. Bonaccorsi, A. (2010). New forms of complementarity in science. Minerva, 48(4), 355–387.
24. Bonaccorsi, A. (2016). Addressing the disenchantment. Universities and regional development in peripheral regions. Journal of Economic Policy Reform, 2, 1–28.
25. Bonaccorsi, A., & Daraio, C. (2005). Exploring size and agglomeration effects on public research productivity. Scientometrics, 63(1), 87–120.
26. Borjas, G. J., & Doran, K. B. (2012). The collapse of the Soviet Union and the productivity of American mathematicians. The Quarterly Journal of Economics, 127(3), 1143–1203.
27. Borjas, G. J., & Doran, K. B. (2014). Which peers matter? The relative impacts of collaborators, colleagues, and competitors. NBER Working Paper no. 20026.
28. Bornmann, L., & Daniel, H. D. (2005). Selection of research fellowship recipients by committee peer review. Reliability, fairness and predictive validity of Board of Trustees decisions. Scientometrics, 63(2), 297–320.
29. Bornmann, L., & Daniel, H. D. (2006). Selecting scientific excellence through committee peer review—A citation analysis of publications previously published to approval or rejection of post-doctoral research fellowship applicants. Scientometrics, 68(3), 427–440.
30. Bornmann, L., Leydesdorff, L., & Van den Besselaar, P. (2010). A meta-evaluation of scientific research proposals: Different ways of comparing rejected to awarded applications. Journal of Informetrics, 4, 211–220.
31. Bornmann, L., Mutz, R., & Daniel, H. D. (2008). Latent Markov modeling applied to grant peer review. Journal of Informetrics, 2, 217–228.
32. Bosquet C., & Combes P.P. (2013). Do large departments make academics more productive? Agglomeration and peer effects in research. SERC Discussion Papers 013, Spatial Economics Research Centre, LSE.
33. Bowden, R. G., & Gonzalez, L. (2012). Faculty appointments and scholarly activity: A changing of the guard? International Journal of Higher Education, 1(2), 166–183.
34. Brandt, T., & Schubert, T. (2014). Is the university model an organizational necessity? Scale and agglomeration effects in science. In A. Bonaccorsi (Ed.), Knowledge, diversity and performance in European higher education (pp. 233–266). Cheltenham: Edward Elgar.
35. Brinkman, P. T., & Leslie, L. L. (1986). Economies of scale in higher education: Sixty years of research. The Review of Higher Education, 10(1), 1–28.
36. Broder, I. E. (1993). Review of NSF economic proposals: Gender and institutional patterns. American Economic Review, 83(4), 964–970.
37. Buchmueller, T. C., Dominitz, J., & Hansen, L. (1999). Graduate training and the early career productivity of Ph.D. economists. Economics of Education Review, 18(1), 65–77.
38. Burke, L. A., & Ron, B. (2010). The research-teaching gap in management. Academy of Management Learning & Education, 9(1), 132–143.
39. Cañibano, C., Otamendi, J., & Andújar, I. (2009). An assessment of selection processes among candidates for public research grants: the case of the Ramón y Cajal Programme in Spain. Research Evaluation, 18(2), 153–161.
40. Carayol, N., & Matt, M. (2006). Individual and collective determinants of academic scientists’ productivity. Information Economics and Policy, 18(1), 55–72.
41. Cherchye, L., & Vanden Abeele, P. (2005). On research efficiency. A micro-analysis of Dutch university research in Economics and Business Management. Research Policy, 34, 495–516.
42. Cicchetti, D. V. (1991). The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation. Behavioral and Brain Sciences, 14(1), 119–135.
43. Clark, M. J., & Centra, J. A. (1985). Influence on the career accomplishments of Ph.D.s. Research in Higher Education, 23(3), 256–268.
44. Cohn, E., & Cooper, S. T. (2004). Multi-product cost functions for universities: economies of scale and scope. In G. Johnes & J. Johnes (Eds.), International handbook on the economics of education (pp. 579–612). Cheltenham: Edward Elgar.
45. Cohn, E., Rhine, S. L. W., & Santos, M. C. (1989). Institutions of higher education as multi-product firms: Economies of scale and scope. The Review of Economics and Statistics, 71(2), 284–290.
46. Cole, S. (1970). Professional standing and the reception of scientific discoveries. American Journal of Sociology, 76, 286–306.
47. Cole, S. (1992). Making science Between nature and society. Cambridge: Harvard University Press.
48. Cole, S., & Cole, J. R. (1967). Scientific output and recognition: A study in the operation of the reward system in science. American Sociological Review, 32, 377–390.
49. Cole, S., & Cole, J. R. (1968). Visibility and the structural bases of awareness in science. American Sociological Review, 33, 397–413.
50. Cole, J. R., & Cole, S. (1973). Social stratification in science. Chicago: The University of Chicago Press.
51. Cox, D. R. (1970). The analysis of binary data. London: Methuen.
52. Crane, D. (1965). Scientists at major and minor universities: A study in productivity and recognition. American Sociological Review, 30, 699–714.
53. Daraio, C., & Bonaccorsi, A. (2016). Beyond university rankings? Generating new indicators on universities by linking data in open platforms, Journal of theAssociation for Information Science and Technology., 68(2), 508–529.
54. Daraio, C., Bonaccorsi, A., & Simar, L. (2015). Efficiency and economies of scale and specialization in European universities: A directional distance approach. Journal of Informetrics, 9, 430–448.
55. De Jong, S. P. L., Van Arensbergen, P., Daemen, F., Van der Meulen, B., & Van den Besselaar, P. (2011). Evaluation of research in context: an approach and two cases. Research Evaluation, 20(2), 61–72.
56. Dubois P., Rochet J.C., & Schlenker J.M. (2013). Productivity and mobility in academic research: Evidence from mathematicians. Toulouse School of Economics, Working Paper, March 2013.
57. Dundar, H., & Lewis, D. R. (1995). Departmental productivity in American universities: economies of scale and scope. Economics of Education Review, 14(2), 119–144.
58. Fairweather, J. S. (2002). The mythologies of faculty productivity: Implications for institutional policy and decision making. Journal of Higher Education, 73(1), 26–48.
59. Faierweather, J. S. (2005). Beyond the rhetoric: Trends in the relative value of teaching and research in faculty salaries. Journal of Higher Education, 76, 401–422.
60. Feldman, K. A. (1987). Research productivity and scholarly accomplishment of college teachers as related to their instructional effectiveness. Research in Higher Education, 26, 227–298.
61. Fender, B. F., Taylor, S. W., & Burke, K. G. (2011). Substitutes or complements: Teaching and scholarship for the active scholar. Journal of Applied Financial Research, 1, 50–61.
62. Gart, J. J., & Zweifel, J. R. (1967). On the bias of various estimators of the logit and its variance with application to quantal bioassay. Biometrika, 54(1–2), 181–187.
63. Geiger, R. L., & Sá, C. M. (2008). Tapping the riches of science. Cambridge: Harvard University Press.
64. Glass, J. C., Mckillop, D. G., & Hyndman, N. S. (1995). The achievement of scale efficiency in UK universities: a multiple-input multiple-output analysis. Education Economics, 3, 249–263.
65. Goddard, J., & Vallance, P. (2013). The university and the city. London: Routledge.
66. Goedegebuure, L. (2012). Mergers and more: The changing tertiary education landscape in the 21st century. University of Oslo HEIK Working Paper Series. 2012/1.
67. Goldstein, H. (2011). Multilevel statistical models. Hoboken: Wiley.
68. Gonzalez-Brambila, C., & Veloso, F. M. (2007). The determinants of research output and impact. A study of Mexican researchers. Research Policy, 36, 1035–1051.
69. Green, R. G. (2008). Tenure and promotion decisions: The relative importance of teaching, scholarship, and service. Journal of Social Work Education, 44(2), 117–128.
70. Grilli, L., & Rampichini, C. (2006). Model building issues in multilevel linear models with endogenous covariates. In Proceedings of IASC-INTERFACE-IFCS Workshop.
71. Hagstrom, W. O. (1971). Inputs, outputs, and the prestige of university science departments. Sociology of Education, 44, 375–397.
72. Hagstrom, W. O. (1974). Competition in science. American Sociological Review, 39, 1–18.
73. Hanchane, S., & Mostafa, T. (2010). Endogeneity problems in multilevel estimation of Education Production Functions: An analysis using PISA data. The Centre for Learning and Life Chances in Knowledge Economies and Societies. http://www.llakes.org.
74. Hargens, L. L., & Hagstrom, W. O. (1967). Sponsored and contest mobility of American academic scientists. Sociology of Education, 40, 24–38.
75. Hargens, L. L., & Hagstrom, W. O. (1982). Scientific consensus and academic status attainment patterns. Sociology of Education, 55, 183–196.
76. Harris, G., & Kaine, G. (1994). The determinants of research performance: A study of Australian university economists. Higher Education, 27(2), 191–201.
77. Hashimoto, K., & Cohn, E. (1997). Economies of scale and scope in Japanese private universities. Education Economics, 5, 107–115.
78. Hattie, J., & Marsh, H. W. (1996). The relationship between research and teaching: A meta-analysis. Review of Educational Research, 66, 507–542.
79. Hodgson, C. M. (1995). Evaluation of cardiovascular grant-in-aid applications by peer review: Influence of internal and external reviewers and committees. Canadian Journal of Cardiology, 11, 864–868.
80. Hoisl, K. (2007). Tracing mobile inventors: The causality between inventor mobility and inventor productivity. Research Policy, 36(5), 619–636.
81. Hoisl, K. (2009). Does mobility increase the productivity of inventors? The Journal of Technology Transfer, 34(2), 212–225.
82. Hornbostel, S., Böhmer, S., Klingsporn, B., Neufeld, J., & von Ins, M. (2009). Funding of young scientist and scientific excellence. Scientometrics, 79(1), 171–190.
83. Horta, H., & Lacy, T. A. (2011). How does size matter for science? Exploring the effects of research unit size on academics’ scientific productivity and information exchange behaviors, Science and Public Policy, 38(6), 449–460.
84. Horta, H., & Santos, J. M. (2016). The impact of publishing during Ph.D. studies on career research publication, visibility, and collaborations. Research in Higher Education, 57, 28–50.
85. Hox, J. J., Moerbeek, M., & van de Schoot, R. (2010). Multilevel analysis: Techniques and applications. London: Routledge.
86. Izadi, H., Johnes, G., Oskrochi, R., & Crouchley, R. (2002). Stochastic frontier estimation of a CES cost function: The case of higher education in Britain. Economics of Education Review, 21, 63–71.
87. Jacob, B. A., & Lefgren, L. (2011). The impact of research grant funding on scientific productivity. Journal of Public Economics, 95, 1168–1177.
88. Jayasinghe, U. W., Marsh, H. W., & Bond, N. (2001). Peer review in the funding of research in higher education: The Australian experience. Educational Evaluation and Policy Analysis, 23(4), 343–364.
89. Johnes, J. (2004), Efficiency measurement, in International Handbook on the Economics of Education, edited by Johnes G. and Johnes J., Edward Elgar Publishing, pp. 613–742.
90. Johnes, J. (2006). Data envelopment analysis and its application to the measurement of efficiency in higher education. Economics of Education Review, 25(3), 273–288.
91. Katz, J. S., & Hicks, D. (1997). How much is a collaboration worth? A calibrated bibliometric model. Scientometrics, 40(3), 541–554.
92. Kay, L. E. (1993). The molecular vision of life. Caltech, the Rockfeller Foundation, and the rise of the new biology. Oxford: Oxford University Press.
93. Kenna, R., & Berche, B. (2012). Critical masses for academic research groups and consequences for higher education research policy and management. Higher Education Management and Policy, 23(3), 1–21.
94. Kenney, M., & Mowery, D. C. (Eds.). (2014). Public universities and regional growth. Insights from the University of California. Stanford University Press: Stanford.
95. Koshal, R. K., & Koshal, M. (1999). Economies of scale and scope in higher education: A case of comprehensive universities. Economics of Education Review, 18, 269–277.
96. Kreft, I. G., Kreft, I., & de Leeuw, J. (1998). Introducing multilevel modeling. Thousand Oaks: Sage.
97. Laband, D. N., & Lentz, B. F. (2003). New estimates of economies of scale and scope in higher education. Southern Economic Journal, 70, 172–183.
98. Lee, S., & Bozeman, B. (2005). The impact of research collaboration on scientific productivity. Social Studies of Science, 35(5), 673–702.
99. Lepori, B., Seeber, M., & Bonaccorsi, A. (2015). Competition for talent. Country and organizational-level effects in the internationalization of European higher education institutions. Research Policy, 44(3), 789–802.
100. Liefner, I. (2003). Funding, resource allocation, and performance in higher education systems. Higher Education, 46, 469–489.
101. Lock, M., Young, A., & Cambrosio, A. (2000). Living and working with the new medical technologies. Intersections of inquiry. Cambridge: Cambridge University Press.
102. Long, J. S. (1978). Productivity and position in the scientific career. American Sociological Review, 43, 899–908.
103. Long, J. S., Allison, P. D., & McGinnis, R. (1979). Entrance into the academic career. American Sociological Review, 44(5), 816–830.
104. Longlong, H., Fengliang, L., & Weifang, M. (2009). Multi-product total cost functions for higher education: The case of Chinese research universities. Economics of Education Review, 28, 505–511.
105. Lopez-Ilescas, C., de Moya-Anegon, F., & Moed, H. F. (2011). A ranking of universities should account for differences in their disciplinary specialization. Scientometrics, 88(2), 563–574.
106. McCaughey, R.A. (1992). Why research and teachers can coexist. Chronicle of Higher Education, 4 August, p. A36.
107. Moed, H. F., de Moya-Anegon, F., Lopez-Ilescas, C., & Visser, M. (2011). Is concentration of university research associated with better research performance? Journal of Informetrics, 5(4), 649–658.
108. Mundlak, Y. (1978). On the pooling of time series and cross-sectional data. Econometrica, 46, 69–86.
109. Musselin, C. (2005a). Le marché des universitaires. France, Allemagne. Paris: Presses de la Fondation National des Sciences Politiques.
110. Musselin, C. (2005b). European academic labor markets in transition. Higher Education, 49, 135–154.
111. Oettl, A. (2012). Reconceptualizing stars. Scientist helpfulness and peer performance. Management Science, 58(6), 1122–1140.
112. Olsen, D., & Simmons, A. (1996). The research versus teaching debate: Untangling the relationships. In Braxton J.M. (ed.) Faculty teaching and research: Is there a conflict? New directions for institutional Research, no. 90, vol. XVIII, no. 2. San Francisco, Jossey-Bass, 31–40.
113. Paglis, L. L., Green, S. G., & Bauer, T. N. (2006). Does adviser mentoring add value? A longitudinal study of mentoring and doctoral student outcomes. Research in Higher Education, 47(4), 451–477.
114. Pinheiro, R., Benneworth, P., & Jones, G. A. (Eds.). (2012). Universities and regional development. London: Routledge.
115. Piro, F., Aksnes, D. W., & Rørstad, K. (2013). A macro analysis of Productivity differences across fields: Challenges in the measurement of scientific publishing. Journal of the American Society for Information Science and Technology, 64(2), 307–320.
116. Rabe-Hesketh, S., & Skrondal, A. (2008). Multilevel and longitudinal modeling using Stata. College Station: STATA Press.
117. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models. Applications and data analysis methods. Thousand Oaks: Sage Publications.
118. Reskin, B. (1977). Scientific productivity and the reward structure of science. American Sociological Review, 42, 491–504.
119. Rossi, F. (2009). Increased competition and diversity in Higher Education: An empirical analysis of the Italian university system. Higher Education Policy, 22, 389–413.
120. Sánchez-Barrioluengo, M. (2014). ‘Turning the tables’: Regions shaping university performance. Regional Studies, Regional Science, 1(1), 276–285.
121. Sarrico, C. S., Teixeira, P. N., Rosa, M. J., & Cardoso, M. F. (2009). Subject mix and productivity in Porguguese universities. European Journal of Operational Research, 197, 287–295.
122. Scaffidi, A. K., & Berman, J. E. (2011). A positive postdoctoral experience is related to quality supervision and career mentoring, collaborations, networking and a nurturing research environment. Higher Education, 62, 685–698.
123. Schaffner, K. F. (1993). Discovery and explanation in biology and medicine. Chicago: The University of Chicago Press.
124. Schimank, U., & Winnes, M. (2000). Beyond Humboldt? The relationship between teaching and research in European university systems. Science and Public Policy, 27(6), 397–408.
125. Seglen, P. O., & Aksnes, D. W. (2000). Scientific productivity and group size: A bibliometric analysis of Norwegian microbiological research. Scientometrics, 49(1), 125–143.
126. Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, Longitudinal and Structural Equation models. Boca Raton, FL: Chapman & Hall/CRC.
127. Snijders, T. A., & Berkhof, J. (2008). Diagnostic checks for multilevel models. In J. De Leeuw, E. Meijer & H. Goldstein (Eds.), Handbook of multilevel analysis (pp. 141–175). New York: Springer.
128. Sullivan, A. V. S. (1996). Teaching norms and publication productivity. In J. M. Braxton (Ed.), Faculty teaching and research: Is there a conflict? New directions for institutional Research, no. 90, vol. XVIII, no. 2. San Francisco, Jossey-Bass, 15–22.
129. Teixeira, P., Rocha, V., Biscaia, R., & Cardoso, M. F. (2014). Public and private higher education in Europe: competition, complementarity or worlds apart? In A. Bonaccorsi (Ed.), Knowledge, diversity and performance in European higher education. Cheltenham: Edward Elgar.
130. Thagard, P. (1999). How scientists explain disease. Princeton: Princeton University Press.
131. Toivanen, O., & Waterson, M. (2013). The effect of department size on quality of research in science: Evidence from the UK research assessment exercise. KU Leuven, Fac. of Economics and Business Working Paper. Available at https://lirias.kuleuven.be/bitstream/123456789/391779/1/MSI_1304.pdf.
132. van Arensbergen, P., & van den Besselaar, P. (2012). The selection of scientific talent in the allocation of research grants. Higher Education Policy, 25, 381–405.
133. van Leeuwen, T. N., & Moed, H. F. (2012). Funding decisions, peer review, and scientific excellence in physical sciences, chemistry and geosciences. Research Evaluation, 21, 189–198.
134. Varga, A. (1998). University research and regional innovation. A spatial econometric analysis of academic technology transfers. Dordrecht: Kluwer Academic Publishers.
135. Viner, N., Powell, P., & Green, R. (2004). Institutionalized biases in the award of research grants: a preliminary analysis revisiting the principle of accumulative advantage. Research Policy, 33, 443–454.
136. Von Tunzelmann, N., Ranga, M., Martin, B., & Geuna, A. (2003). The effects of size on research performance: A SPRU Review, Report prepared for the Office of Science and Technology, Department of Trade and Industry.
137. Waldinger, F. (2011). Peer effects in science. Evidence from the dismissal of scientists in Nazi Germany. Review of Economic Studies, 79(2), 838–861.
138. Weinberg, R. A. (1996). Racing to the beginning of the road. The search for the origin of cancer. New York: Harmony Books.
139. Worthington, A. C., & Higgs, H. (2011). Economies of scale and scope in Australian higher education. Higher Education, 61(4), 387–414.