Are there any frontiers of research performance? Efficiency measurement of funded research projects with the Bayesian stochastic frontier analysis for count data

Journal of Informetrics

Volumn 11, Issue 3, 2017, Pages 613-628

  Mutz, Rüdiger ^a   Bornmann, Lutz ^b   Daniel, Hans Dieter ^a,c  

^a ETH ZURICH   (Switzerland)

^b MAX PLANCK INSTITUTE   (Germany)

^c UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

Data envelopment analysis; Productivity and efficiency analysis; Research funding; Stochastic frontier analysis

Indexed keywords

DATA ENVELOPMENT ANALYSIS; ECONOMICS; EFFICIENCY; PRODUCTIVITY; SAMPLING; STATISTICS;

EFFICIENCY ANALYSIS; EFFICIENCY MEASUREMENT; RESEARCH FUNDING; RESEARCH PERFORMANCE; STATISTICAL APPROACH; STATISTICAL BEHAVIOR; STOCHASTIC FRONTIER ANALYSIS; TECHNICAL EFFICIENCY;

STOCHASTIC SYSTEMS;

EID: 85020712441     PISSN: 17511577     EISSN: 18755879     Source Type: Journal    
DOI: 10.1016/j.joi.2017.04.009     Document Type: Article

References (66)

1. Abbott, M., Doucouliagos, C., The efficiency of Australian universities: A data envelopment analysis. Economics of Education Review 22 (2003), 89–97.
2. Abramo, G., D‘Angelo, C.A., How do you define and measure research productivity?. Scientometrics 101 (2014), 1129–1144.
3. Abramo, G., D‘Angelo, C.A., A farewell to the MNCS and like size-independent indicators. Journal of Informetrics 10:2 (2016), 646–651.
4. Abramo, G., Cicero, T., D‘Angelo, C.A., A field-standardized application of DEA to national-scale research assessment of universities. Journal of Informetrics 5 (2011), 618–628.
5. Abramo, G., D‘Angelo, C.A., Murgia, C.A., The combined effects of age and seniority on research performance of full professors. Science and Public Policy 43:3 (2016), 301–319.
6. Archibald, R.B., Feldman, D.H., Graduation rates and accountability: Regression versus production frontiers. Research in Higher Education 49 (2008), 80–100.
7. Athanassopoulos, A.D., Shale, E., Assessing the comparative efficiency of higher education institutions in the UK by the means of Data Envelopment Analysis. Education Economics 5:2 (1997), 117–134.
8. Ballou, J., Mishkind, M., Mooney, G., van Kammen, W., National science foundation report on efficiency of grant size and duration., 2002 Retrieved from https://www.nsf.gov/pubs/2004/nsf04205/mathematica_nsfrptfinal6.pdf.
9. Bessent, A.M., Bessent, E.W., Charnes, A., Cooper, W.W., Thorogood, N.C., Evaluation of educational-program proposals by means of DEA. Educational Administration Quarterly 19 (1983), 82–107.
10. Bolli, T., Olivares, M., Bonaccorsi, A., Daraio, C., Aracil, A.G., Lepori, B., The differential effects of competitive funding on the production frontier and the efficiency of universities. Economics of Education Review 52 (2016), 91–104.
11. Bonaccorsi, A., Daraio, C., Econometric approaches to the analysis of productivity of R&D systems. Moed, H.F., Glänzel, W., Schmoch, U., (eds.) Handbook of quantitative science and technology research, 2004, Kluwer, New York, 51–74.
12. Bornmann, L., Stefaner, M., de Moya Anegón, F., Mutz, R., Excellence networks in science: A Web-based application based on Bayesian multilevel logistic regression (BMLR) for the identification of institutions collaborating successfully. Journal of Informetrics 10:1 (2016), 312–327.
13. Charnes, A., Cooper, W.W., Rhodes, E., Measuring the efficiency of decision making units. European Journal of Operational Research 2 (1978), 429–444.
14. Chen, C.-M., Delmas, M.A., Lieberman, M.B., Production frontier methodologies and efficiency as a performance measure in strategic management research. Strategic Management Journal 36 (2015), 19–36.
15. Coelli, T.J.C., Rao, D.S.P., O‘Donnell, C.J., Battese, G.E., An introduction to efficiency and productivity analysis. 2nd ed., 2005, Springer, New York.
16. Daniel, H.-D., Methodische Probleme institutsvergleichender Analysen der Forschungsproduktivität [Methodical problems of institutional comparisons of research productivity]. Daniel, H.-D., Fisch, R., (eds.) Evaluation von Forschung, Methoden – Ergebnisse – Stellungnahmen [Evaluation of research, methods – results – statements], 1988, Universitätsverlag, Konstanz, 215–241.
17. Drivas, K., Balafoutis, A.T., Rozakis, S., Research funding and academic output: The case of Agricultural University of Athens (AUA working paper series No. 2014-5). 2014 Retrieved from http://aoatools.aua.gr/RePEc/aua/wpaper/files/2014-5_Drivas.pdf.
18. Drivas, K., Economidou, C., Tsionas, E.G., A Poisson Stochastic frontier model with finite mixture structure (MPRA paper No. 57485). 2014 Retrieved from http://mpra.ub.uni-muenchen.de/57485/.
19. Dundar, H., Lewis, D.R., Determinants of research productivity in higher education. Research in Higher Education 39:6 (1998), 607–631.
20. Ehlers, R.S., Comparison of Bayesian models for production efficiency. Journal of Applied Statistics 38:11 (2011), 2433–2443.
21. Fé, E., Hofler, R., Count data stochastic frontier models, with an application to the patents–R&D relationship. Journal of Productivity Analysis 39:3 (2013), 271–284.
22. Faire, R., Grosskopf, S., Margaritis, D., Efficiency and productivity: Malmquist and more. Fried, H.O., Lovell, C.A.K., Schmid, C.H., (eds.) The measurement of productive efficiency and productive growth, 2008, Oxford University Press, Oxford, 522–621.
23. Falzagic, A., Measuring the intellectual capital of a university. Paper presented at the trends in the management of human resources in higher education, Paris, 2005 https://www.oecd.org/edu/imhe/35322785.pdf.
24. Gingras, Y., Lariviere, V., Macaluso, B., Robitaille, J.P., The effects of aging on researchers’ publication and citation patterns. Plos One, 3(12), 2008, e4048.
25. Glänzel, W., On reliability and robustness of scientometrics indicators based on stochastic models: An evidence-based opinion paper. Journal of Informetrics 4:3 (2010), 313–319.
26. Greene, W.H., Fixed and random effects in stochastic frontier models. Journal of Productivity Analysis 23:1 (2005), 7–32.
27. Greene, W.H., The econometric approach to efficiency analysis. Fried, H.O., Lovell, C.A.K., Schmidt, S.S., (eds.) The measurement of productive efficiency and productivity growth, 2008, Oxford University Press, Oxford, 92–250.
28. Griffin, J.E., Steel, M.F.J., Bayesian stochastic frontier analysis using WinBUGS. Journal of Production Analysis 27 (2007), 163–176.
29. Hicks, D., Performance-based university research funding systems. Research Policy 41 (2012), 251–261.
30. Hilbe, J.M., Negative binomial regression. 2012, Cambridge University Press, Cambridge.
31. Hilbe, J.M., Modeling count data. 2014, Cambridge University Press, New York.
32. Hoff, P.D., A first course in Bayesian statistical methods. 2009, Springer, New York.
33. Horta, H., Lacy, T.A., How does size matter for science? Exploring the effects of research unit size on academics’ performance productivity and information exchange behavior. Science and Public Policy 38:6 (2011), 449–460.
34. Hox, J., Multilevel analysis − Techniques and applications. 2010, Routledge, New York.
35. Johnes, J., Efficiency measurement. Johnes, G., Johnes, J., (eds.) International handbook on the economics of education, 2004, Edward Elgar Publications, Cheltenham, UK, 613–742.
36. Johnes, J., Data envelopment analysis and its application to the measurement of efficiency in higher education. Economics of Education Review 25 (2006), 273–288.
37. Johnson, A.L., Kuosmanen, T., One-stage and two-stage DEA estimation of the effects of contextual variables. European Journal of Operational Research 220 (2012), 559–570.
38. Kaplan, D., Bayesian statistics for the social sciences. 2014, The Guilford Press, London.
39. Kumbhakar, S.C., Wang, H.-J., Horncastle, A.P., A practioner’s guide to stochastic frontier analysis using STATA. 2015, Cambridge University Press, New York.
40. Lariviere, V., Ni, C., Gingras, Y., Cronin, B., Sugimoto, C.R., Bibliometrics: Global gender disparities in science. Nature 504:7479 (2013), 211–2013.
41. Limpert, E., Stahel, W.A., Abbt, M., Log-normal distributions across the sciences: Keys and clues. BioScience 51:5 (2001), 341–352.
42. Lindsey, J.K., Modelling frequency and count data. 1995, Oxford University Press, New York.
43. Liu, J.S., Lu, L.Y.Y., Lu, W.-M., Lin, B.J.Y., Data envelopment analysis 1978–2010: A citation-based literature survey. Omega 41 (2013), 3–15.
44. Liu, J.S., Lu, L.Y.Y., Lu, W.-M., Lin, B.J.Y., A survey of DEA applications. Omega 41 (2013), 893–902.
45. Lunn, D., Jackson, C., Best, N., Thomas, A., Spiegelhalter, D., The BUGS book − A practical introduction to Bayesian analysis. 2013, Chapman & Hall, New York.
46. Mutz, R., Bornmann, L., Daniel, H.-D., Does gender matter in grant peer review? An empirical investigation using the example of the Austrian Science Fund. Zeitschrift für Psychologie 220:2 (2012), 121–129.
47. Mutz, R., Bornmann, L., Daniel, H.-D., Types of research output profiles: A multilevel latent class analysis of the Austrian Science Fund's final project report data. Research Evaluation 22:2 (2013), 118–133.
48. Mutz, R., Bornmann, L., Daniel, H., Funding decision-making systems: An empirical comparison of continuous and dichotomous approaches based on psychometric theory. Research Evaluation 25:4 (2016), 416–426.
49. O‘Hagan, A., The Bayesian approach to statistics. Rudas, T., (eds.) Handbook of probability: Theory and applications, 2008, Sage, London, 85–100.
50. O‘Hara, R.B., Kotze, D.J., Do not log-transform count data. Methods in Ecology and Evolution 1 (2010), 118–122.
51. Olivares, M., Wetzel, H., Competing in the higher education market: Empirical evidence for economics of scale and scope in German higher education institutions. CESifo Economic Studies 60:4 (2014), 653–680.
52. Over, R., Does scholarly impact decline with age?. Scientometrics 13:5–6 (1988), 215–223.
53. Parmeter, C.F., Efficiency analysis: A primer on recent advances. Foundations and Trends in Econometrics 7:3–4 (2014), 191–385.
54. Rabovsky, T., Support for performance-based funding: The role of political ideology, performance, and dysfunctional information environments. Public Administration Review 74:6 (2014), 761–774.
55. Rabovsky, T., Using data to manage for performance at public universities. Public Administration Review 74:2 (2014), 260–272.
56. Ramalho, E.A., Ramalho, J.J.S., Henriques, P.D., Fractional regression models for second stage DEA efficiency analyses. Journal of Production Analysis 34 (2010), 239–255.
57. ®13.2 user's guide. 2014, SAS Institute Inc., Cary, NC.
58. Simar, L., Wilson, P.W., Two-stage DEA: Caveat emptor. Journal of Production Analysis 36 (2011), 205–218.
59. Sinuanystern, Z., Mehrez, A., Barboy, A., Academic departments efficiency via DEA. Computers & Operations Research 21 (1994), 543–556.
60. Stringer, M.J., Sales-Pardo, M., Amaral, L.A.N., Statistical validation of a global model for the distribution of the ultimate number of citations accrued by papers published in a scientific journal. Journal of the American Society for Information Science and Technology 61:7 (2010), 1377–1385.
61. Stroebe, W., The graying of academia. American Psychologist 65:7 (2010), 660–673.
62. Tsionas, E.G., Stochastic frontier model with random coefficients. Journal of Applied Econometrics 17 (2002), 127–147.
63. Waltman, L., van Eck, N.J., van Leeuwen, T.N., Visser, M.S., van Raan, A.F.J., Towards a new crown indicator: An empirical analysis. Scientometrics 87:3 (2011), 467–481.
64. Wang, H.-J., Schmidt, P., One-step and two-step estimation of the effects of exogenous variables on technical efficiency levels. Journal of Productivity Analysis 18 (2002), 129–144.
65. Wijk van, E., Costas-Comesana, R., Bibliometric Study of FWF Austrian Science Fund 2001-2010/2011. 2012 Retrieved from http://www.fwf.ac.at/fileadmin/files/Dokumente/Ueber_den_FWF/Publikationen/FWF-Selbstevaluation/Bibliometric-Study-FWF-2001-2010-Originalstudie.pdf.
66. van den Broek, J., Koop, G., Osiewalski, J., Steel, M.F.J., Stochastic frontier models. Journal of Econometrics 61 (1994), 273–303.