Group size, h-index, and efficiency in publishing in top journals explain expert panel assessments of research group quality and productivity

Research Evaluation

Volumn 22, Issue 4, 2013, Pages 224-236

  Engels, Tim C E ^a,b   Goos, Peter ^a,c   Dexters, Nele ^a   Spruyt, Eric H J ^a  

^a UNIVERSITY OF ANTWERP   (Belgium)

^b ANTWERP MARITIME ACADEMY   (Belgium)

^c Erasmus University Rotterdam   (Netherlands)

Author keywords

Group size; Peer review; Research assessment exercise; Research productivity; Research quality

Indexed keywords

EID: 84887985600     PISSN: 09582029     EISSN: 14715449     Source Type: Journal    
DOI: 10.1093/reseval/rvt013     Document Type: Article

References (41)

1. Abramo, G., D'Angelo, C. A. and Di Costa, F. (2011) 'National Research Assessment Exercises: A Comparison of Peer Review and Bibliometrics Rankings', Scientometrics, 89: 929-41.
2. Adams, J. and Thomson, S. (2011) Funding Research Excellence: Research Group Size, Critical Mass & Peormance. London: University Alliance.
3. Agresti, A. (2002) Categorical Data Analysis, 2nd edn. New York: Wiley.
4. Agresti, A. (2010) Analysis of Ordinal Categorical Data, 2nd edn. New York: Wiley.
5. Aksnes, D. W. and Taxt, R. E. (2004) 'Peer Reviews And Bibliometric Indicators: A Comparative Study At A Norwegian University', Research Evaluation, 13: 33-41.
6. Butler, L. and McAllister, I. (2011) 'Evaluating University Research Peormance Metrics', European Political Science, 10: 44-58.
7. de Andrés, A. (2011) 'Evaluating Research Using Impact and Hirsch Factors: Why Such Large Differences For Different Fields'?, Europhysics News, 42: 29-31.
8. Engels, T. C. E., Dexters, N. and Houben, B. (2012) 'Correlations of Assessments of Research Group Quality and Productivity With Group Size, Output Numbers and Normalized Impact', Proceedings of the 17th Science and Technology Indicators Conference, pp. 247-257. Montréal: STI.
9. European Commission. (2010) Assessing Europe's University- Based Research. Brussels: Expert Group on Assessment of University-Based Research, Directorate-General for Research, European Commission.
10. Franceschet, M. and Constantini, A. (2011) 'The First Italian Research Assessment Exercise: A Bibliometric Perspective', Journal of Informetrics, 5: 275-91.
11. Glänzel, W. and Schubert, A. (2003) 'A New Classification Scheme of Science Fields and Subfields Designed For Scientometric Evaluation Purposes', Scientometrics, 56: 357-67.
12. Goos, P. and Gilmour, S. G. (2012) 'A General Strategy for Analyzing Data From Split-Plot and Multistratum Experimental Designs', Technometrics, 54: 340-54.
13. Hansen, H. F. (2009) Research Evaluation: Methods, Practice, and Experience. Copenhagen: Danish Agency for Science, Technology and Innovation.
14. Hansson, F. (2010) 'Dialogue in or With Peer Review? Evaluating Research Organizations in Order to Promote Organizational Learning', Science and Public Policy, 37: 239-51.
15. Hicks, D. (2012) 'Peormance-based university research funding systems', Research Policy, 41: 251-61.
16. Himanen, L. et al. (2009) 'Influence of Research Funding and Science Policy on University Research Peormance: A Comparison of Five Countries', Science and Public Policy, 36: 417-96.
17. Hirsch, J. E. (2005) 'An Index to Quantify an Individual's Scientific Research Output', Proceedings of the National Academy of Sciences of the United States of America, 102: 16569-72.
18. Jayasinghe, U. W., Marsh, H. W. and Bond, N. (2006) 'A New Reader Trial Approach to Peer Review in Funding Research Grants: An Australian Experiment', Scientometrics, 69: 591-606.
19. Kenna, R. and Berche, B. (2011) 'Critical Mass and the Dependency of Research Quality on Group Size', Scientometrics, 86: 527-40.
20. KNAW. (2011) Quality indicators for research in the Humanities. Amsterdam: Koninklijke Nederlandse Akademie van Wetenschappen.
21. Kretschmer, H. (1985) 'Cooperation Structure, Group Size and Productivity in Research Groups', Scientometrics, 7: 39-53.
22. Langfeldt, L. (2004) 'Expert Panels Evaluating Research: Decision-Making and Sources of Bias', Research Evaluation, 13: 51-62.
23. Larivière, V. and Gingras, Y. (2011) 'Averages of Ratios vs. Ratios of Averages: An Empirical Analysis of Four Levels of Aggregation', Journal of Informetrics, 5: 392-9.
24. Lawrenz, F., Thao, M. and Johnson, K. (2012) 'Expert Panel Reviews of Research Centers: The Site Visit Process', Evaluation and Program Planning, 35: 390-7.
25. Leydesdorff, L. and Bornmann, L. (2011) 'Integrated Impact Indicators (I3) Compared With Impact Factors (Ifs): An Alternative Research Design With Policy Implications', Journal of the American Society for Information Science and Technology, 63: 1370-83.
26. Moed, H. F. (2005) Citation Analysis in Research Evaluation. Dordrecht, the Netherlands: Springer.
27. Ochsner, M., Hug, S. E. and Daniel, H.-D. (2013) 'Four Types of Research in the Humanities: Setting the Stage for Research Quality Criteria in the Humanities', Research Evaluation, 22: 79-92.
28. Olsson, L., Hemlin, S. and Pousette, A. (2012) 'A Multi-Level Analysis of Leader-Member Exchange and Creative Peormance in Research Groups', The Leadership Quarterly, 23: 604-19.
29. Richardson, M. (2011) 'Behind the Data. Two's Company: How Scale Affects Research Groups', Research Trends, 25: 13-14.
30. Rinia, E. J., van Leeuwen, T., van Vuren, H. G. and van Raan, A. F. J. (1998) 'Comparative Analysis of a Set of Bibliometric Indicators and Central Peer Review Criteria: Evaluation of Condensed Matter Physics in the Netherlands', Research Policy, 27: 95-107.
31. Rons, N. (2012) 'Partition-Based Field Normalization: An Approach to Highly Specialized Publication Records', Journal of Informetrics, 6: 1-10.
32. Rons, N., De Bruyn, A. and Cornelis, J. (2008) 'Research Evaluation Per Discipline: A Peer-Review Method and its Outcomes', Research Evaluation, 17: 45-57.
33. Schubert, A., Glänzel, W. and Braun, T. (1988) 'Against Absolute Methods: Relative Scientometric Indicators And Relational Charts As Evaluation Tools'. In: van Raan, A.F.J. (ed.) Handbook of Quantitative Studies of Science and Technology, pp. 137-175. Amsterdam: Elsevier.
34. SCImago Research Group. (2012) SIR World Report. Granada: SCImago Research Group, University of Granada.
35. Spruyt, E. H. J. and Engels, T. C. E. (2008) 'Dient Massaliteit Excellentie in Onderzoek?', Thema: Tijdschrift voor Hoger Onderwijs en Management, 15: 39-47.
36. Spruyt, E. H. J. and Engels, T. C. E.. (2010) 'Onderzoeksevaluaties als beleidsinstrument aan de Universiteit Antwerpen', Thema: Tijdschrift voor Hoger Onderwijs en Management, 3: 31-7.
37. van Drooghe, L. et al. (2013) Twintig jaar onderzoeksevaluatie. den Haag: Rathenau Instituut.
38. van Raan, A. F. J. (2006) 'Comparison of the Hirsch-index with Standard Bibliometric Indicators and with Peer Judgements for 147 Chemistry Research Groups', Scientometrics, 67: 491-502.
39. VSNU. (2003) Standard Evaluation Protocol 2003-2009 for Public Research Organisations. Utrecht/den Haag/Amsterdam: VSNU, NWO and KNAW.
40. VSNU. (2009) Standard Evaluation Protocol 2009-2015. Protocol for Research Assessment in the Netherlands. Utrecht/den Haag/Amsterdam: VSNU, KNAW and NWO.
41. Waltman, L. and van Eck, N. J. (2012) 'The Inconsistency of the H-Index', Journal of the American Society for Information Science and Technology, 63: 406-415.