Measuring the productivity of national R&D systems: Challenges in cross-national comparisons of R&D input and publication output indicators

Science and Public Policy

Volumn 44, Issue 2, 2017, Pages 246-258

  Aksnes, Dag W ^a   Sivertsen, Gunnar ^a   van Leeuwen, Thed N ^b   Wendt, Kaja K ^a  

^a NORDIC INSTITUTE FOR STUDIES IN INNOVATION   (Norway)

^b LEIDEN UNIVERSITY   (Netherlands)

Author keywords

Bibliometric indicators; Frascati Manual; National R D systems; Productivity; R D statistics

Indexed keywords

EFFICIENCY MEASUREMENT; INDUSTRIAL PERFORMANCE; INDUSTRIAL PRODUCTION; INNOVATION; OECD; RESEARCH AND DEVELOPMENT;

EID: 85019772199     PISSN: 03023427     EISSN: None     Source Type: Journal    
DOI: 10.1093/scipol/scw058     Document Type: Article

References (38)

1. Abramo, G., D'Angelo, C. A. and Pugini, F. (2008) 'The measurement of Italian universities' research productivity by a non parametric-bibliometric methodology'. Scientometrics, 76: 225-44.
2. Aksnes, D. W., Schneider, J. W. and Gunnarsson, M. (2012) 'Ranking national research systems by citation indicators. A comparative analysis using whole and fractionalised counting methods'. Journal of Informetrics, 6: 36-43.
3. Archambault, É., Vignola-Gagne, E., Cõté, G., Larivière, V. and Gingras, Y. (2006) 'Benchmarking scientific output in the social sciences and humanities: The limits of existing databases'. Scientometrics, 68: 329-42.
4. Den Hertog, P., Jager, C. J., Vankan, A., te Velde, R. et al. (2014) Science, Technology & Innovation Indicators 2014. Challenging in Measuring the Efficiency of National Science, Technology & Innovation Systems. Utrecht, the Netherlands: Dialogic.
5. Dougherty, S. M., Inklaar R., McGuckin, R. H. and van Ark, B. (2007) 'International comparisons of R&D expenditure: Does an R&D PPP make a difference?' in Hard-to-Measure Goods and Services: Essays in Honor of Zvi Griliches, E. E. Berndt and C. R. Hulten (eds), pp. 291-323. Chicago, IL: University of Chicago Press.
6. Eurostat (2009a) Synthesis of National Quality Reports for 2009 R&D and GBAORD Statistics accessed 1 Sep 2016.
7. Eurostat (2009b) R&D National Quality Reports HES 2009 (unpublished reports).
8. Gauffriau, M., Olesen Larsen, P., Maye, I., Roulin-Perriard, A. and von Ins, M. (2008) 'Comparisons of results of publication counting using different methods'. Scientometrics, 77: 147-76.
9. Godin, B. (2007) 'Science, accounting and statistics: The input-output framework'. Research Policy, 36: 1388-403.
10. ILO. (1990) International Standard Classification of Occupations: ISCO-88. Geneva, Switzerland: International Labour Organization.
11. King, D. A. (2004) 'The scientific impact of nations. What different countries get for their research spending'. Nature, 430(6997): 311-6.
12. Kivinen, O., Hedman, J. and Kaipainen P. (2013) 'Productivity analysis of research in natural sciences, technology and clinical medicine: An input-output model applied in comparison of Top 300 ranked universities of 4North European and 4 EastAsian countries'. Scientometrics, 94: 683-99.
13. Lee, S. and Bozeman, B. (2005) 'The impact of research collaboration on scientific productivity'. Social Studies of Science, 35: 673-702.
14. Leydesdorff, L. and Wagner, C. (2009) 'Macro-level indicators of the relations between research funding and research output'. Journal of Informetrics, 3: 353-62.
15. Luwel, M. (2004) 'The use of input data in the performance analysis of R&D systems: Potentialities and pitfalls' in Handbook of Quantitative Science and Technology Research, H. F. Moed, W. Glänzel and U. Schmoch (eds), pp. 315-38. Dordrecht, the Netherlands: Kluwer.
16. May, R. M. (1997) 'The scientific wealth of nations'. Science, 275(5301): 793-6.
17. May, R. M. (1998) 'The scientific investments of nations'. Science, 281(5373): 49-51.
18. Moed, H. F. (2005) Citation Analysis in Research Evaluation. Dordrecht, the Netherlands: Springer.
19. National Science Board (2012) Science and Engineering Indicators 2012. Arlington VA: National Science Foundation.
20. OECD (2010) Measuring Innovation: A New Perspective. Paris: OECD Publishing.
21. OECD (2012) R&D Sources and Methods Database accessed 1 Sep 2016.
22. OECD (2014) Main Science and Technology Indicators, Volume 2013/2. Paris: OECD Publishing.
23. OECD (2015a) Frascati Manual 2015: Guidelines for Collecting and Reporting Data on Research and ExperimentalDevelopment. Paris:OECDPublishing.
24. OECD (2015b) Working Party of National Experts on Science and Technology Indicators. Revision of Frascati Manual Chapter 9. Higher Education Sector. DSTI/EAS/STP/NESTI(2015)3/CHAP9. Paris: OECD.
25. OECD (2016) Working Party of National Experts on Science and Technology Indicators. The International Data Collection on Resources Devoted to R&D: Options for Implementing the OECD Frascati Manual 2015. DST/EAS/STP/NESTI(2019)/3. Paris: OECD.
26. Piro, F., Aksnes, D. W. and Rørstad, K. (2013) 'A macro analysis of productivity differences across fields: Challenges in the measurement of scientific publishing'. Journal of the American Society for Information Science and Technology, 64: 307-20.
27. Rørstad, K. and Aksnes, D. W. (2015) 'Publication rate expressed by age, gender and academic position: A large-scale analysis of Norwegian academic staff'. Journal of Informetrics, 9: 317-33.
28. Rousseau, S. and Rousseau, R. (1998) The scientific wealth of European nations: Taking effectiveness into account'. Scientometrics, 42: 75-87.
29. Sivertsen, G. (2016) 'Patterns of internationalization and criteria for research assessment in the social sciences and humanities'. Scientometrics, 107: 357-68.
30. Sivertsen, G. and Larsen, B. (2012) 'Comprehensive bibliographic coverage of the social sciences and humanities in a citation index: An empirical analysis of the potential'. Scientometrics, 91: 567-75.
31. UNESCO. (1997) ISCED 1997. International Standard Classification of Education. Paris, France: UNESCO.
32. van Leeuwen, T. N., Moed, H. F., Tijssen, R. J., Visser, M. S. and Raan, A. F. (2001) 'Language biases in the coverage of the Science Citation Index and its consequences for international comparisons of national research performance'. Scientometrics, 51: 335-46.
33. van Leeuwen, T. N., Moed, H. F., Tijssen, R. J., Visser, M. S. and Raan, A. F., Visser, M. S., Moed, H. F., Nederhof, A. J. and Raan, A. F. J. (2003) 'The Holy Grail of science policy: Exploring and combining bibliometric tools in search of scientific excellence'. Scientometrics, 57: 257-80.
34. van Leeuwen, T. (2013) 'Bibliometric research evaluations, Web of Science and the Social Sciences and Humanities: a problematic relationship'? Bibliometrie-Praxis und Forschung, 2: 1-18.
35. van Raan, A. F. J. (2004) 'Measuring science. Capita selecta of current main issues' in Handbook of Quantitative Science and Technology Research, H. F. Moed, W. Glänzel and U. Schmoch (eds), pp. 19-50. Dordrecht, the Netherlands: Kluwer.
36. van Raan, A. F. J., van Leeuwen, T. N. and Visser, M. S. (2011) 'Severe language effect in university rankings: Particularly Germany and France are wronged in citation-based rankings'. Scientometrics, 88: 495-8.
37. Wendt, K. K., Aksnes, D. W., Karlsson, S. and Sivertsen, G. (2012) 'Challenges in cross-national comparisons of R&D expenditure and publication output'. Proceedings of STI 2012 Montreal: 17th International Conference on Science and Technology Indicators, held 5-8 September 2012, Montreal, Canada, pp. 826-35. Montreal, Canada: Science Matrix and OST.
38. Zhou, P. and Leydesdorff, L. (2006) 'The emergence of China as a leading nation in science'. Research Policy, 35: 83-104.