The validation of (advanced) bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000

Journal of Informetrics

Volumn 7, Issue 2, 2013, Pages 286-291

  Bornmann, Lutz ^a   Leydesdorff, Loet ^b  

^a MAX PLANCK INSTITUTE   (Germany)

^b UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

Advanced bibliometric indicators; F1000; InCites; Peer review

Indexed keywords

BIBLIOMETRIC INDICATORS; BIOMEDICAL LITERATURE; COMPARATIVE STUDIES; F1000; INCITES; PEER EVALUATIONS; PEER REVIEW; PEER REVIEW SYSTEM;

CYTOLOGY;

EID: 84872235168     PISSN: 17511577     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.joi.2012.12.003     Document Type: Article

References (38)

1. Allen L., Jones C., Dolby K., Lynn D., Walport M. Looking for landmarks: the role of expert review and bibliometric analysis in evaluating scientific publication outputs. PLoS ONE 2009, 4(6):e5910.
2. Bollen J., Van de Sompel H., Hagberg A., Chute R. A principal component analysis of 39 scientific impact measures. PLoS One 2009, 4(6):e6022.
3. Bornmann L. Towards an ideal method of measuring research performance: Some comments to the Opthof and Leydesdorff (2010) paper. Journal of Inormetrics 2010, 4(3):441-443.
4. Bornmann L. Scientific peer review. Annual Review of Information Science and Technology 2011, 45:199-245.
5. 0 at the micro level using data from chemistry. Journal of the American Society for Information Science and Technology 2009, 60(8):1664-1670.
6. Bornmann L., Mutz R., Daniel H.-D. Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology 2008, 59(5):830-837. 10.1002/asi.20806.
7. Bornmann L., Mutz R., Marx W., Schier H., Daniel H.-D. A multilevel modelling approach to investigating the predictive validity of editorial decisions: Do the editors of a high-profile journal select manuscripts that are highly cited after publication?. Journal of the Royal Statistical Society Series A: Statistics in Society 2011, 174(4):857-879. 10.1111/j.1467-985X.2011.00689.x.
8. Butler L., McAllister I. Evaluating university research performance using metrics. European Political Science 2011, 10(1):44-58. 10.1057/eps.2010.13.
9. Cohen J. Statistical power analysis for the behavioral sciences 1988, Lawrence Erlbaum Associates, Publishers, Hillsdale, NJ, USA. 2nd ed.
10. Informing research choices: Indicators and judgment: The expert panel on science performance and research funding 2012, Council of Canadian Academies, Council of Canadian Academies, Ottawa, Canada.
11. [F1000]. What is F1000? (2012). Accessed 25.10.12. http://f1000.com/about/whatis.
12. Figueredo E. The numerical equivalence between the impact factor of journals and the quality of the articles. Journal of the American Society for Information Science and Technology 2006, 57(11):1561.
13. Franceschet M., Costantini A. The first Italian research assessment exercise: A bibliometric perspective. Journal of Informetrics 2011, 5(2):275-291. 10.1016/j.joi.2010.12.002.
14. Garfield E. Citation indexing-Its theory and application in science, technology, and humanities 1979, John Wiley & Sons, Ltd, New York, NY, USA.
15. Haslam N., Laham S.M. Quality, quantity, and impact in academic publication. European Journal of Social Psychology 2010, 40(2):216-220. 10.1002/ejsp.727.
16. InCites™ Thomson Reuters. (2012a). Report created: 03.09.2012. Data processed: 09.05.2012. Data source: Web of Science. This data is reproduced under a license from Thomson Reuters. Subject area baseline data processed Jan. 1, 1981-Dec. 31, 2011.
17. InCites™ Thomson Reuters. (2012b). Report created: 09.07.2012. Data processed: 27.02.2012. Data source: Web of Science. This data is reproduced under a license from Thomson Reuters. Subject area baseline data processed Jan. 1, 1981-Dec. 31, 2010.
18. InCites™ Thomson Reuters. (2012c). Report created: 29.10.2012. Data processed: 31.12.2011. Data source: Web of Science. This data is reproduced under a license from Thomson Reuters.
19. Kohler U., Kreuter F. Data analysis using Stata 2012, Stata Press, Stata Corporation, College Station, TX, USA. 3rd ed.
20. Kreiman G., Maunsell J.H.R. Nine criteria for a measure of scientific output. Frontiers in computational neuroscience 2011, 5(48):1-6.
21. Leydesdorff L. How are new citation-based journal indicators adding to the bibliometric toolbox?. Journal of the American Society for Information Science and Technology 2009, 60(7):1327-1336.
22. Li X., Thelwall M. F1000, Mendeley and traditional bibliometric indicators. The 17th international conference on science and technology indicators 2012, 541-551. Repro-UQAM, Montreal, Canada. E. Archambault, Y. Gingras, V. Lariviere (Eds.).
23. Mahdi S., d'Este P., Neely A.D. Citation counts: Are they good predictors of RAE scores? A bibliometric analysis of RAE 2001 2008, Advanced Institute of Management Research, London, UK.
24. Martin B.R., Irvine J. Assessing basic research-Some partial indicators of scientific progress in radio astronomy. Research Policy 1983, 12(2):61-90.
25. McKay S. Social policy excellence-Peer review or metrics? Analyzing the 2008 research assessment exercise in social work and social policy and administration. Social Policy and Administration 2012, 46(5):526-543. 10.1111/j.1467-9515.2011.00824.x.
26. Medical Research Council. F1000 evaluation of MRC publication output. (2009). Accessed 24.10.12. http://www.mrc.ac.uk/Achievementsimpact/Outputsoutcomes/MRCe-Val2009/Publications/index.htm.
27. Rehn C., Kronman U., Wadskog D. Bibliometric indicators-Definitions and usage at Karolinska Institutet 2007, Karolinska Institutet University Library, Stockholm, Sweden.
28. Riffenburgh R.H. Statistics in medicine 2012, Elsevier, Oxford, UK. 3rd ed.
29. Ruscio J., Seaman F., D'Oriano C., Stremlo E., Mahalchik K. Measuring scholarly impact using modern citation-based indices. Measurement: Interdisciplinary Research and Perspectives 2012, 10(3):123-146.
30. Schubert A., Braun T. Relative indicators and relational charts for comparative assessment of publication output and citation impact. Scientometrics 1986, 9(5-6):281-291.
31. Sheskin D. Handbook of parametric and nonparametric statistical procedures 2007, Chapman & Hall/CRC, Boca Raton, FL, USA. 4th ed.
32. Smith A., Eysenck M. The correlation between RAE ratings and citation counts in psychology 2002, Department of Psychology, Royal Holloway, University of London, London, UK.
33. Stata statistical software: Release 12 2011, StataCorp, Stata Corporation, College Station, TX, USA.
34. Vinkler P. Evaluation of some methods for the relative assessment of scientific publications. Scientometrics 1986, 10(3-4):157-177.
35. Vinkler P. The evaluation of research by scientometric indicators 2010, Chandos Publishing, Oxford, UK.
36. Wardle D.A. Do 'Faculty of 1000' (F1000) ratings of ecological publications serve as reasonable predictors of their future impact?. Ideas in Ecology and Evolution 2010, 3:11-15.
37. Wets K., Weedon D., Velterop J. Post-publication filtering and evaluation: Faculty of 1000. Learned Publishing 2003, 16(4):249-258.
38. Wouters P., Costas R. Users, narcissism and control-Tracking the impact of scholarly publications in the 21st century 2012, SURF Foundation, Utrecht, The Netherlands.