Evaluating research and researchers by the journal impact factor: Is it better than coin flipping?

Journal of Informetrics

Volumn 13, Issue 1, 2019, Pages 314-324

  Brito, Ricardo ^a   Rodríguez Navarro, Alonso ^a,b  

^a UNIVERSIDAD COMPLUTENSE DE MADRID   (Spain)

^b UNIVERSIDAD POLITÉCNICA DE MADRID   (Spain)

Author keywords

Failure probability; Impact factor; Research evaluation

Indexed keywords

PROBABILITY DISTRIBUTIONS; PUBLISHING;

COMPARATIVE EVALUATIONS; FAILURE PROBABILITY; IMPACT FACTOR; LOG-NORMAL DISTRIBUTION; RESEARCH EVALUATION; RISK OF FAILURE; SCIENTIFIC MERITS;

PAPER;

EID: 85060805177     PISSN: 17511577     EISSN: 18755879     Source Type: Journal    
DOI: 10.1016/j.joi.2019.01.009     Document Type: Article

References (33)

1. G. Abramo, C.A. D'Angelo, and F. Di Costa Citation versus journal impact factot as proxy of quality: could the latter ever be preferable? Scientometrics 84 2010 821 833
2. E. Archambault, and V. Larivière History of the journal impact factor: Contingencies and consequences Scientometrics 79 2009 635 649
3. L. Bornmann, and W. Marx How good is research really? Measuring the citation impact of publications with percentiles increases correct assessments and fair comparisons EMBO Reports 14 2013 226 230
4. L. Bornmann, and W. Marx How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentile citations Scientometrics 98 2014 487 509
5. L. Bornmann, and R. Williams Can the journal impact factor be used as a criterion for the selection of junior researchers? A large-scale empirical study based on ResercherID data Journal of Informetrics 11 2017 788 799
6. L. Bornmann, and R. Williams Use of the journal impact factor as a criterion for the selection of junior researchers: A rejoinder on a comment by Peters (2017) Journal of Informetrics 11 2017 945 947
7. E. Callaway Publishing elite turns against impact factor Nature 535 2016 210 211
8. N. De Bellis Bibliometrics and citation analysis 2009 The Scarecrow Press, Inc. Lanham, Maryland ISBN-13: 978-0-8108-6713-0
9. J. DeCoster, A.M. Iselin, and M. Gallucci A conceptual and empirical examination of justifications for dichotomization Psychological Methods 14 2009 349 366
10. E. Garfield Impact factors, and why they won't go away Nature 411 2001 522
11. B. Hammarfelt, and A.D. Rushforth Indicators as judgment devices: An empirical study of citizen bibliometrics in research evaluation Research Evaluation 26 2017 169 180
12. V. Larivière, V. Kiermer, C.J. MacCallum, M. McNutt, M. Patterson, B. Pulverer, and et al. A simple proposal for the publication of journal citation distributions Preprint at bioRxiv https://doiorg/101101/062109 2016
13. P.A. Lawrence The politcs of publications. Authors, reviewers and editors must act to protect the quality of reserach Nature 422 2003 259 261
14. J.M. Levitt, and M. Thelwall A combined bibliometric indicator to predict article impact Information Processing & Management 47 2011 300 308
15. L. Leydesdorff, L. Bornmann, J.A. Comins, and S. Milojevic Citations: Indicators of quality? The impact fallacy Frontiers in Research Metrics and Analytics 1 2016 1 10.3389/frma.2016.00001
16. L. Leydesdorff, P. Wouters, and L. Bornmann Professional and citizen bibliometrics: Complementarities and ambivalences in the development and use of indicators - A state-of-the-art report Scientometrics 109 2016 2129 2150
17. S. Milojevic, F. Radicchi, and J. Bar-Ilan Citation success index - An intuitive pair-wise journal comparison metric Journal of Informetrics 11 2017 223 231
18. G.-J.Y. Peters Why not to use the journal impact factor as a criterion for the selection of junior researchers: A comment on Bornmann and Williams (2017) Journal of Informetrics 11 2017 888 891
19. A. Rodríguez-Navarro, and R. Brito Double rank analysis for research assessment Journal of Informetrics 12 2018 31 41
20. A. Rodríguez-Navarro, and R. Brito Probability and expected frequency of breakthroughs - A robust method of research assessment based on the double rank property of citation distributions Preprint at arXiv: 1807.07285 2018
21. A. Schubert, and T. Braun Relative indicators and relational charts for comparative assessment of publication output and citation impact Scientometrics 9 1986 281 291
22. A. Schubert, and T. Braun Cross-field normalization of scientometrics indicators Scientometrics 36 1996 311 324
23. P.O. Seglen Why the impact factor of journals should not be used for evaluating research BMJ 314 1997 498 502
24. I. Tahamtan, and L. Bornmann Core elements in the process of citing publications: Conceptual overview of the literature Journal of Informetrics 12 2018 203 216
25. M. Thelwall Citation count distributions for large monodisciplinary journals Journal of Informetrics 10 2016 863 874
26. V. Traag, and L. Waltman Systematic analysis of agreement between metrics and peer review in the UK REF Preprint at arXiv:1808.03491 2018
27. J. Tregoning How will you judge me if not by impact factor? Nature 558 2018 345
28. UK Forum for Responsible Reseaech Metrics UK Progress towards the use of metrics responibly. Three years on from the Metric Tide report 2018 https://wwwuniversitiesukacuk/policy-and-analysis/research-policy/open-science/Pages/forum-for-responsible-research-metricsaspx
29. L. Waltman A review of the literature on citation impact indicators Journal of Informetrics 10 2016 365 391
30. L. Waltman, and V.A. Traag Use of the journal impact factor for assessing individual articles need not be wrong Preprint at arXiv:1703.02334 2017
31. J. Wang, R. Veugelers, and P. Stephan Bias against novelty in science: A cautionary tale for users of bibliometric indicators Research Policy 46 2017 1416 1436
32. J. Wilsdon, L. Allen, E. Belfiore, P. Campbell, S. Curry, S. Hill, and et al. The metric tide: Report of the independent review of the role of metrics in research assessment and management ISBN: 1902369273 2015 10.13140/RG.2.1.4929.1363
33. L. Zhang, R. Rousseau, and G. Sivertsen Science deserves to be judged by its contents, not by its wrapping: Revisiting Seglen's work on journal impact and research evaluation PloS One 12 3 2017 e0174205