How good is research really? Measuring the citation impact of publications with percentiles increases correct assessments and fair comparisons

EMBO Reports

Volumn 14, Issue 3, 2013, Pages 226-230

  Bornmann, Lutz ^a   Marx, Werner ^b  

^a MAX PLANCK INSTITUTE   (Germany)

^b MAX PLANCK INSTITUT FÜR FESTKÖRPERFORSCHUNG   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BIBLIOMETRICS; CITATION ANALYSIS; PRIORITY JOURNAL; RESEARCH; SCIENTIFIC LITERATURE;

BIOMEDICAL RESEARCH; JOURNAL IMPACT FACTOR; PERIODICALS AS TOPIC;

EID: 84874644880     PISSN: 1469221X     EISSN: 14693178     Source Type: Journal    
DOI: 10.1038/embor.2013.9     Document Type: Article

References (41)

1. Greene M (2007) The demise of the lone author. Nature 450: 1165
2. Bornmann L, Marx W, Gasparyan AY, Kitas GD (2012) Diversity, value and limitations of the journal impact factor and alternative metrics. Rheumatol Int 32: 1861-1867
3. Hirsch JE (2005) An index to quantify an individual's scientific research output. Proc Natl Acad Sci USA 102: 16569-16572
4. Bornmann L, Daniel H-D (2007) What do we know about the h index? J Am Soc Inf Sci Technol 58: 1381-1385
5. Vinkler P (2010) The Evaluation of Research by Scientometric Indicators. Oxford, UK: Chandos
6. Schubert A, Braun T (1986) Relative indicators and relational charts for comparative assessment of publication output and citation impact. Scientometrics 9: 281-291
7. Bornmann L, Leydesdorff L, Mutz R (2013) The use of percentiles and percentile rank classes in the analysis of bibliometric data: opportunities and limits. J Informetr 7: 158-165
8. Bornmann L, Mutz R, Marx W, Schier H, Daniel H-D (2011) A multilevel modelling approach to investigating the predictive validity of editorial decisions: do the editors of a high-profile journal select manuscripts that are highly cited after publication? J R Stat Soc Ser A Stat Soc 174: 857-879
9. Waltman L et al (2012) The Leiden Ranking 2011/2012: data collection, indicators, and interpretation. J Am Soc Inf Sci Tec 63: 2419-2432
10. Garfield E (2006) The history and meaning of the journal impact factor. JAMA 295: 90-93
11. Dimitrov JD, Kaveri SV, Bayry J (2010) Metrics: journal's impact factor skewed by a single paper. Nature 466: 179
12. Jackson C (1996) Understanding Psychological Testing. Leicester, UK: British Psychological Society
13. National Science Board (2012) Science and Engineering Indicators. Arlington, Virginia, USA: National Science Foundation
14. Boyack KW (2004) Mapping knowledge domains: characterizing PNAS. Proc Natl Acad Sci USA 101: 5192-5199
15. Belter C (2013) A bibliometric analysis of NOAA's Office of Ocean Exploration and Research. Scientometrics [Epub ahead of print] doi:10.1007/s11192-012-0836-0
16. Bornmann L (2013) The problem of percentile rank scores used with small reference sets. J Am Soc Inf Sci Tec (in the press
17. Leydesdorff L (2012) Accounting for the uncertainty in the evaluation of percentile ranks. J Am Soc Inf Sci Tec 63: 2349-2350
18. Rousseau R (2012) Basic properties of both percentile rank scores and the I3 indicator. J Am Soc Inf Sci Tec 63: 416-420
19. Schreiber M (2012) Inconsistencies of recently proposed citation impact indicators and how to avoid them. J Am Soc Inf Sci Tec 63: 2062-2073
20. Bornmann L (2013) Assigning publications to multiple subject categories for bibliometric analysis: an empirical case study based on percentiles. J Doc (in the press
21. Bornmann L, Mutz R (2013) The advantage of the use of samples in evaluative bibliometric studies. J Informetr 7: 89-90
22. Conroy RM (2012) What hypotheses do 'nonparametric' two-group tests actually test? Stata Journal 12: 1-9
23. Bornmann L (2013) How to analyse percentile citation impact data meaningfully in bibliometrics: The statistical analysis of distributions, percentile rank classes and top-cited papers. J Am Soc Inf Sci Tec (in the press
24. Bornmann L, Daniel H-D (2009) The state of h index research. Is the h index the ideal way to measure research performance? EMBO Rep 10: 2-6
25. Kosmulski M (2012) Modesty-index. J Informetr 6: 368-369
26. Franceschini F, Galetto M, Maisano D, Mastrogiacomo L (2012) The success-index: an alternative approach to the h-index for evaluating an individual's research output. Scientometrics 92: 621-641
27. Kosmulski M (2011) Successful papers: a new idea in evaluation of scientific output. J Informetr 5: 481-485
28. Bornmann L, de Moya Anegón F, Leydesdorff L (2012) The new Excellence Indicator in the World Report of the SCImago Institutions Rankings 2011. J Informetr 6: 333-335
29. Leydesdorff L, Bornmann L, Mutz R, Opthof T (2011) Turning the tables in citation analysis one more time: principles for comparing sets of documents J Am Soc Inf Sci Tec 62: 1370-1381
30. Waltman L, van Eck NJ (2012) The inconsistency of the h-index. J Am Soc Inf Sci Tec 63: 406-415
31. Tijssen R, van Leeuwen T (2006) Centres of research excellence and science indicators. Can 'excellence' be captured in numbers? In Ninth International Conference on Science and Technology Indicators (ed Glänzel W), pp 146-147. Leuven, Belgium: Katholieke Universiteit Leuven
32. Tijssen R, Visser M, van Leeuwen T (2002) Benchmarking international scientific excellence: are highly cited research papers an appropriate frame of reference? Scientometrics 54: 381-397
33. Sahel JA (2011) Quality versus quantity: assessing individual research performance. Sci Transl Med 3: 84cm13
34. Garfield E (1979) Citation Indexing-its Theory and Application in Science, Technology, and Humanities. New York, USA: Wiley
35. SCImago Reseach Group (2012) SIR World Report 2012. Granada, Spain: University of Granada
36. Bornmann L, Mutz R, Neuhaus C, Daniel H-D (2008) Use of citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results. ESEP 8: 93-102
37. Bornmann L (2011) Scientific peer review. Annu Rev Inform Sci 45: 199-245
38. van Noorden R (2010) Metrics: a profusion of measures. Nature 465: 864-866
39. Anon (2006) Cash-per-publication.is an idea best avoided. Nature 441: 785-786
40. Anon (2010) How to improve the use of metrics. Nature 465: 870-872
41. Bornmann L, Mutz R, Hug SE, Daniel H-D (2011) A meta-analysis of studies reporting correlations between the h index and 37 different h index variants. J Informetr 5: 346-359