Which percentile-based approach should be preferred for calculating normalized citation impact values? An empirical comparison of five approaches including a newly developed citation-rank approach (P100)

Journal of Informetrics

Volumn 7, Issue 4, 2013, Pages 933-944

  Bornmann, Lutz ^a   Leydesdorff, Loet ^b   Wang, Jian ^c,d  

^a MAX PLANCK INSTITUTE   (Germany)

^b UNIVERSITY OF AMSTERDAM   (Netherlands)

^c Institute for Research Information and Quality Assurance (iFQ)   (Germany)

^d UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

Citation impact normalization; Citation rank; P100; Percentile; Percentile rank class

Indexed keywords

EID: 84885921289     PISSN: 17511577     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.joi.2013.09.003     Document Type: Article

References (40)

1. Baumgartner S., Leydesdorff L. Group-based trajectory modeling (GBTM) of citations in scholarly literature: Dynamic qualities of Transient and Sticky Knowledge Claims. Journal of the American Society for Information Science and Technology 2013, (in press).
2. Bornmann L. How to analyse percentile citation impact data meaningfully in bibliometrics: The statistical analysis of distributions, percentile rank classes and top-cited papers. Journal of the American Society for Information Science and Technology 2013, 64(3):587-595.
3. Bornmann L. The problem of citation impact assessments for recent publication years in institutional evaluations. Journal of Informetrics 2013, 7(3):722-729. 10.1016/j.joi.2013.05.002.
4. Bornmann L. Assigning publications to multiple subject categories for bibliometric analysis: An empirical case study based on percentiles. Journal of Documentation 2013, (in press).
5. Bornmann L., Daniel H.-D. What do citation counts measure? A review of studies on citing behavior. Journal of Documentation 2008, 64(1):45-80. 10.1108/00220410810844150.
6. Bornmann L., de Moya Anegón F., Leydesdorff L. The new Excellence Indicator in the World Report of the SCImago Institutions Rankings 2011. Journal of Informetrics 2012, 6(2):333-335. 10.1016/j.joi.2011.11.006.
7. Bornmann L., Leydesdorff L., Mutz R. The use of percentiles and percentile rank classes in the analysis of bibliometric data: Opportunities and limits. Journal of Informetrics 2013, 7(1):158-165.
8. Bornmann L., Marx W. Distributions instead of single numbers: Percentiles and beam plots for the assessment of single researchers. Journal of the American Society of Information Science and Technology 2013, (in press).
9. Bornmann L., Mutz R. Further steps towards an ideal method of measuring citation performance: The avoidance of citation (ratio) averages in field-normalization. Journal of Informetrics 2011, 5(1):228-230.
10. Bornmann L., Mutz R., Marx W., Schier H., Daniel H.-D. A multilevel modelling approach to investigating the predictive validity of editorial decisions: Do the editors of a high-profile journal select manuscripts that are highly cited after publication?. Journal of the Royal Statistical Society - Series A (Statistics in Society) 2011, 174(4):857-879. 10.1111/j.1467-985X.2011.00689.x.
11. Bornmann L., Williams R. How to calculate the practical significance of citation impact differences? An empirical example from evaluative institutional bibliometrics using adjusted predictions and marginal effects. Journal of Informetrics 2013, (in press).
12. Cox N.J. Calculating percentile ranks or plotting positions 2005, Retrieved from:. http://www.stata.com/support/faqs/stat/pcrank.html.
13. Garfield E. Citation analysis as a tool in journal evaluation: Journals can be ranked by frequency and impact of citations for science policy studies. Science 1972, 178(4060):471-479.
14. Glänzel W., Thijs B., Schubert A., Debackere K. Subfield-specific normalized relative indicators and a new generation of relational charts: Methodological foundations illustrated on the assessment of institutional research performance. Scientometrics 2009, 78(1):165-188.
15. Guerrero-Bote V.P., de Moya-Anegon F. A further step forward in measuring journals' scientific prestige: The SJR2 indicator. Journal of Informetrics 2012, 6(4):674-688.
16. Hazen A. Storage to be provided in impounding reservoirs for municipal water supply. Transactions of American Society of Civil Engineers 1914, 77:1539-1640.
17. Hyndman R.J., Fan Y.N. Sample quantiles in statistical packages. American Statistician 1996, 50(4):361-365.
18. Leydesdorff L. Accounting for the uncertainty in the evaluation of percentile ranks. Journal of the American Society for Information Science and Technology 2012, 63(11):2349-2350.
19. Leydesdorff L., Bornmann L. Integrated Impact Indicators (I3) compared with Impact Factors (IFs): An alternative research design with policy implications. Journal of the American Society of Information Science and Technology 2011, 62(11):2133-2146.
20. Leydesdorff L., Bornmann L. Percentile ranks and the integrated impact indicator (I3). Journal of the American Society for Information Science and Technology 2012, 63(9):1901-1902. 10.1002/asi.22641.
21. Leydesdorff L., Bornmann L., Mutz R., Opthof T. Turning the tables in citation analysis one more time: Principles for comparing sets of documents. Journal of the American Society for Information Science and Technology 2011, 62(7):1370-1381.
22. Lundberg J. Lifting the crown - Citation z-score. Journal of Informetrics 2007, 1(2):145-154.
23. Moed H.F., De Bruin R.E., Van Leeuwen T.N. New bibliometric tools for the assessment of national research performance - Database description, overview of indicators and first applications. Scientometrics 1995, 33(3):381-422.
24. Nagin D.S. Group-based modeling of development 2005, Harvard University Press, Cambridge, MA, USA.
25. Opthof T., Leydesdorff L. Caveats for the journal and field normalizations in the CWTS (Leiden) evaluations of research performance. Journal of Informetrics 2010, 4(3):423-430.
26. Pudovkin A.I., Garfield E. Percentile rank and author superiority indexes for evaluating individual journal articles and the author's overall citation performance. Paper presented at the fifth international conference on Webometrics, Informetrics & Scientometrics (WIS) 2009.
27. Rousseau R. Basic properties of both percentile rank scores and the I3 indicator. Journal of the American Society for Information Science and Technology 2012, 63(2):416-420. 10.1002/asi.21684.
28. Schreiber M. Inconsistencies of recently proposed citation impact indicators and how to avoid them. Journal of the American Society for Information Science and Technology 2012, 63(10):2062-2073. 10.1002/asi.22703.
29. Schreiber M. Uncertainties and ambiguities in percentiles and how to avoid them. Journal of the American Society for Information Science and Technology 2013, 64(3):640-643. 10.1002/asi.22752.
30. Schreiber M. How much do different ways of calculating percentiles influence the derived performance indicators? A case study. Scientometrics 2013, (in press).
31. Schubert A., Braun T. Relative indicators and relational charts for comparative assessment of publication output and citation impact. Scientometrics 1986, 9(5/6):281-291.
32. SCImago Reseach Group SIR World Report 2012 2012, University of Granada, Granada, Spain.
33. Seglen P.O. The skewness of science. Journal of the American Society for Information Science 1992, 43(9):628-638.
34. StataCorp Stata statistical software: Release 12 2011, Stata Corporation, College Station, TX, USA.
35. van Raan A.F.J., van Leeuwen T.N., Visser M.S., van Eck N.J., Waltman L. Rivals for the crown: Reply to Opthof and Leydesdorff. Journal of Informetrics 2010, 4:431-435.
36. Waltman L., Calero-Medina C., Kosten J., Noyons E.C.M., Tijssen R.J.W., van Eck N.J., et al. The Leiden Ranking 2011/2012: Data collection, indicators, and interpretation. Journal of the American Society for Information Science and Technology 2012, 63(12):2419-2432.
37. Waltman L., Schreiber M. On the calculation of percentile-based bibliometric indicators. Journal of the American Society for Information Science and Technology 2013, 64(2):372-379.
38. Waltman L., van Eck N.J., van Leeuwen T.N., Visser M.S. Some modifications to the SNIP journal impact indicator. Journal of Informetrics 2013, 7(2):272-285. 10.1016/j.joi.2012.11.011.
39. Wang J. Citation time window choice for research impact evaluation. Scientometrics 2013, 94(3):851-872. 10.1007/s11192-012-0775-9.
40. Zhou P., Zhong Y. The citation-based indicator and combined impact indicator-New options for measuring impact. Journal of Informetrics 2012, 6(4):631-638. 10.1016/j.joi.2012.05.004.