Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level

PLoS Biology

Volumn 14, Issue 9, 2016, Pages

  Hutchins, B Ian ^a   Yuan, Xin ^a   Anderson, James M ^a   Santangelo, George M ^a  

^a NATIONAL INSTITUTES OF HEALTH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

HUMAN; JOURNAL IMPACT FACTOR; MEDICAL RESEARCH; MEDLINE; NATIONAL HEALTH ORGANIZATION; QUANTITATIVE STUDY; BIBLIOMETRICS; UNITED STATES; WRITING;

AUTHORSHIP; BIBLIOMETRICS; BIOMEDICAL RESEARCH; HUMANS; NATIONAL INSTITUTES OF HEALTH (U.S.); UNITED STATES;

EID: 84991813254     PISSN: 15449173     EISSN: 15457885     Source Type: Journal    
DOI: 10.1371/journal.pbio.1002541     Document Type: Article

References (78)

1. Couzin-Frankel J, Shaking up science. Science2013; 339: 386–389. doi: 10.1126/science.339.6118.38623349264
2. Hirsch JE, An index to quantify an individual’s scientific research output. Proc Natl Acad Sci U S A2005; 102: 16569–16572. doi: 10.1073/pnas.050765510216275915
3. Seglen PO, Why the impact factor of journals should not be used for evaluating research. BMJ1997; 314: 497–497. doi: 10.1136/bmj.314.7079.497
4. Not-so-deep impact. Nature2005; 435: 1003–1004. doi: 10.1038/4351003b
5. The maze of impact metrics. Nature2013; 502: 271–271. doi: 10.1038/502271a
6. Pulverer B, Impact fact-or fiction?EMBO2013; J 32: 1651–1652. doi: 10.1038/emboj.2013.126
7. Stallings J, Vance E, Yang J, Vannier MW, Liang J, et al. Determining scientific impact using a collaboration index. Proc Natl Acad Sci U S A2013; 110: 9680–9685. doi: 10.1073/pnas.122018411023720314
8. Garfield E, The history and meaning of the journal impact factor. JAMA2006; 295: 90–93. doi: 10.1001/jama.295.1.9016391221
9. Stringer MJ, Sales-Pardo M, Nunes Amaral LA, Effectiveness of journal ranking schemes as a tool for locating information. PLoS ONE2008; 3: e1683. doi: 10.1371/journal.pone.000168318301760
10. Misteli T, Eliminating the impact of the Impact Factor. J Cell Biol2013; 201: 651–652. doi: 10.1083/jcb.20130416223690180
11. Johnston M, We have met the enemy, and it is us. Genetics2013; 194: 791–792. doi: 10.1534/genetics.113.15348623723423
12. Moed HF, Burger WJM, Frankfort JG, Van Raan AFJ, The use of bibliometric data for the measurement of university research performance. Res Policy1985; 14: 131–149. doi: 10.1016/0048-7333(85)90012-5
13. Zitt M, Small H, Modifying the journal impact factor by fractional citation weighting: The audience factor. J Am Soc Inf Sci Technol2008; 59: 1856–1860. doi: 10.1002/asi.20880
14. Bornmann L, Leydesdorff L, The validation of advanced bibliometric indicators through peer assessments: A comparative study using data from InCites and F1000. J Informetr2013; 7: 286–291. doi: 10.1016/j.joi.2012.12.003
15. Opthof T, Leydesdorff L, Caveats for the journal and field normalizations in the CWTS (“Leiden”) evaluations of research performance. J Informetr2010; 4: 423–430. doi: 10.1016/j.joi.2010.02.003
16. van Raan AFJ, van Leeuwen TN, Visser MS, van Eck NJ, Waltman L, Rivals for the crown: Reply to Opthof and Leydesdorff. J Informetr2010; 4: 431–435. doi: 10.1016/j.joi.2010.03.008
17. Waltman L, van Eck NJ, van Leeuwen TN, Visser MS, van Raan AFJ, Towards a new crown indicator: Some theoretical considerations. J Informetr2011; 5: 37–47. doi: 10.1016/j.joi.2010.08.001
18. Waltman L, Yan E, van Eck NJ, A recursive field-normalized bibliometric performance indicator: an application to the field of library and information science. Scientometrics2011; 89: 301–314. doi: 10.1007/s11192-011-0449-z21957321
19. Schubert A, Braun T, Relative indicators and relational charts for comparative assessment of publication output and citation impact. Scientometrics1986; 9: 281–291. doi: 10.1007/BF02017249
20. Bornmann L, Marx W, How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations. Scientometrics2013; 98: 487–509. doi: 10.1007/s11192-013-1161-y
21. Bergstrom CT, West JD, Wiseman MA, The Eigenfactor metrics. J Neurosci2008; 28: 11433–11434. doi: 10.1523/JNEUROSCI.0003-08.200818987179
22. Bergstrom CT, West JD, Assessing citations with the Eigenfactor metrics. Neurology2008; 71: 1850–1851. doi: 10.1212/01.wnl.0000338904.37585.6619047558
23. Moed HF, Measuring contextual citation impact of scientific journals. J Informetr2010; 4: 265–277. doi: 10.1016/j.joi.2010.01.002
24. Bollen J, Van de Sompel H, Hagberg A, Chute R, A principal component analysis of 39 scientific impact measures. PLoS ONE2009; 4: e6022. doi: 10.1371/journal.pone.000602219562078
25. Waltman L, van Eck NJ, van Leeuwen TN, Visser MS, Some modifications to the SNIP journal impact indicator. J Informetr2013; 7: 272–285. doi: 10.1016/j.joi.2012.11.011
26. Waltman L, van Eck NJ, A systematic empirical comparison of different approaches for normalizing citation impact indicators. J Informetr2013; 7: 833–849. doi: 10.1016/j.joi.2013.08.002
27. Wang D, Song C, Barabási A-L, Quantifying long-term scientific impact. Science2013; 342: 127–132. doi: 10.1126/science.123782524092745
28. Radicchi F, Fortunato S, Castellano C, Universality of citation distributions: toward an objective measure of scientific impact. Proc Natl Acad Sci U S A2008; 105: 17268–17272. doi: 10.1073/pnas.080697710518978030
29. Stringer MJ, Sales-Pardo M, Nunes Amaral LA, Statistical validation of a global model for the distribution of the ultimate number of citations accrued by papers published in a scientific journal. J Am Soc Inf Sci Technol JASIST2010; 61: 1377–1385. doi: 10.1002/asi.2133521858251
30. Walker D, Xie H, Yan K-K, Maslov S, Ranking scientific publications using a model of network traffic. J Stat Mech Theory Exp2007: P06010–P06010. doi: 10.1088/1742-5468/2007/06/P06010
31. Crous CJ, Judge research impact on a local scale. Nature2014; 513: 7. doi: 10.1038/513007a25186869
32. Radicchi F, Castellano C, Testing the fairness of citation indicators for comparison across scientific domains: The case of fractional citation counts. J Informetr2012; 6: 121–130. doi: 10.1016/j.joi.2011.09.002
33. Jeong H, Néda Z, Barabási AL, Measuring preferential attachment in evolving networks. Europhys Lett2003; 61: 567–572. doi: 10.1209/epl/i2003-00166-9
34. Eom Y-H, Fortunato S, Characterizing and modeling citation dynamics. PLoS ONE2011; 6: e24926. doi: 10.1371/journal.pone.002492621966387
35. Small H, Co-citation in the scientific literature: A new measure of the relationship between two documents. J Am Soc Inf Sci1973; 24: 265–269. doi: 10.1002/asi.4630240406
36. Mir R, Karim S, Kamal MA, Wilson CM, Mirza Z, Conotoxins: Structure, Therapeutic Potential and Pharmacological Applications. Curr Pharm Des2016; 22: 582–589. doi: 10.2174/138161282266615112423471526601961
37. Lewis J, Ossowski S, Hicks J, Errami M, Garner HR, Text similarity: an alternative way to search MEDLINE. Bioinformatics2006; 22: 2298–2304. doi: 10.1093/bioinformatics/btl38816926219
38. Breiman L, Random Forests. Mach Learn2001; 45: 5–32. doi: 10.1023/A:1010933404324
39. Rousseau R, Leydesdorff L, Simple arithmetic versus intuitive understanding: The case of the impact factor. ISSI Newsl2011: 10–14.
40. Glänzel W, Moed HF, Opinion paper: thoughts and facts on bibliometric indicators. Scientometrics2012; 96: 381–394. doi: 10.1007/s11192-012-0898-z
41. Bornmann L, Haunschild R, Relative Citation Ratio (RCR): A first empirical attempt to study a new field-normalized bibliometric indicator. J Assoc Inf Sci Technol2015; In Press. doi: 10.1002/asi.23729
42. Ke Q, Ferrara E, Radicchi F, Flammini A, Defining and identifying Sleeping Beauties in science. Proc Natl Acad Sci U S A2015; 112: 7426–7431. doi: 10.1073/pnas.142432911226015563
43. Hicks D, Wouters P, Waltman L, de Rijcke S, Rafols I, Bibliometrics: The Leiden Manifesto for research metrics. Nature2015; 520: 429–431. doi: 10.1038/520429a25903611
44. Ribas S, Ueda A, Santos RLT, Ribeiro-Neto B, Ziviani N. Simplified Relative Citation Ratio for Static Paper Ranking: UFMG/LATIN at WSDM Cup 2016. http://arxiv.org/pdf/1603.01336v1.pdf
45. Page L, Brin S, Motwani R, Winograd T. The PageRank Citation Ranking: Bringing Order to the Web. 1999. http://ilpubs.stanford.edu:8090/422/
46. Chen P, Xie H, Maslov S, Redner S, Finding scientific gems with Google’s PageRank algorithm. J Informetr2007; 1: 8–15. doi: 10.1016/j.joi.2006.06.001
47. Yan E, Ding Y, Discovering author impact: A PageRank perspective. Inf Process Manag2011; 47: 125–134. doi: 10.1016/j.ipm.2010.05.002
48. Leydesdorff L, Bornmann L, The operationalization of “fields” as WoS subject categories (WCs) in evaluative bibliometrics: The cases of “library and information science” and “science & technology studies”. J Assoc Inf Sci Technol2015; 67: 707–714. doi: 10.1002/asi.23408
49. Price DDS, A general theory of bibliometric and other cumulative advantage processes. J Am Soc Inf Sci1976; 27: 292–306. doi: 10.1002/asi.4630270505
50. Sullivan MX, Synthetic Culture Media and the Biochemistry of bacterial Pigments. J Med Res1905; 14: 109–160. 19971689
51. Institute of Medicine NA of S and NA of EFacilitating Interdisciplinary Research. Washington, D.C.: National Academies Press2004. doi: 10.17226/11153
52. Huerta MF, Farber GK, Wilder EL, Kleinman D V, Grady PA, et al. NIH Roadmap interdisciplinary research initiatives. PLoS Comput Biol2005; 1: e59. doi: 10.1371/journal.pcbi.001005916322762
53. Mansilla R, Köppen E, Cocho G, Miramontes P, On the behavior of journal impact factor rank-order distribution. J Informetr2007; 1: 155–160. doi: 10.1016/j.joi.2007.01.001
54. Egghe L, Mathematical derivation of the impact factor distribution. J Informetr2009; 3: 290–295. doi: 10.1016/j.joi.2009.01.004
55. Moreira JAG, Zeng XHT, Amaral LAN, The Distribution of the Asymptotic Number of Citations to Sets of Publications by a Researcher or from an Academic Department Are Consistent with a Discrete Lognormal Model. PLoS ONE2015; 10: e0143108. doi: 10.1371/journal.pone.014310826571133
56. Casadevall A, Fang FC, Causes for the persistence of impact factor mania. MBio2014; 5: e00064–14. doi: 10.1128/mBio.00064-1424643863
57. Shaw D, The prisoners’ dilemmas: Authorship guidelines and impact factors: between a rock and a hard place. EMBO Rep2014; 15: 635–637. doi: 10.1002/embr.20133834824781693
58. Bertuzzi S, Drubin DG, No shortcuts for research assessment. Mol Biol Cell2013; 24: 1505–1506. doi: 10.1091/mbc.E13-04-019323686606
59. Suhrbier A, Poland GA, Are Impact Factors corrupting truth and utility in biomedical research?Vaccine2013; 31: 6041–6042. doi: 10.1016/j.vaccine.2013.09.07624184289
60. Alberts B, Impact factor distortions. Science2013; 340: 787. doi: 10.1126/science.124031923687012
61. Schekman R, Patterson M, Reforming research assessment. Elife2013; 2: e00855. doi: 10.7554/eLife.0085523700504
62. Vinkler P, Relations of relative scientometric indicators. Scientometrics2003; 58: 687–694. doi: 10.1023/B:SCIE.0000006888.69146.24
63. Rand DG, Pfeiffer T, Systematic differences in impact across publication tracks at PNAS. PLoS ONE2009; 4: e8092. doi: 10.1371/journal.pone.000809219956649
64. Wilhite AW, Fong EA, Scientific publications. Coercive citation in academic publishing. Science2012; 335: 542–543. doi: 10.1126/science.121254022301307
65. Krell F-T, Should editors influence journal impact factors?Learn Publ2010; 23: 59–62. doi: 10.1087/20100110
66. Salganik MJ, Dodds PS, Watts DJ, Experimental study of inequality and unpredictability in an artificial cultural market. Science2006; 311: 854–856. doi: 10.1126/science.112106616469928
67. Liao H, Xiao R, Cimini G, Medo M, Network-driven reputation in online scientific communities. PLoS ONE2014; 9: e112022. doi: 10.1371/journal.pone.011202225463148
68. Danthi N, Wu CO, Shi P, Lauer M, Percentile ranking and citation impact of a large cohort of National Heart, Lung, and Blood Institute-funded cardiovascular R01 grants. Circ Res2014; 114: 600–606. doi: 10.1161/CIRCRESAHA.114.30265624406983
69. Kaltman JR, Evans FJ, Danthi NS, Wu CO, DiMichele DM, et al. Prior publication productivity, grant percentile ranking, and topic-normalized citation impact of NHLBI cardiovascular R01 grants. Circ Res2014; 115: 617–624. doi: 10.1161/CIRCRESAHA.115.30476625214575
70. Mervis J, Peering into peer review. Science2014; 343: 596–598. doi: 10.1126/science.343.6171.59624503830
71. DePellegrin TA, Johnston M, An Arbitrary Line in the Sand: Rising Scientists Confront the Impact Factor. Genetics2015; 201: 811–813. doi: 10.1534/genetics.115.18226126564899
72. Contopoulos-Ioannidis DG, Alexiou GA, Gouvias TC, Ioannidis JPA, Medicine. Life cycle of translational research for medical interventions. Science2008; 321: 1298–1299. doi: 10.1126/science.116062218772421
73. Weber GM, Identifying translational science within the triangle of biomedicine. J Transl Med2013; 11: 126. doi: 10.1186/1479-5876-11-12623705970
74. Garfield E, Citation analysis as a tool in journal evaluation. Science1972; 178: 471–479. 5079701
75. Hutchins BI, Yuan X, Anderson JM, Santangelo GM, Relative Citation Ratio (RCR): A new metric that uses citation rates to measure influence at the article level. bioRxiv2015; 029629. doi: 10.1101/029629
76. Rose S, Engel D, Cramer N, Cowley W, Berry MW, Kogan J, Autometic keyword extraction from individual documents. In: Text Mining: Applications and Theory. Chichester, UK: John Wiley & Sons, Ltd; 2010.
77. Feinerer I, Hornik K, Meyer D, Text Mining Infrastructure in R. J Stat Softw2008; 25: 1–54. doi: 10.18637/jss.v025.i05
78. Rossner M, Van Epps H, Hill E, Show me the data. J Cell Biol2007; 179: 1091–1092. doi: 10.1083/jcb.20071114018086910