The use and misuse of journal metrics and other citation indicators

Archivum Immunologiae et Therapiae Experimentalis

Volumn 57, Issue 1, 2009, Pages 1-11

  Pendlebury, David A ^a  

^a THOMSON REUTERS   (United States)

Author keywords

Bibliometric indicators; Citation analysis; Impact factor; Peer review

Indexed keywords

ACCURACY; ALGORITHM; ARTIFACT; BIBLIOMETRICS; CITATION ANALYSIS; HEALTH CARE POLICY; HEALTH SERVICE; JOURNAL IMPACT FACTOR; MATHEMATICS; MEDICAL PRACTICE; MEDICAL RESEARCH; NEUROSCIENCE; PEER REVIEW; PERFORMANCE; PRACTICE GUIDELINE; PUBLICATION; QUALITY CONTROL; QUANTITATIVE ANALYSIS; REVIEW; SCIENCE; SOCIOLOGY; DOCUMENTATION; PUBLISHING; STANDARD; STATISTICS;

ABSTRACTING AND INDEXING AS TOPIC; BIBLIOMETRICS; JOURNAL IMPACT FACTOR; MATHEMATICS; NEUROSCIENCES; PEER REVIEW, RESEARCH; PUBLISHING; SCIENCE; SOCIAL SCIENCES;

EID: 59849096849     PISSN: 0004069X     EISSN: 16614917     Source Type: Journal    
DOI: 10.1007/s00005-009-0008-y     Document Type: Review

References (68)

1. Banks MA, Dellavalle R (2008) Emerging alternatives to the impact factor. OCLC Systems & Services 24(3). Available via: http://eprints.rclis.org/ archive/00014614/01/OCLC%2BPaper%2BFinal-v2Feb2008.pdf
2. st century: a review. J Informetrics 2:1-52
3. Bensman SJ (2007a) Garfield and the impact factor. Annu Rev Inf Sci Technol 41:93-155
4. Bensman SJ (2007b) The impact factor, total citations, and better citation mouse traps: a commentary. J Am Soc Inf Sci Technol 58:1904-1908
5. Bergstrom CT (2007) Eigenfactor: measuring the value and prestige of scholarly journals. C&RL News 68:5. Available via: http://www.ala.org/ala/ acrl/acrlpubs/crlnews/backissues2007/may07/eigenfactor.cfm, also see: http://www.eigenfactor.org//whyeigenfactor.htm
6. Bollen J, Rodriguez MA, Van de Sompel H (2006) Journal status. Scientometrics 69:669-687
7. Borgman C, Furner J (2002) Scholarly communication and bibliometrics. Annu Rev Inf Sci Technol 36:3-72
8. Bornmann L, Mutz R, Neuhaus C et al (2008) Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics Sci. Environmental Politics 8:93-102 (Pubitemid 351836496)
9. Braun T, Glanzel W, Schubert A (2005) A Hirsch-type index for journals. Scientist 19:8
10. Brin S, Page L (1998) The anatomy of a large-scale hypertextual web search engine. Comput Networks ISDN Systems 30:107-117 (Pubitemid 128666929)
11. Butler L (2008) Using a balanced approach to bibliometrics: quantitative performance measures in the Australian Research Quality Framework. Ethics Sci Environmental Politics 8:83-92.
12. Egghe L, Rousseau R (2002) A general framework for relative impact factors. Can J Inf Library Sci 21:29-48 (Pubitemid 36769577)
13. Egghe L (2005) Continuous, weighted Lorenz theory and applications to the study of fractional relative impact factors. Inf Processing Management 41:1330-1359
14. Egghe L (2006) Theory and practice of the g-index. Scientometrics 69:131-152
15. Falagas ME, Kouranos VD, Arencibia-Jorge R et al (2008) Comparison of SCImago journal rank indicator with journal impact factor. FASEB J 22:2623-2628
16. Garfield E (1955) Citation indexes for science. A new dimension in documentation through association of ideas. Science 122:108-111
17. Garfield E (1972) Citation analysis as a tool in journal evaluation. Science 178:471-479
18. Garfield E (1976) Significant journals of science. Nature 264:609-615
19. Garfield E (1979) Citation Indexing: Its Theory and Application in Science, Technology, and Humanities. John Wiley & Sons, New York
20. Garfield E (1996) How can impact factors be improved? BMJ 313:411-413
21. Garfield E (1999) Journal impact factor: a brief review. CMAJ 161:979-980
22. Garfield E (2006) The history and meaning of the journal impact factor. JAMA 295:90-93
23. Garfield E, Sher IH (1963) New factors in the evaluation of scientific literature through citation indexing. Am Documentation 14:195-201
24. Glanzel W (2008) Seven myths in bibliometrics: about facts and fiction in quantitative science studies. Proceedings of the WIS 2008, Berlin. Fourth International Conference on Webometrics, Informetrics and Scientometrics & Ninth COLLNET Meeting. Available via: http://www.collnet.de//Berlin-2008/ GlanzelWIS2008smb.pdf
25. Glanzel W, Moed HF (2002) Journal impact measures in bibliometric research. Scientometrics 53:171-193
26. Harnad S (2008) Validating research performance metrics against peer rankings. Ethics Sci. Environmental Politics 8:103-107
27. Harzing A-WK, van der Wal R (2008) Google Scholar as a new source for citation analysis. Ethics Sci. Environmental Politics 8:61-73. Available via: http://www.harzing.com/resources.htm#/pop-hindex.htm
28. Hirsch JE (2005) An index to quantify an individual's scientific research output. Proc Natl Acad Sci U S A 102:16569-16572
29. Hirsch JE (2007) Does the h-index have predictive power? Proc Natl Acad Sci U S A 104:19193-19198
30. Jasco P (2005) As we may search: Comparison of major features of the Web of Science, Scopus, and Google Scholar citation-based and citation-enhanced databases. Curr Sci 89:1537-1547
31. Joint Committee on Quantitative Assessment of Research (2008) Citation Statistics. A report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Institute of Mathematical Statistics (IMS), by Robert Adler, John Ewing (Chair), and Peter Taylor. Available via: http://www.mathunion.org/ fileadmin/IMU//Report/CitationStatistics.pdf
32. Ketcham CM, Crawford JM (2007) The impact of review articles. Lab Invest 87:1174-1185
33. Krauze T (1977) The sociology of science in Poland. In: Merton RK, Gaston J (eds) The Sociology of Science in Europe. Southern Illinois University Press, Carbondale, Illinois, pp. 193-223
34. Leydesdorff L (2008) Caveats in the use of citation indicators in research and journal evaluations. J Am Soc Inf Sci Technol 59:278-287
35. Markpin T, Boonradsamee B, Ruksinsut K et al (2008) Article-count impact factor of materials science journals in SCI database. Scientometrics 75:251-261
36. Meho LI, Yang K (2007) A new era in citation and bibliometric analyses: Web of Science. Scopus, and Google Scholar. J Am Soc Inf Sci Technol 58:2105-2125
37. Moed HF (2002) The impact-factors debate: the ISI's uses and limits. Nature 415:731-732
38. Moed HF (2005a) Citation analysis of scientific journals and journal impact measures. Curr Sci 89:1990-1996
39. Moed HF (2005b) Citation analysis in research evaluation. Springer, Dordrecht
40. Moed HF, Glanzel W, Schmoch U (eds) (2004) Handbook of quantitative science and technology research. The use of publication and patent statistics in studies of S&T systems. Kluwer Academic Publishers, Dordrecht
41. Moed HF, van Leeuwen TN (1995) Improving the accuracy of Institute for Scientific Information's journal impact factors. J Am Soc Inf Sci 46:461-467
42. Moed HF, van Leeuwen TN (1996) Impact factors can mislead. Nature 381:186
43. Moed HF, van Leeuwen TN, Reedijk J (1996) A critical analysis of the journal impact factors of Angewandte Chemie and Journal of the American Chemical Society: inaccuracies in published impact factors based on overall citation counts only. Scientometrics 37:105-116
44. Moed HF, van Leeuwen TN, Reedijk J (1999) Towards appropriate indicators of journal impact. Scientometrics 46:575-589
45. Monastersky R (2005) The number that's devouring science. The Chronicle of Higher Education 52:A12.
46. Narin F (1976) Evaluative bibliometrics: the use of publication and citation analysis in the evaluation of scientific activity. Computer Horizons Inc., Cherry Hill, New Jersey
47. Nicolaisen J (2007) Citation analysis. Annu Rev Inf Sci Technol 41:609-641
48. Ossowska M, Ossowski S (1936) The science of science. Organon 1:1-12
49. Pinski G, Narin F (1976) Citation influence for journal aggregates of scientific publications: theory, with application to the literature of physics. Inf Processing Management 12:297-312
50. th ASIS&T Annual Meeting 41:507-515
51. Rousseau R (2002) Journal evaluation: technical and practical issues. Library Trends 50:418-439
52. Rousseau R (2005) Median and percentile impact factors. Scientometrics 63:431-441
53. Seglen PO (1997) Why the impact factor of journals should not be used for evaluating research. BMJ 314:498-502
54. Sombatsompop N, Markpin T (2005) Making an equality of ISI impact factors for different subject fields. J Am Soc Inf Sci Technol 56:676-683 (Pubitemid 40751658)
55. Sombatsompop N, Markpin T, Premkamolnetr N (2004) A modified method for calculating the impact factors of the journals in ISI Journal Citation Reports: polymer science category in 1997-2001. Scientometrics 60:217-235
56. Sombatsompop N, Markpin T, Yochai W et al (2005) An evaluation of research performance for different subject categories using impact factor point average (IFPA) index: Thailand case study. Scientometrics 65:293-305 (Pubitemid 41433504)
57. Stringer MJ, Sales-Pardo M, Nunes Amaral LA (2008) Effectiveness of journal ranking schemes as a tool for locating information. PLoS One 3:e1683. Available via: http://www.plosone.org/article/info:doi%2F10.1371%2Fjournal.pone. 0001683
58. Thomson Reuters, http://scientific.thomson.com/products/jcr/
59. Thomson Reuters, http://scientific.thomsonreuters.com/products/jpi/
60. Thomson Reuters, http://scientific.thomson.com/products/esi/
61. Vaidya JS (2005) V-index: a fairer index to quantify an individual's research output capacity. BMJ Rapid Response. Available via: http://bmj.bmjjournals.com/cgi/eletters/331//7528/1339-c#123188
62. van Leeuwen TN, Moed HF (2002) Development and application of journal impact measures in the Dutch science system. Scientometrics 53:249-266
63. van Leeuwen TN, Moed HF (2005) Characteristics of journal impact factors: the effects of uncitedness and citation distribution on the understanding of journal impact factors. Scientometrics 63:357-371
64. van Leeuwen TN, Moed HF, Reedijk J (1999) Critical comments on Institute for Scientific Information impact factors: a sample of inorganic molecular chemistry journals. J Inf Sci 25:489-498
65. Vinkler P (2004) Characterization of the impact of sets of scientific papers: the Garfield (impact) factor. J Am Soc Inf Sci Technol 55:431-435
66. Wouters P (1999) The Citation Culture. University of Amsterdam, Amsterdam, pp. 82-83
67. Zitt M, Bassecoulard E (2008) Challenges for scientometric indicators: data demining, knowledge-flow measurements and diversity issues. Ethics Sci. Environmental Politics 8:49-60 (Pubitemid 351796585)
68. Zitt M, Small H (2008) Modifying the journal impact factor by fractional citation weighting: the audience factor. J Am Soc Inf Sci Technol 59:1856-1860