Caveats for the use of citation indicators in research and journal evaluations

Journal of the American Society for Information Science and Technology

Volumn 59, Issue 2, 2008, Pages 278-287

  Leydesdorff, Loet ^a  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

FORMAL LANGUAGES; INFORMATION RETRIEVAL; NATURAL SCIENCES COMPUTING;

CITATION INDICATORS; DELINEATION; JOURNAL EVALUATIONS;

ELECTRONIC PUBLISHING;

EID: 38849110046     PISSN: 15322882     EISSN: 15322890     Source Type: Journal    
DOI: 10.1002/asi.20743     Document Type: Article

References (90)

1. Adams, J., & Smith, D. (2003). Funding research diversity. Leeds: Evidence Ltd, available at http://www.evidence.co.uk/downloads/funding_tech.pdf Accessed on 27 March 2007.
2. Ahlgren, P., Jarneving, B., & Rousseau, R. (2003). Requirement for a cocitation similarity measure, with special reference to Pearson's correlation coefficient. Journal of the American Society for Information Science and Technology, 54(6), 550-560.
3. Anderson, J., Collins, P.M.D., Irvine, J., Isard, P.A., Martin, B.R., Narin, F., et al. (1988). On-line approaches to measuring national scientific output - A cautionary tale. Science and Public Policy 15, 153-161.
4. Ashby, R. (1958). Requisite variety and its implications for the control of complex systems. Cybernetica, 1(2), 1-17.
5. Bar-Ilan, J. (2006). H-index for Price medalists revisited. ISSI Newsletter, 2(1), 3-5.
6. Bensman, S.J. (2007). Garfield and the impact factor. Annual Review of Information Science and Technology, 41, 93-155.
7. Bensman, S.J. & Wilder, S.J. (1998). Scientific and technical serials holdings optimization in an inefficient market: A LSU Serials Redesign Project Exercise. Library Resources and Technical Services, 42(3), 147-242.
8. Bonaccorsi, A. (2005). Search regimes and the industrial dynamics of science. Paper presented at the PRIME General Assembly, Manchester, UK, 7-9 January 2005.
9. Boyack, K.W., Klavans, R., & Börner, K. (2005). Mapping the backbone of science. Scientometrics, 64(3), 351-374.
10. Bornmann, L., Leydesdorff, L., & Marx, W. (2007). Citation environment of Angewandte Chemie. CHIMIA, 61(3), 104-109.
11. Braam, R. (2007). Everything about genes: Some results on the dynamics of genomics research. In D. Torres-Salinas & H. Moed (Eds.), Proceedings of the 11th International Conference of Scientometrics and Informetrics, CSIC, Madrid, 21-25 June 2007, Vol. 1, 136-148.
12. Bradford, S.C. (1934). Sources of information on specific subjects. Engineering, 137, 85-86.
13. Braun, T., Glänzet, W., & Schubert, A. (2006). A Hirsch-type index for journals. Scientometrics, 69(1), 169-173.
14. Bruckner, E., Ebeling, W., Montaño, M.A.J., & Scharnhorst, A. (1994). Hyperselection and innovation described by a stochastic model of technological evolution. In L. Leydesdorff & P. Van den Besselaar (Eds.), Evolutionary economics and chaos theory: New directions in technology studies (pp. 79-90). London: Pinter.
15. Burrell, Q.L. (2007). Hirsch index or Hirsch rate? Some thoughts arising from Liang's data. Scientometrics, 73(1), 19-28.
16. Christensen, F.H., Ingwersen, P., & Wormell, I. (1997). Online determination of the Journal impact factor and its international properties. Scientometrics, 40(3), 529-540.
17. Collins, H.M. (1985). The possibilities of science policy. Social Studies of Science, 15, 554-558.
18. Egghe, L. (2006a). How to improve the h-index. The Scientist, 20(3), 315-321.
19. Egghe, L. (2006b). An improvement of the H-index: The G-index. ISSI Newsletter, 2(1), 8-9.
20. Egghe, L. & Rousseau, R. (1990). Introduction to informetrics. Amsterdam: Elsevier.
21. Frandsen, T.F. & Rousseau, R. (2005). Article impact calculated over arbitrary periods. Journal of the American Society for Information Science and Technology, 56(1), 58-62.
22. Freeman, L.C. (1977). A set of measures of centrality based on betweenness. Sociometry, 40(1), 35-41.
23. Fujigaki, Y. (1998). Filling the gap between discussions on science and scientists' everyday activities: Applying the autopoiesis system theory to scientific knowledge. Social Science Information, 37(1), 5-22.
24. Garfield, E. (1971). The mystery of the transposed journal lists - wherein Bradford's Law of Scattering is generalized according to Garfield's Law of Concentration. Current Contents, 3(33), 5-6.
25. Garfield, E. (1972). Citation analysis as a tool in journal evaluation. Science 178(4060), 471-479.
26. Garfield, E. (1979). Citation indexing: Its theory and application in science, technology, and humanities. New York: John Wiley & Sons.
27. Garfield, E. (Ed.) (1980). SCI journal citation reports: A bibliometric analysis of science journals in the ISI data base. Science Citation Index 1979 annual, v. 14. Philadelphia: Institute for Scientific Information.
28. Garfield, E. (1990). How the ISI selects journals for coverage: Quantitative and qualitative considerations. Current Contents (28 May), 5-13.
29. Garfield, E. (1996). How can impact factors be improved? British Medical Journal, 313(7054), 411-413.
30. Garfield, E. (1998a). Long-term vs. short-term journal impact: Does it matter? The Scientist, 12(3), 11-12.
31. Garfield, E. (1998b). The multiple meanings of impact factors. Journal of the American Society for Information Science, 49(8), 768.
32. Garfield, E. (2003). The meaning of the impact factor. Revista Internacional de Psicologia Clinica y de la Salud, 3(2), 363-369.
33. Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P., & Trow, M. (1994). The new production of knowledge: the dynamics of science and research in contemporary societies. London: Sage.
34. Glänzel, W. & Schubert, A. (2003). A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics, 56(3), 357-367.
35. Granovetter, M.S. (1973). The strength of weak ties. American Journal of Sociology, 78(6), 1360-1380.
36. Hanneman, R.A. & Riddle, M. (2005). Introduction to social network methods. Riverside: University of California, Riverside; available at http://www.faculty.ucr.edu/~hanneman/ Accessed on 19 February 2007.
37. Hellsten, I., Lambiotte, R., Scharnhorst, A., & Ausloos, M. (2007). Self-citations, co-authorships and keywords: A new approach to scientists' field mobility? Scientometrics, 72(3), 469-486.
38. Hicks, D. (1999). The difficulty of achieving full coverage of international social science literature and the bibliometric consequences. Scientometrics, 44(2), 193-215.
39. Hirsch, J.E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences of the USA, 102(46), 16569-16572.
40. Jin, B.H. (2006). H-index: An evaluation indicator proposed by scientist. Science Focus, 1(1), 8-9.
41. Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007). The R- and AR-indices: Complementing the h-index. Chinese Science Bulletin, 52(6), 855-863.
42. King, D.A. (2004). The scientific impact of nations. Nature, 430(15 July 2004), 311-316.
43. Kosmulski, M. (2006). A new Hirsch-type index saves time and works equally well as the original h-index. ISSI Newsletter, 2(3), 4-6.
44. Kreft, G.G. & De Leeuw, E.D. (1988). The see-saw effect: A multilevel problem. Quality and Quantity, 22(2), 127-137.
45. Leydesdorff, L. (1998). Theories of citation? Scientometrics, 43(1), 5-25.
46. Leydesdorff, L. (2006a). Can scientific journals be classified in terms of aggregated journal-journal citation relations using the journal citation reports? Journal of the American Society for Information Science & Technology, 57(5), 601-613.
47. Leydesdorff, L. (2006b). The knowledge-based economy: modeled, measured, simulated. Boca Raton, FL: Universal Publishers.
48. Leydesdorff, L. (2007a). Visualization of the citation impact environments of scientific journals: An online mapping exercise. Journal of the American Society of Information Science and Technology, 58(1), 207-222.
49. Leydesdorff, L. (2007b). Mapping interdisciplinarity at the interfaces between the science citation index and the social science citation index. Scientometrics, 71(3), 391-405.
50. Leydesdorff, L. (2007c). "Betweenness centrality" as an indicator of the "interdisciplinarity" of scientific journals. Journal of the American Society for Information Science and Technology, 58(9), 1303-1309.
51. Leydesdorff, L. & Van der Schaar, P. (1987). The use of scientometric indicators for evaluating national research programmes. Science & Technology Studies, 5, 22-31.
52. Leydesdorff, L. & Amsterdamska, O. (1990). Dimensions of citation analysis. Science, Technology & Human Values, 15, 305-335.
53. Leydesdorff, L. & Van den Besselaar, P. (1997). Scientometrics and communication theory: Towards theoretically informed indicators. Scientometrics, 38, 155-174.
54. Leydesdorff, L. & Jin, B. (2005). Mapping the Chinese science citation database in terms of aggregated journal-journal citation relations. Journal of the American Society for Information Science and Technology, 56(14), 1469-1479.
55. Leydesdorff, L. & Wagner, C.S. (2007). Is the United States losing ground in science? A global perspective on the world science system. Scientometrics (forthcoming).
56. Leydesdorff, L. & Zhou, P. (2005). Are the contributions of China and Korea upsetting the world system of science? Scientometrics, 63(3), 617-630.
57. Li, Z.X. (2006). How to establish a first-class international scientific journal in China? World Journal of Gastroenterology, 12(43), 6905-6908.
58. Maricic, S. (1997). The mainstream-peripheral science communication. Technoscience, 10 (Winter 1997), available at http://cybermetrics.cindoc.csic. es/cybermetrics/pdf/394.pdf Accessed on 19 February 2007.
59. Martin, B. & Irvine, J. (1983). Assessing basic research: Some partial indicators of scientific progress in radio astronomy. Research Policy, 12, 61-90.
60. Martyn, J. & Gilchrist, A. (1968). An evaluation of British scientific periodicals. Aslib Occasional Publication No. 1. London: Aslib.
61. Marx, W. (2001). Angewandte Chemie in light of the science citation index. Angewandte Chemie International Edition, 40(1), 139-143.
62. Meho, L.I. & Yang, K. (forthcoming). A new era in citation and bibliometric analyses: Web of Science, Scopus, and Google Scholar. Journal of the American Society for Information Science and Technology.
63. Moed, H.F. (2005). Citation analysis in research evaluation. Dordrecht: Springer.
64. Moed, H.F., Burger, W.J.M., Frankfort, J.G., & Van Raan, A.F.J. (1985). The use of bibliometric data for the measurement of university research performance. Research Policy, 14, 131-149.
65. Moravcsik, M.J. (1985). Applied scientometrics: An assessment methodology for developing countries. Scientometrics, 7(3), 165-176.
66. National Research Council (2007). Research doctorate programs. The National Academies, Washington, DC; available at http://www7.nationalacademies. org/resdoc/index.html Accessed on 10 February 2007.
67. Nederhof, A.J., Zwaan, R.A., Bruin, R.E., & Dekker, P.J. (1989). Assessing the usefulness of bibliometric indicators for the humanities and the social sciences: A comparative study. Scientometrics, 15, 423-436.
68. Nowotny, H., Scott, P., & Gibbons, M. (2001). Re-thinking science: knowledge and the public in an age of uncertainty. Cambridge: Polity.
69. Park, H.W., Hong, H.D., & Leydesdorff, L. (2005). A comparison of the knowledge-based innovation systems in the economies of South Korea and the Netherlands using triple helix indicators. Scientometrics, 65(1), 3-27.
70. Park, H.W., & Leydesdorff, L. (2007). Korean journals in the Science Citation Index: What do they reveal about the intellectual structure of S&T in Korea? Scientometrics (forthcoming).
71. Pinski, G. & Narin, F. (1976). Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics. Information Processing and Management, 12(5), 297-312.
72. Price, D.J. de Solla (1965). Networks of scientific papers. Science, 149, 510-515.
73. Price, D.J. de Solla (1970). Citation measures of hard science, soft science, technology, and nonscience. In C.E. Nelson & D.K. Pollock (Eds.), Communication among scientists and engineers (pp. 3-22). Lexington, MA: Heath.
74. Rafols, I. & Meyer, M. (2007). How cross-disciplinary is bionanotechnology? Explorations in the specialty of molecular motors. Scientometrics, 70(3), 633-650.
75. Rousseau, R. (2004). Impact factors and databases as instruments for research evaluation. Paper presented at the 4th International Conference on University Evaluation and Research Evaluation, Wuhan University (China), September 27-October 1, 2004, Available at http://www.vub.ac.be/BIB-LIO/itp/ lecturers/ronald_rousseau/ronald_roussea_stim4_impact_factors_ as_instruments.pdf Accessed on 19 March 2007.
76. Rousseau, R. (2006a). Simple models and the corresponding h- and g-index. Science Technology Development, 2006 (forthcoming).
77. Rousseau, R. (2006b). Timelines in citation research. Journal of the American Society for Information Science and Technology, 57(10), 1404-1405.
78. Salton, G., & McGill, M.J. (1983). Introduction to modern information retrieval. Auckland: McGraw-Hill.
79. Seglen, P.O. (1997). Why the impact factor of journals should not be used for evaluating research. British Medical Journal, 314, 498-502.
80. Sher, I.H. & Garfield, E. (1965). New tools for improving and evaluating the effectiveness of research. Second conference on Research Program Effectiveness, Washington, DC July, 27-29.
81. Stegmann, J. (1997). How to evaluate journal impact factors. Nature, 390, 550.
82. Stegmann, J. (1999). Building a list of journals with constructed impact factors. Journal of Documentation, 55(3), 310-324.
83. Studer, K.E. & Chubin, D.E. (1980). The cancer mission. Social contexts of biomedical research. Beverly Hills, CA: Sage.
84. Testa, J. (1997). The ISI database: The journal selection process (Philadelphia: Thomson-ISI). Available at http://scientific.thomson.com/free/ essays/selectionofmaterial/journalselection/Accessed on 19 March 2007.
85. Tijssen, R.J.W. & Van Leeuwen, T.N. (1995). On generalising scientometric journal mapping beyond the ISI's journal and citation databases. Scientometrics, 33(1), 93-116.
86. Van den Besselaar, P. (2001). The cognitive and the social structure of science & technology studies. Scientometrics, 51, 441-460.
87. Von Tunzelmann, N., Ranga, M., Martin, B., & Geuna, A. (2003). The effects of size on research performance: A SPRU review. Available at http://www.sussex.ac.uk/spru/documents/geunaost.pdf Accessed on 27 March 2007.
88. Whitley, R.D. (1984). The intellectual and social organization of the sciences. Oxford: Oxford University Press.
89. Zhou, P. & Leydesdorff, L. (2007a). A comparison between the China Scientific and Technical Papers and Citations Database and the Science Citation Index in terms of journal hierarchies and inter-journal citation relations. Journal of the American Society for Information Science and Technology 58(2), 223-236.
90. Zhou, P. & Leydesdorff, L. (2007b). The citation impacts and citation environments of Chinese journals in mathematics. Scientometrics 72(2), 185-200.