Integrated impact indicators compared with impact factors: An alternative research design with policy implications

Journal of the American Society for Information Science and Technology

Volumn 62, Issue 11, 2011, Pages 2133-2146

  Leydesdorff, Loet ^a   Bornmann, Lutz ^b  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b MAX PLANCK INSTITUTE   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

AMERICAN SOCIETY FOR INFORMATION SCIENCE AND TECHNOLOGIES; BIBLIOMETRICS; CITATION IMPACT; DOCUMENT SETS; IMPACT FACTOR; IMPACT INDICATORS; INSTITUTE FOR SCIENTIFIC INFORMATIONS; LIBRARY AND INFORMATION SCIENCE; LIBRARY SCIENCE; MULTIDISCIPLINARY SCIENCE; PARADIGM SHIFTS; POLICY IMPLICATIONS; REFERENCE SET; RESEARCH DESIGNS;

INDEXING (OF INFORMATION); INFORMATION SCIENCE; INTEGRATION;

PUBLIC POLICY;

EID: 80054713361     PISSN: 15322882     EISSN: 15322890     Source Type: Journal    
DOI: 10.1002/asi.21609     Document Type: Article

References (43)

1. Ahlgren, P., Jarneving, B., &, Rousseau, R., (2003). Requirement for a cocitation similarity measure, with special reference to Pearson's correlation coefficient. Journal of the American Society for Information Science and Technology, 54 (6), 550-560.
2. Bensman, S.J., (2007). Garfield and the impact factor. Annual Review of Information Science and Technology, 41 (1), 93-155.
3. Bensman, S.J., &, Wilder, S.J., (1998). Scientific and technical serials holdings optimization in an inefficient market: A LSU Serials Redesign Project Eexercise. Library Resources and Technical Services, 42 (3), 147-242. (Pubitemid 128395669)
4. Bornmann, L., (2010). Towards an ideal method of measuring research performance: Some comments to the Opthof and Leydesdorff (2010) paper. Journal of Informetrics, 4 (3), 441-443.
5. Bornmann, L., &, Mutz, R., (2011). Further steps towards an ideal method of measuring citation performance: The avoidance of citation (ratio) averages in field-normalization. Journal of Informetrics, 5 (1), 228-230.
6. Bornmann, L., Mutz, R., Neuhaus, C., &, Daniel, H.D., (2008). Citation counts for research evaluation: Standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in Science and Environmental Politics, 8 (1), 93-102. (Pubitemid 351836496)
7. Bornmann, L., Leydesdorff, L., &, Van den Besselaar, P., (2010). A meta-evaluation of scientific research proposals: Different ways of comparing rejected to awarded applications. Journal of Informetrics, 4 (3), 211-220.
8. Bornmann, L., Schier, H., Marx, W., &, Daniel, H.-D., (2011). Is interactive open access publishing able to identify high-impact submissions? A study on the predictive validity of atmospheric chemistry and physics by using percentile rank classes. Journal of the American Society for Information Science and Technology, 62 (1), 61-71.
9. Brumback, R. A., (2008a). Response to correspondence, "Worshiping false idols: The impact factor dilemma": Correcting the record. Journal of Child Neurology, 23 (9), 1092.
10. Brumback, R. A., (2008b). Worshiping false idols: The impact factor dilemma. Journal of Child Neurology, 23 (4), 365.
11. Garfield, E., (1972). Citation analysis as a tool in journal evaluation. Science, 178 (4060), 471-479.
12. Garfield, E., &, Sher, I.H., (1963). New factors in the evaluation of scientific literature through citation indexing. American Documentation, 14, 195-201.
13. Gingras, Y., &, Larivière, V., (2011). There are neither "king" nor "crown" in scientometrics: Comments on a supposed "alternative" method of normalization. Journal of Informetrics, 5 (1), 226-227.
14. Glänzel, W., (2010). On reliability and robustness of scientometrics indicators based on stochastic models. An evidence-based opinion paper. Journal of Informetrics, 4 (3), 313-319.
15. Hirsch, J.E., (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences, USA, 102 (46), 16569-16572. (Pubitemid 41688817)
16. Kamada, T., &, Kawai, S., (1989). An algorithm for drawing general undirected graphs. Information Processing Letters, 31 (1), 7-15.
17. King, D.A., (2004, July 15). The scientific impact of nations. Nature, 430, 311-316. (Pubitemid 38937732)
18. Levine, G., (1991). A guide to SPSS for analysis of variance. Hillsdale, NJ: Erlbaum.
19. Leydesdorff, L., (2008). Caveats for the use of citation indicators in research and journal evaluation. Journal of the American Society for Information Science and Technology, 59 (2), 278-287. (Pubitemid 351205033)
20. Leydesdorff, L., (2009). How are new citation-based journal indicators adding to the bibliometric toolbox ? Journal of the American Society for Information Science and Technology, 60 (7), 1327-1336.
21. Leydesdorff, L., Bornmann, L., Mutz, R., &, Opthof, T., (2011). Turning the tables in citation analysis one more time: Principles for comparing sets of documents. Journal of the American Society for Information Science and Technology, 62 (7), 1370-1381.
22. Leydesdorff, L., &, Opthof, T., (2011). Remaining problems with the "New Crown Indicator" (MNCS) of the CWTS. Journal of Informetrics, 5 (1), 224-225.
23. Leydesdorff, L., &, Rafols, I., (2011). Indicators of the interdisciplinarity of journals: Diversity, centrality, and citations. Journal of Informetrics, 5 (1), 87-100.
24. Lundberg, J., (2007). Lifting the crown-Citation z-score. Journal of Informetrics, 1 (2), 145-154.
25. Moed, H.F., &, van Leeuwen, T.N., (1996). Impact factors can mislead. Nature, 381 (6579), 186.
26. Narin, F., (1976). Evaluative bibliometrics: The use of publication and citation analysis in the evaluation of scientific activity. Washington, DC: National Science Foundation.
27. National Science Board. (2010). Science and Engineering Indicators. Washington, DC: National Science Foundation. Available at
28. Nisonger, T.E., (1999). JASIS and library and information science journal rankings: A review and analysis of the last half century. Journal of the American Society for Information Science, 50 (11), 1004-1019.
29. Nisonger, T.E., &, Davis, C.H., (2005). The perception of library and information science journals by LIS education deans and ARL library directors: A replication of the Kohl-Davis study. College and Research Libraries, 66 (4), 341-377. (Pubitemid 41137921)
30. Opthof, T., &, Leydesdorff, L., (2010). Caveats for the journal and field normalizations in the CWTS ("Leiden") evaluations of research performance. Journal of Informetrics, 4 (3), 423-430.
31. Pringle, J., (2008). "Worshiping false idols: The impact factor dilemma": Correcting the record. Journal of Child Neurology, 23 (9), 1092.
32. Pudovkin, A.I., &, Garfield, E., (2002). Algorithmic procedure for finding semantically related journals. Journal of the American Society for Information Science and Technology, 53 (13), 1113-1119. (Pubitemid 35359180)
33. Pudovkin, A.I., &, Garfield, E., (2009). Percentile rank and author superiority indexes for evaluating individual journal articles and the author's overall citation performance. CollNet Journal of Scientometrics and Information Management, 3 (2), 3-10.
34. Rafols, I., &, Leydesdorff, L., (2009). Content-based and algorithmic classifications of journals: Perspectives on the dynamics of scientific communication and indexer effects. Journal of the American Society for Information Science and Technology, 60 (9), 1823-1835.
35. Rossner, M., Epps, H.V., &, Hills, E., (2007). Show me the data. Journal of Cell Biology, 179 (6), 1091-1092. (Pubitemid 350277727)
36. Rossner, M., Van Epps, H., &, Hill, E., (2008). Irreproducible results-A response to Thomson Scientific. The Journal of General Physiology, 131 (2), 183-184. (Pubitemid 351185798)
37. Rousseau, R., &, Leydesdorff, L., (2011). Simple arithmetic versus intuitive understanding: The case of the impact factor. ISSI Newsletter, 7 (1), 10-14.
38. Seglen, P. O., (1992). The skewness of science. Journal of the American Society for Information Science, 43 (9), 628-638.
39. Small, H., &, Garfield, E., (1985). The geography of science: Disciplinary and national mappings. Journal of Information Science, 11 (4), 147-159.
40. Van den Besselaar, P., &, Leydesdorff, L., (2009). Past performance, peer review, and project selection: A case study in the social and behavioral sciences. Research Evaluation, 18 (4), 273-288.
41. van Raan, A.F.J., van Leeuwen, T.N., Visser, M.S., van Eck, N.J., &, Waltman, L., (2010). Rivals for the crown: Reply to Opthof and Leydesdorff. Journal of Informetrics, 4 (3), 431-435.
42. van Raan, A.F.J., van Eck, N.J., van Leeuwen, T.N., Visser, M.S., &, Waltman, L., (2010, September). The new set of bibliometric indicators of CWTS. Paper presented at the 11th International Conference on Science and Technology Indicators (pp. 291-293), Leiden, The Netherlands.
43. Zhao, D., &, Strotmann, A., (2008). Evolution of research activities and intellectual influences in information science 1996-2005: Introducing author bibliographic-coupling analysis. Journal of the American Society for Information Science and Technology, 59 (13), 2070-2086.