Assessing scientific research performance and impact with single indices

Scientometrics

Volumn 81, Issue 3, 2009, Pages 635-670

  Panaretos, John ^a   Malesios, Chrisovaladis ^a  

^a ATHENS UNIVERSITY OF ECONOMICS AND BUSINESS   (Greece)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77956615014     PISSN: 01389130     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11192-008-2174-9     Document Type: Article

References (94)

1. ADLER, R., EWING, J., TAYLOR, P. (2008), Citation Statistics. Joint IMU/ICIAM/IMS-Committee on Quantitative Assessment of Research. (Available at: http://www.mathunion.org/fileadmin/IMU/Report /CitationStatistics.pdf).
2. AMIN, M., MABE, M. (2000), Impact factor: Use and abuse. Perspectives in Publishing, 1: 1-6. (Available at: http://www.elsevier.com/framework_editors/pdfs/Perspectives1.pdf).
3. ARENCIBIA-JORGE, R., BARRIOS-ALMAGUER, I., FERNANDEZ-HERNANDEZ, S., CARVAJAL-ESPINO, R. (2008), Applying successive H indices in the institutional evaluation: A case study. Journal of the American Society for Information Science and Technology, 59 (1): 155-157. (Available at: http://eprints.rclis.org /archive/00013532/01/English_version.pdf).
4. BALL, P. (2005), Index aims for fair ranking of scientists. Nature, 436: 900. (Available at: http://dx.doi.org /10.1038/436900a).
5. BANKS, M. G. (2006), An extension of the Hirsch Index: Indexing scientific topics and compounds. Scientometrics, 69 (1): 161-168. (Available at: http://www.springerlink.com/content/cq62k71162274586/).
6. BARENDSE, W. (2007), The strike rate index: A new index for journal quality based on journal size and the h-index of citations. Biomedical Digital Libraries, 4: 3. (Available at: http://www.pubmedcentral.nih.gov/picrender.fcgi?artid=1868756&blobtype=pdf).
7. BATISTA, P. D., CAMPITELI, M. G., KINOUCHI, O., MARTINEZ, A. S. (2005), Universal Behaviour of a Research Productivity Index. ArXiv:physics/0510142. (Available at: http://arxiv.org/ftp/physics/papers /0509 /0509048.pdf).
8. BEIRLANT, J., EINMAHL, J. H. J. (2007), Asymptotics for the Hirsch Index. (Available at: http://econpapers.repec.org /paper/dgrkubcen/200786.htm).
9. BORNMANN, L., DANIEL, H. D. (2007), What do we know about the h index? Journal of the American Society for Information Science and Technology, 58 (9): 1381-1385. (Available at: http://dx.doi.org/10.1002/asi.20609).
10. BORNMANN, L., MUTZ, R., DANIEL, H. D. (2008), Are there better indices for evaluation purposes than the h-index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59 (5): 1-8. (Available at: http://dx.doi.org/10.1002/asi.20806).
11. BRAUN, T., GLÄNZEL, W., SCHUBERT, A. (2005), A Hirsch-type Index for journals. The Scientist, 19: 8. (Available at: http://www.steunpuntoos.be/WG_Papers/Scientist_19_22_8.pdf).
12. BRAUN, T., GLÄNZEL, W., SCHUBERT, A. (2006), A Hirsch-type Index for journals. Scientometrics, 69 (1): 169-173. (Available at: http://dx.doi.org/10.1007/s11192-006-0147-4).
13. BURRELL, Q. L. (1992), A simple model for linked informetric processes. Information Processing and Management, 28: 637-645. (Available at: http://dx.doi.org/10.1016/0306-4573(92)90033-V).
14. BURRELL, Q. L. (2007a), Hirsch's h-index: A stochastic model. Journal of Informetrics, 1: 16-25. (Available at: http://dx.doi.org/10.1016/j.joi.2006.07.001).
15. BURRELL, Q. L. (2007b), On the h-index, the size of the Hirsch core and Jin's A-index. Journal of Informetrics, 1: 170-177. (Available at: http://dx.doi.org/10.1016/j.joi.2007.01.003).
16. CHAI, J.-C., HUA, P.-H., ROUSSEAU, R., WAN, J.-K. (2008), The Adapted Pure h-Index. Proccedings of WIS 2008, Berlin Fourth International Conference on Webometrics, Informetrics and Scientometrics & Ninth COLLNET Meeting. (Available at: http://www.collnet.de/Berlin-2008/ChaiWIS2008aph.pdf).
17. CHAPRON, G., HUSTÉ, A. (2006), Open, fair, and free journal ranking for researchers. BioScience, 56 (7): 558-559. (Available at: http://www.carnivoreconservation.org/files/guillaume/bioscience_2006.pdf).
18. COLE, S., COLE, J. R. (1973), Social Stratification in Science. University of Chicago Press, Chicago.
19. COSTAS, R., BORDONS, M. (2007), The h-index: Advantages, limitations and its relation with other bibliometric indicators at the micro level. Journal of Informetrics, 1: 193-203. (Available at: http://dx.doi.org /10.1016/j.joi.2007.02.001).
20. CRONIN, B., MEHO, L. (2006), Using the h-index to rank influential information scientists. Journal of the American Society for Information Science and Technology, 59: 1275-1278. (Available at: http://www.slis.indiana.edu/faculty/meho/cronin-meho-01.pdf).
21. COZZENS, S. E. (1989), What do citations count? The rhetoric-first model. Scientometrics, 15 (5): 437-447. (Available at: http://dx.doi.org/10.1007/BF02017064).
22. EGGHE, L. (2005), Power Laws in the Information Production Process: Lotkaian Informetrics. Elsevier, Oxford.
23. EGGHE, L. (2006a), How to improve the h-index. The Scientist, 20 (3): 14. (Available at: http://www.accessmylibrary.com/coms2/summary_0286-14029249_ITM).
24. EGGHE, L. (2006b), An improvement of the H-index: the G-index. ISSI Newsletter, 2 (1): 8-9. (Available at: https://wiki-sop.inria.fr/wiki/pub/Ce/IndicateurEvaluation/animprovement.pdf).
25. EGGHE, L. (2006c), Theory and practice of the g-index. Scientometrics, 69 (1): 131-152. (Available at: http://dx.doi.org/10.1007/s11192-006-0144-7).
26. EGGHE, L. (2008a), Modelling successive h-indices. Scientometrics (to appear). (Available at: http://dx.doi.org/10.1007/s11192-007-1968-5).
27. EGGHE, L. (2008b), Dynamic h-index: The Hirsch Index in function of time. Journal of the American Society for Information Science and Technology (to appear). (Available at: http://dx.doi.org/10.1002/asi.v58:3).
28. EGGHE, L. (2008c), Mathematical theory of the h- and g-Index in case of fractional counting of authorship. Journal of the American Society for Information Science and Technology, 59 (10): 1608-1616. (Available at: http://dx.doi.org/10.1002/asi.20845).
29. EGGHE, L., RAO, R. (2008), Study of different h-indices for groups of authors. Journal of the American Society for Information Science and Technology, 59 (8): 1276-1281. (Available at: http://dx.doi.org/10.1002/asi.20809).
30. EGGHE, L., ROUSSEAU, R. (2006), An informetric model for the Hirsch Index. Scientometrics, 69 (1): 121-129. (Available at: http://dx.doi.org/10.1007/s11192-006-0143-8).
31. EGGHE, L., ROUSSEAU, R. (2007), An h-index weighted by citation impact. Information Processing & Management, 44 (2): 770-780. (Available at: 10.1016/j.ipm.2007.05.003).
32. EGGHE, L., ROUSSEAU, R., VAN HOOYDONK, G. (2000), Methods for accrediting publications to authors or countries: Consequences for evaluation studies. Journal of the American Society for Information Science, 51 (2): 145-157. (Available at: http://dx.doi.org/10.1002/(SICI)1097-4571(2000)51:2145::AID-ASI63.0.CO2-9).
33. EVIDENCE REPORT (2007), The Use of Bibliometrics to Measure Research Quality in the UK Higher Education System (A Report Produced for the Research Policy Committee of Universities, UK, by Evidence Ltd., a Company Specializing in Research Performance Analysis and Interpretation). (Available at: http://bookshop.universitiesuk.ac.uk/downloads/bibliometrics.pdf).
34. GUMBEL, E. J. (1958), Statistics of Extremes. Columbia University Press.
35. GARFIELD, E. (1955), Citation indexes to science: A new dimension in documentation through association of ideas. Science, 122: 108-111. (Available at: http://ije.oxfordjournals.org/cgi/reprint/35/5/1123).
36. GARFIELD, E. (2006), The history and meaning of the journal impact factor. Journal of the American Medical Association, 295: 90-93. (Available at: http://jama.ama-assn.org/cgi/reprint/295/1/90).
37. GLÄNZEL, W. (2006), On the opportunities and limitations of the H-index. Science Focus, 1 (1): 10-11. (Available at: http://eprints.rclis.org/archive/00009535/01/H_Index_opprtunities.pdf).
38. GOLDSTEIN, H., SPIEGELHALTER, D. J. (1996), League tables and their limitations: statistical issues in comparisons of institutional performance. Journal of the Royal Statistical Society, Series A. 159 (3): 385-443. (Available at: http://www.cmm.bristol.ac.uk/team/HG_Personal/limitations-of-league-tables.pdf).
39. HIRSCH, J. E. (2005), An Index to Quantify an Individual's Scientific Research Output. Proceedings of the National Academy of Sciences, USA, 102 (46): 16569-16572. (Available at: http://arxiv.org/PS_cache/physics/pdf/0508/0508025v5.pdf).
40. HIRSCH, J. E. (2007), Does the h-index have a Predictive Power? (Available at: http://www.pnas.org/content/104/49/19193.full.pdf+html).
41. IGLESIAS, J. E., PECHARROMÁN, C. (2007a), Scaling the h-index for different scientific ISI fields. Scientometrics, 73 (3): 303-320. (Available at: http://arxiv.org/ftp/physics/papers/0607/0607224.pdf).
42. th Conference of the International Society for Scientometrics and Informetrics. Madrid, Spain: Spanish Research Council (CSIC), 862-863.
43. IMPERIAL, J., RODRIGUEZ-NAVARRO, A. (2007), Usefulness of Hirsch's h-index to evaluate scientific research in Spain. Scientometrics, 71 (2): 271-282. (Available at: http://dx.doi.org/10.1007/s11192-007-1665-4).
44. JACSÓ, P. (2008a), Testing the calculation of a realistic h-index in Google Scholar, Scopus, and Web of Science for F. W. Lancaster. Library Trends. (Available at: http://findarticles.com/p/articles/mi_m1387 /is_/ai_n28076518).
45. JACSÓ, P. (2008b), The pros and cons of computing the h-index using web of science. Online Information Review, 32 (5): 673-688. (Available at: http://dx.doi.org/10.1108/14684520810914043).
46. JIN, B.-H. (2006), H-index: An evaluation indicator proposed by scientist. Science Focus, 1 (1): 8-9.
47. JIN, B.-H., LIANG, L., ROUSSEAU, R., EGGHE, L. (2007), The R- and AR- indices: Complementing the h-index. Chinese Science Bulletin, 52: 855-863. (Available at: http://dx.doi.org/10.1007/s11434-007-0145-9).
48. KELLY, C. D., JENNIONS, M. D. (2006), The h index and career assessment by numbers. Trends in Ecology and Evolution, 21 (4): 167-170. (Available at: http://dx.doi.org/10.1016/j.tree.2006.01.005).
49. KINNEY, A. L. (2007), National scientific facilities and their science impact on nonbiomedical research. Proceedings of the National Academy of Sciences USA, 104 (46): 17943-17947. (Available at: http://dx.doi.org/10.1073/pnas.0704416104).
50. KOSMULSKI, M. (2006), A new Hirsch-type Index saves time and works equally well as the original h-index. ISSI Newsletter, 2 (3): 4-6.
51. KOSMULSKI, M. (2007), MAXPROD - A new index for assessment of the scientific output of an individual, and a comparison with the h-index. International Journal of Scientometrics, Informetrics and Bibliometrics, 11 (1). (Available at: http://cybermetrics.cindoc.csic.es/articles/v11i1p5.pdf).
52. LEHMANN, S., JACKSON, A. D., LAUTRUP, B. E. (2006), Measures for Measures. Nature, 444: 1003-1004. (Available at: http://dx.doi.org/10.1038/4441003a).
53. LEHMANN, S., JACKSON, A. D., LAUTRUP, B. E. (2005), Measures and Missmeasures of Scientific Quality (to appear). (Available at: http://arxiv.org/abs/physics/0512238).
54. LIANG, L. (2006), H-index sequence and H-index matrix: Constructions and applications. Scientometrics, 69 (1): 153-159. (Available at: http://dx.doi.org/10.1007/s11192-006-0145-6).
55. LIANG L., ROUSSEAU, R., EGGHE, L. (2007), The R- and AR-indices: Complementing the h-index. Chinese Science Bulletin, 52 (6): 855-863. (Available at: http://dx.doi.org/10.1007/s11434-007-0145-9).
56. LIU Y., ROUSSEAU, R. (2007), Hirsch-type Indices and Library Management: The Case of Tongji University Library. Proceedings of ISSI 2007. Madrid: CINDOC-CSIC, 514-522. (Available at: http://users.pandora.be/ronald.rousseau/Liu_Rousseau_ISSI_2007.pdf).
57. MACROBERTS, M. H., MACROBERTS, B. R. (1989), Problems of citation analysis: A critical review. Journal of the American Society for Information Science, 40: 342-349. (Available at: http://dx.doi.org/10.1002/(SICI)1097-4571(198909)40:5342::AID-ASI73.0.CO2-U).
58. MARTIN, B. R., IRVINE, J. (1983), Assessing basic research: Some partial indicators of scientific progress in radio astronomy. Research Policy, 12: 61-90. (Available at: http://dx.doi.org/10.1016/0048-7333(93)90049-N-abstract only).
59. MEHO, L. I. (2007), The rise and rise of citation analysis. Physics World, 29 (1): 32-36. (Available at: http://dlist.sir.arizona.edu/1703/).
60. MEHO, L. I., YANG, K. (2007), Impact of data sources on citation counts and rankings of lis faculty: Web of science vs. scopus and google scholar. Journal of the American Society for Information Science and Technology, 58 (13): 2105-2125. (Available at: http://www3.interscience.wiley.com/journal/116328907/abstract).
61. MILLER, C. W. (2006), Superiority of the h-index Over the Impact Factor for Physics. Eprint arXiv:physics/0608183. (Available at: http://arxiv.org/PS_cache/physics/pdf/0608/0608183v1.pdf).
62. MOED, H., VAN LEEUWEN, T. (1996), Impact factors can mislead. Nature, 381: 186.
63. MOLINARI, J.-F., MOLINARI, A. (2008), A new methodology for ranking scientific institutions. Scientometrics, 75 (1): 163-174. (Available at: http://dx.doi.org/10.1007/s11192-007-1853-2).
64. ORBAY, M., KARAMUSTAFAOöLU, O., ÖNER, F. (2007), What does hirsch index evolution explain us? A Case Study: Turkish Journal of Chemistry. (Available at: http://arxiv.org/ftp/arxiv/papers/0704/0704.1602.pdf).
65. PANARETOS, J., MALESIOS, C. C. (2008), Assessing Scientific Research Performance and Impact with Single Indices. Technical Report no 235 (March 2008), Department of Statistics, Athens University of Economics and Business.
66. PERSSON, O., GLÄNZEL, W., DANELL, R. (2008), Inflationary bibliometric values: The role of scientometric collaboration and the need for relative indicators in evaluative studies. Scientometrics (to appear). (Available at: http://dx.doi.org/10.1023/B:SCIE.0000034384.35498.7d).
67. PODLUBNY, I. (2005), Comparison of scientific impact expressed by the number of citations in different fields of science. Scientometrics, 64 (1): 95-99. (Available at: http://dx.doi.org/10.1007/s11192-005-0240-0).
68. PODLUBNY, I., KASSAYOVA, K. (2006), Towards a better list of citation superstars: compiling a multidisciplinary list of highly cited researchers. Research Evaluation, 15 (3): 154-162 (Available at: http://uk.arxiv.org/abs/math.ST/0603024).
69. PRATHAP, G. (2006), Hirsch-type indices for ranking institutions' scientific research output. Current Science, 91 (11): 1439. (Available at: http://www.ias.ac.in/currsci/dec102006/1439.pdf).
70. PUBLISH OR PERISH (2007), User's Manual, http://www.harzing.com/resources.htm#/pop.htm, accessed on June 12, 2007.
71. ROEDIGER, H. L. (2006), The h-index in science: A new measure of scholarly contribution. Observer: The Academic Observer, 19 (4): 37-40.
72. ROUSSEAU, R. (2006a), Simple models and the corresponding h- and g-index. Science Technology Development (to appear). (Available at: http://eprints.rclis.org/archive/00006153/).
73. ROUSSEAU, R. (2006b), New developments related to the Hirsch index. Science Focus, 1 (4): 23-25 (in Chinese). (English version available at: http://eprints.rclis.org/archive/00006376/).
74. ROUSSEAU, R. (2007a), A case study: Evolution of JASIS' Hirsch index, Science Focus. (Available at: http://eprints.rclis.org/archive/00005430/01/Evolution_of_h_JASIS_rev.pdf).
75. ROUSSEAU, R. (2007b), The influence of missing publications on the Hirsch index. Journal of Informetrics, 1: 2-7. (Available at: http://dx.doi.org/10.1016/j.joi.2006.05.001).
76. SAAD, G. (2006), Exploring the h-index at the author and journal levels using bibliometric data of productive consumer scholars and business-related journals respectively. Scientometrics, 69 (1): 117-120. (Available at: http://dx.doi.org/10.1007/s11192-006-0142-9).
77. SAISANA, M., D'HOMBRES, B. (2008), Higher Education Rankings: Robustness Issues and Critical Assessment, JRC Scientific and Technical Reports 23487 EN. (Available at: http://crell.jrc.ec.europa.eu/Publications /CRELL%20Research%20Papers/EUR23487.pdf).
78. SANDERSON, M. (2008), Revisiting h measured on UK LIS Academics. Journal of the American Society for Information Science and Technology, 59 (7): 1184-1190. (Available at: http://dx.doi.org/10.1002/asi.20771).
79. SYMONDS M. R., GEMMELL N. J., BRAISHER T. L., GORRINGE K. L., ELGAR M. A. (2006), Gender differences in publication output: Towards an unbiased metric of research performance. PLoS ONE 1 e127: 1-5. (Available at: http://dx.doi.org/10.1371/journal.pone.0000127).
80. SCHREIBER, M. (2007a), Self-citation corrections for the Hirsch index. EPL 78 30002, 1-6. (Available at: http://dx.doi.org/10.1209/0295-5075/78/30002).
81. SCHREIBER, M. (2007b), The Influence of Self-citations on Egghe's g Index. (Available at: http://arxiv.org /PS_cache/arxiv/pdf/0707/0707.4577v1.pdf).
82. SCHREIBER, M. (2008a), A modification of the h-index: The hm-index accounts for multi-authored manuscripts. Journal of Informetrics, 2 (3): 211-216. (Available at: http://dx.doi.org/10.1016/j.joi.2008.05.001).
83. SCHREIBER, M. (2008b), To share the fame in a fair way, hm modifies h for multi-authored manuscripts. New Journal of Physics, 10: 9. (Available at: http://dx.doi.org/10.1088/1367-2630/10/4/040201).
84. SCHUBERT, A. (2007), Successive h-indices. Scientometrics, 70 (1): 201-205. (Available at: http://dx.doi.org /10.1007/s11192-007-0112-x).
85. SCHUBERT, A., GLÄNZEL, W. (2007), A systematic analysis of Hirsch-type indices for journals. Journal of Informetrics, 1: 179-184. (Available at: http://dx.doi.org/10.1016/j.joi.2006.12.002).
86. SIDIROPOULOS, A., KATSAROS, D., MANOLOPOULOS, Y. (2006), Generalized h-index for disclosing latent facts in citation networks. Scientometrics, 72 (2): 253-280. (Available at: http://dx.doi.org/10.1007/s11192-007-1722-z).
87. TOL, R. S. J. (2007), Of the h-index and its Alternatives: An Application to the 100 Most Prolific Economists. Working paper FNU-146. (Available at: http://www.fnu.zmaw.de/fileadmin/fnu-files/publication/working-papers/alternativeindexwp.pdf).
88. TORO-ALVES, N., HERCULANO, R. D., TERÇARIOL, C. A. S., FILHO, O. K., GRAEFF, C. F. O. (2007), Hirsch's Index: Case Study Conducted at the Faculdade de Filosofia, Ciências e Letras de Ribeirão Preto, Universidade de São Paulo. Brazilian Journal of Medical and Biological Research, 40: 1529-1536. (Available at: http://dx.doi.org/10.1590/S0100-879X2007001100013).
89. VAN RAAN, A. J. (2005), Measurement of central aspects of scientific research: performance, Interdisciplinary, structure. Measurement: Interdisciplinary Research and Perspectives, 3 (1): 1-19. (Available at: http://www.cwts.nl/TvR/documents/AvR-Measurem.pdf).
90. VAN RAAN, A. J. (2006), Comparison of the Hirsch-index with standard bibliometric indicators and with Peer Judgment for 147 Chemistry Research Groups. Scientometrics, 67 (3): 491-502. (Available at: http://www.cwts.nl/cwts/Hirsch.pdf).
91. VANCLAY, J. K. (2007), On the robustness of the h-index. Journal of the American Society for Information Science and Technology, 58 (10): 1547-1550. (Available at: http://dx.doi.org/10.1002/asi.20616).
92. VINKLER, P. (2007), Eminence of scientists in the light of the h-index and other scientometric indicators. Journal of Information Science, 33: 481-491. (Available at: http://dx.doi.org/10.1177/0165551506072165).
93. WAN, J.-K., HUA, P.-H., ROUSSEAU, R. (2007), The Pure h-index: Calculating an Author's h-index by Taking Co-authors Into Account. Preprint. (Available at: http://eprints.rclis.org/archive/00011401/01/pure_h.pdf).
94. WU, Q. (2008), The w-index: A significant improvement of the h-index. (Available at: http://arxiv.org/ftp/arxiv/papers/0805/0805.4650.pdf).