The objectivity of national research foundation peer review in South Africa assessed against bibliometric indexes

Scientometrics

Volumn 97, Issue 2, 2013, Pages 177-206

  Fedderke, J W ^a  

^a UNIVERSITY OF THE WITWATERSRAND   (South Africa)

Author keywords

Bibliometric measures; Peer review

Indexed keywords

EID: 84885856782     PISSN: 01389130     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11192-013-0981-0     Document Type: Review

References (65)

1. Abramo, G., & D'Angelo, C. A. (2011). Evaluating research: From informed peer review to bibliometrics. Scientometrics, 87(3), 499-514.
2. Abramo, G., D'Angelo, C. A., & Di Costa, F. (2011). National research assessment exercises: Acomparison of peer review and bibliometrics rankings. Scientometrics, 89(3), 929-941.
3. Adler, N., Elmquist, M., & Norrgre, F. (2009). The challenge of managing boundary-spanning research activities: Experiences from the Swedish context. Research Policy 38, 1136-1149.
4. Archambault, E., & Gagné, E. V. (2004). The use of bibliometrics in social sciences and Humanities. Montreal: Social Sciences and Humanities Research Council of Canada (SSHRCC).
5. Bar-Ilan, J. (2008). Which h-index?: A comparison of WoS, Scopus and Google Scholar. Scientometrics, 74(2), 257-271.
6. Batista, P. D., Campiteli, M. G., Kinouchi, O., & Martinez, A. S. (2006). Is it possible to compare researchers with different scientific interests?. Scientometrics, 68(1), 179-189.
7. Bedeian, A. G. (2003). The manuscript review process: The proper role of authors, referees, and editors. Journal of Management Inquiry, 12(4), 331-338.
8. Belew, R. K. (2005). Scientific impact quantity and quality: Analysis of two sources of bibliographic data, arXiv: cs. IR/0504036 v1, 11 August 2005.
9. Benner, M., & Sandström, U. (2000). Institutionalizing the triple helix: research funding and norms in the academic system. Research Policy, 29, 291-301.
10. Bornmann, L. (2011). Scientific peer review. Annual Review of Information Science and Technology, 45, 199-245.
11. Bornmann, L., & Daniel, H. -D. (2005). Does the h-index for ranking of scientists really work? Scientometrics, 65(3), 391-392.
12. Bornmann, L., & Daniel, H. -D. (2007). What do we know about the h index? Journal of the American Society for Information Science and Technology, 58(9), 1381-1385.
13. Bornmann, L., Mutz, R., & Daniel, H. -D. (2008). Are there better indices for evaluation purposes than the h index? A comparison of nine different variants of the h index using data from biomedicine. Journal of the American Society for Information Science and Technology, 59(5), 830-837.
14. Bornmann, L., Mutz, R., Daniel, H. -D., Wallon, G., & Ledin, A. (2009a). Are there really two types of h index variants? A validation study by using molecular life sciences data. Research Evaluation, 18(3), 185-190.
15. Bornmann, L., Marx, W., Schier, H., Rahm, E., Thor, A., & Daniel, H. D. (2009b). Convergent validity of bibliometric Google Scholar data in the field of chemistry. Citation counts for papers that were accepted by Angewandte Chemie International Edition or rejected but published elsewhere, using Google Scholar, Science Citation Index, Scopus, and Chemical Abstracts. Journal of Informetrics, 3(1), 27-35.
16. Bosman, J, Mourik, I. van, Rasch, M.; Sieverts, E., & Verhoeff, H. (2006). Scopus reviewed and compared. The coverage and functionality of the citation database Scopus, including comparisons with Web of Science and Google Scholar. Utrecht: Utrecht University Library. Retrieved from http://igitur-archive. library. uu. nl/DARLIN/2006-1220-200432/Scopusdoorgelicht&vergeleken-translated. pdf.
17. Butler, L. (2006). RQF pilot study project: History and political science methodology for citation analysis, November 2006. Retrieved from http://www. chass. org. au/papers/bibliometrics/CHASS_Methodology. pdf.
18. Cronin, B., & Meho, L. (2006). Using the h-Index to rank Influential Information scientists. Journal of the American Association for Information Science and Technology, 57(9), 1275-1278.
19. Debackere, K., & Glänzel, W. (2003). Using a bibliometric approach to support research policy making: The case of the Flemish BOF-key. Scientometrics, 59(2), 253-276.
20. Demicheli, V., & Pietrantonj, C. (2007). Peer review for improving the quality of grant applications. Cochrane Database of Systematic Reviews, 2, Art. No.: MR000003. doi: 10. 1002/14651858. MR000003. pub2.
21. Derrick, G. E., Sturk, H., Haynes, A. S., Chapman, S., & Hall, W. D. (2010). A cautionary bibliometric tale of two cities. Scientometrics, 84(2), 317-320.
22. Egghe, L. (2006). Theory and practice of the g-index. Scientometrics, 69 (1), 131-152.
23. Egghe, L., & Rousseau, R. (2006). An informetric model for the Hirsch-index. Scientometrics, 69(1), 121-129.
24. Eysenck, H. J., & Eysenck, S. B. G. (1992). Peer review: Advice to referees and contributors. Personality and Individual Differences, 13(4), 393-399.
25. Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., & Pappas, G. (2008). Comparison of PubMed, Scopus, Web of Science, and Google Scholar: Strengths and weaknesses. The FASEB Journal, 22, 338-342 doi: 10. 1096/fj. 07-9492LSF.
26. Frey, B. S. (2003). Publishing and prostitution? Choosing between one's own ideas and academic success. Public Choice, 116(1-2), 205-223.
27. García-Aracil, A., Gracia, A. G., & Pérez-Marín, M. (2006). Analysis of the evaluation process of the research performance: An empirical case. Scientometrics, 67(2), 213-230.
28. García-Pérez, M. A. (2010). Accuracy and completeness of publication and citation records in the Web of Science, PsycINFO, and Google Scholar: A case study for the computation of h Indices in psychology. Journal of the American Society for Information Science and Technology, 61(10), 2070-2085.
29. Glänzel, W. (2006). On the opportunities and limitations of the H-index. Science Focus, 1(1), 10-11.
30. Goodman, S. N., Berlin, J., Fletcher, S. W., & Fletcher, R. H. (1994). Manuscript quality before and after peer review and editing at annals of internal medicine. Annals of Internal Medicine, 121(1), 11-21.
31. Gray, J. E., Hamilton, M. C., Hauser, A., Janz, M. M., Peters, J. P., & Taggart, F. (2012). Scholarish: Google Scholar and its value to the sciences. Issues in Science and Technology Librarianship, 70(Summer). doi:: 10: 5062/F4MK69T9.
32. Harzing, A.-W. (2007-2008). Google Scholar: Anew data source for citation analysis. Retrieved from http://www. harzing. com/pop_gs. htm.
33. Harzing, A.-W. (2008). Reflections on the h-index. Retrieved from http://www. harzing. com/pop_hindex. htm.
34. Hirsch, J. E. (2005). An index to quantify an individual's scientific research output, arXiv: physics/0508025 v5, 29 Sep 2006.
35. Horrobin, D. F. (1990). The philosophical basis for peer review and the suppression of innovation. Journal of the American Medical Association, 263(10), 1438-1441.
36. Iglesias, J. E., & Pecharromán, C. (2007). Scaling the h-index for different scientific ISI fields. Scientometrics, 73(3), 303-320.
37. Jacsó, P. (2005). Google Scholar: The pros and the cons. Online Information Review, 29(2), 208-214.
38. Jacsó, P. (2006a). Dubious hit counts and cuckoo's eggs. Online Information Review, 30(2), 188-193.
39. Jacsó, P. (2006b). Deflated, inflated and phantom citation counts. Online Information Review, 30(3), 297-309.
40. Jacsó, P. (2010). Metadata mega mess in Google Scholar. Online Information Review, 34(1), 175-191.
41. Jin, B. (2007). The AR-index: Complementing the h-index. ISSI Newsletter, 3(1), 6.
42. Kulkarni, A. V., Aziz, B., Shams, I., & Busse, J. W. (2009). Comparisons of citations in Web of Science, Scopus, and Google Scholar for articles published in general medical journals. Journal of the American Medical Association, 302(10), 1092-1096.
43. Kousha, K., & Thelwall, M. (2007). Google Scholar citations and Google Web/URL citations: A multi-discipline exploratory analysis. Journal of the American Society for Information Science and Technology, 58(7), 1055-1065.
44. Kousha, K, & Thelwall, M. (2008). Sources of Google Scholar citations outside the Science Citation index: A comparison between four science disciplines. Scientometrics, 74(2), 273-294.
45. Leydesdorff, L. & Etzkowitz, H. (1996). Emergence of a Triple-Helix of university-industry-government relations. Science and Public Policy, 23(5), 279-296.
46. Meho, L. I., & Yang, K. (2007). A new era in citation and bibliometric analyses: Web of Science, Scopus, and Google Scholar. Journal of the American Society for Information Science and Technology, 58(13), 2105-2125.
47. Moed, H. F. (2002). The impact-factors debate: The ISI's uses and limits. Nature, 415, 731-732.
48. Moxam, H., & Anderson, J. (1992). Peer review. A view from the inside. Science and Technology Policy, 5(1), 7-15.
49. Nisonger, T. E. (2004). Citation autobiography: An investigation of ISI database coverage in determining author citedness. College & Research Libraries, 65(2), 152-163.
50. Pendlebury, D. A. (2008). Using bibliometrics in evaluating research. Philadelphia, PA: Research Department, Thomson Scientific.
51. Pendlebury, D. A. (2009). The use and misuse of journal metrics and other citation indicators. Scientometrics, 57(1), 1-11.
52. Pierie, J. P. E. N., Walvoort, H. C., & Overbeke, A. J. P. M. (1996). Reader's evaluation of effect of peer review and editing on quality of articles in the Nederlans Tijdschrift voor Geneeskunde. Lancet, 348(9040), 1480-1483.
53. Rehn, C., Kronman, U., & Wadskog, D. (2007). Bibliometric indicators: Definitions and usage at Karolinska Institutet, Stickholm. Sweden: Karolinska Institutet University Library.
54. Roediger III, H. L. (2006). The h index in Science: A new measure of scholarly contribution. APS Observer: The Academic Observer, 19, 4.
55. Saad, G. (2006). Exploring the h-index at the author and journal levels using bibliometric data of productive consumer scholars and business-related journals respectively. Scientometrics, 69(1), 117-120.
56. m modifies h for multi-authored manuscripts. New Journal of Physics, 10, 040201-1-8.
57. Shatz, D. (2004). Peer review: A critical inqury. Lanham, MD: Rowman and Littlefield.
58. Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2006). Generalized h-index for disclosing latent facts in citation networks, arXiv: cs. DL/0607066 v1, 13 Jul 2006.
59. Testa, J. (2004). The Thomson scientific journal selection process. Retrieved from http://scientific. thomson. com/free/essays/selectionofmaterial/journalselection/.
60. Thor, A., & Bornmann, L. (2011). The calculation of the single publication h index and related performance measures: A web application based on Google Scholar data. Online Information Review, 35(2), 291-300.
61. van Raan, A. F. J. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62(1), 133-143.
62. van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgement for 147 chemistry research groups. Scientometrics, 67(3), 491-502.
63. Vaughan, L., & Shaw, D. (2008). A new look at evidence of scholarly citations in citation indexes and from web sources. Scientometrics, 74(2), 317-330.
64. Wessely, S. (1998). Peer review of grant applications: What do we know?. Lancet, 352(9124), 301-305.
65. Zhang, C.-T. (2009). The e-index, complementing the h-index for excess citations. PLoS ONE, 5(5), e5429.