Reviewers' Ratings and Bibliometric Indicators: Hand in Hand When Assessing Over Research Proposals?

PLoS ONE

Volumn 8, Issue 6, 2013, Pages

  Cabezas Clavijo, Álvaro ^a   Robinson García, Nicolás ^a   Escabias, Manuel ^a   Jiménez Contreras, Evaristo ^a  

^a UNIVERSITY OF GRANADA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BIBLIOMETRICS; CORRELATION ANALYSIS; DATA BASE; HUMAN; INFORMATION PROCESSING; LOGISTIC REGRESSION ANALYSIS; MEDICAL RESEARCH; MULTIVARIATE LOGISTIC REGRESSION ANALYSIS; PEER REVIEW; POLICY; PUBLISHING; SOCIOLOGY; STATISTICAL ANALYSIS; WEB BROWSER;

BIBLIOMETRICS; BIOMEDICAL RESEARCH; DATABASES, FACTUAL; FINANCING, ORGANIZED; HUMANS; PUBLICATIONS; PUBLISHING; RESEARCH DESIGN; RESEARCH PERSONNEL; SPAIN;

EID: 84879510693     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0068258     Document Type: Article

References (41)

1. Bornmann L, (2011) Scientific peer review. Annu Rev Infor Sci 45: 199-245.
2. Campanario JM, (1996) Have referees rejected some of the most-cited articles of all times? J Am Soc Inform Sci 47: 302-310. doi:10.1002/(SICI)1097-4571(199604)47:4.
3. Abdoul H, Perrey C, Amiel P, Tubach F, Gottot S, Durand-Zaleski et al,. (2012) Peer Review of Grant Applications: Criteria Used and Qualitative Study of Reviewer Practices. PLOS ONE 7: e46054. doi:10.1371/journal.pone.0046054. PubMed: 23029386.
4. Abramo G, D'Angelo CA, Costa FD, (2011) National research assessment exercises: a comparison of peer review and bibliometric rankings. Scientometrics 89: 929-941. doi:10.1007/s11192-011-0459-x.
5. Bornmann L, Daniel H-D, (2005) Selection of research fellowship recipients by committee peer review. Reliability, fairness and predictive validity of Board of Trustees' decisions. Scientometrics 63: 297-320. doi:10.1007/s11192-005-0214-2.
6. Wessely S, (1998) Peer review of grant applications: what do we know? Lancet 352: 301-305. doi:10.1016/S0140-6736(97)11129-1. PubMed: 9690424.
7. Benda WGG, Engels TCE, (2010) The predictive validity of peer review: A selective review of the judgmental forecasting qualities of peers, and implications for innovation in science. Int J Forecast 27: 166-182.
8. Smith R, (2006) Peer review: a flawed process at the heart of science and journals. J R Soc Med 99: 178-182. doi:10.1258/jrsm.99.4.178. PubMed: 16574968.
9. eds. (2010) Assessing assessment. Nature 465: 845. doi:10.1038/465845b. PubMed: 20559339.
10. Moed HF, (2008) UK Research Assessment Exercises: Informed judgments on research quality or quantity? Scientometrics 74: 153-161. doi:10.1007/s11192-008-0108-1.
11. Haslam N, Koval P, (2010) Possible research area bias in the Excellence in Research for Australia (ERA) draft journal rankings. Aust J Psychol 62: 112-114. doi:10.1080/00049530903334489.
12. Allen L, Jones C, Dolby K, Lynn D, Walport M, (2009) Looking for landmarks: the role of expert review and bibliometric analysis in evaluating scientific publication outputs. PLOS ONE 4: e5910. doi:10.1371/journal.pone.0005910. PubMed: 19536339.
13. Costas R, Bordons M, (2005) Bibliometric indicators at the micro-level: some results in the area of natural resources at the Spanish CSIC. Res Evaluat 14: 110-120. doi:10.3152/147154405781776238.
14. Alonso S, Cabrerizo FJ, Herrera-Viedma E, Herrera F, (2009) H-Index: A review focused in its variants, computation and standardization for different scientific fields. J Informetr 3: 273-289. doi:10.1016/j.joi.2009.04.001.
15. Seglen PO, (1992) The skewness of science. J Am Soc Inform Sci 43: 628-638. doi:10.1002/(SICI)1097-4571(199210)43:9.
16. Abramo G, Dangelo C, Caprasecca A, (2009) Allocative efficiency in public research funding: Can bibliometrics help? Res Policy 38: 206-215. doi:10.1016/j.respol.2008.11.001.
17. Van Raan AFJ, (1996) Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises. Scientometrics 36: 397-420. doi:10.1007/BF02129602.
18. Aksnes DW, Taxt RE, (2004) Peer reviews and bibliometric indicators: a comparative study at a Norwegian university. Res Evaluat 13: 33-41. doi:10.3152/147154404781776563.
19. Larivière V, Macaluso B, Archambault É, Gingras Y, (2010) Which scientific elites? On the concentration of research funds, publications and citations. Res Evaluat 19: 45-53. doi:10.3152/095820210X492495.
20. Norris M, Oppenheim C, (2003) Citation counts and the Research Assessment Exercise V: Archaeology and the 2001 RAE. J Doc 59: 709-730. doi:10.1108/00220410310698734.
21. Reale E, Barbara A, Costantini A, (2007) Peer review for the evaluation of academic research: lessons from the Italian experience. Res Evaluat 16: 216-228. doi:10.3152/095820207X227501.
22. Van Leeuwen TN, Moed HF, (2012) Funding decisions, peer review, and scientific excellence in physical sciences, chemistry, and geosciences. Res Evaluat 21: 189-198. doi:10.1093/reseval/rvs009.
23. Fernández-Esquinas M, Pérez-Yruela M, Merchán-Hernández C, (2006) El sistema de incentivos y recompensas en la ciencia pública española. In: Sebastián J, Muñoz E, eds. Radiografía de la investigación pública en España. Madrid: Biblioteca Nueva. pp. pp. 148-206.
24. Fernández-Esquinas M, Díaz-Catalán C, Ramos-Vielba I, (2011) Evaluación y política científica en España: el origen y la implantación de las prácticas de evaluación científica en el sistema público de I+D (1975-1994). In: González de la Fe T, López Peláez A, eds. Innovación tecnológica, conocimiento científico y cambio social. Madrid: Centro de Investigaciones Sociológicas. pp. pp. 93-130.
25. Jiménez-Contreras E, Moya Anegón F, Delgado-López-Cózar E, (2003) The evolution of research activity in Spain: The impact of the National Commission for the Evaluation of Research Activity(CNEAI). Res Policy 32: 123-142.
26. Diest PV, Holzel H, Burnett D, Crocker J, (2001) Impactitis: new cures for an old disease. J Clin Pathol 54: 817-819. doi:10.1136/jcp.54.11.817. PubMed: 11684711.
27. Rossner M, Van Epps H, Hill E, (2007) Show me the data. J Cell Biol 179: 1091-1092. doi:10.1083/jcb.200711140. PubMed: 18086910.
28. Gordillo V, González Marqués J, Muñiz J, (2004) La evaluación de proyectos de investigación por la Agencia Nacional de Evaluación y Prospectiva. Psicothema 16: 343-349.
29. Gibbons JD, (1985) Nonparametric statistical inference. New York: Marcel Dekker.
30. Randles RH, Wolfe DA, (1979) Introduction to the theory of nonparametric statistics. New York: John Wiley and Sons.
31. Siegel S, Castellan NJ, (1988) Nonparametric statistics for the behavioral sciences. New York: McGraw-Hill.
32. Draper NR, (1998) Applied regression analysis. New York: John Wiley and Sons.
33. Hosmer DW, Lemeshow S, (2000) Applied Logistic Regression. Jonh Wiley and Sons: New York.
34. Agresti A, (2002) Categorical Data Analysis. New York: John Wiley and Sons.
35. Kleinbaum DG, (2002) Logistic regression: a self-learning text. New York: Springer.
36. Moed HF, (2005) Citation analysis in research evaluation. Dordrecht: Springer.
37. Sanz-Menéndez L, (1995) Research actors and the state: Research evaluation and evaluation of science and technology policies in Spain. Res Evaluat 5: 79-88.
38. Delgado López-Cózar E, Jiménez-Contreras E, Ruiz-Pérez R, (2009) España y los 25 grandes de la ciencia mundial en cifras (1992-2008). Prof Inform 18: 81-86.
39. Torres-Salinas D, Robinson-Garcia N, Jimenez-Contreras E, Delgado López-Cózar E, (2012) Towards a 'Book Publishers Citation Reports'. First approach using the 'Book Citation Index'. Rev Esp Doc Cient 35: 615-620.
40. Jiménez-Contreras E, Robinson-García N, Cabezas-Clavijo Á, (2011) Productivity and impact of Spanish researchers: reference thresholds within scientific areas. Rev Esp Doc Cient 34. pp. 505-525.
41. Abramo G, Tindaro C, D'Angelo CA, (2012) National peer-review research assessment exercises for the hard sciences can be a complete waste of money: the Italian Case. Scientometrics. doi:10.1007/s11192-012-0875-6.