The validation of peer review through research impact measures and the implications for funding strategies

PLoS ONE

Volumn 9, Issue 9, 2014, Pages

  Gallo, Stephen A ^a   Carpenter, Afton S ^a   Irwin, David ^a   McPartland, Caitlin D ^a   Travis, Joseph ^b   Reynders, Sofie ^a   Thompson, Lisa A ^a   Glisson, Scott R ^a  

^a American Institute of Biological Sciences   (United States)

^b FLORIDA STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; COST EFFECTIVENESS ANALYSIS; DISCRIMINANT VALIDITY; FUNDING; MEDICAL INFORMATICS; MEDICAL RESEARCH; PEER REVIEW; PREDICTIVE VALUE; PUBLICATION; RETROSPECTIVE STUDY; STATISTICAL ANALYSIS; TIME; TIME ANALYSIS; TOTAL RELATIVE CITATION; VALIDATION PROCESS; AWARDS AND PRIZES; FINANCIAL MANAGEMENT; HUMAN; ORGANIZATION; STANDARDS;

ACADEMIES AND INSTITUTES; AWARDS AND PRIZES; FINANCIAL SUPPORT; HUMANS; PEER REVIEW; PEER REVIEW, RESEARCH; PUBLICATIONS;

EID: 84906962311     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0106474     Document Type: Article

References (31)

1. Cole S, Cole JR, Simon GA (1981) Chance and consensus in peer review. Science 214 (4523): 881-886
2. Mervis J (2013) Proposed change in awarding grants at NSF spurs partisan sniping. Science 340(6133): 670
3. Berg J (2013) On deck chairs and life boats. ASBMB Today 12: 3-6
4. Graves N, Barnett AG, Clarke P (2011) Funding grant proposals for scientific research: retrospective analysis of scores by members of grant review panel. BMJ 343: d4797
5. Mayo NE, Brophy J, Goldberg MS, Klein MB, Miller S, et al. (2006) Peering at peer review revealed high degree of chance associated with funding of grant applications. J Clin Epidem 59(8): 842-8
6. Wood F, Wessely S (2003) Peer review of grant applications: a systematic review. In: Peer Review in Health Sciences (ed Godlee, Jefferson). BMJ Publications, London 14-31
7. Jackson JL, Srinivasan M, Rea J, Fletcher KE, Kravitz RL (2011) The validity of peer review in a general medicine journal. PLoS ONE 6(7)
8. Van Raan AFJ (2006) Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgement for 147 chemistry research groups. Scientometrics 67(3): 491-502
9. Demicheli V, Di Pietrantonj C (2007) Peer review for improving the quality of grant applications. Cochrane Database of Systematic Reviews Library. John Wiley and Sons, LTD. 2: MR000003
10. Claveria LE, Guallar E, Cami J, Conde J, Pastor R, et al. (2000) Does peer review predict the performance of research projects in health sciences? Scientometrics 47: 11-23
11. Bornmann L, Daniel HD (2004) Reliability, fairness and predictive validity of committee peer review. BIF Futura 19: 7-19
12. Fang D, Meyer RE (2003) Effect of two Howard Hughes Medical Institute research training programs for medical students on the likelihood of pursuing research careers. Academic Medicine 78(12): 1271-1280
13. Escobar-Alvarez SN, Myers ER (2013) The Doris Duke clinical scientist development award: implications for early-career physician scientists. Academic Medicine 88(11): 1-7
14. Berg JM (2013) Scientific approaches to science policy. Molecular Biology of the Cell 24: 3273-3274
15. Danthi N, Wu CO, Shi P, Lauer MS (2014) Percentile ranking and citation impact of a large cohort of NHLBI-funded cardiovascular R01 grants. Circ. Res. 114(4): 600-6
16. Gallo SA, Carpenter AS, Glisson SR (2013) Teleconference versus face-to-face scientific peer review of grant application: effects on review outcomes. PLoS ONE 8(8)
17. Citation Averages, 2000-2010, By Fields and Years http://www. timeshighereducation.co.uk/415643.article (2011) Accessed 2013 March 10
18. Fortin JM, Currie DJ (2013) Big science vs. little science: how scientific impact scales with funding. PLoS ONE 8(6)
19. Wadman M (2010) Study says middle-sized labs do best. Nature 468: 356-357
20. Glenn JF (2010) Uses and needs for peer review in army medical research. Technology and Innovation 12: 241-247
21. Markowitz HM (1991) Foundations of portfolio theory. The Journal of Finance 46: 469-477
22. Galis ZS, Hoots WK, Kiley JP, Lauer MS (2012) On the value of portfolio diversity in heart, lung, and blood research. Am J Respir Crit Care Med 186(7): 575-578
23. Ioannidis JPA (2011) More time for research: fund people not projects. Nature 477(7366): 529-531
24. Lauer MS (2014) Personal reflections on big science, small science, or the right mix. Circ. Res. 114: 1080-1082
25. Van Raan AFJ (2004) Measuring science. In: Handbook of Quantitative Science and Technology Research (eds, HF Moed et al.), Kluwer Academic Publishers, Dordrecht, Netherlands. 19-50
26. Braun T, Bergstrom CT, Frey BS, Osterloh M, West JD, et al. (2010) How to improve the use of metrics. Nature 465(17) 870-872
27. Cabezas-Clavijo A, Robinson-Garcia N, Escabias M, Jiménez-Contreras E (2013) Reviewers' ratings and bibliometric indicators: hand in hand when assessing over research proposals? PLoS ONE 8(6)
28. Van Noorden R (2010) Metrics: a profusion of measures. Nature 465(7300): 864-866
29. de Winter JCF, Zadpoor AA, Dodou D (2014) The expansion of google scholar versus web of science: a longitudinal study. Scientometrics 98(2): 1547-1565
30. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, et al. (2008) Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS ONE 3(8)
31. Lane J (2010) Let's make science metrics more scientific Nature 464(25): 488-489.