NIH peer review percentile scores are poorly predictive of grant productivity

eLife

Volumn 5, Issue FEBRUARY2016, 2016, Pages

  Fang, Ferric C ^a   Bowen, Anthony ^b   Casadevall, Arturo ^c  

^a UNIVERSITY OF WASHINGTON SCHOOL OF MEDICINE   (United States)

^b ALBERT EINSTEIN COLLEGE OF MEDICINE   (United States)

^c JOHNS HOPKINS BLOOMBERG SCHOOL OF PUBLIC HEALTH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

PEER REVIEW; PRODUCTIVITY; MEDICAL RESEARCH; NATIONAL HEALTH ORGANIZATION; UNITED STATES;

BIOMEDICAL RESEARCH; EFFICIENCY; NATIONAL INSTITUTES OF HEALTH (U.S.); PEER REVIEW, RESEARCH; UNITED STATES;

EID: 84961942967     PISSN: None     EISSN: 2050084X     Source Type: Journal    
DOI: 10.7554/eLife.13323     Document Type: Article

References (22)

1. Abdoul H, Perrey C, Amiel P, Tubach F, Gottot S, Durand-Zaleski I, Alberti C, Gagnier JJ. 2012. Peer review of grant applications: criteria used and qualitative study of reviewer practices. PLoS ONE 7: e46054. doi: 10.1371/journal.pone.0046054
2. Berg J. 2011. Productivity metrics and peer review scores. NIGMS Feedback Loop Blog https://loop. nigms.nih.gov/2011/06/productivity-metrics-and- peer-review-scores/. accessed 15 Dec 2015.
3. Berg J. 2013. On deck chairs and lifeboats. ASBMB Today:-. Available at: http://www.asbmb.org/asbmbtoday/asbmbtoday_article.aspx?id=32362.
4. Costello LC. 2010. Is NIH funding the "best science by the best scientists"? A critique of the NIH R01 research grant review policies. Academic Medicine 85:775-779. doi: 10.1097/ACM.0b013e3181d74256
5. Danthi N, Wu CO, Shi P, Lauer M. 2014. Percentile ranking and citation impact of a large cohort of national heart, lung, and blood institute-funded cardiovascular R01 grants. Circulation Research 114: 600-606. doi: 10.1161/CIRCRESAHA.114.302656
6. Fang FC, Casadevall A. 2009. NIH peer review reform-change we need, or lipstick on a pig? Infection and Immunity 77:929-932. doi: 10.1128/IAI.01567-08
7. Fang FC, Casadevall A. 2014. Taking the powerball approach to medical research. Wall Street Journal http://www.wsj.com/articles/SB1000142405270230353270457 9477530153771424. accessed 15 Dec 2015.
8. GAO. 2009. National institutes of health: completion of comprehensive risk management program essential to effective oversight. GAO-09-687.
9. Germain RN. 2015. Healing the NIH-funded biomedical research enterprise. Cell 161:1485-1491. doi: 10.1016/j.cell.2015.05.052
10. Graves N, Barnett AG, Clarke P. 2011. Funding grant proposals for scientific research: retrospective analysis of scores by members of grant review panel. BMJ 343:d4797. doi: 10.1136/bmj.d4797.explorer-grants. accessed 15 Dec 2015.
11. Johnson VE. 2008. Statistical analysis of the National Institutes of Health peer review system. Proceedings of the National Academy of Sciences of the USA 105:11076-11080. doi: 10.1073/pnas.0804538105
12. Kaplan D, Lacetera N, Kaplan C, Tregenza T. 2008. Sample size and precision in NIH peer review. PLoS ONE 3:e2761. doi: 10.1371/journal.pone.0002761
13. Lauer MS, Nakamura R. 2015. Reviewing peer review at the NIH. New England Journal of Medicine 373: 1893-1895. doi: 10.1056/NEJMp1507427
14. Li D, Agha L. 2015. Big names or big ideas: do peer-review panels select the best science proposals? Science 348:434-438. doi: 10.1126/science.aaa0185
15. Liaw A, Wiener M. 2002. Classification and regression by randomForest. R News 2:18-22.
16. Lindner MD, Nakamura RK, Smalheiser NR. 2015. Examining the predictive validity of NIH peer review scores. PLoS ONE 10:e0126938. doi: 10.1371/journal.pone.0126938
17. Mayo NE, Brophy J, Goldberg MS, Klein MB, Miller S, Platt RW, Ritchie J. 2006. Peering at peer review revealed high degree of chance associated with funding of grant applications. Journal of Clinical Epidemiology 59:842-848. doi: 10.1016/j.jclinepi.2005.12.007
18. Mervis J. 2015. NIH’s peer review stands up to scrutiny. Science 348:384. doi: 10.1126/science.348.6233.384
19. Pagano M. 2006. American idol and NIH grant review. Cell 126:637-638. doi: 10.1016/j.cell.2006.08.004
20. Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez J-C, Muller M. 2011. PROC: an open-source package for r and s+ to analyze and compare ROC curves. BMC Bioinformatics 12:77. doi: 10.1186/1471-2105-12-77
21. Wallace ML, Larivière V, Gingras Y. 2009. Modeling a century of citation distributions. Journal of Informetrics 3:296-303. doi: 10.1016/j.joi.2009.03.010
22. Williams RS. 2015. Study: peer review predicts success. The Scientist http://www.the-scientist.com/?articles.view/articleNo/42787/title/Study-Peer-Review-Predicts-Success/. accessed 15 Dec 2015.