Research funding: The case for a modified lottery

mBio

Volumn 7, Issue 2, 2016, Pages

  Fang, Ferric C ^a   Casadevall, Arturo ^b  

^a UNIVERSITY OF WASHINGTON SCHOOL OF MEDICINE   (United States)

^b JOHNS HOPKINS BLOOMBERG SCHOOL OF PUBLIC HEALTH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

APPLICATION SYSTEM; COMMERCIAL PHENOMENA; COST BENEFIT ANALYSIS; EDITORIAL; FINANCE; FINANCIAL MANAGEMENT; FUND PROPOSAL; FUNDING; FUNDING ALLOCATION; FUNDING CRISIS; FUNDING LOTTERY; GENDER BIAS; HOPE; HUMAN; MEDICAL RESEARCH; MODIFIED LOTTERY; PEER REVIEW; PRIORITY JOURNAL; RACISM; REFORM; RESEARCH; RESEARCH FUNDING; UNITED STATES; ECONOMICS; PROCEDURES; STANDARDS;

BIOMEDICAL RESEARCH; FINANCING, ORGANIZED; HUMANS; PEER REVIEW, RESEARCH;

EID: 84965131648     PISSN: 21612129     EISSN: 21507511     Source Type: Journal    
DOI: 10.1128/mBio.00422-16     Document Type: Editorial

References (69)

1. Oliver J. 2014. Last Week Tonight. Episode 24, 9 November 2014. https://www.youtube.com/watch?v_9PK-netuhHA.
2. Szabo L. 23 April 2014. NIH director: budget cuts put U.S. science at risk. USA Today, McLean, VA. http://www.usatoday.com/story/news/nation/2014/04/23/nih-budget-cuts/8056113/.
3. Mervis J. 18 December 2015. Budget agreement boosts science. Science Insider. American Association for the Advancement of Science, Washington, DC. http://www.sciencemag.org/news/2015/12/updated-budgetagreement-boosts-us-science.
4. Fang FC, Casadevall A. 14 April 2014. Taking the Powerball approach to funding medical research. Wall Street Journal. http://www.wsj.com/articles/SB10001424052702303532704579477530153771424.
5. Li D, Agha L. 2015. Research funding. Big names or big ideas: do peerreview panels select the best science proposals? Science 348:434–438. http://dx.doi.org/10.1126/science.aaa0185.
6. Fang FC, Bowen A, Casadevall A. 2016. NIH peer review percentile scores are poorly predictive of grant productivity. Elife 5:e13323. http://dx.doi.org/10.7554/eLife.13323.
7. Danthi N, Wu CO, Shi P, Lauer M. 2014. Percentile ranking and citation impact of a large cohort of National Heart, Lung, and Blood Institutefunded cardiovascular R01 grants. Circ Res 114:600–606. http://dx.doi.org/10.1161/CIRCRESAHA.114.302656.
8. Berg J. 2013. On deck chairs and lifeboats. ASBMB Today 12(2):3–6.
9. Kaplan D, Lacetera N, Kaplan C. 2008. Sample size and precision in NIH peer review. PLoS One 3:e2761. http://dx.doi.org/10.1371/journal.pone.0002761.
10. Mayo NE, Brophy J, Goldberg MS, Klein MB, Miller S, Platt RW, Ritchie J. 2006. Peering at peer review revealed high degree of chance associated with funding of grant applications. J Clin Epidemiol 59: 842–848. http://dx.doi.org/10.1016/j.jclinepi.2005.12.007.
11. Graves N, Barnett AG, Clarke P. 2011. Funding grant proposals for scientific research: retrospective analysis of scores by members of grant review panel. BMJ 343:d4797. http://dx.doi.org/10.1136/bmj.d4797.
12. Marsh HW, Jayasinghe UW, Bond NW. 2008. Improving the peerreview process for grant applications: reliability, validity, bias, and generalizability. Am Psychol 63:160–168. http://dx.doi.org/10.1037/0003-066X.63.3.160.
13. Fogelholm M, Leppinen S, Auvinen A, Raitanen J, Nuutinen A, Väänänen K. 2012. Panel discussion does not improve reliability of peer review for medical research grant proposals. J Clin Epidemiol 65:47–52. http://dx.doi.org/10.1016/j.jclinepi.2011.05.001.
14. Abdoul H, Perrey C, Amiel P, Tubach F, Gottot S, Durand-Zaleski I, Alberti C. 2012. Peer review of grant applications: criteria used and qualitative study of reviewer practices. PLoS One 7:e46054. http://dx.doi.org/10.1371/journal.pone.0046054.
15. Tetlock P. 2005. Expert political judgment: how good is it? How can we know? Princeton University Press, Princeton, NJ.
16. Nicholson JM, Ioannidis JP. 2012. Research grants: conform and be funded. Nature 492:34–36. http://dx.doi.org/10.1038/492034a.
17. Kuhn TS. 1962. The Structure of Scientific Revolutions. University of Chicago Press, Chicago, IL.
18. Lee C. 28 May 2007. Slump in NIH funding is taking toll on research. Washington Post, Washington, DC. http://www.washingtonpost.com/wp-dyn/content/article/2007/05/27/AR2007052700794.html.
19. Casadevall A, Fang FC. 2016. Revolutionary science. mBio 7:e00158-16. http://dx.doi.org/10.1128/mBio.00158-16.
20. Bornmann L, Mutz R, Daniel H. 2007. Gender differences in grant peer review: a meta-analysis. J Informetr 1:226–238. http://dx.doi.org/10.1016/j.joi.2007.03.001.
21. Pohlhaus JR, Jiang H, Wagner RM, Schaffer WT, Pinn VW. 2011. Sex differences in application, success, and funding rates for NIH extramural programs. Acad Med 86:759–767. http://dx.doi.org/10.1097/ACM.0b013e31821836ff.
22. Ginther DK, Schaffer WT, Schnell J, Masimore B, Liu F, Haak LL, Kington R. 2011. Race, ethnicity, and NIH research awards. Science 333: 1015–1019. http://dx.doi.org/10.1126/science.1196783.
23. Hayden EC. 2015. Racial bias continues to haunt NIH grants. Nature 527:286–287. http://dx.doi.org/10.1038/527286a.
24. Wessely S. 1998. Peer review of grant applications: what do we know? Lancet 352:301–305. http://dx.doi.org/10.1016/S0140-6736(97)11129-1.
25. Day TE. 2015. The big consequences of small biases: a simulation of peer review. Res Policy 44:1266–1270. http://dx.doi.org/10.1016/j.respol.2015.01.006.
26. Cole S, Cole JR, Simon GA. 1981. Chance and consensus in peer review. Science 214:881–886. http://dx.doi.org/10.1126/science.7302566.
27. Hargrave-Thomas E, Yu B, Reynisson J. 2012. Serendipity in anticancer drug discovery. World J Clin Oncol 3:1–6. http://dx.doi.org/10.5306/wjco.v3.i1.1.
28. Mandel HG, Vesell ES. 2008. Declines in NIH R01 research grant funding. Science 322:189a. http://dx.doi.org/10.1126/science.322.5899.189a.
29. Siliciano R. 19 May 2007. Testimony before the Senate Committee on Appropriations: Subcommittee on Labor, Health and Human Services, Education, and Related Agencies. U.S. Senate Committee on Appropriations, Washington, DC.
30. Price M. 4 June 2013. A shortcut to better grantsmanship. Science Careers. American Association for the Advancement of Science, Washington, DC. http://sciencecareers.sciencemag.org/career_magazine/previous_issues/articles/2013_06_04/caredit.a1300119.
31. Benezra R. 2013. Grant applications: undo NIH policy to ease effect of cuts. Nature 493:480. http://dx.doi.org/10.1038/493480e.
32. Kaiser J. 2014. Biomedical funding. At NIH, two strikes policy is out. Science 344:350. http://dx.doi.org/10.1126/science.344.6182.350.
33. Atkinson RD, Ezell SJ, Giddings LV, Stewart LA, Andes SM. 2012. Leadership in decline: assessing U.S. international competitiveness in biomedical research. Information Technology and Innovation Foundation and United for Medical Research, Washington, DC. http://www2.itif.org/2012-leadership-in-decline.pdf.
34. Von Hippel T, von Hippel C. 2015. To apply or not to apply: a survey analysis of grant writing costs and benefits. PLoS One 10:e0118494. http://dx.doi.org/10.1371/journal.pone.0118494.
35. Clotfelter CT, Cook PJ. 1989. Selling hope: state lotteries in America. Harvard University Press, Cambridge, MA.
36. Basken P, Voosen P. 24 February 2014. Strapped scientists abandon research and students. Chronicle of Higher Education, Washington, DC. http://chronicle.com/article/Strapped-Scientists-Abandon/144921/.
37. Staw BM. 1976. Knee-deep in the big muddy: a study of escalating commitment to a chosen course of action. Organ Behav Hum Perform 16: 27–44. http://dx.doi.org/10.1016/0030-5073(76)90005-2.
38. Casadevall A, Fang FC. 2014. Specialized science. Infect Immun 82: 1355–1360. http://dx.doi.org/10.1128/IAI.01530-13.
39. Xue G, He Q, Lei X, Chen C, Liu Y, Chen C, Lu ZL, Dong Q, Bechara A. 2012. The gambler’s fallacy is associated with weak affective decision making but strong cognitive ability. PLoS One 7:e47019. http://dx.doi.org/10.1371/journal.pone.0047019.
40. Foster R. 26 September 2013. Optimism is a sine qua non for scientists. Times Higher Education, London, United Kingdom. https://www.timeshighereducation.com/features/optimism-is-a-sine-qua-nonfor-scientists/2007563.article.
41. Sharot T, Korn CW, Dolan RJ. 2011. How unrealistic optimism is maintained in the face of reality. Nat Neurosci 14:1475–1479. http://dx.doi.org/10.1038/nn.2949.
42. Mervis J. 2014. Peering into peer review. Science 343:596–598. http://dx.doi.org/10.1126/science.343.6171.596.
43. Fang FC, Casadevall A. 2009. NIH peer review reform—change we need, or lipstick on a pig? Infect Immun 77:929–932. http://dx.doi.org/10.1128/IAI.01567-08.
44. Bollen J, Crandall D, Junk D, Ding Y, Börner K. 2014. From funding agencies to scientific agency: collective allocation of science funding as an alternative to peer review. EMBO Rep 15:131–133. http://dx.doi.org/10.1002/embr.201338068.
45. Pagano M. 2006. American Idol and NIH grant review. Cell 126:637–638. http://dx.doi.org/10.1016/j.cell.2006.08.004.
46. Kaltman JR, Evans FJ, Danthi NS, Wu CO, DiMichele DM, Lauer MS. 2014. Prior publication productivity, grant percentile ranking, and topicnormalized citation impact of NHLBI cardiovascular R01 grants. Circ Res 115:617–624. http://dx.doi.org/10.1161/CIRCRESAHA.115.304766.
47. Ioannidis JPA. 2011. More time for research: fund people not projects. Nature 477:529–531. http://dx.doi.org/10.1038/477529a.
48. Malkiel BG. 1973. ARandom Walk down Wall Street. W. W. Norton, New York, NY.
49. Sommer J. 14 March 2015. How many mutual funds routinely rout the market? None. New York Times, New York, NY. http://www.nytimes.com/2015/03/15/your-money/how-many-mutual-fundsroutinely- rout-the-market-zero.html?_r_0.
50. Buffett WE. 2014. Berkshire Hathaway 2013 annual report, p 20. Berkshire Hathaway, Omaha, NE.
51. Taleb NN. 2007. The Black Swan: the Impact of the Highly Improbable. Random House, New York, NY.
52. Biondo AE, Pluchino A, Rapisarda A, Helbing D. 2013. Are random trading strategies more successful than technical ones? PLoS One 8:e68344. http://dx.doi.org/10.1371/journal.pone.0068344.
53. Knapp A. 22 March 2013. Computer simulation suggests that the best investment strategy is a random one. Forbes, Jersey City, NJ. http://www.forbes.com/sites/alexknapp/2013/03/22/computer-simulationsuggests-that-the-best-investment-strategy-is-a-random-one/#204abcd97e7c.
54. Scannell JW, Blanckley A, Boldon H, Warrington B. 2012. Diagnosing the decline in pharmaceutical R&D efficiency. Nat Rev Drug Discov 11: 191–200. http://dx.doi.org/10.1038/nrd3681.
55. Bowen A, Casadevall A. 2015. Increasing disparities between resource inputs and outcomes, as measured by certain health deliverables, in biomedical research. Proc Natl Acad Sci U S A 112:11335–11340. http://dx.doi.org/10.1073/pnas.1504955112.
56. National Institutes of Health. 2016. New and early stage investigator policies. Office of Extramural Research, National Institutes of Health, Bethesda, MD. http://grants.nih.gov/grants/new_investigators/.
57. Druss BG, Marcus SC. 2005. Tracking publication outcomes of National Institutes of Health grants. Am J Med 118:658–663. http://dx.doi.org/10.1016/j.amjmed.2005.02.015.
58. Merton RK. 1968. The Matthew effect in science: the reward and communication systems of science are considered. Science 159:56–63. http://dx.doi.org/10.1126/science.159.3810.56.
59. O’Sullivan F. 21 June 2010. Charter school lotteries leave lots behind. Care2, Redwood City, CA. http://www.care2.com/causes/charterlotteries-leave-lots-behind.html.
60. Persad G, Wertheimer A, Emanuel EJ. 2009. Principles for allocation of scarce medical interventions. Lancet 373:423–431. http://dx.doi.org/10.1016/S0140-6736(09)60137-9.
61. Annas GJ. 1985. The prostitute, the playboy, and the poet: rationing schemes for organ transplantation. AmJ Public Health 75:187–189. http://dx.doi.org/10.2105/AJPH.75.2.187.
62. McLachlan HV. 2012. A proposed non-consequentialist policy for the ethical distribution of scarce vaccination in the face of an influenza pandemic. J Med Ethics 38:317–318. http://dx.doi.org/10.1136/medethics-2011-100031.
63. Greenberg D. 11 February 2008. Peer review at NIH: a lottery would be better. Chronicle of Higher Education, Washington, DC. http://chronicle.com/blogs/brainstorm/peer-review-at-nih-a-lottery-wouldbe-better/5696.
64. Health Research Council of New Zealand. 2015. Funding opportunities. Explorer grants. Health Research Council of New Zealand, Auckland, NZ. http://www.hrc.govt.nz/funding-opportunities/researcher-initiatedproposals/explorer-grants.
65. National Science Board. 2013. Report to the National Science Board on the National Science Foundation’s merit review process fiscal year 2012. NSB-13-33. National Science Board, Arlington, VA.
66. Wilson EO. 2014. The Meaning of Human Existence. Liveright, New York, NY.
67. Freedman DH. November 2010. Lies, damned lies, and medical science. The Atlantic, Washington, DC. http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/308269/.
68. Fang FC, Casadevall A. 2012. Reforming science: structural reforms. Infect Immun 80:897–901. http://dx.doi.org/10.1128/IAI.06184-11.
69. Casadevall A, Fang FC. 2012. Reforming science: methodological and cultural reforms. Infect Immun 80:891–896. http://dx.doi.org/10.1128/IAI.06183-11.