Cost of the nserc science grant peer review system exceeds the cost of giving every qualified researcher a baseline grant

Accountability in Research

Volumn 16, Issue 1, 2009, Pages 13-40

  Gordon, Richard ^a   Poulin, Bryan J ^b  

^a UNIVERSITY OF MANITOBA   (Canada)

^b LAKEHEAD UNIVERSITY   (Canada)

Author keywords

Discovery Grants Program; Excellence; NSERC; Peer review; Positive feedback

Indexed keywords

ARTICLE; CANADA; COST BENEFIT ANALYSIS; ECONOMICS; FINANCIAL MANAGEMENT; HUMAN; PEER REVIEW; PERSONNEL; STATISTICS;

CANADA; COST-BENEFIT ANALYSIS; FINANCING, GOVERNMENT; HUMANS; PEER REVIEW, RESEARCH; RESEARCH PERSONNEL; RESEARCH SUPPORT AS TOPIC;

EID: 68249122947     PISSN: 08989621     EISSN: 15455815     Source Type: Journal    
DOI: 10.1080/08989620802689821     Document Type: Article

References (87)

1. AUCC. (2007). Trends in Higher Education. Volume 2. Faculty. Ottawa: Association of Universities and Colleges of Canada. Available at: http://www.aucc.ca/-pdf/ english/publications/trends-2007-vol2-e.pdf, accessed January 20, 2009.
2. AUCC. (2008). Trends in Higher Education. Volume 3. Finance. Ottawa: Association of Universities and Colleges of Canada. Available at: http://www.aucc.ca/-pdf/ english/publications/trends-2008-vol3-e.pdf, accessed January 20, 2009.
3. Belousov, B. P. (1958). Oscillation reaction and its mechanisms [Russian]. In: Collection of Abstracts on Radiation Medicine. Moscow: Medgiz, pp. 145.
4. Benos, D. J., Bashari, E., Chaves, J. M., Gaggar, A., Kapoor, N., LaFrance, M., Mans, R., Mayhew, D., McGowan, S., Polter, A., Qadri, Y., Sarfare, S., Schultz, K., Splittgerber, R., Stephenson, J., Tower, C., Walton, R. G., and Zotov, A. (2007). The ups and downs of peer review. Adv Physiol Educ, 31: 145-152.
5. Berezin, A. A., and Hunter, G. (1994). Myth of competition and NSERC policy of selectivity. Canadian Chemical News, 46: 4-5.
6. Berezin, A. A., Gordon, R., and Hunter, G. (1995a). Anonymous peer review and the QWERTY effect. Amer. Physics Soc. News, 4: 7.
7. Berezin, A. A., Gordon, R., and Hunter, G. (1995b). Forum: Lifting the pernicious veil of secrecy; peer reviewers should shed the mask of anonymity argue three Canadian academics. New Scientist, 145: 46-47.
8. Berezin, A. A., Hunter, G., Pear, J. J., and Rangacharyulu, C. (1998). Canadian research councils need fundamental reform. OCUFA FORUM (Ontario Confederation of University Faculty Associations), (Fall): 8-13.
9. Berezin, A. A., Gordon, R., Hunter, G., and Poulin, B. J. (2001). Sliding funding scale deserves a try. CAUT Bulletin, (September). Available at: http:// www.cautbulletin.ca/en-article.asp?SectionID=684&SectionName= Letters& VolID=178&VolumeName=No%207&VolumeStartDate=689/681/ 2001& EditionID=2020&EditionName=Vol%2048&EditionStartDate=2001/ 2001/ 2001&ArticleID=1764, accessed January 20, 2009.
10. Birenbaum, R. (1993). The fine art of grantsmanship. University Affairs, Association of Universities and Colleges of Canada, 34: 6-8.
11. Braben, D. W. (2004). Pioneering Research: A Risk Worth Taking. New York: John Wiley and Sons.
12. Brainard, J. (2007). Teaching math and science may get boost from Congress. Chronicle of Higher Education, 53. Available at: http://chronicle.com/weekly/ v53/i49/49a00101.htm, accessed January 20, 2009.
13. Catano, V. M., Haines, T., Kirpalani, H., Shannon, H., Stringer, B., and Lozanski, L. (2007). Occupational Stress Among Canadian University Academic Staff. Available at: http://www.caut.ca/en/publications/healthandsafety/ CAUTStress Study-EN.pdf. Ottawa: Canadian Association of University Teachers.
14. Catano, V. M., and Turk, J. (2007). Fraud and misconduct in scientific research: A definition and procedures for investigation. Med Law, 26: 465-476.
15. Churchill, W. (1947). 11 November. The Official Report, House of Commons (5th Series), 444(Nov. 11): 206-207.
16. CIHR. (2007). 45th Meeting of Governing Council. Available at: http://www. irsc-cihr.gc.ca/e/34357.html, accessed January 20, 2009.
17. Cole, J. R., and Stephen, C. (1981). Peer Review in the National Science Foundation: Phase Two of a Study. Washington, D.C.: National Academy Press.
18. Cole, S., Cole, J. R., and Simon, G. A. (1981). Chance and consensus in peer review. Science, 214: 881-886.
19. de Solla Price, D. (1975). Science Since Babylon. New Haven: Yale University Press.
20. Drucker, P. F. (1973). Management: Tasks, Responsibilities, Practices. New York: Harper and Row.
21. Elgar, F. J. (2003). PhD Completion in Canadian Universities: Final Report. Halifax: Dalhousie University. Available at: http://careerchem.com/ CAREER-INFO-ACADEMIC/Frank-Elgar.pdf. Graduate Students Association of Canada.
22. Ethics Resource Center. (2006). How Employees Perceive Ethics at Work: 2005 National Business Ethics Survey. Washington, DC: Ethics Resource Center.
23. Ferrell, O. C., Fraedrich, J., and Ferrell, L. (2008). Business Ethics: Ethical Decision Making and Cases. New York: Houghton Mifflin.
24. Fletcher, J. M. (1992). Venture capital in Canada: Redesigning the pipeline. Canadian Investment Review, (Spring): 99-103.
25. Forsdyke, D. R. (1989). A systems analyst asks about AIDS research funding. Lancet, 2: 1382-1384.
26. Forsdyke, D. R. (2000). Tomorrow's Cures Today? How to Reform the Health Research System. London: Gordon and Breach.
27. Galanski, M., and Keppler, B. K. (2007). Searching for the magic bullet: anticancer platinum drugs which can be accumulated or activated in the tumor tissue. Anticancer Agents Med Chem, 7: 55-73.
28. Garfield, E., and Welljams-Dorof, A. (1990). The impact of fraudulent research on the scientific literature. The Stephen E. Breuning case. JAMA, 263: 1424-1426.
29. Glassford, R. G., Moriarty, D., and Redmond, G. (1980). Grantsmanship. Vanier, Ontario: CAHPER Research Committee.
30. Goonatilake, S. (1984). Aborted Discovery, Science & Creativity in the Third World. London: Zed Books.
31. Gordon, R. (1993). Grant agencies versus the search for truth. Accountability in Research: Policies and Quality Assurance, 2: 297-301.
32. Gordon, R. (1996). Viewpoint: A rough analysis of funding inequity in the Faculty of Medicine, University of Manitoba. Prairie Med. J., 66: 131-132.
33. Gordon, R. (1999). The Hierarchical Genome and Differentiation Waves: Novel Unification of Development, Genetics and Evolution. Singapore and London: World Scientific and Imperial College Press. Available at: http://www.wspc.com.sg/ books/lifesci/2755.html, accessed January 21, 2009.
34. Gordon, R., and Poulin, B. J. (1999). Testimony at Standing Committee on Health, Canadian Parliament, December 1, 1999, on Bill C-13, An Act to establish the Canadian Institutes of Health Research, to repeal the Medical Research Council Act and to make consequential amendments to other Acts, The Minister of Health. Available at: http://www.parl.gc.ca/InfoComDoc/36/ 2/HEAL/Meetings/ Evidence/healev07-e.htm, accessed September 12, 2000.
35. Gozlan, M. (2005). Turning Medical Dogma on Its Head: Nobel Prize Winner Barry Marshall, MBBS, Medscape Med Students. 7(2). Available at: http:// www.medscape.com/viewarticle/514219, accessed January 21, 2009.
36. Hawkins, R. W., Langford, C. H., and Sidhu, K. S. (2006). University research in an 'innovation society'. In: Blue Sky II, 2006 Conference, What Indicators for Science, Technology and Innovation Policies in the 21st Century, Ottawa, Canada, 26 September. Eds. Paris, Organisation for Economic Co-operation and Development (OECD). Available at: http://www.oecd.org/dataoecd/ 23/50/37436286.pdf, accessed January 21, 2009.
37. Hodges, A. (1983). Alan Turing: The Enigma. New York: Simon and Schuster.
38. Hodgson, C. (1997). How reliable is peer review? An examination of operating grant proposals simultaneously submitted to two similar peer review systems. Journal of Clinical Epidemiology, 50: 1189-1195.
39. Hughes, P. (1994). The meteorologist who started a revolution. Weatherwise, 47: 1-29. Available at: http://pangaea.org/wegener.htm, accessed January 21, 2009.
40. Humphreys, L. G., Hower, J., Westley, J., Kyburg Jr., H. E., Cole, S., Cole, J. R., and Simon, G. A. (1982). NSF Peer Review (Continued). Science, 215: 346-348.
41. Kapur, D. M. J. (2005). Give Us Your Best and Brightest: The Global Hunt for Talent and Its Impact on the Developing World. Washington, D.C.: Center for Global Development. Available at: http://www.cgdev.org/content/ publications/detail/4415/, accessed January 21, 2009.
42. Langford, C. H., Hall, J., Josty, P., Matos, S., and Jacobson, A. (2006). Indicators and outcomes of Canadian university research: Proxies becoming goals? Research Policy, 35: 1586-1598.
43. Lauffer, A. D. M. L. W. S. E., Dluhy, M., Lawrence, W., and Snyder, E. (1977). Grantsmanship. Beverly Hills, CA: Sage Publications.
44. Laznicka, P. (1993). The dark side of grantsmanship. University Affairs (Association of Universities and Colleges of Canada), 34: 26.
45. Leavitt, D. (2006). The Man Who Knew too Much: Alan Turing and the Invention of the Computer. New York: W. W. Norton.
46. Lewin, B. (1989). Fraud and the fabric of science. Cell, 57: 699-700.
47. Manning, P. (2007). You want liberation? Globe and Mail, (February 23): A20.
48. Marchildon, L. (2007). NSERC reviews Discovery Grant Program and GSC Structure/CAP Position Paper on NSERC International Review of the Discovery Grants Program. CAP News (Canadian Association of Physicists), 7(4): 1.
49. Mayo, N. E., Brophy, J., Goldberg, M. S., Klein, M. B., Miller, S., Platt, R. W., and Ritchie, J. (2006). Peering at peer review revealed high degree of chance associated with funding of grant applications. Journal of Clinical Epidemiology, 59: 842-848.
50. Meikle, M., Wilson, E., and Gordon, R. (1992). The Science Game: Interview of Richard Gordon (Quirks and Quarks). Audio tape, February 15, 1992. Toronto: Canadian Broadcasting Corp. Available at: http://www.cbc.ca/quirks/archives/92-93/oct1092.htm, accessed January 21, 2009.
51. Meikle, M., Wilson, E., Osmond, D., and Forsdyke, D. R. (1995). Quirks and Quarks: Interview of Dan Osmond and Don Forsdyke. Audio tape, May 13, 1995. Toronto: Canadian Broadcasting Corp.
52. Milsum, J. H., ed. (1968). Positive Feedback: A General Systems Approach to Positive/Negative Feedback and Mutual Causality. Oxford: Pergamon Press.
53. Mitchell, J. (2006). A Review of NSERC and SSHRC. Ottawa, Industry Canada. Available at: http://www.ryerson.ca/ors/news-archive/download/Federal- Review-of%20NSERC-and%20SSHRC.doc, accessed January 21, 2009.
54. Munro, M. (2008). Cancer researcher fired over faked data: Ottawa institute discovers past misconduct. National Post, (June 19): A10.
55. Nicholson, P., Davies, G., Kaspi, V., Mayer, L. A., Olsen, K., Robson, A., Winnacker, E.-L., Salcudean, M., and Tibshirani, R. (2008). Report of the International Review Committee on the Discovery Grants Program. Ottawa: Natural Sciences and Engineering Research Council. Available at: http://www.nserc.gc.ca/about/PDF/international-review-e.pdf, accessed June 10, 2008.
56. NIH. (2006a). Average Cost of Competing and Non-Competing Research Grants. Available at: http://grants.nih.gov/grants/award/research/RGAVG06.htm, accessed January 21, 2009.
57. NIH. (2006b). Research Project Success Rates by Type and Activity for 2006. Available at: http://grants.nih.gov/grants/award/success/Success- ByActivity.cfm, accessed January 21, 2009.
58. NSERC. (2005). Facts and Figures: Discovery Grants Competition Results, 1995-2005. Available at: http://www.nserc.gc.ca/about/stats/2004-2005/table show-e.asp?tableValue=27, accessed October 3, 2007.
59. NSERC. (2006). Peer Review Manual 2006: Appendix 1. Guidelines Governing Membership of Selection Committees and Panels. Available at: http:// www.nserc.gc.ca/commit/prm2006/app1-e.pdf, accessed September 27, 2007.
60. NSERC (2007a). NSERC Discovery Grants Information Session. Ottawa: Natural Sciences and Engineering Research Council Canada.
61. NSERC. (2007b). Inter-agency Funding Mechanism. Available at: http://www.nserc.gc.ca/about/initiatives/Tri-Agency/2007-06-06-e. htm#interagency, accessed October 3, 2007.
62. NSERC (2008a). Discovery Programs Budget 2007-08. Ottawa: Natural Sciences and Engineering Research Council of Canada. Available at: http://www.nserc.gc.ca/about/2006-2007-figures-e.pdf, accessed September 23, 2008.
63. NSERC. (2008b). Eligibility Criteria. Available at: http://www.nserc.gc. ca/professors-e.asp?nav=profnav&lbi=11a1, accessed June 11, 2008.
64. NSERC. (2008c). List of Eligible Institutions. Available at: http://www.nserc.gc.ca/institution/list-inst-e.htm, accessed September 24, 2008.
65. OECD (2007). Education at a Glance 2007: OECD Indicators. Paris: Organisation for Economic Co-Operation and Development. Available at: http://www.oecd.org/ dataoecd/4/55/39313286.pdf, accessed January 21, 2009.
66. Office of Inspector General (2006). Annual Audit Plan, FY 2007. Washington, DC, National Science Foundation. Available at: http://www.nsf.gov/ oig/FY2007-Annual-Audit-Plan.pdf, accessed October 3, 2007.
67. Pharaon, S. (2007). Open peer review: A route to democracy. J R Soc Med, 100: 9.
68. Polanyi, J. C. (2007). You want innovation? Liberate Canada's scientists. Globe and Mail, (February 21): A21.
69. Poulin, B. J., and Gordon, R. (2001). How to organize science funding: The new Canadian Institutes for Health Research (CIHR), an opportunity to vastly increase innovation. Canadian Public Policy, 27: 95-112.
70. Pouris, A. (2007). Estimating R&D expenditures in the higher education sector. Minerva, 45: 3-16.
71. Robinson, G. G. C., and Lewthwaite, B. (2007). Centres for Research in Youth, Science Training and Learning (CRYSTAL). Available at: http://www.umanitoba.ca/outreach/crystal/index.html, accessed January 21, 2009.
72. Root-Bernstein, R. S. (1989). Breaking faith. Sciences (New York), 29: 8-11.
73. Sedra, A. (2008). Recommendations to the Natural Sciences and Engineering Research Council, Grant Selection Committee Structure Review Advisory Committee. Ottawa: Natural Sciences and Engineering Research Council. Available at: http:// www.nserc.gc.ca/about/GSC-final-report-e.pdf, accessed June 10, 2008.
74. Smith, S. C. (1997). The postdoc trap. University Affairs (Association of Universities and Colleges of Canada), 38: 16-17.
75. Smith, S. H., and McLean, D. D. (1988). ABC's of Grantsmanship. Reston, VA: CAHPERD.
76. Statistics Canada (2004). Salaries and Salary Scales of Full-time Teaching Staff at Canadian Universities, 2003-2004. Ottawa: Postsecondary and Adult Learning Section, Statistics Canada. Available at: http://www.statcan.ca/english/research/81-595-MIE/81-595-MIE2004019.pdf, accessed January 21, 2009.
77. Taylor, M., and Yasmeen, G. (2008). Results of the grants and fellowships competitions held between September 2007 and March 2008. Available at: http:// www.sshrc.ca/web/whatsnew/peer-review-letter-e.asp#results, accessed January 21, 2009.
78. Tibbetts, J. (2007). Canada trails world in science grads. Falls below OECD average, risks being left behind in knowledged-based economy: Study. Ottawa Citizen, (September 19). Available at: http://www.canada.com/ ottawacitizen/news/ story.html?id=9c05ed71-04cb01-04deb-af83- dd6385f6389abc6382&k=52985, accessed September 19, 2007.
79. Titus, S. L., Wells, J. A., and Rhoades, L. J. (2008). Repairing research integrity: A survey suggests that many research misconduct incidents in the United States go unreported to the Office of Research Integrity. Nature 453: 980-982.
80. Tornow, J. (2006). Award Size, Duration and Success Rates. Available at: http:// www.nsf.gov/od/ipamm/ipamm-jtornow-nsb-update-0605.pdf, accessed January 21, 2009.
81. Tornow, J. (2007). Impact of Proposal and Award Management Mechanisms. U.S. National Science Foundation. Available at: http://www.nsf.gov/pubs/2007/ nsf0745/nsf0745.pdf, accessed January 21, 2009.
82. Turing, A. M. (1952). The chemical basis of morphogenesis. Phil. Trans. Roy. Soc. London, B237: 37-72.
83. Varmus, H. (2008). Advancing Global Health by Encouraging Medical Science Worldwide. Available at: http://www.fcihr.ca/news, http://mms//windows. stream.yorku.ca/faculty/abel/as/abelfriesen.wmv, accessed September 24, 2008.
84. Vella, F., Mietchen, D., and Gabaldon, T. (2004). Perceptions about postdocs. EMBO Rep, 5: 1104.
85. Warwick, A. (1998). Exercising the student body: Mathematics and Athleticism in Victorian Cambridge. In: Science Incarnate: Historical Embodiments of Natural Knowledge. C. Lawrence and S. Shapin, eds. Chicago: University of Chicago Press, pp. 288-326.
86. Winfree, A. T. (1984). The prehistory of the Belousov-Zhabotinsky oscillator. J. Chem. Ed., 61: 661-663.
87. Yin, R. K. (1989). Case Study Research: Design and Methods. Newbury Park, CA: Sage Publications.