Peer review of grant applications in biology and medicine. Reliability, fairness, and validity

Scientometrics

Volumn 81, Issue 3, 2009, Pages 789-809

  Reinhart, Martin ^a  

^a UNIVERSITY OF BASEL   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77956619851     PISSN: 01389130     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11192-008-2220-7     Document Type: Article

References (54)

1. ARMSTRONG, P. W., CAVERSON, M. M., ADAMS, L., TAYLOR, M., OLLEY, P. M. (1997), Evaluation of the heart and stroke foundation of canada research scholarship program: Research productivity and impact, Canadian Journal of Cardiology, 13 (5): 507-516.
2. BAKANIC, V., MCPHAIL, C., SIMON, R. J. (1987), The manuscript review and decision-making process, American Sociological Review, 52 (5): 631-642.
3. BORNMANN, L., DANIEL, H. D. (2003), Begutachtung durch Fachkollegen in der Wissenschaft. Stand der Forschung zur Reliabilität, Fairness und Validität des Peer-Review-Verfahrens. In: SCHWARZ, S., TEICHLER, U. (Eds), Universität auf dem Prüfstand. Konzepte und Befunde der Hochschulforschung. Campus, Frankfurt am Main, pp. 211-230.
4. BORNMANN, L., DANIEL, H. D. (2005), Selection of research fellowship recipients by committee peer review. Reliability, fairness and predictive validity of Board of Trustees' decisions, Scientometrics, 63 (2): 297-320.
5. BORNSTEIN, R. F. (1991), The predictive validity of peer review: A neglected issue, Behavioral and Brain Sciences, 14 (1): 138-9.
6. BRODY, T., CARR, L., GINGRAS, Y., HAJJEM, C., HARNAD, S., SWAN, A. (2007), Incentivizing the open access research web: Publication-archiving, data-archiving and scientometrics, CTWatch Quarterly, 3 (3): 17-18.
7. CARTER, G. M. (1978), The Consequences of Unfunded NIH Applications for the Investigator and His Research, Rand Corporation, Santa Monica.
8. CARTER, G. M. (1974), Peer Review, Citations, and Biomedical Research Policy: NIH Grants to Medical School Faculty, Rand Corporation, Santa Monica.
9. CARTER, G. M. (1978), A Citation Study of the NIH Peer Review Process, Rand Corporation, Santa Monica.
10. CHAPMAN, G. B., MCCAULEY, C. (1994), Predictive validity of quality ratings of National Science Foundation graduate fellows, Educational and Psychological Measurement, 54 (2): 428-438.
11. CICCHETTI, D. V. (1991A), The reliability of peer review for manuscript and grant submissions: A cross- disciplinary investigation, Behavioral and Brain Sciences, 14 (1): 119-135.
12. CICCHETTI, D. V. (1991B), Reflections from the peer-review mirror, Behavioral and Brain Sciences, 14: 167-186.
13. CICCHETTI, D. V., SPARROW, S. A. (1981), Developing criteria for establishing interrater reliability of specific items: Applications to assessment of adaptive behavior. American Journal of Mental Deficiency, 86 (2): 127-137.
14. CLAVERIA, L. E., GUALLAR, E., CAMI, J., CONDE, J., PASTOR, R., RICOY, J. R., RODRIGUEZ, E., RUIZ- PALOMO, F., MUNOZ, E. (2000), Does peer review predict the performance of research projects in health sciences? Scientometrics, 47 (1): 11-23.
15. COLE, J. R., COLE, S. (1981), Peer Review in the National Science Foundation: Phase Two of a Study, National Academy Press, Washington D.C.
16. COLE, S., FIORENTINE, R. (1991), Discrimination against women in science: The confusion of outcome with process, In: H. ZUCKERMAN, J. R. COLE, J. T. BRUER (Eds), The Outer Circle: Women in the Scientific Community, Yale University Press, 205-226.
17. COLE, S., RUBIN, L., COLE, J. R. (1978), Peer Review in the National Science Foundation: Phase One of a Study, National Academy Press, Washington D.C.
18. DANIEL, H. D. (1993), Guardians of Science, VCH, New York.
19. DEMICHELI, V., PIETRANTONJ, C. (2004), Peer review for improving the quality of grant applications, The Cochrane Library, 4.
20. DEUTSCHE FORSCHUNGSGEMEINSCHAFT (2007), Jahresbericht 2006, Bonn.
21. DIRK, L. (1999), A measure of originality: The elements of science, Social Studies of Science, 29 (5): 765-776.
22. GARFIELD, E. (1979), Citation Indexing - Its Theory and Application in Science, Technology, and Humanities, John Wiley and Sons, New York.
23. GUETZKOW, J., LAMONT, M., MALLARD, G. (2004), What is originality in the humanities and the social sciences? American Sociological Review, 69 (2): 190-212.
24. HARNAD, S. (1983), Peer Commentary on Peer Review: A Case Study in Scientific Quality Control. Cambridge University Press.
25. HARNAD, S. (1985), Rational disagreement in peer review, Science, Technology, & Human Values, 10 (3): 55-62.
26. HARTMANN, I. (1990), Begutachtung in der Forschungsförderung - Die Argumente der Gutachter in der deutschen Forschungsgemeinschaft. Rita G. Fischer, Frankfurt am Main.
27. HARTMANN, I., NEIDHARDT, F. (1990), Peer review at the Deutsche Forschungsgemeinschaft, Scientometrics, 19(5): 419-425.
28. HEMLIN, S. (1993), Scientific quality in the eyes of the scientist. A questionnaire study, Scientometrics, 27 (1): 3-18.
29. HIRSCHAUER, S. (2004), Peer Review Verfahren auf dem Prüfstand: Zum Soziologiedefizit der Wissenschaftsevaluation, Zeitschrift für Soziologie, 33 (1): 62-83.
30. HOFFMANN, H., JOYE, D., KUHN, F., METRAL, G. (2002), Der SNF im Spiegel der Forschenden. SIDOS, Neuchâtel.
31. HOSMER, D. W., LEMESHOW, S. (2000), Applied Logistic Regression, John Wiley and sons, New York.
32. HOWARD, L., WILKINSON, G. (1999), Peer review and editorial decision-making, Neuroendocrinology Letters, 20 (5): 256-260.
33. KALTHOFF, H. (1999), Die Herstellung von Gewissheit: Firmenkredite und Risikoanalyse in Mitteleuropa. Frankfurter Institut für Transformationsstudien, Europa-Universität Viadrina.
34. KELLE, U., PREIN, G., BIRD, K. (1995), Computer-Aided Qualitative Data Analysis: Theory, Methods and Practice, Sage Publications.
35. LAKATOS, I. (1970), Falsification and the methodology of scientific research programmes, Criticism and the Growth of Knowledge, 4: 91-195.
36. LANGFELDT, L. (2001), The decision-making constraints and processes of grant peer review, and their effects on the review outcome, Social Studies of Science, 31 (6): 820-841.
37. LONGINO, H. E. (2002), The Fate of Knowledge, Princeton University Press.
38. LONKILA, M. (1995), Grounded theory as an emerging paradigm for CAQDAS. In: U. KELLE, G. PREIN, K. BIRD (Eds), Computer-Aided Qualitative Data Analysis. London: Sage.
39. MERTON, R. K. (1973), The normative structure of science. In: The Sociology of Science, University of Chicago Press, pp. 267-279.
40. NATIONAL SCIENCE FOUNDATION (n.d.), US NSF - Budget, Retrieved August 6, 2008, from http://www.nsf.gov/about/budget/.
41. NEIDHARDT, F. (1988), Selbststeuerung in der Forschungsförderung: Das Gutachterwesen der DFG, Westdeutscher Verlag.
42. OPPENHEIM, C. (1996), Do citations count? Citation indexing and the Research Assessment Exercise (RAE), Serials: The Journal for the Serials Community, 9 (2): 155-161.
43. PETERS, D. P., CECI, S. J. (1982), Peer-review practices of psychological journals: The fate of published articles, submitted again, Behavioral and Brain Sciences, 5 (2): 187-195.
44. REINHART, M. (forthcoming), Peer review and quality criteria in science funding: access point versus boundary organization.
45. REINHART, M. (2006), Peer Review, Retrieved June 26, 2008, from http://www.forschungsinfo.de/iq/agora/Peer%20Review/peer_review.html.
46. REINAHRT, M., SIRTES, D. (2006), Wieviel Intransparenz ist für Entscheidungen über exzellente Wissenschaft notwendig? IfQ Working Paper, 1: 27-36.
47. SCHWEIZERISCHER NATIONALFONDS. (2002), Stiftungsurkunde/Statuten, Retrieved August 14, 2007, from http://www.snf.ch/SiteCollectionDocuments/por_org_statuten_d.pdf.
48. SCHWEIZERISCHER NATIONALFONDS. (2007), Jahresbericht 2006, Bern.
49. SMITH, A., EYSENCK, M. (2002), The Correlation Between Rae Ratings and Citation Counts In Psychology, Department of Psychology, Royal Holloway, University of London, UK.
50. SOLOMON, M. (2001), Social Empiricism, MIT Press.
51. STRICKER, L. J. (1991), Disagreement among journal reviewers: No cause for undue alarm, Behavioral and Brain Sciences, 14: 163-164.
52. STRULIK, T. (2007), Evaluationen in der Wirtschaft - Rating-Agenturen und das Management des Beobachtetwerdens, Leviathan Sonderheft, 24: 288-314.
53. WELLER, A. C. (2001), Editorial Peer Review: Its Strengths and Weaknesses, Information Today.
54. WOOD, F., WESSELY, S. (1999), Peer review of grant applications: A systematic review. In: F. GODLEE, T. JEFFERSON (Eds), Peer Review in Health Sciences. BMJ Books, London, 14-31.