Analysis of national institutes of health r01 application critiques, impact, and criteria scores: Does the sex of the principal investigator make a difference'

Academic Medicine

Volumn 91, Issue 8, 2016, Pages 1080-1088

  Kaatz, Anna ^a   Lee, You Geon ^a   Potvien, Aaron ^b   Magua, Wairimu ^a   Filut, Amarette ^a   Bhattacharya, Anupama ^a   Leatherberry, Renee ^a   Zhu, Xiaojin ^b   Carnes, Molly ^a,c  

^a UNIVERSITY OF WISCONSIN MADISON   (United States)

^b University of Wisconsin   (United States)

^c WILLIAM S MIDDLETON MEMORIAL VETERANS HOSPITAL   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FEMALE; HUMAN; MALE; MEDICAL RESEARCH; NATIONAL HEALTH ORGANIZATION; PEER REVIEW; PERSONNEL; SEX DIFFERENCE; SEXISM; STATISTICS AND NUMERICAL DATA; UNITED STATES;

BIOMEDICAL RESEARCH; FEMALE; HUMANS; MALE; NATIONAL INSTITUTES OF HEALTH (U.S.); PEER REVIEW, RESEARCH; RESEARCH PERSONNEL; SEX FACTORS; SEXISM; UNITED STATES;

EID: 84973375278     PISSN: 10402446     EISSN: 1938808X     Source Type: Journal    
DOI: 10.1097/ACM.0000000000001272     Document Type: Article

References (72)

1. Lautenberger O, Dandar V, Raezel C, Sloane R. The State of Women in Academic Medicine: The Pipeline and Pathways to Leadership 2013-2014. Washington, DC: Association of American Medical Colleges; 2014.
2. Committee on Bridges to Independence: Identifying Opportunities for and Challenges to Fostering the Independence of Young Investigators in the Life Sciences; Board on Life Sciences; Division on Earth and Life Studies; National Research Council. Bridges to Independence: Fostering the Independence of New Investigators in Biomedical Research. Washington, DC: National Academies Press; 2005.
3. Ley TJ, Hamilton BH. Sociology. The gender gap in NIH grant applications. Science. 2008; 322: 1472-1474.
4. Office of Extramural Research, National Institutes of Health. NIH research project grant program (R01). http://grants.nih.gov/.grants/funding/r01.htm. Accessed April 19, 2016.
5. Office of Extramural Research, National Institutes of Health. Grants and funding: New and early stage investigator policies. http://grants.nih.gov/grants/new-.investigators/. Accessed April 19, 2016.
6. Pohlhaus JR, Jiang H, Wagner RM, Schaffer WT, Pinn VW. Sex differences in application, success, and funding rates for NIH extramural programs. Acad Med. 2011; 86: 759-767.
7. National Institutes of Health. R01-equivalent grants: Success rates, by gender and type of application. NIH IMPAC, success rate file. http://report.nih.gov/NIHDatabook/.Charts/Default.aspx'showm = Y&chartId = 178&catId = 15. Accessed April 19, 2016.
8. Office of Extramural Research, National Institutes of Health. Peer review process. http://grants.nih.gov/grants/peer-review-.process.htm. Accessed April 19, 2016.
9. Biernat M. Stereotypes and shifting standards: Forming, communicating and translating person impressions. In: Devine P, Plant A, eds. Advances in Experimental Social Psychology. Vol 45. San Diego, Calif: Academic Press; 2012.
10. Biernat M, Kobrynowicz D. Gender- And race-based standards of competence: Lower minimum standards but higher ability standards for devalued groups. J Pers Soc Psychol. 1997; 72: 544-557.
11. Carnes M, Geller S, Fine E, Sheridan J, Handelsman J. NIH Director's Pioneer Awards: Could the selection process be biased against women'. J Womens Health (Larchmt). 2005; 14: 684-691.
12. Castilla EJ. Gender, race, and meritocracy in organizational careers. AJS. 2008; 113: 1479-1526.
13. Cleveland J, Stockdale MS, Murphy KR. Women and Men in Organizations: Sex and Gender Issues at Work. Mahwah, NJ: Lawrence Erlbaum Associates; 2000.
14. Foschi M. Double standards in the evaluation of men and women. Soc Psychol Q. August 1996: 237-254.
15. Heilman ME, Haynes MC. Subjectivity in the appraisal process: A facilitator of gender bias in work settings. In: Borgida E, Fiske ST, eds. Beyond Common Sense: Psychological Science in the Courtroom. Malden, Mass: Blackwell Publishers; 2008.
16. Kaatz A, Magua W, Zimmerman DR, Carnes M A quantitative linguistic analysis of National Institutes of Health R01 application critiques from investigators at one institution. Acad Med. 2015; 90: 69-75.
17. Lyness KS, Heilman ME. When fit is fundamental: Performance evaluations and promotions of upper-level female and male managers. J Appl Psychol. 2006; 91: 777-785.
18. Madera JM, Hebl MR, Martin RC. Gender and letters of recommendation for academia: Agentic and communal differences. J Appl Psychol. 2009; 94: 1591-1599.
19. Moss-Racusin CA, Dovidio JF, Brescoll VL, Graham MJ, Handelsman J. Science faculty's subtle gender biases favor male students. Proc Natl Acad Sci U S A. 2012; 109: 16474-16479.
20. Murphy KR, Cleveland JN. Understanding Performance Appraisal: Social, Organizational, and Goal-Based Perspectives. Thousand Oaks, Calif: Sage Publications, Inc.; 1995.
21. National Academy of Sciences, National Academy of Engineering, and Institute of Medicine Committee on Maximizing the Potential of Women in Academic Science and Engineering. Beyond Bias and Barriers: Fulfilling the Potential of Women in Academic Science and Engineering. Washington, DC: National Academies Press; 2007.
22. Schmader T, Whitehead J, Wysocki VH A linguistic comparison of letters of recommendation for male and female chemistry and biochemistry job applicants. Sex Roles. 2007; 57: 509-514.
23. Steinpreis RE, Anders KA. The impact of gender on the review of the curricula vitae of job applicants and tenure candidates: A national empirical study. Sex Roles. 1999; 41: 509-528.
24. Trix F, Psenka C. Exploring the color of glass: Letters of recommendation for female and male medical faculty. Discourse Soc. 2003; 14: 191-220.
25. Vescio TK, Gervais SJ, Snyder M, Hoover A. Power and the creation of patronizing environments: The stereotype-based behaviors of the powerful and their effects on female performance in masculine domains. J Pers Soc Psychol. 2005; 88: 658-672.
26. Waisbren SE, Bowles H, Hasan T, et al. Gender differences in research grant applications and funding outcomes for medical school faculty. J Womens Health (Larchmt). 2008; 17: 207-214.
27. Wenneras C, Wold A. Nepotism and sexism in peer-review. Nature. 1997; 387: 341-343.
28. Wilson KY. An analysis of bias in supervisor narrative comments in performance appraisal. Hum Relat. 2010; 63: 1903-1933.
29. Heilman ME. Description and prescription: How gender stereotypes prevent women's ascent up the organizational ladder. J Soc Issues. 2001; 57: 657-674.
30. Heilman ME, Haynes MC. No credit where credit is due: Attributional rationalization of women's success in male-female teams. J Appl Psychol. 2005; 90: 905-916.
31. Devine PG. Stereotypes and prejudice: Their automatic and controlled components. J Per Soc Psychol. 1989; 56: 5-18.
32. Valian V. Why So Slow'. The Advancement of Women. Cambridge, Mass: MIT Press; 1999.
33. Nosek BA, Smyth FL, Hansen JJ, et al. Pervasiveness and correlates of implicit attitudes and stereotypes. Eur Rev Soc Psychol. 2007; 18: 36-88.
34. National Institutes of Health. Side-by-side comparison of enhanced and former review criteria (research grants and cooperative agreements). http://grants. nih.gov/grants/peer/guidelines-general/.comparison-of-review-criteria.pdf. Accessed April 19, 2016.
35. National Institutes of Health. Enhancing peer review at NIH. http://enhancing-peer-review. nih.gov/index.html. Accessed April 19, 2016.
36. National Institutes of Health. New and early stage investigator policies. http://grants.nih. gov/grants/new-investigators/index.htm. Accessed April 19, 2016.
37. Jagsi R, Motomura AR, Griffith KA, Rangarajan S, Ubel PA. Sex differences in attainment of independent funding by career development awardees. Ann Intern Med. 2009; 151: 804-811.
38. Pennebaker J, Chung C, Ireland M, Gonzales A, Booth R. The Development and Psychometric Properties of LIWC2007. Austin, Tex: LIWC Net; 2007.
39. Isaac C, Chertoff J, Lee B, Carnes M. Do students' and authors' genders affect evaluations' A linguistic analysis of medical student performance evaluations. Acad Med. 2011; 86: 59-66.
40. Belotti F, Deb P, Manning WG, Norton EC. Twopm: Two-part models. Stata J. 2015; 15: 3-20.
41. Fletcher D, Mackenzie D, Villouta E. Modelling skewed data with many zeros: A simple approach combining ordinary and logistic regression. Env Ecol Stat. 2005; 12: 45-54.
42. Papke LE, Wooldridge JM. Econometric methods for fractional response variables with an application to 401 (K) plan participation rates. J Appl Econ. 1996; 11: 619-632.
43. Baulch B, Quisumbing A. Testing and adjusting for attrition in household panel data. Chronic Poverty Research Centre (CPRC) toolkit note. 2011. http://www.chronicpoverty. org/publications/details/testing-Andadjusting-for-Attrition-in-household-paneldata. Accessed April 25, 2016.
44. Cuddeback G, Wilson E, Orme JG, Combs-Orme T. Detecting and statistically correcting sample selection bias. J Soc Ser Res. 2004; 30: 19-33.
45. Fitzgerald J, Gottschalk P, Moffitt RA. An Analysis of Sample Attrition in Panel Data: The Michigan Panel Study of Income Dynamics. Cambridge, Mass: National Bureau of Economic Research; 1998.
46. Foster EM, Fang GY; Conduct Problems Research Group. Alternative methods for handling attrition: An illustration using data from the Fast Track evaluation. Eval Rev. 2004; 28: 434-464.
47. Miller R, Hollist C, Neil J. Attrition bias. In: Salkind N, ed. Encyclopedia of Measurement and Statistics. Vol. 1. Thousand Oaks, Calif: Sage; 2007.
48. National Institutes of Health. Definitions of criteria and considerations for research project grant (RPG/X01/R01/R03/R21/R33/.R34) critiques. http://grants.nih.gov/grants/.peer/critiques/rpg-D.htm#rpg-overall. Accessed April 19, 2016.
49. Uhlmann EL, Cohen GL. I think it, therefore it's true: Effects of self-perceived objectivity on hiring discrimination. Organ Behav Hum Decis Process. 2007; 104: 207-223.
50. Carnes M, Bland C. Viewpoint: A challenge to academic health centers and the National Institutes of Health to prevent unintended gender bias in the selection of clinical and translational science award leaders. Acad Med. 2007; 82: 202-206.
51. Kawakami K, Dovidio JF, van Kamp S. Kicking the habit: Effects of nonstereotypic association training and correction processes on hiring decisions. J Exp Soc Psychol. 2005; 41: 68-75.
52. Nosek BA, Smyth FL, Sriram N, et al. National differences in gender-science stereotypes predict national sex differences in science and math achievement. Proc Natl Acad Sci U S A. 2009; 106: 10593-10597.
53. Biernat M, Collins EC, Katzarska-Miller I, Thompson ER. Race-based shifting standards and racial discrimination. Pers Soc Psychol Bull. 2009; 35: 16-28.
54. Glick P, Fiske ST. The ambivalent sexism inventory: Differentiating hostile and benevolent sexism. J Person Soc Psychol. 1996; 70: 491-512.
55. Glick P, Fiske ST. An ambivalent alliance: Hostile and benevolent sexism as complementary justifications for gender inequality. In: Dixon J, Levine M, eds. Beyond Prejudice. Cambridge, Mass: Cambridge University Press; 2012.
56. King EB, Botsford W, Hebl MR, Kazama S, Dawson JF, Perkins A. Benevolent sexism at work: Gender differences in the distribution of challenging developmental experiences. J Manage. 2012; 38: 1835-1866.
57. Biernat M, Tocci MJ, Williams JC. The language of performance evaluations: Gender-based shifts in content and consistency of judgment. Soc Psychol Personal Sci. 2012; 3: 186-192.
58. Biernat M, Vescio TK. She swings, she hits, she's great, she's benched: Implications of gender-based shifting standards for judgment and behavior. Pers Soc Psychol Bull. 2002; 28: 66-77.
59. Heilman M. Sex bias in hiring women. Organ Behav Hum Perform. 1980; 26: 386-395.
60. van Ommeren J, de Vries RE, Russo G, van Ommeren M. Context in selection of men and women in hiring decisions: Gender composition of the applicant pool. Psychol Rep. 2005; 96: 349-360.
61. Isaac C, Lee B, Carnes M. Interventions that affect gender bias in hiring: A systematic review. Acad Med. 2009; 84: 1440-1446.
62. Madera JM, Hebl MR, Martin RC. Gender and letters of recommendation for academia: Agentic and communal differences. J Appl Psychol. 2009; 94: 1591-1599.
63. Pang B, Lee L. Opinion mining and sentiment analysis. Found Trend Inf Retr. 2008; 2: 1-35.
64. Glass G. Primary, secondary, and metaanalysis of research. Educ Res. 1976; 5: 3-5.
65. Lipsey MW, Wilson DB. The efficacy of psychological, educational, and behavioral treatment. Confirmation from meta-Analysis. Am Psychol. 1993; 48: 1181-1209.
66. Martell R, Lane D, Emrich C. Male-female differences: A computer simulation. Am Psychol. 1996; 51: 157-158.
67. Carnes M, Morrissey C, Geller SE. Women's health and women's leadership in academic medicine: Hitting the same glass ceiling'. J Womens Health (Larchmt). 2008; 17: 1453-1462.
68. Isaac C, Byars-Winston A, McSorley R, Schultz A, Kaatz A, Carnes ML A qualitative study of work-life choices in academic internal medicine. Adv Health Sci Educ Theory Pract. 2014; 19: 29-41.
69. Carnes M, Devine PG, Baier Manwell L, et al. The effect of an intervention to break the gender bias habit for faculty at one institution: A cluster randomized, controlled trial. Acad Med. 2015; 90: 221-230.
70. Bauer CC, Baltes BB. Reducing the effects of gender stereoytpes on performance evaluations. Sex Roles. 2002; 47: 465-476.
71. Duguid MM, Thomas-Hunt MC. Condoning stereotyping'. How awareness of stereotyping prevalence impacts expression of stereotypes. J Appl Psychol. 2015; 100: 343-359.
72. Kaatz A, Gutierrez B, Carnes M. Threats to objectivity in peer review: The case of gender. Trends Pharmacol Sci. 2014; 35: 371-373.