Reviewer bias in single- versus double-blind peer review

Proceedings of the National Academy of Sciences of the United States of America

Volumn 114, Issue 48, 2017, Pages 12708-12713

  Tomkins, Andrew ^a   Zhang, Min ^b   Heavlin, William D ^a  

^a GOOGLE INC   (United States)

^b TSINGHUA UNIVERSITY   (China)

Author keywords

Double blind; Peer review; Scientific method

Indexed keywords

ACCESS TO INFORMATION; ARTICLE; AUTHOR; CONTROLLED STUDY; DATA MINING; DOUBLE BLIND PROCEDURE; FEMALE; HUMAN; INTERPRETATION BIAS; MALE; PEER REVIEW; PRIORITY JOURNAL; SINGLE BLIND PROCEDURE; UNIVERSITY; ETHICS; PROCEDURES; PUBLISHING; STATISTICAL MODEL; TRENDS; WRITING;

AUTHORSHIP; DOUBLE-BLIND METHOD; EDITORIAL POLICIES; FEMALE; HUMANS; MALE; MODELS, STATISTICAL; PEER REVIEW, RESEARCH; PUBLICATION BIAS; SINGLE-BLIND METHOD;

EID: 85035244720     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1707323114     Document Type: Article

References (30)

1. Tomkins A, Zhang M, Heavlin WD (2017) Single versus double blind reviewing at WSDM 2017. arXiv:1702.00502.
2. Lamont M (2010) How Professors Think: Inside the Curious World of Academic Judgment (Harvard Univ Press, Cambridge, MA).
3. Snodgrass R (2006) Single-versus double-blind reviewing: An analysis of the literature. ACM Sigmod Rec 35:8–21.
4. Largent EA, Snodgrass RT (2016) Blind peer review by academic journals. Blinding as a Solution to Bias: Strengthening Biomedical Science, Forensic Science, and Law, eds Robertson C, Kesselheim A (Academic, Cambridge, MA), pp 75–95.
5. Snodgrass RT (2007) Editorial: Single- versus double-blind reviewing. ACM Trans Database Syst (TODS) 32:1–29.
6. McKinley KS (2008) Improving publication quality by reducing bias with double-blind reviewing and author response. ACM Sigplan Not 43:5–9.
7. Schulzrinne H (2009) Double-blind reviewing: More placebo than miracle cure? ACM SIGCOMM Comput Commun Rev 39:56–59.
8. Walker R, Rocha da Silva P (2015) Emerging trends in peer review—a survey. Front Neurosci 9:169.
9. Budden AE, et al. (2008) Double-blind review favours increased representation of female authors. Trends Ecol Evol 23:4–6.
10. Webb TJ, O’Hara B, Freckleton RP (2008) Does double-blind review benefit female authors? Trends Ecol Evol 23:351–353.
11. Madden S, DeWitt D (2006) Impact of double-blind reviewing on SIGMOD publication rates. SIGMOD Rec 35:29–32.
12. Tung AKH (2006) Impact of double blind reviewing on SIGMOD publication: A more detail analysis. SIGMOD Rec 35:6–7.
13. Hill S, Provost F (2003) The myth of the double-blind review?: Author identification using only citations. SIGKDD Explor Newsl 5:179–184.
14. Jaschik S (2011) Rejecting double blind. Available at https://www.insidehighered.com/news/2011/05/31/american-economic-association-abandons-double-blind-journal-reviewing. Accessed January 29, 2017.
15. Blank RM (1991) The effects of double-blind versus single-blind reviewing: Experimental evidence from the American economic review. Am Econ Rev 81:1041–1067.
16. Roberts SG, Verhoef T (2016) Double-blind reviewing at EvoLang 11 reveals gender bias. J Lang Evol 1:163–167.
17. Okike K, Hug KT, Kocher MS, Leopold SS (2016) Single-blind vs double-blind peer review in the setting of author prestige. JAMA 316:1315–1316.
18. Knobloch-Westerwick S, Glynn CJ, Huge M (2013) The Matilda effect in science communication. Sci Commun 35:603–625.
19. Merton RK (1968) The Matthew effect in science. Science 159:56–63.
20. Vrettas G, Sanderson M (2015) Conferences versus journals in computer science. J Assoc Inf Sci Technol 66:2674–2684.
21. CORPORATE Committee on Academic Careers for Experimental Scientists and CORPORATE Commission on Physical Sciences, Mathematics & Applications (1994) Academic Careers for Experimental Computer Scientists and Engineers (Natl Acad Press, Washington, DC).
22. Eckles D, Kizilcec RF, Bakshy E (2016) Estimating peer effects in networks with peer encouragement designs. Proc Natl Acad Sci USA 113:7316–7322.
23. Price S, Flach PA (2017) Computational support for academic peer review: A perspective from artificial intelligence. Commun ACM 60:70–79.
24. Tomkins A, Zhang M (2017) 2017 call for papers. Available at www.wsdmconference.org/2017/calls/papers/. Accessed January 29, 2017.
25. Luxburg U, Guyon I (2016) 2016 call for papers. Available at https://nips.cc/Conferences/2016/CallForPapers. Accessed February 13, 2017.
26. Peters DP, Ceci SJ (1982) Peer-review practices of psychological journals: The fate of published articles, submitted again. Behav Brain Sci 5:187–195.
27. Peters DP, Ceci SJ (2014) The Peters & Ceci study of journal publications. Available at https://thewinnower.com/discussions/7-the-peters-ceci-study-of-journal-publications. Accessed January 29, 2017.
28. Rothwell PM, Martyn CN (2000) Reproducibility of peer review in clinical neuroscience: Is agreement between reviewers any greater than would be expected by chance alone? Brain 123:1964–1969.
29. Lawrence N (2015) Get your NIPS reviews in! Available at inverseprobability.com/ 2015/07/23/get-your-nips-review-in. Accessed January 29, 2017.
30. Lawrence N (2015) NIPS experiment analysis. Available at inverseprobability.com/ 2015/03/30/nips-experiment-analysis. Accessed January 29, 2017.