Understanding and supporting anonymity policies in peer review

Journal of the Association for Information Science and Technology

Volumn 68, Issue 4, 2017, Pages 957-971

  Nobarany, Syavash ^a   Booth, Kellogg S ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER SUPPORT; PEER REVIEW; PEER-REVIEW PROCESS; SUPPORT SYSTEMS; TRADE OFF;

ECONOMIC AND SOCIAL EFFECTS;

HUMAN; IDENTITY; INFORMATION TECHNOLOGY; PEER REVIEW; PRACTICE GUIDELINE; PUBLICATION;

EID: 84971671398     PISSN: 23301635     EISSN: 23301643     Source Type: Journal    
DOI: 10.1002/asi.23711     Document Type: Article

References (61)

1. Abramowitz, S.I., Gomes, B., & Abramowitz, C.V. (1975). Publish or politic: Referee bias in manuscript review. Journal of Applied Social Psychology, 5(3), 187–200. http://doi.org/10.1111/j.1559-1816.1975.tb00675.x
2. Altman, D.G., Chalmers, I., & Herxheimer, A. (1994). Is there a case for an international medical scientific press council? JAMA: The Journal of the American Medical Association, 272(2), 166–167.
3. Bargh, J.A., McKenna, K.Y.A., & Fitzsimons, G.M. (2002). Can you see the real me? Activation and expression of the “true self” on the internet. Journal of Social Issues, 58(1), 33–48. http://doi.org/10.1111/1540-4560.00247
4. Bernstein, M.S., Little, G., Miller, R.C., Hartmann, B., Ackerman, M.S., Karger, D.R., … Panovich, K. (2010). Soylent: A word processor with a crowd inside. In Proceedings of the 23rd Annual ACM Symposium on User Interface Software and Technology (pp. 313–322). New York: ACM. http://doi.org/10.1145/1866029.1866078
5. Bingham, C.M., Higgins, G., Coleman, R., & Van Der Weyden, M.B. (1998). The Medical Journal of Australia internet peer-review study. The Lancet, 352(9126), 441–445. http://doi.org/10.1016/S0140-6736(97)11510-0
6. Blank, R.M. (1991). The effects of double-blind versus single-blind reviewing: Experimental evidence from the American economic review. The American Economic Review, 81(5), 1041–1067. http://doi.org/10.2307/2006906
7. Bornmann, L., & Daniel, H.-D. (2010). Reliability of reviewers' ratings when using public peer review: A case study. Learned Publishing, 23(2), 124–131. http://doi.org/10.1087/20100207
8. Bradley, J.V. (1981). Pernicious publication practices. Bulletin of the Psychonomic Society, 18(1), 31–34.
9. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. http://doi.org/10.1191/1478088706qp063oa
10. Briand, L., & van der Hoek, A. (2014). Insights and lessons learned from analyzing ICSE 2014 survey and review data (No. TR-SnT-2014-6). University of Luxembourg. Retrieved from: http://orbilu.uni.lu/handle/10993/16345
11. Bundy, A. (1983). How to programme chair IJCAI. Retrieved February 19, 2016, from: http://homepages.inf.ed.ac.uk/bundy/how-tos/ijcai-how-pc.txt
12. Campanario, J.M. (1998). Peer review for journals as it stands today—part 1 and 2. Science Communication, 19(3), 181–211. http://doi.org/10.1177/1075547098019003002
13. Chen, V.H.-H., Duh, H.B.-L., & Ng, C.W. (2009). Players who play to make others cry: The influence of anonymity and immersion. In Proceedings of the International Conference on Advances in Computer Enterntainment Technology (pp. 341–344). New York: ACM. http://doi.org/10.1145/1690388.1690454
14. Cohen, A.L., Cash, D., & Muller, M.J. (2000). Designing to support adversarial collaboration. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (pp. 31–39). New York: ACM. http://doi.org/10.1145/358916.358948
15. Derlega, V.J., & Chaikin, A.L. (1977). Privacy and self-disclosure in social relationships. Journal of Social Issues, 33(3), 102–115. http://doi.org/10.1111/j.1540-4560.1977.tb01885.x
16. Diakopoulos, N., & Naaman, M. (2011). Towards quality discourse in online news comments. In Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work (pp. 133–142). New York: ACM. http://doi.org/10.1145/1958824.1958844
17. Dourish, P., & Anderson, K. (2006). Collective information practice: Exploring privacy and security as social and cultural phenomena. Human–Computer Interaction, 21(3), 319–342. http://doi.org/10.1207/s15327051hci2103_2
18. Erickson, T., & Kellogg, W.A. (2000). Social translucence: An approach to designing systems that support social processes. ACM Transactions on Computer-Human Interaction, 7(1), 59–83. http://doi.org/10.1145/344949.345004
19. Ernst, E., & Resch, K.L. (1994). Reviewer bias: A blinded experimental study. The Journal of Laboratory and Clinical Medicine, 124(2), 178–182.
20. Godlee, F., Gale, C.R., & Martyn, C.N. (1998). Effect on the quality of peer review of blinding reviewers and asking them to sign their reports. JAMA: The Journal of the American Medical Association, 280(3), 237–240. http://doi.org/10.1001/jama.280.3.237
21. Grudin, J. (2013). Varieties of Conference Experience. The Information Society, 29(2), 71–77. http://doi.org/10.1080/01972243.2012.757263
22. Isenberg, S.J., Sanchez, E., & Zafran, K.C. (2009). The effect of masking manuscripts for the peer-review process of an ophthalmic journal. British Journal of Ophthalmology, 93(7), 881–884. http://doi.org/10.1136/bjo.2008.151886
23. Justice, A.C., Cho, M.K., Winker, M.A., Berlin, J.A., & Rennie, D. (1998). Does masking author identity improve peer review quality?: A randomized controlled trial. JAMA: The Journal of the American Medical Association, 280(3), 240–242.
24. Kang, R., Brown, S., & Kiesler, S. (2013). Why do people seek anonymity on the internet?: Informing policy and design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2657–2666). New York: ACM. Retrieved from: http://doi.org/10.1145/2470654.2481368
25. Katz, D.S., Proto, A.V., & Olmsted, W.W. (2002). Incidence and nature of unblinding by authors: Our experience at two radiology journals with double-blinded peer review policies. American Journal of Roentgenology, 179(6), 1415–1417. http://doi.org/10.2214/ajr.179.6.1791415
26. Kilner, P.G., & Hoadley, C.M. (2005). Anonymity options and professional participation in an online community of practice. In Proceedings of the 2005 Conference on Computer Support for Collaborative Learning: Learning 2005: The Next 10 Years! (pp. 272–280). Taipei, Taiwan: International Society of the Learning Sciences. Retrieved from: http://dl.acm.org/citation.cfm?id=1149293.1149328
27. Lampson, B.W. (1974). Protection. SIGOPS Operating Systems Review, 8(1), 18–24. http://doi.org/10.1145/775265.775268
28. Lee, C.J., Sugimoto, C.R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology, 64(1), 2–17. http://doi.org/10.1002/asi.22784
29. Lipworth, W., & Kerridge, I. (2011). Shifting power relations and the ethics of journal peer review. Social Epistemology, 25(1), 97–121. http://doi.org/10.1080/02691728.2010.534567
30. Mahoney, M.J. (1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research, 1(2), 161–175. http://doi.org/10.1007/BF01173636
31. Marchionini, G., Saracevic, T., Carroll, J., Kraft, D., Hersh, W., Mothe, J., … Schwartz, C. (2007). Reviewer merits. Information Processing & Management, 43(1), 1–2. http://doi.org/10.1016/j.ipm.2006.05.012
32. McCarty, R. (2002). Science, politics, and peer review: An editors dilemma. American Psychologist, 57(3), 198–201. http://doi.org/10.1037/0003-066X.57.3.198
33. McDonald, D.W., Gokhman, S., & Zachry, M. (2012). Building for social translucence: A domain analysis and prototype system. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (pp. 637–646). New York: ACM. Retrieved from: http://doi.org/10.1145/2145204.2145301
34. McNutt, R.A., Evans, A.T., Fletcher, R.H., & Fletcher, S.W. (1990). The effects of blinding on the quality of peer review. A randomized trial. JAMA: The Journal of the American Medical Association, 263(10), 1371–1376.
35. Melero, R., & López-Santoveña, F. (2001). Referees' attitudes toward open peer review and electronic transmission of papers. Food Science and Technology International, 7(6), 521 –527. http://doi.org/10.1106/0MXD-YM6F-3LM6-G9EB
36. Millen, D.R., & Patterson, J.F. (2003). Identity disclosure and the creation of social capital. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (pp. 720–721). New York: ACM. http://doi.org/10.1145/765891.765950
37. Mulligan, A., Hall, L., & Raphael, E. (2012). Peer review in a changing world: An international study measuring the attitudes of researchers. Journal of the American Society for Information Science and Technology, 64(1), 132–161. http://doi.org/10.1002/asi.22798
38. Nanavati, M., Taylor, N., Aiello, W., & Warfield, A. (2011). Herbert west: Deanonymizer. In Proceedings of the Sixth USENIX Conference on Hot Topics in Security (pp. 6–6). Berkeley, CA: USENIX Association. Retrieved from: http://dl.acm.org/citation.cfm?id=2028040.2028046
39. Nature's trial of open peer review. (2006). Retrieved from: http://www.nature.com/nature/peerreview/debate/nature05535.html
40. Outstanding referees program. (2013). Retrieved from: http://publish.aps.org/OutstandingReferees
41. Patton, M.Q. (1990). Qualitative evaluation and research methods. Newbury Park, CA: SAGE Publications. Retrieved from: http://psycnet.apa.org/psycinfo/1990-97369-000
42. Peters, D.P., & Ceci, S.J. (1982). Peer-review practices of psychological journals: The fate of published articles, submitted again. Behavioral and Brain Sciences, 5(2), 187–195. http://doi.org/10.1017/S0140525X00011183
43. Postmes, T., Spears, R., & Lea, M. (1998). Breaching or building social boundaries? SIDE-effects of computer-mediated communication. Communication Research, 25(6), 689–715. http://doi.org/10.1177/009365098025006006
44. Rainie, L., Kiesler, S., Kang, R., Madden, M., Duggan, M., Brown, S., & Dabbish, L. (2013). Anonymity, privacy, and security online. Pew Research Center. Retrieved from: http://www.davidellis.ca/wp-content/uploads/2014/09/PIP_AnonymityOnline_090513.pdf
45. Reiter, M.K., & Rubin, A.D. (1999). Anonymous web transactions with crowds. Communications of the ACM, 42(2), 32–48. http://doi.org/10.1145/293411.293778
46. Riggs, J.E. (1995). Priority, rivalry, and peer review. Journal of Child Neurology, 10(3), 255–256. http://doi.org/10.1177/088307389501000325
47. Ross, J.S., Gross, C.P., Desai, M.M., Hong, Y., Grant, A.O., Daniels, S.R., … Krumholz, H.M. (2006). Effect of blinded peer review on abstract acceptance. JAMA: The Journal of the American Medical Association, 295(14), 1675–1680. http://doi.org/10.1001/jama.295.14.1675
48. Roy, S.B., De Cock, M., Mandava, V., Savanna, S., Dalessandro, B., Perlich, C., … Hamner, B. (2013). The Microsoft Academic Search Dataset and KDD Cup 2013. In Proceedings of the 2013 KDD Cup 2013 Workshop (pp. 1:1–1:6). New York, NY, USA: ACM. http://doi.org/10.1145/2517288.2517299
49. Shatz, D. (2004). Peer review: A critical inquiry. Lanham, MD: Rowman & Littlefield.
50. Siegel, J., Dubrovsky, V., Kiesler, S., & McGuire, T.W. (1986). Group processes in computer-mediated communication. Organizational Behavior and Human Decision Processes, 37(2), 157–187. http://doi.org/10.1016/0749-5978(86)90050-6
51. Simon, R.J., Bakanic, V., & McPhail, C. (1986). Who complains to journal editors and what happens*. Sociological Inquiry, 56(2), 259–271. http://doi.org/10.1111/j.1475-682X.1986.tb00087.x
52. Stevens, G., & Wulf, V. (2002). A new dimension in access control: Studying maintenance engineering across organizational boundaries. In Proceedings of the 2002 ACM Conference on Computer Supported Cooperative Work (pp. 196–205). New York: ACM. http://doi.org/10.1145/587078.587106
53. Stuart, H.C., Dabbish, L., Kiesler, S., Kinnaird, P., & Kang, R. (2012). Social transparency in networked information exchange: A theoretical framework. In Proceedings of the ACM 2012 Conference on Computer Supported Cooperative Work (pp. 451–460). New York: ACM. http://doi.org/10.1145/2145204.2145275
54. Teddlie, C., & Yu, F. (2007). Mixed methods sampling a typology with examples. Journal of Mixed Methods Research, 1(1), 77–100. http://doi.org/10.1177/2345678906292430
55. Van Rooyen, S., Delamothe, T., & Evans, S.J.W. (2010). Effect on peer review of telling reviewers that their signed reviews might be posted on the web: Randomised controlled trial. BMJ, 341(7782), c5729. http://doi.org/10.1136/bmj.c5729
56. Van Rooyen, S., Godlee, F., Evans, S., Black, N., & Smith, R. (1999). Effect of open peer review on quality of reviews and on reviewers' recommendations: A randomised trial. BMJ, 318(7175), 23–27.
57. Van Rooyen, S., Godlee, F., Evans, S., Smith, R., & Black, N. (1998). Effect of blinding and unmasking on the quality of peer review. JAMA: The Journal of the American Medical Association, 280(3), 234–237. http://doi.org/10.1001/jama.280.3.234
58. Vrooman, S.S. (2002). The art of invective performing identity in cyberspace. New Media & Society, 4(1), 51–70. http://doi.org/10.1177/14614440222226262
59. Walsh, E., Rooney, M., Appleby, L., & Wilkinson, G. (2000). Open peer review: A randomised controlled trial. The British Journal of Psychiatry, 176(1), 47–51. http://doi.org/10.1192/bjp.176.1.47
60. Weicher, M. (2008). Peer review and secrecy in the “Information Age.” Proceedings of the American Society for Information Science and Technology, 45(1), 1–12. http://doi.org/10.1002/meet.2008.14504503155
61. Weller, A.C. (2001). Editorial peer review: Its strengths and weaknesses. Medford, NJ: Information Today.