The Hawthorne effect in journal peer review

Scientometrics

Volumn 91, Issue 3, 2012, Pages 857-862

  Bornmann, Lutz ^a  

^a MAX PLANCK INSTITUTE   (Germany)

Author keywords

Editorial peer review; Hawthorne effect; Manuscript rejection

Indexed keywords

EID: 84860562222     PISSN: 01389130     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11192-011-0547-y     Document Type: Review

References (48)

1. Abelson, P. H. (1980). Scientific communication. Science, 209(4452), 60-62.
2. Bakanic, V., McPhail, C., & Simon, R. J. (1987). The manuscript review and decision-making process. American Sociological Review, 52(5), 631-642.
3. Benda, W. G. G., & Engels, T. C. E. (2011). The predictive validity of peer review: a selective review of the judgmental forecasting qualities of peers, and implications for innovation in science. International Journal of Forecasting, 27(1), 166-182. doi: 10. 1016/j. ijforecast. 2010. 03. 003.
4. Bornmann, L. (2008). Scientific peer review. An analysis of the peer review process from the perspective of sociology of science theories. Human Architecture-Journal of the Sociology of Self-Knowledge, 6(2), 23-38.
5. Bornmann, L. (2011). Scientific peer review. Annual Review of Information Science and Technology, 45, 199-245.
6. Bornmann, L., & Daniel, H.-D. (2008a). The effectiveness of the peer review process: inter-referee agreement and predictive validity of manuscript refereeing at Angewandte Chemie. Angewandte Chemie International Edition, 47(38), 7173-7178. doi: 10. 1002/anie. 200800513.
7. Bornmann, L., & Daniel, H.-D. (2008b). Selecting manuscripts for a high impact journal through peer review: a citation analysis of Communications that were accepted by Angewandte Chemie International Edition, or rejected but published elsewhere. Journal of the American Society for Information Science and Technology, 59(11), 1841-1852. doi: 10. 1002/asi. 20901.
8. Bunting, C. (2005, 25 February). Early careers spent grinding teeth, not cutting them. Times Higher Education Supplement, 18.
9. Callaham, M., & McCulloch, C. (2011). Longitudinal trends in the performance of scientific peer reviewers. [Proceedings Paper]. Annals of Emergency Medicine, 57(2), 141-148. doi: 10. 1016/j. annemergmed. 2010. 07. 027.
10. Chew, F. S. (1991). Fate of manuscripts rejected for publication in the AJR. American Journal of Roentgenology, 156(3), 627-632.
11. Chiesa, M., & Hobbs, S. (2008). Making sense of social research: how useful is the Hawthorne Effect? [Article]. European Journal of Social Psychology, 38(1), 67-74. doi: 10. 1002/ejsp. 401.
12. Cole, S. (1992). Making science, between nature and society. Cambridge: Harvard University Press.
13. Cronin, B., & McKenzie, G. (1992). The trajectory of rejection. Journal of Documentation, 48(3), 310-317.
14. Daniel, H.-D. (1993). Guardians of science, fairness and reliability of peer review. Weinheim: Wiley-VCH.
15. de Vries, D. R., Marschall, E. A., & Stein, R. A. (2009). Exploring the peer review process: What is it, does it work, and can it be improved? Fisheries, 34(6), 270-279.
16. Eisenhart, M. (2002). The paradox of peer review: Admitting too much or allowing too little? Research in Science Education, 32(2), 241-255.
17. Fogg, L., & Fiske, D. W. (1993). Foretelling the judgments of reviewers and editors. American Psychologist, 48(3), 293-294.
18. French, J. R. P. (1953). Experiments in field settings. In L. Festinger & D. Katz (Eds.), Research methods in the behavioral sciences (pp. 98-135). New York: Holt, Rinehart and Winston.
19. Gans, J. S., & Shepherd, G. B. (1994). How are the mighty fallen-rejected classic articles by leading economists. Journal of Economic Perspectives, 8(1), 165-179.
20. Gordon, M. D. (1984). How authors select journals-a test of the reward maximization model of submission behavior. Social Studies of Science, 14(1), 27-43.
21. Gorman, G. E. (2007). The Oppenheim effect in scholarly journal publishing. Online Information Review, 31(4), 417-419. doi: 10. 1108/14684520710780386.
22. Hojat, M., Gonnella, J. S., & Caelleigh, A. S. (2003). Impartial judgment by the "gatekeepers" of science: fallibility and accountability in the peer review process. Advances in Health Sciences Education, 8(1), 75-96.
23. Jayasinghe, U. W., Marsh, H. W., & Bond, N. (2001). Peer review in the funding of research in higher education: the Australian experience. Educational Evaluation and Policy Analysis, 23(4), 343-346.
24. Khosla, A., McDonald, R. J., Bornmann, L., & Kallmes, D. F. (2011). Getting to yes: the fate of neuroradiology manuscripts rejected by Radiology over a 2-year period. Radiology, 260(1), 3-5. doi: 10. 1148/radiol. 11110490.
25. Lipworth, W. L., Kerridge, I. H., Carter, S. M., & Little, M. (2011). Journal peer review in context: a qualitative study of the social and subjective dimensions of manuscript review in biomedical publishing. [Review]. Social Science and Medicine, 72(7), 1056-1063. doi: 10. 1016/j. socscimed. 2011. 02. 002.
26. Lock, S. (1985). A difficult balance: editorial peer review in medicine. Philadelphia: ISI Press.
27. Marsh, H. W., Jayasinghe, U. W., & Bond, N. W. (2008). Improving the peer-review process for grant applications-reliability, validity, bias, and generalizability. American Psychologist, 63(3), 160-168. doi: 10. 1037/0003-066x. 63. 3. 160.
28. Martin, B. (2000). Research grants: problems and options. Australian Universities' Review, 43(2), 17-22.
29. McCook, A. (2006). Is peer review broken? The Scientist, 20(2), 26.
30. McDonald, R. J., Cloft, H. J., & Kallmes, D. F. (2007). Fate of submitted manuscripts rejected from the American Journal of Neuroradiology: outcomes and commentary. American Journal of Neuroradiology, 28(8), 1430-1434. doi: 10. 3174/Ajnr. A0766.
31. Opthof, T., Furstner, F., van Geer, M., & Coronel, R. (2000). Regrets or no regrets? No regrets! The fate of rejected manuscripts. Cardiovascular Research, 45(1), 255-258.
32. Owen, R. (1982). Reader bias. Journal of the American Medical Association, 247(18), 2533-2534.
33. Petty, R. E., & Fleming, M. A. (1999). The review process at PSPB: correlates of interreviewer agreement and manuscript acceptance. Personality and Social Psychology Bulletin, 25(2), 188-203.
34. Pruthi, S., Jain, A., Wahid, A., Mehra, K., & Nabi, S. A. (1997). Scientific community and peer review system-a case study of a central government funding scheme in India. Journal of Scientific & Industrial Research, 56(7), 398-407.
35. Pulverer, B. (2010). Transparency showcases strength of peer review. [10. 1038/468029a]. Nature, 468(7320), 29-31.
36. Ray, J., Berkwits, M., & Davidoff, F. (2000). The fate of manuscripts rejected by a general medical journal. American Journal of Medicine, 109(2), 131-135.
37. Ross, P. F. (1980). The sciences' self-management: manuscript refereeing, peer review, and goals in science. Lincoln: The Ross Company.
38. Sharp, D. W. (1990). What can and should be done to reduce publication bias-the perspective of an editor. Journal of the American Medical Association, 263(10), 1390-1391.
39. Shatz, D. (2004). Peer review: a critical inquiry. Lanham: Rowman & Littlefield.
40. Sternberg, R. J., Hojjat, M., Brigockas, M. G., & Grigorenko, E. L. (1997). Getting in: criteria for acceptance of manuscripts in Psychological Bulletin, 1993-1996. Psychological Bulletin, 121(2), 321-323.
41. van Rooyen, S., Godlee, F., Evans, S., Smith, R., & Black, N. (1998). Effect of blinding and unmasking on the quality of peer review-a randomized trial. Journal of the American Medical Association, 280(3), 234-237.
42. Weller, A. C. (1996). Editorial peer review: a comparison of authors publishing in two groups of US medical journals. Bulletin of the Medical Library Association, 84(3), 359-366.
43. Weller, A. C. (2002). Editorial peer review: its strengths and weaknesses. Medford: Information Today, Inc.
44. Whitman, N., & Eyre, S. (1985). The pattern of publishing previously rejected articles in selected journals. Family Medicine, 17(1), 26-28.
45. Wood, F. Q., & Wessely, S. (2003). Peer review of grant applications: a systematic review. In F. Godlee & T. Jefferson (Eds.), Peer review in health sciences (2nd ed., pp. 14-44). London: BMJ Books.
46. Ziman, J. (1968). Public knowledge: an essay concerning the social dimension of science. Cambridge: Cambridge University Press.
47. Ziman, J. (2000). Real science, what it is, and what it means. Cambridge: Cambridge University Press.
48. Zuckerman, H., & Merton, R. K. (1971). Patterns of evaluation in science: institutionalisation, structure and functions of the referee system. Minerva, 9(1), 66-100. doi: 10. 1007/bf01553188.