From black box to white box at open access journals: Predictive validity of manuscript reviewing and editorial decisions at Atmospheric Chemistry and Physics

Research Evaluation

Volumn 19, Issue 2, 2010, Pages 105-118

  Bornmann, Lutz ^a   Marx, Werner ^b   Schier, Hermann ^b   Thor, Andreas ^c   Daniel, Hans Dieter ^a,d  

^a Social Psychology and Research on Higher Education   (Switzerland)

^b MAX PLANCK INSTITUT FÜR FESTKÖRPERFORSCHUNG   (Germany)

^c UNIVERSITY OF LEIPZIG   (Germany)

^d UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77955583799     PISSN: 09582029     EISSN: None     Source Type: Journal    
DOI: 10.3152/095820210X510089     Document Type: Article

References (58)

1. Allison, P D, J S Long and T K Krauze 1982. Cumulative advantage and inequality in science. American Sociological Review, 47(5), 615-625.
2. Bloom, T 2006. Systems: online frontiers of the peer-reviewed literature. The Internet is allowing much more interactive science publishing. Retrieved 6 July 2006, from urlhttp://www.nature.com/nature/peerreview/debate/nature05030.html, last accessed 1 May 2010.
3. Borgman, C L 2007. Scholarship in the Digital Age: Information, Infrastructure, and the Internet. Cambridge, MA, USA: MIT Press.
4. Bornmann, L in press. Scientific peer review. Annual Review of Information Science and Technology.
5. Bornmann, L and H-D Daniel 2007. Multiple publication on a single research study: does it pay? The influence of number of research articles on total citation counts in biomedicine', Journal of the American Society for Information Science and Technology, 58(8), 1100-1107.
6. Bornmann, L and H-D Daniel 2008a. The effectiveness of the peer review process: inter-referee agreement and predictive validity of manuscript refereeing at Angewandte Chemie. Angewandte Chemie International Edition, 47(38), 7173-7178.
7. Bornmann, L and H-D Daniel 2008b. Functional use of frequently and infrequently cited articles in citing publications. A content analysis of citations to articles with low and high citation counts. European Science Editing, 34(2), 35-38.
8. Bornmann, L and H-D Daniel 2008c. Selecting manuscripts for a high impact journal through peer review: a citation analysis of Communications that were accepted by Angewandte Chemie International Edition, or rejected but published elsewhere. Journal of the American Society for Information Science and Technology, 59(11), 1841-1852.
9. Bornmann, L and H-D Daniel 2008d. What do citation counts measure? A review of studies on citing behavior. Journal of Documentation, 64(1), 45-80.
10. Bornmann, L and H-D Daniel 2010. Reliability of reviewers' ratings at an interactive open access journal using public peer review: a case study on Atmospheric Chemistry and Physics. Learned Publishing, 23(2), 124-131.
11. Bornmann, L, L Leydesdorff and W Marx 2007. Citation environment of Angewandte Chemie. CHIMIA, 61(3), 104-109.
12. Bornmann, L, R Mutz, C Neuhaus and H-D Daniel 2008. Use of citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in Science and Environmental Politics, 8, 93-102.
13. Bornmann, L, W Marx, H Schier, E Rahm, A Thor and H-D Daniel 2009. Convergent validity of bibliometric Google Scholar data in the field of chemistry. Citation counts for papers that were accepted by Angewandte Chemie International Edition or rejected but published elsewhere, using Google Scholar, Science Citation Index, Scopus, and Chemical Abstracts. Journal of Informetrics, 3(1), 27-35.
14. Bornstein, R F 1991. The predictive validity of peer-review: a neglected issue. Behavioral and Brain Sciences, 14(1), 138-139.
15. Braun, T, I Dióspatonyi, S Zsindely and E Zádor 2007. Gatekeeper index versus impact factor of science journals. Scientometrics, 71(3), 541-543.
16. Cameron, A C and P K Trivedi 1998. Regression Analysis of Count Data. Cambridge, UK: Cambridge University Press.
17. Cohen, J 1988. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ, USA: Lawrence Erlbaum Associates, Publishers.
18. Craig, I D, A M Plume, M E McVeigh, J Pringle and M Amin 2007. Do open access articles have greater citation impact? A critical review of the literature. Journal of Informetrics, 1(3), 239-248.
19. Cronin, B and G McKenzie 1992. The trajectory of rejection. Journal of Documentation, 48(3), 310-317.
20. Daniel, H-D 1993. Guardians of Science: Fairness and Reliability of Peer Review. Weinheim, Germany: Wiley-VCH.
21. DeCoursey, T 2006. Perspective: The pros and cons of open peer review. Should authors be told who their reviewers are? Retrieved 20 June 2006, from urlhttp://www.nature.com/nature/peerreview/debate/nature04991.html, last accessed 1 May 2010.
22. Garfield, E 2006. The history and meaning of the Journal Impact Factor. Journal of the American Medical Association, 295(1), 90-93.
23. Gölitz, P 2010. Twitter, Facebook, and Open Access. Angewandte Chemie International Edition, 49(1), 4-6.
24. Grant, B 2009. Merck published fake journal. Retrieved 25 March 2010, from urlhttp://www.the-scientist.com/blog/display/55671/, last accessed 1 May 2010.
25. Hames, I 2007. Peer Review and Manuscript Management of Scientific Journals: Guidelines for Good Practice. Oxford UK: Blackwell.
26. th Conference of the International Society for Scientometrics and Informetrics, eds D Torres-Salinas and H F Moed, pp. 27-33.
27. Madrid, Spain: Spanish Research Council (CSIC). Herbertz, H 1995. Does it pay to cooperate? A bibliometric casestudy in molecular-biology. Scientometrics, 33(1), 117-122.
28. Jacso, P 2009a. Database source coverage: hypes, vital signs and reality checks. Online Information Review, 33(5), 997-1007.
29. Jacso, P 2009b. Five-year impact factor data in the Journal Citation Reports. Online Information Review, 33(3), 603-614.
30. Jennings, C G 2006. Quality and value: the true purpose of peer review. What you can't measure, you can't manage: the need for quantitative indicators in peer review. Retrieved 6 July 2006, from urlhttp://www.nature.com/nature/peerreview/debate/nature05032.html, last accessed 5 May 2010.
31. Joint Information Systems Committee 2004. Journal Authors Survey Report. Key Perspectives Ltd: Truro, UK.
32. Koop, T and U Pöschl 2006). Systems: an open, two-stage peerreview journal. The editors of Atmospheric Chemistry and Physics explain their journal's approach. Retrieved 26 June 2006, from http://www.nature.com/nature/peerreview/debate/nature04988.html, last accessed 5 May 2010.
33. Leimu, R and J Koricheva 2005. What determines the citation frequency of ecological papers? Trends in Ecology and Evolution, 20(1), 28-32.
34. Lindsey, D 1989 Using citation counts as a measure of quality in science: measuring what's measurable rather than what's valid. Scientometrics, 15(3-4), 189-203.
35. Long, J S and J Freese 2006. Regression Models for Categorical Dependent Variables Using Stata. Stata Corporation, College Station, TX, USA: Stata Press.
36. Marsh, H W, L Bornmann, R Mutz, H-D Daniel and A O'Mara 2009. Gender effects in the peer reviews of grant proposals: a comprehensive meta-analysis comparing traditional and multilevel approaches. Review of Educational Research, 79(3), 1290-1326.
37. Martin, B R and J Irvine 1983. Assessing basic research: some partial indicators of scientific progress in radio astronomy. Research Policy, 12(2), 61-90.
38. McDonald, R J, H J Cloft and D F Kallmes 2009. Fate of manuscripts previously rejected by the American Journal of Neuroradiology: a follow-up analysis. American Journal of Neuroradiology, 30(2), 253-256.
39. Neuhaus, C and H-D Daniel 2008. Data sources for performing citation analysis: an overview. Journal of Documentation, 64(2), 193-210.
40. Oppenheim, C 2008. Electronic scholarly publishing and open access. Journal of Information Science, 34(4), 577-590.
41. Opthof, T, F Furstner, M van Geer and R Coronel 2000. Regrets or no regrets? No regrets! The fate of rejected manuscripts. Cardiovascular Research, 45(1), 255-258.
42. Pöschl, U 2004. Interactive journal concept for improved scientific publishing and quality assurance. Learned Publishing, 17(2), 105-113.
43. Pöschl, U 2010. Interactive open access publishing and peer review: the effectiveness and perspectives of transparency and self-regulation in scientific communication and evaluation. Liber Quarterly, 19(3/4), 293-314.
44. Rabe-Hesketh, S and B Everitt 2004. A Handbook of Statistical Analyses Using Stata. Boca Raton, FL, USA: Chapman & Hall/CRC.
45. Relman, A S 1978. Are journals really quality filters? In Coping with the Biomedical Literature Explosion: a Qualitative Approach, eds W Goffman, J T Bruer and K S Warren, pp. 54-60. New York, NY, USA: Rockefeller Foundation.
46. Schultz, D M 2010. Rejection rates for journals publishing atmospheric science. Bulletin of the American Meteorological Society, 91(2), 231-243.
47. Seglen, P O 1997. Why the impact factor of journals should not be used for evaluating research. British Medical Journal, 314(7079), 498-502.
48. Shashok, K 2005. Standardization vs diversity: how can we push peer review research forward? Medscape General Medicine, 7(1), 11.
49. Smith, L C 1981. Citation analysis. Library Trends, 30(1), 83-106.
50. StataCorp. 2008. Stata Statistical Software: Release 11. College Station, TX, USA: Stata Corporation.
51. Taylor, M, P Perakakis and V Trachana 2008. The siege of science. Ethics in Science and Environmental Politics, 8, 17-40.
52. Thorngate, W, R M Dawes and M Foddy 2009. Judging Merit. New York, NY, USA: Psychology Press.
53. Vieira, E and J Gomes 2009. A comparison of Scopus and Web of Science for a typical university. Scientometrics, 81(2), 587-600.
54. Vieira, E S and J A N F Gomes 2010 Citations to scientific articles: its distribution and dependence on the article features. Journal of Informetrics, 4(1), 1-13.
55. Von Eye, A and E Y Mun 2005. Analyzing Rater Agreement: Manifest Variable Methods. Mahwah, NJ, USA: Lawrence Erlbaum Associates.
56. Weller, A C 2002. Editorial Peer Review: Its Strengths and Weaknesses. Medford, NJ, USA: Information Today, Inc.
57. Whitley, K M 2002. Analysis of SciFinder scholar and web of science citation searches. Journal of the American Society for Information Science and Technology, 53(14), 1210-1215.
58. Wilson, J D 1978. Peer review and publication. Journal of Clinical Investigation, 61(4), 1697-1701.