Referees, editors, and publication practices: Improving the reliability and usefulness of the peer review system

Science and Engineering Ethics

Volumn 3, Issue 1, 1997, Pages 51-62

  Cicchetti, Domenic V ^a  

^a YALE UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

Acceptance rates; Behavioral sciences; Chemistry; Fair; Grants; Innovative original; Manuscripts; Medical sciences; Objective; Peer review; Physical sciences; Physics; Publication practices; Rejection rates; Reliability; Training reviewers; Validity

Indexed keywords

EID: 0012320236     PISSN: 13533452     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11948-997-0016-4     Document Type: Review

References (34)

1. Armstrong, J.S. & Hubbard, R. (1991) Does the need for agreement among reviewers inhibit the publication of controversial findings? Behavioral & Brain Sciences 14:136-137.
2. Bailar, J.C. (1991) Reliability, fairness, objectivity and other inappropriate goals in peer review. Behavioral & Brain Sciences 14:137-138.
3. Cicchetti, D.V. (1976) Assessing inter-rater reliability for rating scales: Resolving some basic issues. British Journal of Psychiatry 129:452-456.
4. Cicchetti, D.V. (1982) On peer review: 〉We have met the enemy and he is us." Behavioral & Brain Sciences 5:205.
5. Cicchetti, D.V. (1991a) The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation. Behavioral and Brain Sciences 14:119-135.
6. Cicchetti, D.V. (1991b) Author's response: Reflections from the peer review mirror. Behavioral & Brain Sciences 14:167-186.
7. Cicchetti, D.V. (1997) Peer review: Agreement and disagreement. Behavioral and Brain Sciences. In press.
8. Cicchetti, D.V., & Conn, H.O. (1976) A statistical analysis of reviewer agreement and bias in evaluating medical abstracts. Yale Journal of Biology & Medicine 49:373-383.
9. Cicchetti, D.V. & Eron, L.D. (1979) The reliability of manuscript reviewing for the Journal of Abnormal Psychology. Proceedings of the American Statistical Association (Social Statistics Section) 22:596-600.
10. Cicchetti, D.V. & Fleiss, J.L. (1977) Comparison of the null distributions of weighted kappa and the C ordinal statistic. Applied Psychological Methods 1: 195-201.
11. Cicchetti, D.V. & Sparrow, S.S. (1981) Developing criteria for establishing inter-rater reliability of specific items: Applications to assessment of adaptive behavior. American Journal of Mental Deficiency 86: 127-137.
12. Cicchetti, D.V., Volkmar, F., Klin, A. & Showalter, D. (1995) Diagnosing autism using ICD-10 criteria: A comparison of neural networks and standard multivariate procedures. Child Neuropsychology, 1: 26-37.
13. Cohen, J. (1960) A coefficient of agreement for nominal scales. Educational & Psychological Measurement 20: 37-46.
14. Cooper, G.S. & Zangwill, L. (1989) An analysis of the quality of research reports in the Journal of General Internal Medicine. Journal of General Internal Medicine 4:232-236.
15. Daniel, H. D. (1993) Guardians of Science: Fairness and reliability of peer review. VHC, New York.
16. Fleiss, J. L. (1975) Measuring agreement between two judges on the presence or absence of a trait. Biometrics 31: 651-659.
17. Fleiss, J. L. (1981) Statistical methods for rates and proportions, Wiley (2nd ed.), New York.
18. Fleiss, J .L. & Cohen, J. (1973) The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational & Psychological Measurement 33: 613-619.
19. Fleiss, J. L., Cohen, J. & Everitt, B. S. (1969) Large sample standard errors of kappa and weighted kappa. Psychological Bulletin 72: 323-327.
20. Grove, W. M., Andreason, N. C., McDonald-Scott, P., Keller, M. B. & Shapiro, R. W. (1981) Reliability studies of psychiatric diagnoses: Theory and practice. Archives of General Psychiatry 38:408-413.
21. Hargens, L. L. (1988) Scholarly consensus and journal rejection rates. American Sociological Review 53: 139-151.
22. Hargens, L. L. (1990) Variation in journal peer review systems: possible causes and consequences. Journal of the American Medical Association 263:1348-1352.
23. Ingelfinger, F. J. (1974). Peer review in biomedical publication. American Journal of Medicine 56: 686-692.
24. Kemper, K. J., McCarthy, P. L. & Cicchetti, D. V. (1996) Improving participation and interrater agreement in scoring APA abstracts: how well have we succeeded? Archives of Pediatrics and Adolescent Medicine 150: 380-383.
25. Kiesler, C. A. (1991) Confusion between reviewer reliability and wise editorial and funding decisions. Behavioral & Brain Sciences 14:151-152.
26. Lock, S. (1985) A difficult balance: Editorial peer review in medicine. ISI Press.
27. McGraw, K. O. & Wong, S. P. (1996) Forming inferences about some intraclass correlation coefficients. Psychological Methods 1: 30-46.
28. Oxman, A. D., Guyatt, G. H., Singer, J., Goldsmith, C. H., Hutchison, B. G., Milner, R. A. & Streiner, D. L. (1991) Agreement among reviewers of review articles. Journal of Clinical Epidemiology 44: 91-98.
29. Relman, A. S. (1978) Are journals really quality filters? Rockefeller working papers (conference, May 22-23, 1977, New York, NY). Coping with the biomedical literature explosion: A qualitative approach.
30. Rourke, B. P. (1991) Toward openness and fairness in the review process. Behavioral & Brain Sciences 14: 161.
31. Shrout, P. E. & Fleiss, J. L. (1979) Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin 86: 420-428.
32. Solomon, M. J. & McLeod, R. S. (1993) Clinical studies in surgical journals - have we improved? Diseases of the Colon and Rectum 36: 43-48.
33. Strayhorne, Jr., J., McDermott, Jr., J. F. & Tanguay, P. (1993) An Intervention to improve the reliability of manuscript reviews for the Journal of the American Academy of Child & Adolescent Psychiatry. American Journal of Psychiatry 150: 947-952.
34. Yalow, R. S. (1978). Radioimmunoassay: A probe for the fine structure of biology systems. In: Les prix nobel en 1977, Nobel Foundation, Almqvist & Wiksell, Stockholm, pp. 243-264.