From Manuscript Evaluation to Article Valuation: The Changing Technologies of Journal Peer Review

Human Studies

Volumn 38, Issue 1, 2015, Pages 57-79

  Pontille, David ^a   Torny, Didier ^b  

^a CENTRE DE SOCIOLOGIE DE L INNOVATION   (France)

^b Universite Paris Dauphine   (France)

Author keywords

Academic journals; Anonymity; Evaluation; Peer review; Valuation studies

Indexed keywords

EID: 84940000100     PISSN: 01638548     EISSN: 1572851X     Source Type: Journal    
DOI: 10.1007/s10746-014-9335-z     Document Type: Article

References (78)

1. American Psychological Association. (1965). Publications in APA Journals: advices from the editors. American Psychologist,20(9), 711f.
2. Auranen, O., & Nieminen, M. (2010). University research funding and publication performance—An international comparison. Research Policy,39(6), 822–834.
3. American Sociological Review. (1955). Notice to contributors, 20(3), 341.
4. Archambault, E., Amyot, D., Deschamps, E., Nicol, A., Rebout, L., & Roberge, G. (2013). Proportion of open access peer-reviewed papers at the European and world levels—2004–2011. Science Metrix for the European Commission DG Research & Innovation.
5. Baruch, Y., Konrad, A. M., Aguinis, H., & Starbuck, W. H. (Eds.). (2008). Opening the black box of editorship. Palgrave Macmillan.
6. Bazerman, C. (1988). Shaping written knowledge: the genre and activity of the experimental article in science. Madison, WI: The University of Wisconsin Press.
7. Benedek, E. P. (1976). Editorial practices of psychiatric and related journals: implications for women. American Journal of Psychiatry,133(1), 89–92.
8. Berg, L. D. (2001). Masculinism, emplacement, and positionality in peer review. The Professional Geographer,53(4), 511–521.
9. Blank, R. M. (1991). The effects of double-blind versus single-blind reviewing: Experimental evidence from the American Economic Review. American Economic Review,81(5), 1041–1067.
10. Bohlin, I. (2004). Communication regimes in competition: The current transition in scholarly communication seen through the lens of the sociology of technology. Social Studies of Science,34(3), 365–391.
11. Bollen, J., Van De Sompel, H., Hagberg, A., & Chute, R. (2009). A principal component analysis of 39 scientific impact measures. PLoS ONE,4(6), e6022.
12. Boltanski, L., & Thévenot, L. (2006). On justification: Economies of worth. New Jersey: Princeton university Press.
13. Bornmann, L. (2011). Scientific peer review. Annual Review of Information Science and Technology,45(1), 197–245.
14. Broad, W., & Wade, N. (1982). Betrayers of the truth. Fraud and deceit in the hall of science. New York: Simon & Schuster.
15. Bruno, I., & Didier, E. (2013). Benchmarking: l’état sous pression statistique. Paris: La Découverte.
16. Burnham, J. C. (1990). The evolution of editorial peer review. Journal of the American Medical Association,263(10), 1323–1329.
17. Butler, L., & McAllister, I. (2009). Metrics or peer review? Evaluating the 2001 UK research assessment exercise in political science. Political Studies Review,7(1), 3–17.
18. Campanario, J. M. (1998). Peer review for journals as it stands today—Part 1. Science Communication,19(3), 181–211.
19. Chubin, D. E., & Hackett, E. J. (1990). Peerless science: Peer review and U.S. science policy. Albany: State University of New York Press.
20. Cicchetti, D. V. (1991). The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation. Behavioral and Brain Sciences,14(1), 119–135.
21. Cicchetti, D. V., & Conn, H. O. (1976). A statistical analysis of reviewer agreement and bias in evaluating medical abstracts. The Yale Journal of Biology and Medicine,49(4), 373–383.
22. Cole, S., Cole, J. R., & Simon, G. A. (1981). Chance and consensus in peer review. Science,214(4523), 881–886.
23. Crane, D. (1967). The gate-keepers of science: Some factors affecting the selection of articles for scientific journals. The American Sociologist,2(1), 195–201.
24. Cronin, B., & Sugimoto, C. R. (2014). Beyond bibliometrics. Harnessing multidimensional indicators of scholarly impact. Cambridge: MIT Press.
25. DeBakey, L. (1976). The scientific journal: editorial policies and practices: guidelines for editors, reviewers, and authors. Saint Louis: CV Mosby Company.
26. Donovan, C. (2007). Introduction: Future pathways for science policy and research assessment: metrics vs peer review, quality vs impact. Science and Public Policy,34(8), 538–542.
27. Douglas-Wilson, I. (1974). Twilight of the medical journal? British Medical Journal,3(5926), 326–327.
28. Erikson, M. G., & Erlandson, P. (2014). A taxonomy of motives to cite. Social Studies of Science,44(4), 625–637.
29. Espeland, W. N., & Stevens, M. L. (1998). Commensuration as a social process. Annual review of sociology,24, 313–343.
30. Eysenbach, G. (2011). Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. Journal of Medical Internet Research,13(4), e123.
31. Frey, B. (2003). Publishing as prostitution? Choosing between one’s own ideas and academic success. Public Choice,116(1–2), 205–223.
32. Glenn, N. D. (1976). The journal article review process: Some proposals for change. The American Sociologist,11(3), 179–185.
33. Godlee, F., Gale, C. R., & Martyn, C. N. (1998). Effect on the quality of peer review of blinding reviewers and asking them to sign their reports. Journal of the American Medical Association,280(3), 237–240.
34. Gunnarsdottir, K. (2005). Scientific journal publications: On the role of electronic preprint exchange in the distribution of scientific literature. Social Studies of Science,35(4), 549–579.
35. Hargens, L. L. (1988). Scholarly consensus and journal rejection rates. American Sociological Review,53(1), 139–151.
36. Harnad, S. (1979). Creative disagreement. The Sciences,19, 18–20.
37. Helgesson, C.-F., & Muniesa, F. (2013). For what it’s worth: An introduction to valuation studies. Valuation Studies,1(1), 1–10.
38. Hicks, D., & Wang, J. (2011). Coverage and overlap of the new social sciences and humanities journal lists. Journal of the American Society for Information Science and Technology,62(2), 284–294.
39. Hirschauer, S. (2010). Editorial judgments: A praxeology of ‘voting’ in peer review. Social Studies of Science,40(1), 71–103.
40. Ingelfinger, F. J. (1969). Definition of ‘sole contribution’. New England Journal of Medicine,281(12), 676–677.
41. Jones, R. (1974). Rights, wrongs and referees. New Scientist,61(890), 758–759.
42. Kennefick, D. (2005). Einstein Versus The Physical Review. Physics Today,58(9), 43–48.
43. Knox, F. G. (1981). No unanimity about anonymity. Journal of Laboratory and Clinical Medicine,97(1), 1–3.
44. Kronick, D. A. (1962). A history of scientific and technical periodicals: the origins and development of the scientific and technical press, 1665-1790. Metuchen, N.J.: The Scarecrow Press.
45. Lamont, M. (2009). How professors think: Inside the curious world of academic judgment. Cambridge, Mass.; London: Harvard University Press.
46. Lamont, M. (2012). Toward a comparative sociology of valuation and evaluation. Annual Review of Sociology,38(1), 201–221.
47. Lancester, F. W. (1995). Attitudes in academia toward feasibility and desirability of networked scholarly publishing. Library Trends,43(4), 741–752.
48. Latour, B. (1987). Science in action: How to follow scientists and engineers through society. Milton Keynes: Open University Press.
49. Latour, B., & Woolgar, S. (1979). Laboratory life: the social construction of scientific facts. Beverly Hills: Sage.
50. Lee, C. J., Sugimoto, C. R., Zhang, G., & Cronin, B. (2013). Bias in peer review. Journal of the American Society for Information Science and Technology,64(1), 2–17.
51. Lowry, R. P. (1967). Communications to the editors. The American Sociologist,2(4), 220.
52. Macdonald, S., & Kam, J. (2007). Aardvark et al.: quality journals and gamesmanship in management studies. Journal of Information Science,33(6), 702–717.
53. Mahoney, M. J. (1977). Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive Therapy and Research,1(2), 161–175.
54. Merton, R. K. (1942). Science and technology in a democratic order. Journal of Legal and Political Sociology,1, 115–126.
55. Morgan, P. P. (1984). Anonymity in medical journals. Canadian Medical Association Journal,131(9), 1007f.
56. Nature. (1974). In defence of the anonymous referee. Nature,249(5458), 601f.
57. Peters, D. P., & Ceci, S. J. (1982). Peer-review practices of psychological journals: The fate of published articles, submitted again. Behavioral and Brain Sciences,5(2), 187–195.
58. Pontille, D., & Torny, D. (2010). The controversial policies of journal ratings: Evaluating social sciences and humanities. Research Evaluation,19(5), 347–360.
59. Pontille, D., & Torny, D. (2012). Behind the scenes of scientific articles: Defining categories of fraud and regulating cases. Revue d’Épidémiologie et de Santé Publique,60(4), 247–253.
60. Pontille, D., & Torny, D. (2013). La manufacture de l’évaluation scientifique: algorithmes, jeux de données, outils bibliométriques. Réseaux,177, 25–61.
61. Pontille, D., & Torny, D. (2014). The blind shall see! The question of anonymity in journal peer review. Ada: A Journal of Gender, New Media, and Technology, 4, doi:10.7264/N3542KVW.
62. Porter, J. R. (1964). The Scientific Journal - 300th Anniversary. Bacteriological Reviews,28(3), 211–230.
63. Priem, J., & Costello, K. L. (2010). How and why scholars cite on Twitter. Proceedings of the American Society for Information Science and Technology,47(1), 1–4.
64. Schroter, S., Black, N., Evans, S., Godlee, F., Osorio, L., & Smith, R. (2008). What errors do peer reviewers detect, and does training improve their ability to detect them? Journal of the Royal Society of Medicine,101(10), 507–514.
65. Shapin, S., & Schaffer, S. (1985). Leviathan and the air-pump: Hobbes, Boyle, and the experimental life. Princeton, N.J.: Princeton University Press.
66. Shapiro, B. J. (2000). A culture of fact: England, 1550-1720. Ithaca and London: Cornell University Press.
67. Smith, R. (2006). Peer review: a flawed process at the heart of science and journals. Journal of the Royal Society of Medicine,99(4), 178–182.
68. Speck, B. W. (1993). Publication peer review: An annotated bibliography. Westport, CT: Greenwood Press.
69. Van Noorden, R. (2011). Science publishing: The trouble with retractions. Nature,478(7367), 26–28.
70. Van Rooyen, S., Godlee, F., Evans, S., Smith, R., & Black, N. (1998). Effect of blinding and unmasking on the quality of peer review. Journal of General Internal Medicine,14(10), 622–624.
71. Ward, W. D., & Goudsmit, S. A. (1967). Reviewer and author anonymity. Physics Today,20(1), 12.
72. Ware, M., & Monkman, M. (2008). Peer review in scholarly journals: Perspective of the scholarly community - an international study. UK, Mark Ware Consulting: Bristol.
73. Weller, A. (2001). Editorial peer review: Its strengths and weaknesses. Medford: Information Today, Inc.
74. Wilhite, A. W., & Fong, E. A. (2012). Coercive citation in academic publishing. Science, 335(6068), 542f.
75. Wilson, J. D. (1978). Peer review and publication. Presidential address before the 70th annual meeting of the American Society for Clinical Investigation, San Francisco, California, 30 April 1978. Journal of Clinical Investigation,61(6), 1697–1701.
76. Wouters, P. (1999). The citation culture. Amsterdam: University of Amsterdam.
77. Wouters, P., & Costas, R. (2012). Users, narcissism and control – tracking the impact of scholarly publications in the 21st century (p. 50). Utrecht, SURFfoundation.
78. Zuckerman, H. A., & Merton, R. K. (1971). Patterns of evaluation in science: institutionalisation, structure and functions of the referee system. Minerva,9(1), 66–100.