Mechanism change in a simulation of peer review: From junk support to elitism

Scientometrics

Volumn 99, Issue 3, 2014, Pages 663-688

  Paolucci, Mario ^a   Grimaldo, Francisco ^b  

^a INSTITUTE OF COGNITIVE SCIENCES AND TECHNOLOGIES   (Italy)

^b UNIVERSITY OF VALENCIA   (Spain)

Author keywords

Agent based simulation; BDI approach; Mechanism change; Peer review; Rational cheating; Restrained cheaters

Indexed keywords

EID: 84900519358     PISSN: 01389130     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11192-014-1239-1     Document Type: Article

References (63)

1. Allesina, S. (2012). 'Modeling peer review: An agent-based approach'. Ideas in Ecology and Evolution 5(2), 27-35.
2. Antonijevic, S., Dormans, S., & Wyatt, S. (2012). Working in virtual knowledge: Affective labor in scholarly collaboration. In Wouters P., Beaulieu A., Scharnhorst A., Wyatt S., (eds.), Virtual knowledge-experimenting in the humanities and the social sciences. Cambridge: MIT press.
3. Axelrod, R. (1997). The complexity of cooperation: Agent-based models of competition and collaboration, 1st printing edn. Princeton: Princeton University Press.
4. Bollen, J., van de Sompel, H., Hagberg, A., & Chute, R. (2009) A principal component analysis of 39 scientific impact measures. PloS One 4(6), e6022+.
5. Bordini, R. H., Hübner, J. F. & Wooldridge, M. (2007). Programming multi-agent systems in AgentSpeak using Jason. Chichester: Wiley.
6. Börner, K. (2010). Atlas of science: Visualizing what we know. Cambridge, Mass: MIT Press.
7. Bornmann, L. (2011). Scientific peer review. Annual Review of Information Science & Technology 45(1), 197-245.
8. Bornmann, L. (2013). A better alternative to the h index. Journal of Informetrics 7(1), 100+, doi: 10. 1016/j. joi. 2012. 09. 004.
9. Bornmann, L., & Daniel, H.-D. (2005). Selection of research fellowship recipients by committee peer review. Reliability, fairness and predictive validity of Board of Trustees' decisions. Scientometrics 63(2), 297-320.
10. Bornmann, L. & Daniel, H.-D. (2009). The luck of the referee draw: the effect of exchanging reviews. Learned Publishing 22(2), 117-125.
11. Bornmann, L., Nast, I., & Daniel, H.-D. (2008). Do editors and referees look for signs of scientific misconduct when reviewing manuscripts? A quantitative content analysis of studies that examined review criteria and reasons for accepting and rejecting manuscripts for publication. Scientometrics 77(3), 415-432.
12. Bratman, M. E. (1999). Intention, plans, and practical reason. Cambridge: Cambridge University Press.
13. Bruckner, E., Ebeling, W. & Scharnhorst, A. (1990). The application of evolution models in scientometrics. Scientometrics 18, 21-41.
14. Bunge, M. (2004). How does it work?: The search for explanatory mechanisms. Philosophy of the Social Sciences 34(2), 182-210.
15. Callahan, D. (2004). Rational cheating: Everyone's doing It. Journal of Forensic Accounting. pp. 575+.
16. Camussone, P., Cuel, R. & Ponte, D. (2010). ICT and Innovative Review Models: Implications For The Scientific Publishing Industry. In 'Proceedings of: WOA 2010, Bologna, 16-18 giugno 2010', pp. 1-14.
17. Cohen, M. R. (1933). Scientific method. In Seligman E. R. A., Johnson A., (eds.), Encyclopeadia of the social sciences. New York: MacMillan and Co., pp. 389-386.
18. Conte, R. & Paolucci, M. (2011). On Agent Based Modelling and Computational Social Science. Social Science Research Network Working Paper Series.
19. Dennett, D. C. (1987). The intentional stance (Bradford Books). reprint edn, Cambridge: The MIT Press.
20. Eckberg, D. L. (1991). When nonreliability of reviews indicates solid science. Behavioral and Brain Sciences 14, 145-146.
21. Edmonds, B. & Hales, D. (2003). Replication, replication and replication: Some hard lessons from model alignment. Journal of Artificial Societies and Social Simulation 6(4).
22. Edmonds, B. & Moss, S. (2005). From KISS to KIDS - An 'Anti-simplistic' Modelling Approach, Vol. 3415 of Lecture Notes in Computer Science, Berlin: Springer, pp. 130-144.
23. Edwards, M., Huet, S., Goreaud, F. & Deffuant, G. (2003). Comparing an individual-based model of behaviour diffusion with its mean field aggregate approximation. Journal of Artificial Societies and Social Simulation 6(4).
24. Egghe, L. & Rousseau, R. (1990). Introduction to informetrics: quantitative methods in library, documentation and information science. Amsterdam: Elsevier Science Publishers.
25. Gilbert, N. (1997). A simulation of the structure of academic science. Sociological Research 2(2), 1-25.
26. Gilbert, N. & Troitzsch, K. G. (2005). Simulation for the Social Scientist, 2nd edition. Buckingham: Open University Press.
27. Goffman, W. (1966). Mathematical approach to the spread of scientific ideas-the history of mast cell research. Nature 212(5061), 449-452.
28. Grimaldo, F. & Paolucci, M. (2013). A simulation of disagreement for control of rational cheating in peer review. Advances in Complex Systems pp. 1350004+.
29. Grimaldo, F., Paolucci, M. & Conte, R. (2012). Agent simulation of peer review: The PR-1 model. In Villatoro, D., Sabater-Mir, J., & Sichman, J. S., (eds.), Multi-agent-based simulation XII, Vol. 7124 of lecture notes in computer science chapter 1. Springer Berlin / Heidelberg, Berlin: Heidelberg, pp. 1-14.
30. Helbing, D. (2010). Quantitative sociodynamics: stochastic methods and models of social interaction processes. Berlin: Springer.
31. Helbing, D., Bishop, S., Conte, R., Lukowicz, P. & McCarthy, J. B. (2012). FuturICT: Participatory computing to understand and manage our complex world in a more sustainable and resilient way. European Physical Journal 214(1), 11-39.
32. Hojat, M., Gonnella, J. & Caelleigh, A. (2003). Impartial judgment by the "Gatekeepers" of science: Fallibility and accountability in the peer review process. Advances in Health Sciences Education 8(1), 75-96.
33. Jayasinghe, U. W., Marsh, H. W. & Bond, N. (2003). A multilevel cross-classified modelling approach to peer review of grant proposals: The effects of assessor and researcher attributes on assessor ratings. Journal of the Royal Statistical Society - Series A - Statistics in Society 166, 279-300.
34. Jefferson, T., Alderson, P., Wager, E. & Davidoff, F. (2002). Effects of Editorial Peer Review: A Systematic Review. JAMA 287(21), 2784-2786.
35. Jefferson, T. & Godlee, F. (2003). Peer Review in Health Sciences. London: Wiley.
36. Kostoff, R. N. (1995). Federal research impact assessmentaxioms, approaches, applications. Scientometrics 2(34), 163-206.
37. Kuhn, T. S. (1996). The structure of scientific revolutions, 3rd edn, Chicago: University of Chicago Press.
38. Lamont, M. (2009). How Professors Think: Inside the Curious World of Academic Judgment. Cambridge: Harvard University Press.
39. Lamont, M. & Huutoniemi, K. (2011). Opening the black box of evaluation: How quality is recognized by peer review panels. Bulletin SAGW 2, 47-49.
40. Lyons, W. (1997). Approaches to intentionality. Oxford: Oxford University Press.
41. Marcus, A. & Oransky, I. (2011). Science publishing: The paper is not sacred. Nature 480(7378), 449-450.
42. Moss, S. & Edmonds, B. (2005). Sociology and simulation: Statistical and qualitative cross-validation. American Journal of Sociology 110, 1095-1131.
43. Nowak, M. A. & Sigmund, K. (1992). Tit for tat in heterogeneous populations. Nature 355, 250-253.
44. Paolucci, M. (2012). Two scenarios for Crowdsourcing Simulation. In Paglieri, F., Tummolini, L., Falcone, R. & Micel, M., (eds), The goals of cognition: Essays in honour of Cristiano Castelfranchi. London: College Publications.
45. Paolucci, M., Kossman, D., Conte, R., Lukowicz, P., Argyrakis, P., Blandford, A., Bonelli, G., Anderson, S., Freitas, S., Edmonds, B., Gilbert, N., Gross, M., Kohlhammer, J., Koumoutsakos, P., Krause, A., Linnér, B. O., Slusallek, P., Sorkine, O., Sumner, R. W. & Helbing, D. (2013). Towards a living earth simulator. The European Physical Journal Special Topics 214(1), 77-108.
46. Payette, N. (2011). For an integrated approach to agent-based modeling of science. Journal of Artificial Societies and Social Simulation 14(4), 9.
47. Ragone, A., Mirylenka, K., Casati, F. & Marchese, M. (2013). On peer review in computer science: analysis of its effectiveness and suggestions for improvement. Scientometrics pp. 1-40.
48. Rao, A. S. (1996). AgentSpeak(L): BDI agents speak out in a logical computable language, in 'Proc. of MAAMAW'96', number 1038 in 'LNAI', Heidelberg: Springer, pp. 42-55.
49. Roebber, P. J. & Schultz, D. M. (2011). Peer review, program officers and science funding. PLoS One 6(4), e18680+.
50. Scharnhorst, A., Börner, K. & van den Besselaar, P., eds (2012). Models of Science Dynamics: Encounters Between Complexity Theory and Information Sciences. Berlin: Springer.
51. Schultz, D. M. (2010). Are three heads better than two? how the number of reviewers and editor behavior affect the rejection rate. Scientometrics 84(2), 277-292.
52. Searle, J. (1979). Expression and meaning: Studies in the theory of speech acts. Cambridge: Cambridge University Press.
53. Smith, R. (2006). Peer review: a flawed process at the heart of science and journals. JRSM 99(4), 178-182.
54. Snow, C. P. (2012). The two cultures. Cambridge: Cambridge University Press.
55. Spier, R. (2002). The history of the peer-review process. Trends in Biotechnology 20(8), 357-358.
56. Squazzoni, F. (2012). Agent-Based Computational Sociology. Chichester: Wiley.
57. Squazzoni, F., Bravo, G. & Takács, K. (2013) Does incentive provision increase the quality of peer review? an experimental study. Research Policy 42(1), 287 - 294.
58. Squazzoni, F. & Takács, K. (2011). Social simulation that 'peers into peer review. Journal of Artificial Societies and Social Simulation 14(4), 3.
59. Sterman, J. D. (1985). The growth of knowledge: Testing a theory of scientific revolutions with a formal model. Technological Forecasting and Social Change 28(2), 93 - 122.
60. Thurner, S. & Hanel, R. (2011). Peer-review in a world with rational scientists: Toward selection of the average. European Physical Journal B-Condensed Matter 84(4), 707.
61. Wicherts, J. M. (2011). Psychology must learn a lesson from fraud case. Nature 480(7375), 7.
62. Wilensky, U. & Rand, W. (2007). Making models match: Replicating an agent-based model. Journal of Artificial Societies and Social Simulation 10(4), 2.
63. Wooldridge, M. (2009). An introduction to MultiAgent systems, 2nd edn. Chichester: Wiley.