Letting the daylight in: Reviewing the reviewers and other ways to maximize transparency in science

Frontiers in Computational Neuroscience

Volumn , Issue MARCH 2012, 2012, Pages 1-31

  Wicherts, Jelte M ^a   Kievit, Rogier A ^a   Bakker, Marjan ^a   Borsboom, Denny ^a  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

HIGHER-DEGREE; ONLINE DATA; ONLINE PUBLICATIONS; ONLINE PUBLISHING; PEER REVIEW; REPLICABILITY; SCIENTIFIC RESEARCHES; STATISTICAL ERRORS;

TRANSPARENCY;

REVIEWS;

INTELLIGENCE QUOTIENT; MEDICAL RESEARCH; ONLINE SYSTEM; PEER REVIEW; PROBLEM SOLVING; PUBLICATION; REVIEW; SCIENTIFIC LITERATURE; STATISTICAL ANALYSIS;

EID: 84859069563     PISSN: 16625188     EISSN: None     Source Type: Journal    
DOI: 10.3389/fncom.2012.00020     Document Type: Review

References (51)

1. Abbott, P., Abe, J., Alcock, J., Alizon, S., Alpedrinha, J. A. C., Andersson, M., et al. (2011). Inclusive fitness theory and eusociality. Nature, 471, E1-E5. doi: 10.1038/nature09831
2. Alberts, B. (2011). Editor's Note. Science, 332, 1149. doi: 10.1126/science.1208877
3. American Psychological Association. (2010). Publication Manual of the American Psychological Association. Sixth Edition. Washington, DC: American Psychological Association.
4. Bakker, M., & Wicherts, J. M. (2011). The (mis)reporting of statistical results in psychology journals. Behavior Research Methods, 43, 666-678. doi: 10.3758/s13428-011-0089-5
5. Benos, D. J., Bashari, E., Chaves, J. M., Gaggar, A., Kapoor, N., LaFrance, M., et al. (2007). The ups and downs of peer review Advances in Physiology Education 31, 145-152. doi: 10.1152/advan.00104.2006
6. Berle, D., & Starcevic, V. (2007). Inconsistencies between reported test statistics and p-values in two psychiatry journals. International Journal of Methods in Psychiatric Research, 16, 202-207. doi: 10.1002/mpr.225
7. Darley, J. M., & Latané, B. (1968). Bystander intervention in emergencies: diffusion of responsibility. Journal of Personality and Social Psychology, 8, 377-383.
8. Garcia-Berthou, E., & Alcaraz, C. (2004). Incongruence between test statistics and P values in medical papers. BMC Medical Research Methodology, 4, 13. doi: 10.1186/1471-2288-4-13
9. Gelade, G. A. (2008). The geography of IQ. Intelligence, 36, 495-501. doi: 10.1016/j.intell.2008.04.003.
10. Gerber, A. S., & Malhotra, N. (2008a). Do statistical reporting standards affect what is published? Publication bias in two leading political science journals. Quarterly Journal of Political Science, 3, 313-326. doi: 10.1561/100.00008024
11. Gerber, A. S., & Malhotra, N. (2008b). Publication bias in empirical sociological research - Do arbitrary significance levels distort published results? Sociological Methods & Research, 37, 3-30. doi: 10.1177/0049124108318973
12. Godlee, F., Gale, C. R., & Martyn, C. N. (1998). Effect on the quality of peer review of blinding reviewers and asking them to sign their reports a randomized controlled trial. Jama-Journal of the American Medical Association, 280, 237-240.
13. Godlee, F., Smith, J., & Marcovitch, H. (2011). Wakefield's article linking MMR vaccine and autism was fraudulent. British Medical Journal, 342, 64-66. doi: 10.1136/bmj.c7452
14. Gottfredson, L. S. (2010). Lessons in academic freedom as lived experience. Personality and Individual Differences, 49, 272-280. doi: 10.1016/j.paid.2010.01.001
15. Hanson, B., Sugden, A., & Alberts, B. (2011). Making data maximally available. Science, 331, 649. doi: 10.1126/science.1203354
16. Hassal, C., & Sherrat, T. (2011). Statistical inference and spatial patterns in correlates of IQ. Intelligence, 39, 303-310. doi: 10.1016/j.intell.2011.05.001
17. Hunt, E. B., & Carlson, J. S. (2007). Considerations relating to the study of group differences in intelligence. Perspectives on Psychological Science, 2, 194-213. doi: 10.1111/j.1745-6916.2007.00037.x
18. Hunt, M. (1999). The new know-nothings: The political foes of the scientific study of human nature. New Brunswick, NJ: Transaction Publishers.
19. ICH. (1996). Good Clinical Practice: Consolidated Guidance. Geneva, Switzerland: International Conference on Harmonisation.
20. Ioannidis, J. P. A., & Trikalinos, T. A. (2007). An exploratory test for an excess of significant findings. Clinical Trials, 4, 245-253. doi: 10.1177/1740774507079441
21. Kanazawa, S. (2004). General intelligence as a domain-specific adaptation. Psychological Review, 111, 512-523. doi: 10.1037/0033-295X.111.2.512
22. Kanazawa, S. (2008). Temperature and evolutionary novelty as forces behind the evolution of general intelligence. Intelligence, 36, 99-108. doi: 10.1016/j.intell.2007.04.001
23. Kriegeskorte, N., Simmons, W. K., Bellgowan, P. S. F., & Baker, C. I. (2009). Circular analysis in systems neuroscience: the dangers of double dipping. Nature Neuroscience, 12, 535-540. doi: 10.1038/nn.2303
24. Lynn, R. (2010). Consistency of race differences in intelligence over millennia: A comment on Wicherts, Borsboom and Dolan. Personality and Individual Differences, 48, 100-101. doi: 10.1016/j.paid.2009.09.007
25. Mayor, J. (2010). Are scientists nearsighted gamblers? The misleading nature of impact factors. Frontiers in Psychology, 1, 215. doi: 10.3389/fpsyg.2010.00215
26. Moxham, H., & Anderson, J. (1992). Peer review: A view from the inside. Science and Technology Policy, 5, 7-15.
27. Murphy, J. R. (2004). Statistical errors in immunologic research. Journal of Allergy and Clinical Immunology, 114, 1259-1264. doi: 10.1016/j.jaci.2004.09.023
28. Nieuwenhuis, S., Forstmann, B. U., & Wagenmakers, E. J. (2011). Erroneous analyses of interactions in neuroscience: a problem of significance. Nature Neuroscience, 14, 1105-1107. doi: 10.1038/nn.2886
29. Nowak, M. A., Tarnita, C. E., & Wilson, E. O. (2010). The evolution of eusociality. Nature, 466, 1057-1062. doi: 10.1038/nature09205
30. Oppenheimer, S. (2004). Out of Eden: The peopling of the world London: Constable & Robinson Ltd.
31. Redfield, R. (2010). Arsenic-associated bacteria (NASA's claims) Retrieved April, 15, 2011, from http://rrresearch.blogspot.com/2010/12/arsenic-associatedbacteria-nasas.html
32. Ridley, J., Kolm, N., Freckelton, R. P., & Gage, M. J. G. (2007). An unexpected influence of widely used significance thresholds on the distribution of reported P-values. Journal of Evolutionary Biology, 20, 1082-1089. doi: 10.1111/j.1420-9101.2006.0291.x
33. Rossi, J. S. (1987). How often are our statistics wrong - a statistics class exercise. Teaching of Psychology, 14, 98-101.
34. Rushton, J. P. (2010). Brain size as an explanation of national differences in IQ, longevity, and other life-history variables Personality and Individual Differences, 48, 97-99. doi: 10.1016/j.paid.2009.07.029
35. Savage, C. J., & Vickers, A. J. (2009). Empirical study of data sharing by authors publishing in PLoS journals. PLoS One, 4, e7078. doi: 10.1371/journal.pone.0007078
36. Sellke, T., Bayarri, M. J., & Berger, J. O. (2001). Calibration of p values for testing precise null hypotheses. The American Statistician, 55, 62-71.
37. Smith, R. (2006). Peer review: a flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99, 178-182. doi: 10.1258/jrsm.99.4.178
38. Smith, R. W. (2009). In search of an optimal peer review system. Journal of Participatory Medicine, 1, e13.
39. Strasak, A. M., Zaman, Q., Marinell, G., Pfeiffer, K. P., & Ulmer, H. (2007). The use of statistics in medical research: A comparison of The New England Journal of Medicine and Nature Medicine. American Statistician, 61, 47-55. doi: 10.1198/000313007X170242
40. Templer, D. I. (2010). Can't see the forest because of the trees Personality and Individual Differences, 48, 102-103. doi: 10.1016/j.paid.2009.08.011
41. Templer, D. I., & Arikawa, H. (2006). Temperature, skin color, per capita income, and IQ: An international perspective. Intelligence, 34, 121-139. doi: 10.1016/j.intell.2005.04.002
42. Van Raan, A. F. J. (1996). Advanced bibliometric methods as quantitative core of peer review based evaluation and foresight exercises. Scientometrics, 36, 397-420.
43. Vul, E., Harris, C., Winkielman, P., & Pashler, H. (2009). Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition. Perspectives on Psychological Science, 4, 274-290. doi: 10.1111/j.1745-6924.2009.01125.x
44. Wakefield, A. J., Murch, S. H., Anthony, A., Linnell, J., Casson, D. M., Malik, M., et al. (1998). [Retracted] Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children Lancet, 351, 637-641.
45. Wicherts, J. M. (2011). Psychology must learn a lesson from fraud case. Nature, 480, 7. doi: 10.1038/480007a
46. Wicherts, J. M., & Bakker, M. (2012). Publish (your data) or (let the data) perish! Why not publish your data too? Intelligence. doi: 10.1016/j.intell.2012.01.004
47. Wicherts, J. M., Bakker, M., & Molenaar, D. (2011). Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PLoS ONE, 6, e26828. doi: 10.1371/journal.pone.0026828
48. Wicherts, J. M., Borsboom, D., & Dolan, C. V. (2010a). Evolution, brain size, and the national IQ of peoples around 3,000 years B.C. Personality and Individual Differences, 48, 104-106. doi: 10.1016/j.paid.2009.08.020
49. Wicherts, J. M., Borsboom, D., & Dolan, C. V. (2010b). Why national IQs do not support evolutionary theories of intelligence. Personality and Individual Differences, 48, 91-96. doi: 10.1016/j.paid.2009.05.028
50. Wicherts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006). The poor availability of psychological research data for reanalysis. American Psychologist, 61, 726-728. doi: 10.1037/0003-066X.61.7.726
51. Wolfe-Simom, F., Blum, J. S., Kulp, T. R., Gordon, G. W., Hoeft, S. E., Pett-Ridge, J., et al. (2011). A bacterium that can grow by using arsenic instead of phosphorus Science. doi: 10.1126/science.1197258