60th Anniversary Essay: How Journals Could Improve Research Practices in Social Science

Administrative Science Quarterly

Volumn 61, Issue 2, 2016, Pages 165-183

  Starbuck, William H ^a  

^a UNIVERSITY OF OREGON   (United States)

Author keywords

editorial policies; editors; empirical findings; hypotheses; reviewing; statistical analyses

Indexed keywords

EID: 84964805512     PISSN: 00018392     EISSN: 19303815     Source Type: Journal    
DOI: 10.1177/0001839216629644     Document Type: Article

References (70)

1. A.AcharyaA.VerstakH.SuzukiS.HendersonM.IakhiaevC.ChiungY.LinN.Shetty2014 “Rise of the rest: The growing impact of non-elite journals.” Google, Inc., October9. arXiv:1410.2217.
2. J.S.Armstrong1980 “Unintelligible management research and academic prestige.” Interfaces, 10: 80–86.
3. J.C.Bailar1991 “Reliability, fairness, objectivity and other inappropriate goals in peer review.” Behavioral and Brain Sciences, 14: 137–138.
4. A.G.Bedeian2004 “Peer review and the social construction of knowledge in the management discipline.” Academy of Management Learning and Education, 3: 198–216.
5. A.G.Bedeian2008 “Balancing authorial voice and editorial omniscience: The ‘It’s my paper and I’ll say what I want to’/’Ghostwriters in the sky’ minuet.” In (eds.), Opening the Black Box of Editorship: 134–142. Basingstoke, UK: Palgrave Macmillan.
6. A.G.BedeianM.C.SturmanD.L.Streiner2009 “Decimal dust, significant digits, and the search for stars.” Organizational Research Methods, 12: 687–694.
7. A.G.BedeianS.G.TaylorA.N.Miller2010 “Management science on the credibility bubble: Cardinal sins and various misdemeanors.” Academy of Management Learning and Education, 9: 715–725.
8. R.A.BettisS.EthirajA.GambardellaC.HelfatW.Mitchell2015 “Creating repeatable cumulative knowledge in strategic management.” Strategic Management Journal (forthcoming), published online ahead of print. DOI: 10.1002/smj.2477.
9. R.M.Blank1991 “The effects of double-blind versus single-blind reviewing: Experimental evidence from the American Economic Review.” American Economic Review, 81: 1041–1067.
10. A.K.Bones2012 “We knew the future all along: Scientific hypothesizing is much more accurate than other forms of precognition—A satire in one part.” Perspectives on Psychological Science, 7: 307–309.
11. L.BornmannP.Mungra2011 “Improving peer review in scholarly journals.” European Science Editing, 37 (2): 41–43.
12. M.Branch2014 “Malignant side-effects of null-hypothesis testing.” Theory and Psychology, 24: 256–277.
13. B.BrembsK.ButtonM.Munafo2013 “Deep impact: Unintended consequences of journal rank.” Frontiers in Human Neuroscience, 7 (291). DOI: 10.3389/fnhum.2013.00291.
14. D.V.Cicchetti2003 “The peer review of scientific documents: Suggestions for improvements.” Presentation to the Committee on Research in Education, National Research Council, February 26.
15. G.Cumming2013 “Dance of the p Values” (video). http://www.youtube.com/watch?v=5OL1Rq.
16. G.CummingF.FidlerM.LeonardP.KalinowskiA.ChristiansenA.KleinigJ.LoN.McMenaminS.Wilson2007 “Statistical reform in psychology: Is anything changing?” Psychological Science, 18: 230–232.
17. H.J.EinhornR.M.Hogarth1975 “Unit weighting schemes for decision making.” Organizational Behavior and Human Performance, 13: 171–192.
18. G.Ellison2002 “The slowdown of the economics publishing process.” Journal of Political Economy, 110: 947–993.
19. I.ErevG.Barron2005 “On adaptation, maximization, and reinforcement learning among cognitive strategies.” Psychological Review, 112: 912–931.
20. K.Eriksson2012 “The nonsense math effect.” Judgment and Decision Making, 7: 746–749.
21. F.Fidler2005 “From statistical significance to effect estimation: Statistical reform in psychology, medicine and ecology.” Doctoral dissertation, University of Melbourne. Accessed at http://www.latrobe.edu.au/psy/staff/fidler.html.
22. F.FidlerG.CummingM.BurgmanN.Thomason2004 “Statistical reform in medicine, psychology and ecology.” Journal of Socio-Economics, 33: 615–630.
23. B.ForguesS.Liarte2013 “Academic publishing: Past and future.” M@n@gement, 16: 739–756.
24. G.FrancisJ.TanzmanW.J.Matthews2014 “Excess success for psychology articles in the journalScience.” PLoS One, 9 (12): e114255.
25. H.G.GauchJr.2006 “Winning the accuracy game.” American Scientist, 94: 135–143.
26. S.D.Gottfredson1978 “Evaluating psychological research reports: Dimensions, reliability, and correlates of quality judgments.” American Psychologist, 33: 920–934.
27. L.L.HargensJ.R.Herting1990 “Neglected considerations in the analysis of agreement among journal referees.” Scientometrics, 19: 91–106.
28. M.HeintzelmanD.Nocetti2009 “Where should we submit our manuscript? An analysis of journal submission strategies.” Berkeley Electronic Journal of Economic Analysis and Policy. Available at http://ssrn.com/abstract=1314850.
29. J.R.HollenbeckM.J.Mannor2007 “Career success and weak paradigms: The role of activity, resiliency, and true scores.” Journal of Organizational Behavior, 28: 933–942.
30. K.D.Hoover1995 “In defense of data mining: Some preliminary thoughts.” In (eds.), Monetarism and the Methodology of Economics: Essays in Honour of Thomas Mayer: 242–257. Cheltenham, UK: Edward Elgar.
31. R.Hubbard2016Corrupt Research: The Case for Reconceptualizing Empirical Management and Social Science. Thousand Oaks, CA: Sage.
32. R.HubbardR.M.Lindsay2008 “Why p values are not a useful measure of evidence in statistical significance testing.” Theory and Psychology, 18: 69–88.
33. J.P.A.Ioannidis2005 “Why most published research findings are false.” PLoS Med, 2 (8): e124. DOI: 10.1371/journal.pmed.0020124.
34. S.KepesM.A.McDaniel2013 “How trustworthy is the scientific literature in industrial and organizational psychology?” Industrial and Organizational Psychology, 6: 252–268.
35. C.A.Kiesler1991 “Confusion between reviewer reliability and wise editorial and funding decisions.” Behavioral and Brain Sciences, 14: 151–152.
36. D.N.LabandM.J.Piette1994 “Favoritism versus search for good papers: Empirical evidence regarding the behavior of journal editors.” Journal of Political Economy, 102: 194–203.
37. E.Leahey2005 “Alphas and asterisks: The development of statistical significance testing standards in sociology.” Social Forces, 84: 1–24.
38. S.LichtensteinB.FischhoffL.Phillips1982 “Calibration probabilities: The state of the art to 1980.” In (eds.), Judgment under Uncertainty: Heuristics and Biases: 306–333. Cambridge: Cambridge University Press.
39. M.C.Lovell1983 “Data mining.” Review of Economics and Statistics, 65: 1–12.
40. M.J.MahoneyA.E.KazdinM.Kenigsberg1978 “Getting published.” Cognitive Therapy and Research, 2: 69–70.
41. T.Mayer1993Truth versus Precision in Economics, chapter 10. Cheltenham, UK: Edward Elgar.
42. J.J.MazzolaJ.K.Deuling2013 “Forgetting what we learned as graduate students: HARKing and selective outcome reporting in I–O journal articles.” Industrial and Organizational Psychology, 6: 279–284.
43. M.F.McBrideF.FidlerM.A.Burgman2012 “Evaluating the accuracy and calibration of expert predictions under uncertainty: Predicting the outcomes of ecological research.” Diversity and Distributions, 18: 782–794.
44. B.B.McShaneD.Gal2015 “Blinding us to the obvious? The effect of statistical training on the evaluation of evidence.” Management Science (forthcoming), published online ahead of print. Available at http://dx.doi.org/10.1287/mnsc.2015.2212.
45. M.H.Medoff2003 “Editorial favoritism in economics?” Southern Economic Journal, 70: 425–434.
46. C.C.Miller2006 “Peer review in the organizational and management sciences: Prevalence and effects of reviewer hostility, bias, and dissensus.” Academy of Management Journal, 49: 425–431.
47. C.C.MillerA.Van de Ven2015 “Peer review, root canals, and other amazing life events.” Academy of Management Discoveries (forthcoming), published online ahead of print. DOI: 10.5465/amd.2015.0039.
48. R.R.NelsonS.G.Winter2002 “Evolutionary theorizing in economics.” Journal of Economic Perspectives, 16 (2): 23–46.
49. M.NylennaP.RiisY.Karlsson1994 “Multiple blinded reviews of the same two manuscripts: Effects of referee characteristics and publication language.” JAMA, 272: 149–151.
50. Open Science Collaboration2015 “Estimating the reproducibility of psychological science.” Science, 349 (6251). DOI: 10.1126/science.aac4716.
51. P.N.PantW.H.Starbuck1990 “Innocents in the forest: Forecasting and research methods.” Journal of Management, 16: 433–460.
52. D.P.PetersS.J.Ceci1982 “Peer-review practices of psychological journals: The fate of published articles, submitted again.” Behavioral and Brain Sciences, 5: 187–255.
53. B.RosenbergM.Houglet1974 “Error rates in CRSP and Compustat data bases and their implications.” Journal of Finance, 29: 1303–1310.
54. J.G.San Miguel1977 “The reliability of R&D data in Compustat and 10-K reports.” Accounting Review, 52: 638–641.
55. A.SchwabE.AbrahamsonW.H.StarbuckF.Fidler2011 “Researchers should make thoughtful assessments instead of null-hypothesis significance tests.” Organization Science, 22: 1105–1120.
56. K.Shashok2008 “Content and communication: How can peer review provide helpful feedback about the writing?” BMC Medical Research Methodology, 8 (3): 1–9.
57. P.Shrout1997 “Should significance tests be banned? Introduction to a special section exploring the pros and cons.” Psychological Science, 8: 1–2.
58. K.SilerK.LeeL.Bero2015 “Measuring the effectiveness of scientific gatekeeping.” PNAS (Proceedings of the National Academy of Sciences), 112 (2): 360–365.
59. J.P.SimmonsL.D.NelsonU.Simonsohn2011 “False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant.” Psychological Science, 22: 1359–1366.
60. W.H.Starbuck2005 “How much better are the most prestigious journals? The statistics of academic publication.” Organization Science, 16: 180–200.
61. W.H.Starbuck2006The Production of Knowledge. Oxford: Oxford University Press.
62. W.H.Starbuck2009 “Cognitive reactions to rare events: Perceptions, uncertainty, and learning.” Organization Science, 20: 925–937.
63. W.H.Starbuck2013 “Why and where do academics publish?” M@n@gement, 16: 707–718.
64. W.H.StarbuckH.AguinisA.M.KonradY.Baruch2008 “Tradeoffs among editorial goals in complex publishing environments.” In (eds.), Opening the Black Box of Editorship: 250–270. Basingstoke, UK: Palgrave Macmillan.
65. C.M.TardyP.K.Matsuda2009 “The construction of author voice by editorial board members.” Written Communication, 26: 32–52.
66. M.Ware2008 “Peer review: Benefits, perceptions and alternatives.” London: Publishing Research Consortium.
67. T.J.WebbB.O’HaraR.P.Freckleton2008 “Does double-blind review benefit female authors?” Trends in Ecology and Evolution, 23: 351–353.
68. E.J.WebsterW.H.Starbuck1988 “Theory building in industrial and organizational psychology.” In (eds.), International Review of Industrial and Organizational Psychology: 93–138. London: Wiley.
69. A.Yankauer1991 “How blind is blind review?” American Journal of Public Health, 81: 843–845.
70. N.S.YoungJ.P.A.IoannidisO.Al-Ubaydli2008 “Why current publication practicesmaydistort science.” PLoS Med, 5 (10): e201. DOI: 10.1371/journal.pmed.0050201.