Questions About Questionable Research Practices in the Field of Management: A Guest Commentary

Journal of Management

Volumn 42, Issue 1, 2016, Pages 5-20

  Banks, George C ^a   O’Boyle, Ernest H ^b   Pollack, Jeffrey M ^c   White, Charles D ^d   Batchelor, John H ^e   Whelpley, Christopher E ^f   Abston, Kristie A ^e   Bennett, Andrew A ^g   Adkins, Cheryl L ^d  

^a UNIVERSITY OF NORTH CAROLINA AT CHARLOTTE   (United States)

^b UNIVERSITY OF IOWA   (United States)

^c NORTH CAROLINA STATE UNIVERSITY   (United States)

^d LONGWOOD UNIVERSITY   (United States)

^e UNIVERSITY OF WEST FLORIDA   (United States)

^f VIRGINIA COMMONWEALTH UNIVERSITY   (United States)

^g UNIVERSITY OF ALABAMA   (United States)

Author keywords

philosophy of science; questionable research practices (QRPs); research methodology

Indexed keywords

EID: 84949294162     PISSN: 01492063     EISSN: 15571211     Source Type: Journal    
DOI: 10.1177/0149206315619011     Document Type: Editorial

References (59)

1. H.AguinisR.K.GottfredsonH.Joo2013. Best-practice recommendations for defining, identifying, and handling outliers. Organizational Research Methods, 16: 270-301.
2. A.A.Alsheikh-AliW.QureshiM.H.Al-MallahJ.P.Ioannidis2011. Public availability of published research data in high-impact journals. PLoS ONE, 6(9): e24357. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0024357
3. M.S.AndersonB.C.MartinsonR.De Vries2007. Normative dissonance in science: Results from a national survey of US scientists. Journal of Empirical Research on Human Research Ethics, 2: 3-14.
4. M.S.AndersonE.A.RonningR.De VriesB.C.Martinson2007. The perverse effects of competition on scientists’ work and relationships. Science and Engineering Ethics, 13: 437-461.
5. S.AndraszewiczB.ScheibehenneJ.RieskampR.GrasmanJ.VerhagenE.-J.Wagenmakers2014. An introduction to Bayesian hypothesis testing for management research. Journal of Management, 41: 521-543.
6. M.BakkerA.van DijkJ.M.Wicherts2012. The rules of the game called psychological science. Perspectives on Psychological Science, 7: 543-554.
7. M.BakkerJ.M.Wicherts2011. The (mis)reporting of statistical results in psychology journals. Behavior Research, 43: 666-678. doi:10.3758/s13428-011-0089-5
8. M.BakkerJ.M.Wicherts2014. Outlier removal and the relation with reporting errors and quality of psychological research. PLoS ONE, 9(7): e103360. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0103360
9. G.C.BanksS.KepesM.A.McDaniel2012. Publication bias: A call for improved meta-analytic practice in the organizational sciences. International Journal of Selection and Assessment, 20: 182-196. doi:10.1111/j.1468-2389.2012.00591.x
10. G.C.BanksS.KepesM.A.McDaniel2015. Publication bias: Understand the myths concerning threats to the advancement of science. In C.E.LanceR.J.Vandenberg (Eds.), More statistical and methodological myths and urban legends:36-64. New York: Routledge.
11. G.C.BanksM.A.McDaniel2011. The kryptonite of evidence-based I-O psychology. Industrial and Organizational Psychology: Perspectives on Science and Practice, 4: 40-44. doi:10.1111/j.1754-9434.2010.01292.x
12. G.C.BanksE.H.O’BoyleJr.2013. Why we need industrial-organizational psychology to fix industrial-organizational psychology. Industrial and Organizational Psychology: Perspectives on Science and Practice, 6: 291-294.
13. A.G.Bedeian2003. The manuscript review process: The proper roles of authors, referees, and editors. Journal of Management Inquiry, 12: 331-338. doi:10.1177/1056492603258974
14. A.G.BedeianS.G.TaylorA.N.Miller2010. Management science on the credibility bubble: Cardinal sins and various misdemeanors. Academy of Management Learning & Education, 9: 715-725. doi:10.5465/amle.2010.56659889
15. F.A.BoscoH.AguinisJ.G.FieldC.A.PierceD.R.Daltonin press. HARKing’s threat to organizational research: Evidence from primary and meta-analytic sources. Personnel Psychology. doi:10.1111/peps.12111
16. J.M.Cortina2015. A revolution with a solution. Opening plenary presented at the meeting of the Society for Industrial/Organizational Psychology, Philadelphia.
17. R.De VriesM.S.AndersonB.C.Martinson. 2006. Normal misbehavior: Scientists talk about the ethics of research. Journal of Empirical Research on Human Research Ethics, 1: 43-50.
18. E.Eich2014. Business not as usual. Psychological Science, 25: 3-6.
19. Elsevier. 2015. Publishing and research ethics: Discussing the major types of scientific misconduct and how to avoid them. https://www.publishingcampus.elsevier.com/pages/63//ethics/Publishing-ethics.html. Accessed August 20, 2015.
20. G.B.EmersonW.J.WarmeF.M.WolfJ.D.HeckmanR.A.BrandS.S.Leopold2010. Testing for the presence of positive-outcome bias in peer review: A randomized controlled trial. Archives of Internal Medicine, 170: 1934-1939. doi:10.1001/archinternmed.2010.406
21. D.Fanelli2009. How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS ONE, 4(5): e5738. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0005738
22. D.Fanelli2013. Redefine misconduct as distorted reporting. Nature, 494: 149.
23. F.C.FangA.Casadevall2011. Retracted science and the retraction index. Infection and Immunity, 79: 3855-3859.
24. F.C.FangA.Casadevall2015. Competitive science: Is competition ruining science?Infection and Immunity, 83: 1229-1233.
25. E.J.FinkelP.W.EastwickH.T.Reis2015. Best research practices in psychology: Illustrating epistemological and pragmatic considerations with the case of relationship science. Journal of Personality and Social Psychology, 108: 275-297.
26. G.FrancisJ.TanzmanW.J.Matthews2014. Excess success for psychology articles in the journal Science. PLoS ONE, 9(12): e114255. http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0114255
27. A.FrancoN.MalhotraG.Simonovits2014. Publication bias in the social sciences: Unlocking the file drawer. Science, 345: 1502-1505.
28. D.C.Hambrick2007. The field of management’s devotion to theory: Too much of a good thing?Academy of Management Journal, 50: 1346-1352.
29. J.S.HarrisonG.C.BanksJ.M.PollackE.H.O’BoyleJr.J.C.Shortin press. Publication bias in strategic management research. Journal of Management. doi:10.1177/0149206314535438
30. C.HitchcockC.Sober2004. Prediction versus accommodation and the risk of overfitting. The British Journal for the Philosophy of Science, 55: 1-34.
31. R.L.Hotz2015. How many scientists does it take to write a paper? Apparently, thousands. Wall Street Journal. http://www.wsj.com/articles/how-many-scientists-does-it-take-to-write-a-paper-apparently-thousands-1439169200. Accessed November 11, 2015.
32. L.K.JohnG.LoewensteinD.Prelec2012. Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23: 524-532. doi:10.1177/0956797611430953
33. S.KepesG.C.BanksM.A.McDanielD.L.Whetzel2012. Publication bias in the organizational sciences. Organizational Research Methods, 15: 624-662. doi:10.1177/1094428112452760
34. S.KepesM.A.McDaniel2013. How trustworthy is the scientific literature in I-O psychology?Industrial and Organizational Psychology: Perspectives on Science and Practice, 6: 252-268.
35. N.L.Kerr1998. HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2: 196-217. doi:10.1207/s15327957pspr0203_4
36. J.T.LeekR.D.Peng2015. P values are just the tip of the iceberg. Nature, 520: 612.
37. K.Leung2011. Presenting post hoc hypotheses as a priori: Ethical and theoretical issues. Management and Organizational Review, 7: 471-479.
38. E.A.Locke2007. The case for inductive theory building. Journal of Management, 33: 867-890.
39. B.C.MartinsonM.S.AndersonA.L.CrainR.De Vries2006. Scientists’ perceptions of organizational justice and self-reported misbehaviors. Journal of Empirical Research on Human Research Ethics, 1: 51-66.
40. C.Matlack2013. Research fraud allegations trail a German B-school wunderkind. Bloomberg Businessweek. http://www.businessweek.com/articles/2013-06-24/research-fraud-allegations-trail-a-german-b-school-wunderkind. Accessed November 11, 2015.
41. J.J.MazzolaJ.K.Deuling2013. Forgetting what we learned as graduate students: HARKing and selective outcome reporting in I-O journal articles. Industrial and Organizational Psychology: Perspectives on Science and Practice, 6: 279-284.
42. B.A.NosekG.AlterG.C.BanksD.BorsboomS.D.BowmanS.BrecklerS.BuckC.ChambersG.ChinG.ChristensenM.ContestabileA.DafoeE.EichJ.FreeseR.GlennersterD.GoroffD.P.GreenB.HeeseM.HumphreysJ.IshiyamaD.KarlanA.KrautA.LupiaP.MarbryT.MadonN.MalhotraE.M.WilsonM.McNuttE.MiguelE.L.PaluckU.SimonsohnC.SoderbergB.A.SpellmanJ.TornowJ.TurittoG.R.VandenBosS.VazireE.J.WagenmakersR.WilsonT.Yarkoni2015. Promoting an open research culture: Author guidelines for journals to promote transparency, openness, and reproducibility. Science, 348: 1422-1425.
43. B.A.NosekY.Bar-Anan2012. Scientific utopia: I. Opening scientific communication. Psychological Inquiry, 23: 217-243.
44. B.A.NosekJ.R.SpiesM.Motyl2012. Scientific utopia: II. Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7: 615-631. doi:10.1177/1745691612459058
45. M.B.NuijtenC.H.J.HartgerinkM.A.L.M.Van AssenS.EpskampJ.M.Wichertsin press. The prevalence of statistical reporting errors in psychology (1985-2013). Behavior Research Methods. doi:10.3758/s13428-015-0664-2
46. E.H.O’BoyleG.C.BanksK.CarterS.WalterZ.Yuan2015. A 20-year review of outcome reporting bias in moderated multiple regression. Paper presented at the annual meeting of the Academy of Management, Vancouver.
47. E.H.O’BoyleJr.G.C.BanksE.Gonzalez-Mule. in press. The chrysalis effect: How ugly initial results metamorphosize into beautiful articles. Journal of Management. doi:10.1177/0149206314527133
48. Open Science Collaboration. 2015. Estimating the reproducibility of psychological science. Science, 349: 6251. doi:10.1126/science.aac4716
49. T.D.PigottJ.C.ValentineJ.R.PolaninR.T.WilliamsD.D.Canada2013. Outcome-reporting bias in education research. Educational Researcher, 42: 424-432. doi:10.3102/0013189X13507104
50. J.W.Riley1958. Proceedings of the thirteenth conference on public opinion research. Public Opinion Quarterly, 22: 169-216.
51. F.L.SchmidtJ.E.Hunter2015. Methods of meta-analysis: Correcting error and bias in research findings (3rd ed.). Thousand Oaks, CA: Sage.
52. K.SijtsmaC.L.VeldkampJ.M.Wichertsin press. Improving the conduct and reporting of statistical analysis in psychology. Psychometrika. doi:10.1007/s11336-015-9444-2
53. J.P.SimmonsL.D.NelsonU.Simonsohn2011. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22: 1359-1366. doi:10.1177/0956797611417632
54. E.-J.WagenmakersR.WetzelsD.BorsboomH.L.Van Der Maas2011. Why psychologists must change the way they analyze their data: The case of psi: Comment on Bem (2011). Journal of Personality and Social Psychology, 100: 426-432.
55. E.-J.WagenmakersR.WetzelsD.BorsboomH.L.Van Der MaasR.A.Kievit2012. An agenda for purely confirmatory research. Perspectives on Psychological Science, 7: 632-638.
56. J.M.WichertsD.BorsboomJ.KatsD.Molenaar2006. The poor availability of psychological research data for reanalysis. American Psychologist, 61: 726-728.
57. C.W.WieseJ.M.Fullick2014. The fantastic four years: Recommendations for industrial–organizational programs. Industrial and Organizational Psychology: Perspectives on Science and Practice, 7: 21-26.
58. D.H.WigboldusR.Dotschin press. Encourage playing with data and discourage questionable reporting practices. Psychometrika. doi:10.1007/s11336-015-9445-1
59. M.J.ZyphurF.L.Oswald2015. Bayesian estimation and inference: A user’s guide. Journal of Management, 41: 390-420.