REGULAR ISSUE PAPER What happened to Popperian falsification? Publishing neutral and negative findings : Moving away from biased publication practices

Cross Cultural and Strategic Management

Volumn 23, Issue 3, 2016, Pages 481-508

  Witteloostuijn, Arjen Van ^a,b,c  

^a TILBURG UNIVERSITY   (Netherlands)

^b ANTWERP MANAGEMENT SCHOOL   (Belgium)

^c CARDIFF UNIVERSITY   (United Kingdom)

Author keywords

Falsification; Publication bias; Replication; Research malpractices

Indexed keywords

EID: 85018170294     PISSN: 20595794     EISSN: None     Source Type: Journal    
DOI: 10.1108/CCSM-03-2016-0084     Document Type: Article

References (116)

1. Adler, N.J. and Harzing, A.-W. (2009), "When knowledge wins: transcending the sense and nonsense of academic rankings", Academy of Management Learning & Education, Vol. 8, pp. 72-95.
2. Aguinis, H.A., Suarez-Gonzalez, I., Lannelongue, G. and Joo, H. (2012), "Scholarly impact revisited", Academy of Management Perspectives, Vol. 26, pp. 105-132.
3. Barker, V.L. and Mone, M.A. (1994), "Retrenchment: cause of turnaround or consequence of decline?", Strategic Management Journal, Vol. 15, pp. 395-405.
4. Baum, J.A. (2011), "Free-riding on power laws: questioning the validity of the impact factor as a measure of research quality in organization studies", Organization, Vol. 18, pp. 449-466.
5. Bedeian, A.G. (2003), "The manuscript review process: the proper roles of authors, referees, and editors", Journal of Management Inquiry, Vol. 12, pp. 331-338.
6. Bedeian, A.G., Taylor, S.G. and Miller, A.N. (2010), "Management science on the credibility bubble: cardinal sins and various misdemeanors", Academy of Management Learning & Education, Vol. 9, pp. 715-725.
7. Begg, C.B. and Berlin, J.A. (1988), "Publication bias: a problem in interpreting medical data", Journal of the Royal Statistical Society: Series A (Statistics in Society), Vol. 151, pp. 419-463.
8. Bettis, R.A. (2012), "The search for asterisks: comprised statistical tests and flawed theories", Strategic Management Journal, Vol. 33, pp. 108-113.
9. Birkinshaw, J., Healey, M.P., Suddaby, R. and Weber, K. (2014), "Debating the future of management research", Journal of Management Studies, Vol. 51, pp. 38-55.
10. Blaug, M. (1992), The Methodology of Economics: Or How Economists Explain, Cambridge University Press, New York, NY.
11. Bogaert, S., Boone, C., Negro, G. and van Witteloostuijn, A. (2016), "Organizational form emergence: a meta-analysis of the ecological theory of legitimation", Journal of Management, Vol. 42, pp. 1344-1373.
12. Bradford De Long, J. and Lang, K. (1992), "Are all economic hypotheses false?", Journal of Political Economy, Vol. 100, pp. 1257-1272.
13. Brodeur, A., Lé, M., Sangnier, M. and Zylberberg, Y. (2012), "Star wars: the empirics strike back", Working Paper No. 2012-29, Paris School of Economics, Paris.
14. Callaham, M. and McCulloch, C. (2011), "Longitudinal trends in the performance of scientific peer reviewers", Annals of Emergency Medicine, Vol. 57, pp. 141-148.
15. Callaham, M.L. and Tercier, J. (2007), "The relationship of previous training and experience of journal peer reviewers to subsequent review quality", PLoS Medicine, Vol. 4, pp. 32-40.
16. Card, D. and Krueger, A.B. (1995), "Time-series minimum wage studies: a meta-analysis", American Economic Review Papers and Proceedings, Vol. 85, pp. 238-243.
17. Coase, R.H. (1937), "The nature of the firm", Economica, Vol. 16, pp. 386-405.
18. Colquitt, J.A. and Zapata-Phelan, C.P. (2007), "Trends in theory building and theory testing: a five-decade study of the Academy of Management Journal", Academy of Management Journal, Vol. 50, pp. 1281-1303.
19. Corley, K.G. and Gioia, D.A. (2011), "Building theory about theory building: what constitutes a theoretical contribution?", Academy of Management Review, Vol. 36, pp. 12-32.
20. Davis, G.F. (2015), "Editorial essay: what is organizational research for?", Administrative Science Quarterly, Vol. 60, pp. 179-188.
21. Delmar, F. (2015), "A response to Honig and Samuelson (2014)", Journal of Business Venturing Insights, Vol. 3, pp. 1-4.
22. Delbridge, R. and Fiss, P.C. (2013), "Editors' comments: styles of theorizing and the social organization of knowledge", Academy of Management Review, Vol. 38, pp. 325-331.
23. Dewald, W.G., Thursby, J.G. and Anderson, R.G. (1986), "Replication in empirical economics: the journal of money, credit and banking project", American Economic Review, Vol. 76, pp. 587-603.
24. Dikova, D. and van Witteloostuijn, A. (2007), "Acquisition versus greenfield foreign entry: diversification mode choice in Central and Eastern Europe", Journal of International Business Studies, Vol. 38, pp. 1013-1033.
25. DiMaggio, P. and Powell, W.W. (1983), "The iron cage revisited: collective rationality and institutional isomorphism in organizational fields", American Sociological Review, Vol. 48, pp. 147-160.
26. Doucouliagos, H. and Stanley, T.D. (2009), "Publication bias in minimum-wage research? A meta-regression analysis", British Journal of Industrial Relations, Vol. 47, pp. 406-428.
27. Duval, S. and Tweedie, R. (2000), "A nonparametric 'Trim and Fill' method of accounting for publication bias in meta-analysis", Journal of American Statistical Association, Vol. 95, pp. 89-98.
28. Eden, D. (2002), "Replication, meta-analysis, scientific progress, and AMJ's publication policy", Academy of Management Journal, Vol. 45, pp. 841-846.
29. Evanschitzky, H., Baumgarth, C., Hubbard, R. and Armstrong, J.S. (2007), "Replication research's disturbing trend", Journal of Business Research, Vol. 60, pp. 411-415.
30. Ferguson, C.J. and Heene, M. (2012), "A vast graveyard of undead theories: publication bias and psychological science's aversion to the null", Perspectives on Psychological Science, Vol. 7, pp. 555-561.
31. Geyskens, I., Krishnan, R., Steenkamp, J.-B.E.M. and Cunha, P.V. (2009), "A review and evaluation of meta-analysis practices in management research", Journal of Management, Vol. 35, pp. 393-419.
32. Ghoshal, S. (2005), "Bad management theories are destroying good management practices", Academy of Management Learning & Education, Vol. 4, pp. 75-91.
33. Golden, M.A. (1995), "Replication and non-quantitative research", PS: Political Science and Politics, Vol. 28, pp. 481-483.
34. Grey, C. (2010), "Organizing studies: publications, politics and polemic", Organization Studies, Vol. 31, pp. 677-694.
35. Hambrick, D.C. (2007), "The field of management's devotion to theory: too much of a good thing?", Academy of Management Journal, Vol. 50, pp. 1346-1352.
36. Hand, D., Mannila, H. and Smyth, P. (2001), Principles of Data Mining, MIT Press, Cambridge, MA.
37. Hannan, M.T. and Freeman, J. (1977), "The population ecology of organizations", American Journal of Sociology, Vol. 82, pp. 929-964.
38. Hannan, M.T. and Freeman, J. (1984), "Structural inertia and organizational change", American Sociological Review, Vol. 49, pp. 149-164.
39. Head, M.L., Holman, L., Lanfear, R., Kahn, A.T. and Jennions, M.D. (2015), "The extent and consequences of P-hacking in science", PLoS Biology, Vol. 13, e10022106.
40. Helfat, C.E. (2007), "Stylized facts, empirical research and theory development in management", Strategic Organization, Vol. 5, pp. 185-192.
41. Hoenig, J.M. and Heisey, D.M. (2001), "The abuse of power: the pervasive fallacy of power calculations for data analysis", The American Statistician, Vol. 55, pp. 1-6.
42. Honig, B. and Samuelsson, M. (2014), "Data replication and extension: a study of business planning and venture-level performance", Journal of Business Venturing Insights, Vols 1-2, pp. 18-25.
43. Hubbard, R. and Armstrong, J.S. (1997), "Publication bias against null results", Psychological Reports, Vol. 80, pp. 337-338.
44. Hubbard, R. and Vetter, D.E. (1996), "An empirical comparison of published replication research in accounting, economics, finance, management, and marketing", Journal of Business Research, Vol. 35, pp. 153-164.
45. Hubbard, R., Vetter, D.E. and Little, E.L. (1998), "Replication in strategic management: scientific testing for validity, generalizability, and usefulness", Strategic Management Journal, Vol. 19, pp. 243-254.
46. Imbens, G.W. and Rubin, D.B.D.B. (2015), Causal Inference in Statistics, Social, and Biomedical Sciences, Cambridge University Press, Cambridge.
47. Institute of Chartered Accountants in England and Wales (2015), The Effects of Mandatory IFRS Adoption in the EU: A Review of Empirical Research, ICAEW, London.
48. Ioannidis, J.P.A. (2005), "Why most published research findings are false", PLoS Medicine, Vol. 2, e124.
49. Ioannidis, J.P.A. (2012), "Why science is not necessarily self-correcting", Perspectives on Psychological Science, Vol. 7, pp. 645-654.
50. Isaeva, N., Bachmann, R., Bristow, A. and Saunders, M.N.K. (2015), "Why the epistemologies of trust researchers matter", Journal of Trust Research, forthcoming.
51. Jefferson, T., Alderson, P., Wagner, E. and Davidoff, F. (2002), "Effects of editorial peer review: a systematic review", Journal of the American Medical Association, Vol. 287 No. 21, pp. 1-4.
52. Jonas, K.J. and Cesario, J. (2015), "How can preregistration contribute to research in our field?", Comprehensive Results in Social Psychology. doi: 10.1080/23743603.2015.1070611.
53. Kacmar, K.M. (2009), "From the editors: an ethical quiz", Academy of Management Journal, Vol. 52, pp. 432-434.
54. Kahneman, D. (2011), Thinking, Fast and Slow, Farrar, Straus, and Giroux, New York, NY.
55. Kane, E.J. (1984), "Why journal editors should encourage the replication of applied econometrics research", Quarterly Journal of Business & Economics, Vol. 23, pp. 3-8.
56. Kerr, N.L. (1998), "HARKing: hypothesizing after the results are known", Personality and Social Psychology, Vol. 2, pp. 196-217.
57. Kuhn, T.S. (1962), The Structure of Scientific Revolutions, University of Chicago Press, Chicago, IL.
58. Leamer, E.E. (1978), Specification Searches: Ad Hoc Inference with Nonexperimental Data, Wiley, New York, NY.
59. Levy, O., Taylor, S., Boyacigiller, N.A., Bodner, T.E., Peiperl, M.A. and Beechler, S. (2015), "Perceived senior leadership opportunities in MNCs: the effect of social hierarchy and capital", Journal of International Business Studies, Vol. 46, pp. 285-307.
60. Lexchin, J., Bero, L.A., Djulbegovic, B. and Clark, O. (2003), "Pharmaceutical industry sponsorship and research outcome and quality: systematic review", British Medical Journal, Vol. 326, pp. 1167-1170.
61. Lindebaum, D. (2015), "Critical essay: building new management theories on sound data? The case of neuroscience", Human Relations, forthcoming.
62. McCullough, B.D., McGeary, K.A. and Harrison, T.D. (2006), "Lessons from the JMCB archive", Journal of Money, Credit and Banking, Vol. 38, pp. 1093-1107.
63. McCullough, B.D., McGeary, K.A. and Harrison, T.D. (2008), "Do economics journal archives promote replicable research?", Canadian Journal of Economics, Vol. 41, pp. 1406-1420.
64. Makel, M., Plucker, J. and Hegarty, B. (2012), "Replications in psychology research: how often do they really occur?", Perspectives in Psychological Science, Vol. 7, pp. 537-542.
65. Mayer, M. and Whittington, R. (2003), "Diversification in context: a cross-national and cross-temporal extension", Strategic Management Journal, Vol. 24, pp. 773-781.
66. Melander, H., Ahlqvist-Rastad, J., Meijer, G. and Beermann, B. (2003), "Evidence b(i)ased medicine - selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications", British Medical Journal, Vol. 326, pp. 1171-1173.
67. Mezias, S.J. and Regnier, M.O. (2007), "Walking the walk as well as talking the talk: replication and the normal science paradigm in strategic management research", Strategic Organization, Vol. 5, pp. 283-296.
68. Miller, D. (2007), "Paradigm prisons, or in praise of atheoretic research", Strategic Organization, Vol. 5, pp. 177-184.
69. Million, M. and Raoult, D. (2012), "Publication biases in probiotics", European Journal of Epidemiology, Vol. 27, pp. 885-886.
70. Mingers, J. and Willmott, H. (2013), "Taylorizing business school research: on the "one best way" performative effects of journal ranking lists", Human Relations, Vol. 66, pp. 1051-1073.
71. Mittelstaedt, R.A. and Zorn, T.S. (1984), "Economic replications: lessons from the experimental sciences", Quarterly Journal of Business and Economics, Vol. 23, pp. 9-15.
72. Moreno, S.G., Sutton, A.J., Ades, A.E., Cooper, N.J. and Abrams, K.R. (2011), "Adjusting for publication biases across similar interventions performed well when compared with gold standard data", Journal of Clinical Epidemiology, Vol. 64, pp. 1230-1241.
73. Moreno, S.G., Sutton, A.J., Turner, E.H., Abrams, K.R., Cooper, N.J., Palmer, T.M. and Ades, A.E. (2009), "Novel methods to deal with publication biases: secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications", British Medical Journal, Vol. 339, b2981.
74. Neuroskeptic (2012), "The nine circles of scientific hell", Perspectives on Psychological Science, Vol. 7, pp. 643-644.
75. Nuijten, M.B., van Assen, M.A.L.M., Veldkamp, C.L.S. and Wicherts, J.M. (2015), "The replication paradox: combining studies can decrease accuracy of effect size estimates", Review of General Psychology, forthcoming.
76. Nuzzo, R. (2014), "P Values, the 'gold standard' of statistical validity, are not as reliable as many scientists assume", Nature, available at: www.nature.com/news
77. Olson, M. (1965), The Logic of Collective Action, Harvard University Press, Cambridge, MA.
78. Open Science Collaboration (2015), "Estimating the reproducibility of psychological science", Science, Vol. 349, aac4716-1.
79. Parker, M. and Thomas, R. (2011), "What is a critical journal?", Organization, Vol. 18, pp. 419-427.
80. Pashler, H. and Wagenmakers, E.-J. (2012), "Editors' introduction to the special section on replicability in psychological science: a crisis of confidence?", Perspectives on Psychological Science, Vol. 7, pp. 528-530.
81. Pfeffer, J. (1993), "Barriers to the advance of organizational science: paradigm development as a dependent variable", Academy of Management Review, Vol. 18, pp. 599-620.
82. Pfeffer, J. and Fong, C.T. (2005), "Building organization theory from first principles", Organization Science, Vol. 16, pp. 372-388.
83. Popper, K. (1959), The Logic of Scientific Discovery, Routledge, Oxford.
84. Renkewitz, F., Fuchs, H.M. and Fiedler, S. (2011), "Is there evidence of publication bias in JDM research?", Judgement and Decision Making, Vol. 6, pp. 870-881.
85. Rosenthal, R. (1979), "An introduction to the file drawer problem", Psychological Bulletin, Vol. 86, pp. 638-641.
86. Sarasvathy, S. (2003), "Entrepreneurship as a science of the artificial", Journal of Economic Psychology, Vol. 24, pp. 203-220.
87. Schooler, J. (2011), "Unpublished results hide the decline effect", Nature, Vol. 470 p. 437.
88. Schroter, S., Black, N., Evans, S., Godlee, F., Osorio, L. and Smith, R. (2008), "What errors do peer reviewers detect, and does training improve their ability to detect them?", Journal of the Royal Society of Medicine, Vol. 101, pp. 507-514.
89. Schultz, M. and Hatch, M.J. (1996), "Living with multiple paradigms: the case of paradigm interplay in organizational culture studies", Academy of Management Review, Vol. 21, pp. 529-557.
90. Shapiro, D.L., Kirkman, R.L. and Courtney, H.G. (2007), "Perceived causes and solution of the translation problem in management research", Academy of Management Journal, Vol. 50, pp. 249-266.
91. Shleifer, A. (2012), "Psychologists at the gate: a review of Daniel Kahneman's thinking, fast and slow", Journal of Economic Literature, Vol. 50, pp. 1080-1091.
92. Simmons, J.P., Nelson, L.D. and Simonsohn, U. (2011), "False-positive psychology undisclosed flexibility in data collection and analysis allows presenting anything as significant", Psychological Science, Vol. 22, pp. 1359-1366.
93. Singh, K., Ang, S.H. and Leong, S.M. (2003), "Increasing replication for knowledge accumulation in strategy research", Journal of Management, Vol. 29, pp. 533-549.
94. Sorge, A.M. and Rothe, K. (2011), "Resource dependence and construction, and macro and micro politics in transnational enterprises and alliances: the case of jet engine manufacturers in Germany", in Geppert, M. and Dörrenbächer, C. (Eds), Politics and Power in the Multinational Corporation: The Role of Institutions, Interests and Identities, Cambridge University Press, Cambridge, pp. 41-71.
95. Stanley, T.D. (2001), "Wheat from chaff: meta-analysis as quantitative literature review", Journal of Economic Perspectives, Vol. 15, pp. 131-150.
96. Stanley, T.D. (2008), "Meta-regression methods for detecting and estimating empirical effects in the presence of publication selection", Oxford Bulletin of Economics and Statistics, Vol. 70, pp. 103-127.
97. Starbuck, W.H. (2003), "Turning lemons into lemonade: where is the value in peer reviews?", Journal of Management Inquiry, Vol. 12, pp. 344-351.
98. Starbuck, W.H. (2004), "Why I stopped trying to understand the real world", Organization Studies, Vol. 25, pp. 1233-1254.
99. Stengers, I. (2000), The Invention of Modern Science, Vol. 19, University of Minnesota Press, Minneapolis, MI.
100. Sterling, T.D. (1959), "Publication decisions and their possible effects on inferences drawn from tests of significance", Journal of the American Statistical Association, Vol. 54, pp. 30-34.
101. Sterne, J.A.C., Egger, M. and Smith, G.D. (2001), "Systematic reviews in health care: investigating and dealing with publication and other biases in meta-analysis", British Medical Journal, Vol. 323, pp. 101-105.
102. Sutton, R.I. and Staw, B.M. (1995), "What theory is not", Administrative Science Quarterly, Vol. 40, pp. 371-384.
103. The Economist (2013), "Trouble at the lab", The Economist, available at: www.economist.com/ news/briefing/21588057-scientists-think-science-self-correcting-alarming-degree-it-nottrouble (accessed July 30, 2015).
104. Tsang, E.W.K. and Kwan, K.M. (1999), "Replication and theory development in organizational science: a critical realist perspective", Academy of Management Review, Vol. 24, pp. 759-780.
105. Tsui, A.S. (2013), "On compassion in scholarship: why should we care?", Academy of Management Review, Vol. 38, pp. 167-181.
106. Tsui, A.S. (2015), "Reconnecting with the business world: socially responsible scholarship", EFMD Global Focus, Vol. 9 No. 1, pp. 36-39.
107. Tsui, A.S. (2016), "Reflections on the so-called value-free ideal: values and science in the business schools", Cross-Cultural and Strategic Management ( formerly known as Cross Cultural Management), Vol. 23, pp. 4-28.
108. Tsui, A.S. and Hollenbeck, J.R. (2009), "Successful authors and effective reviewers balancing supply and demand in the organizational sciences", Organizational Research Methods, Vol. 12, pp. 259-275.
109. Van de Ven, A.H., Ang, S., Arino, A., Bamberger, P., LeBaron, C., Miller, C. and Milliken, F. (2015), "Welcome to the Academy of Management Discoveries (AMD)", Academy of Management Discoveries, Vol. 1, pp. 1-4.
110. Van Maanen, J. (1989), "Some notes on the importance of writing in organization studies", Harvard Business School Research Colloquium, Harvard Business School, Cambridge, MA, pp. 27-33.
111. van Witteloostuijn, A. (2015), "Toward experimental international business: unraveling fundamental causal linkages", Cross-Cultural and Strategic Management (formerly known as Cross Cultural Management), Vol. 22, pp. 530-544.
112. Walsh, J.P., Weber, K. and Margolis, J.D. (2003), "Social issues and management: our lost cause found", Journal of Management, Vol. 29, pp. 859-881.
113. Wicherts, J.M., Bakker, M. and Molenaar, D. (2011), "Willingness to share research data is related to the strength of the evidence and the quality of the reporting of statistical results", PLoS ONE, Vol. 6, e26828.
114. Wigboldus, D.H.J. and Dotsch, R. (2015), "Encouraging playing with data and discourage questionable reporting practices", Psychometrika, forthcoming.
115. Yong, E. (2013), "Psychologists strike a blow for reproducibility: thirty-six labs collaborate to check 13 earlier findings", Nature News, available at: www.nature.com/news/
116. Zimmerman, C. (2015), "On the need for a replication journal", Working Paper No. 2015-016A, Federal Reserve Bank of St. Louis, St. Louis, MO.