Science's reproducibility and replicability crisis: International business is not immune

Journal of International Business Studies

Volumn 48, Issue 6, 2017, Pages 653-663

  Aguinis, Herman ^a   Cascio, Wayne F ^b   Ramani, Ravi S ^a  

^a GEORGE WASHINGTON UNIVERSITY   (United States)

^b UNIVERSITY OF COLORADO   (United States)

Author keywords

capitalization on chance; credibility; quantitative methods; replicability; reproducibility

Indexed keywords

EID: 85019270161     PISSN: 00472506     EISSN: 14786990     Source Type: Journal    
DOI: 10.1057/s41267-017-0081-0     Document Type: Review

References (79)

1. Aguinis, H., Culpepper, S. A. 2015. An expanded decision making procedure for examining cross-level interaction effects with multilevel modeling. Organizational Research Methods, 18(2): 155-176.
2. Aguinis, H., Gottfredson, R. K., Joo, H. 2013. Best-practice recommendations for defining, identifying, and handling outliers. Organizational Research Methods, 16(2): 270-301.
3. Aguinis, H., Pierce, C. A., Bosco, F. A., Muslin, I. S. 2009. First decade of Organizational Research Methods: Trends in design, measurement, and data-analysis topics. Organizational Research Methods, 12(1): 69-112.
4. Aguinis, H., Pierce, C. A., Culpepper, S. A. 2009. Scale coarseness as a methodological artifact: Correcting correlation coefficients attenuated from using coarse scales. Organizational Research Methods, 12(4): 623-652.
5. Aguinis, H., Shapiro, D. L., Antonacopoulou, E., Cummings, T. G. 2014. Scholarly impact: A pluralist conceptualization. Academy of Management Learning and Education, 13(4): 623-639.
6. Aguinis, H., Vandenberg, R. J. 2014. An ounce of prevention is worth a pound of cure: Improving research quality before data collection. Annual Review of Organizational Psychology and Organizational Behavior, 1(1): 569-595.
7. Aguinis, H., Werner, S., Abbott, J. L., Angert, C., Park, J. H., Kohlhausen, D. 2010. Customer-centric science: Reporting significant research results with rigor, relevance, and practical impact in mind. Organizational Research Methods, 13(3): 515-539.
8. Andersson, U., Cuervo-Cazurra, A., Nielsen, B. B. 2014. From the editors: Explaining interaction effects within and across levels of analysis. Journal of International Business Studies, 45(9): 1063-1071.
9. Bakker, M., van Dijk, A., Wicherts, J. M. 2012. The rules of the game called psychological science. Perspectives on Psychological Science, 7(6): 543-554.
10. Bakker, M., Wicherts, J. M. 2011. The (mis) reporting of statistical results in psychology journals. Behavior Research Methods, 43(3): 666-678.
11. Bamberger, P., Ang, S. 2016. The quantitative discovery: What is it and how to get it published. Academy of Management Discoveries, 2(1): 1-6.
12. Banks, G. C., O'Boyle, Jr., E. H. et al. 2016. Questions about questionable research practices in the field of management: A guest commentary. Journal of Management, 42(1): 5-20.
13. Banks, G. C., Rogelberg, S. G., Woznyj, H. M., Landis, R. S., Rupp, D. E. 2016. Evidence on questionable research practices: The good, the bad, and the ugly. Journal of Business and Psychology, 31(3): 323-338.
14. Becker, T. E. 2005. Potential problems in the statistical control of variables in organizational research: A qualitative analysis with recommendations. Organizational Research Methods, 8(3): 274-289.
15. Bedeian A. G., Taylor, S. G., Miller, A. N. 2010. Management science on the credibility bubble: Cardinal sins and various misdemeanors. Academy of Management Learning and Education, 9(4): 715-725.
16. Bentler, P. M. 2007. On tests and indices for evaluating structural models. Personality and Individual Differences, 42(5): 825-829.
17. Bergh, D. D., Sharp, B., Li, M. 2017. Tests for identifying "red flags" in empirical findings: Demonstration and recommendations for authors, reviewers and editors. Academy of Management Learning & Education, 16(1): 110-124.
18. Bernerth, J., Aguinis, H. 2016. A critical review and best-practice recommendations for control variable usage. Personnel Psychology, 69(1): 229-283.
19. Bettis, R. A., Ethiraj, S., Gambardella, A., Helfat, C., Mitchell, W. 2016. Creating repeatable cumulative knowledge in strategic management. Strategic Management Journal, 37(2): 257-261.
20. Bobko, P. 2001. Correlation and regression (2nd edn). Thousand Oaks, CA: Sage.
21. Boellis, A., Mariotti, S., Minichilli, A., Piscitello, L. 2016. Family involvement and firms' establishment mode choice in foreign markets. Journal of International Business Studies, 47(8): 929-950.
22. Bosco, F. A., Aguinis, H., Field, J. G., Pierce, C. A., Dalton, D. R. 2016. HARKing's threat to organizational research: Evidence from primary and meta-analytic sources. Personnel Psychology, 69(3): 709-750.
23. Breaugh, J. A. 2008. Important considerations in using statistical procedures to control for nuisance variables in non-experimental studies. Human Resource Management Review, 18(4): 282-293.
24. Butler, N., Delaney, H., Spoelstra, S. 2017. The grey zone: Questionable research practices in the business school. Academy of Management Learning & Education, 16(1): 94-109.
25. Carlson, K. D., Wu, J. 2012. The illusion of statistical control: Control variable practice in management research. Organizational Research Methods, 15(3): 413-435.
26. Cascio, W. F. 2012. Methodological issues in international HR management research. International Journal of Human Resource Management, 23(12): 2532-2545.
27. Cascio, W. F., Aguinis, H. 2005. Test development and use: New twists on old questions. Human Resource Management, 44(3): 219-235.
28. Cascio, W. F., Zedeck, S. 1983. Open a new window in rational research planning: Adjust alpha to maximize statistical power. Personnel Psychology, 36(3), 517-526.
29. Chang, S. J., van Wittleloostuijn, A., Eden, L. 2010. From the editors: Common method variance in international business research. Journal of International Business Studies, 41(2): 178-184.
30. Chen, E. E., Wojcik, S. P. 2016. A practical guide to big data research in psychology. Psychological Methods, 21(4): 458-474.
31. Cortina, J. M. 2002. Big things have small beginnings: An assortment of "minor" methodological misunderstandings. Journal of Management, 28(3): 339-362.
32. Cortina, J. M., Aguinis, H., DeShon, R. P. 2017. Twilight of dawn or of evening? A century of research methods in the Journal of Applied Psychology. Journal of Applied Psychology, 102(3): 274-290.
33. Cortina, J. M., Green, J. P., Keeler, K. R., Vandenberg, R. J. 2016. Degrees of freedom in SEM: Are we testing the models that we claim to test? Organizational Research Methods. doi: 10.1177/1094428116676345.
34. Credé, M., Harms, P. D. 2015. 25 years of higher-order confirmatory factor analysis in the organizational sciences: A critical review and development of reporting recommendations. Journal of Organizational Behavior, 36(6): 845-872.
35. Cuervo-Cazurra, A., Andersson, U., Brannen, M.Y., Nielsen, B., Reuber, A. R. 2016. From the editors: Can I trust your findings? Ruling out alternative explanations in international business research. Journal of International Business Studies, 47(8): 881-997.
36. Dalton, D. R., Aguinis, H., Dalton, C. A., Bosco, F. A., Pierce, C. A. 2012. Revisiting the file drawer problem in meta-analysis: An empirical assessment of published and non-published correlation matrices. Personnel Psychology, 65(2): 221-249.
37. Davis, G. F. 2015. What is organizational research for? Administrative Science Quarterly, 60(2): 179-188.
38. Edwards, J. R., Berry, J. W. 2010. The presence of something or the absence of nothing: Increasing theoretical precision in management research. Organizational Research Methods, 13(4): 668-689.
39. Eysenck, H. J. 1978. An exercise in mega-silliness. American Psychologist, 33(5), 517.
40. Fanelli, D. 2009. How many scientists fabricate and falsify research? A systematic review and meta-analysis of survey data. PLoS ONE, 4, e5738. doi: 10.1371/journal.pone.0005738.
41. Fischer, R., Mansell, A. 2009. Commitment across cultures: A meta-analytical approach. Journal of International Business Studies, 40(8): 1339-1358.
42. Fisher, G., Aguinis, H. 2017. Using theory elaboration to make theoretical advancements. Organizational Research Methods. doi: 10.1177/1094428116689707.
43. Fitzsimmons, S., Liao, Y., Thomas, D. 2017. From crossing cultures to straddling them: An empirical examination of outcomes for multicultural employees. Journal of International Business Studies, 48(1): 63-89.
44. Friedman, D., Sunder, S. 1994. Experimental methods: A primer for economists. New York, NY: Cambridge University Press.
45. Freese, J. 2007. Replication standards for quantitative social science: Why not sociology. Sociological Methods & Research, 36(2): 153-172.
46. Fung, S. K., Zhou, G., Zhu, X. J. 2016. Monitor objectivity with important clients: Evidence from auditor opinions around the world. Journal of International Business Studies, 47(3): 263-294.
47. Funk, C. A., Arthurs, J. D., Trevinõ, L. J., Joireman, J. 2010. Consumer animosity in the global value chain: The effect of international production shifts on willingness to purchase hybrid products. Journal of International Business Studies, 41(4): 639-651.
48. George, G. 2014. Rethinking management scholarship. Academy of Management Journal, 57(1): 1-6.
49. Goldfarb, B., King, A. A. 2016. Scientific apophenia in strategic management research: Significance tests & mistaken inference. Strategic Management Journal, 37(1): 167-176.
50. Harlow, L. L., Oswald, F. L. 2016. Big data in psychology: Introduction to the special issue. Psychological Methods, 21(4): 447-457.
51. Hollenbeck, J. H., Wright, P. M. 2016. Harking, sharking, and tharking: Making the case for post hoc analysis of scientific data. Journal of Management, 43(1): 5-18.
52. Hunter, J. E., Schmidt, F. L. 2015. Methods of meta-analysis: Correcting error and bias in research findings (3rd edn). Thousand Oaks, CA: Sage.
53. Hurley, A. E. et al. 1997. Exploratory and confirmatory factor analysis: Guidelines, issues, and alternatives. Journal of Organizational Behavior, 18(6): 667-683.
54. Ioannidis, J. P. A. 2005. Why most published research findings are false. PLoS Med, 2(8): E124. doi: 10.1371/journal.pmed.0020124.
55. John, L. K, Loewenstein, G., Prelec, D. 2012. Measuring the prevalence of questionable research practices with incentives for truth-telling. Psychological Science, 23(5), 524-532.
56. Kepes, S., McDaniel, M. A. 2013. How trustworthy is the scientific literature in industrial and organizational psychology? Industrial and Organizational Psychology, 6(3): 252-268.
57. Kerr, N. L. 1998. HARKing: Hypothesizing after the results are known. Personality and Social Psychology Review, 2(3): 196-217.
58. Lisak, A., Erez, M., Sui, Y., Lee, C. 2016. The positive role of global leaders in enhancing multicultural team innovation. Journal of International Business Studies, 47(6): 655-673.
59. Macaskill, P., Walter, S., Irwig, L. 2001. A comparison of methods to detect publication bias in meta-analysis. Statistics in Medicine, 20(4), 641-654.
60. MacCallum, R. C., Roznowski, M., Necowitz, L. B. 1992. Model modification in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin, 111(3): 490-504.
61. Meyer, K. E., van Witteloostuijn, A., Beugelsdijk, S. 2017. What's in a p? Reassessing best practices for conducting and reporting hypothesis-testing research. Journal of International Business Studies. doi: 10.1057/s41267-017-0078-8.
62. Nosek, B. A., Spies, J. R., Motyl, M. 2012. Scientific utopia II: Restructuring incentives and practices to promote truth over publishability. Perspectives on Psychological Science, 7(6): 615-631.
63. Nuijten, M. B., Hartgerink, C. H., Assen, M. A., Epskamp, S., Wicherts, J. M. 2015. The prevalence of statistical reporting errors in psychology (1985-2013). Behavior Research Methods, 48(4): 1-22.
64. O'Boyle, E. H., Banks, G. C., Gonzalez-Mulé, E. 2017. The chrysalis effect: How ugly initial results metamorphosize into beautiful articles. Journal of Management, 43(2): 376-399.
65. Open Science Collaboration. 2015. Estimating the reproducibility of psychological science. Science, 349(6251): Aac4716. doi: 10.1126/science.aac4716.
66. Orr, J. M., Sackett, P. R., DuBois, C. L. Z. 1991. Outlier detection and treatment in I/O psychology: A survey of researcher beliefs and an empirical illustration. Personnel Psychology, 44(3): 473-486.
67. Reeb, D., Sakakibara, M., Mahmood, I. P. 2012. Endogeneity in international business research. Journal of International Business Studies, 43(3): 211-218.
68. Richard, P. J., Devinney, T. M., Yip, G. S., Johnson, G. 2009. Measuring organizational performance: Towards methodological best practice. Journal of Management, 35(3): 718-804.
69. Rousseeuw, P. J., Leroy, A. M. 2003. Robust regression and outlier detection. Hoboken, NJ: Wiley.
70. Schwab, A., Starbuck, W. H. 2017. A call for openness in research reporting: How to turn covert practices into helpful tools. Academy of Management Learning & Education, 16(1): 125-141.
71. Simmons, J. P., Nelson, L. D., Simonsohn, U. 2011. False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11): 1359-1366.
72. Sijtsma, K. 2016. Playing with data-Or how to discourage questionable research practices and stimulate researchers to do things right. Psychometrika, 81(1): 1-15.
73. Spector, P. E., Brannick, M. T. 2011. Methodological urban legends: The misuse of statistical control variables. Organizational Research Methods, 14(2): 287-305.
74. Stahl, G. K., Maznevski, M. L., Voigt, A., Jonsen, K. 2010. Unraveling the effects of cultural diversity in teams: A meta-analysis of research on multicultural work groups. Journal of International Business Studies, 41(4): 690-709.
75. Starbuck, W. H. 2016. 60th anniversary essay: How journals could improve research practices in social science. Administrative Science Quarterly, 61(2): 165-183.
76. van Essen, M., Heugens, P. P., Otten, J., van Oosterhout, J. 2012. An institution-based view of executive compensation: A multilevel meta-analytic test. Journal of International Business Studies, 43(4): 396-423.
77. Verbeke, A., Von Glinow, M. Y., Luo, Y. 2017. Becoming a great reviewer: Four actionable guidelines. Journal of International Business Studies, 48(1): 1-9.
78. Waldman, I. D., Lilienfeld, S. O. 2016. Thinking about data, research methods, and statistical analyses: Commentary on Sijtsma's (2014) "Playing with Data". Psychometrika, 81(1): 16-26.
79. Wright, P. M. 2016. Ensuring research integrity: An editor's perspective. Journal of Management, 42(5): 1037-1043.