Mend It or End It: Redirecting the Search for Interactions in the Organizational Sciences

Organizational Research Methods

Volumn 20, Issue 4, 2017, Pages 549-573

  Murphy, Kevin R ^a   Russell, Craig J ^b  

^a Veterinary Dentistry and Oral Surgery   (United States)

^b UNIVERSITY OF OKLAHOMA   (United States)

Author keywords

ANOVA methods; interactions; linear techniques including categorical dependent variables; measurement models; quantitative; quantitative research

Indexed keywords

EID: 85029516956     PISSN: 10944281     EISSN: 15527425     Source Type: Journal    
DOI: 10.1177/1094428115625322     Document Type: Article

References (106)

1. Aguinis H., (1995). Statistical power problems with moderated multiple regression in management research. Journal of Management, 21, 1141–1158.
2. Aguinis H., (2002). Estimation of interaction effects in organizational studies. Organizational Research Methods, 5, 207–211.
3. Aguinis H., Beaty J. C., Boik R. J., Pierce C. A., (2005). Effect size and power in assessing moderating effects of categorical variables using multiple regression: A 30-year review. Journal of Applied Psychology, 90, 94–107.
4. Aguinis H., Boik R. J., Pierce C. A., (2001). A generalized solution for approximating the power to detect effects of categorical moderator variables using multiple regression. Organizational Research Methods, 4, 291–323.
5. Aguinis H., Gottfredson R. K., (2010). Best-practice recommendations for estimating Interaction effects using moderated multiple regression. Journal of Organizational Behavior, 31, 776–786.
6. Aguinis H., Gottfredson R. K., Culpepper S. A., (2013). Best-practice recommendations for estimating cross-level interaction effects using multilevel modeling. Journal of Management, 39, 1490–1528.
7. Aguinis H., Gottfredson R. K., Wright T. A., (2011). Best-practice recommendations for estimating interaction effects using meta-analysis. Journal of Organizational Behavior, 32, 1033–1043.
8. Aguinis H., O’Boyle E., (2014). Star performers in twenty-first-century organizations. Personnel Psychology, 67, 313–350.
9. Aguinis H., Pierce C. A., (1998). Testing moderator variable hypotheses meta-analytically. Journal of Management, 24, 577–592.
10. Aguinis H., Stone-Romero E. F., (1997). Methodological artifacts in moderated multiple regression and their effects on statistical power. Journal of Applied Psychology, 82, 192–206.
11. Alf E. F., Abrahams N. M., (1974). Let’s give the devil his due: A response to Birnbaum. Psychological Bulletin, 81, 72–73.
12. Ambrose M. L., Kulik C. T., (1999). Old friends, new faces: Motivation research in the 1990s. Journal of Management, 25, 231–292.
13. Anderson N. H., (1970). Functional measurement and psychophysical judgment. Psychological Review, 77, 153–170.
14. Anderson N. H., (1976). How functional measurement can yield validated interval scales of mental quantities. Journal of Applied Psychology, 61, 677–692.
15. Anderson N. H., Shanteau J. C., (1977). Weak inference with linear models. Psychological Bulletin, 84, 1155–1170.
16. Andersson U., Cuervo-Cazurra A., Nielsen B. B., (2014). From the editors: Explaining interaction effects within and across levels of analysis. Journal of International Business Studies, 45, 1063–1071.
17. Arnold H. J., (1981). A test of the validity of the multiplicative hypothesis of expectancy-valence theories of work motivation. Academy of Management Journal, 24, 128–141.
18. Arnold H. J., House R. J., (1980). Methodological and substantive extensions to the job characteristics model of motivation. Organizational Behavior and Human Performance, 25, 161–183.
19. Bacharach S. B., (1989). Organizational theories: Some criteria for evaluation, Academy of Management Review, 14, 496–515.
20. Barrick M. R., Mount M. K., (1991). The big five personality dimensions and job performance: A meta-analysis. Personnel Psychology, 44, 1–26.
21. Baum J. R., Locke E. A., Smith K. G., (2001). A multidimensional model of venture growth. Academy of Management Journal, 44, 292–303.
22. Beck J. W., Beatty A. S., Sackett P. R., (2014). On the distribution of job performance: The role of measurement characteristics in observed departures from normality. Personnel Psychology, 67, 531–566.
23. Birnbaum M. H., (1973). The devil rides again: Correlation as an index of fit. Psychological Bulletin, 79, 239–242.
24. Birnbaum M. H., (1974). Reply to the devil’s advocates: Don’t confound model testing and measurement. Psychological Bulletin, 81, 854–855.
25. Blettner D. P., Chaddad F. R., Bettis R. A., (2012). Research notes and commentaries the CEO performance effect: Statistical issues and a complex fit perspective, Strategic Management Journal, 33, 986–999.
26. Bobko P., (1986). A solution to some dilemmas when testing hypotheses about ordinal interactions. Journal of Applied Psychology, 71, 323–326.
27. Bobko P., Russell C. J., (1994). On theory, statistics, and the search for interactions in the organizational sciences. Journal of Management, 20, 193–200.
28. Bosco F. A., Aguinis H., Singh K., Field J. G., Pierce C. A., (2015). Correlational effect size benchmarks. Journal of Applied Psychology, 100, 431–449.
29. Boyd B. K., Gove S., Hitt M. A., (2005a). Consequences of measurement problems in strategic management research: The case of Amihud and Lev. Strategic Management Journal, 26, 367–375.
30. Boyd B. K., Gove S., Hitt M. A., (2005b). Construct measurement in strategic management research: Illusion or reality? Strategic Management Journal, 26, 239–257.
31. Boyd B. K., Reuning-Elliott E., (1998). A measurement model of strategic planning. Strategic Management Journal, 19, 181–192.
32. Brahma S. S., (2009). Assessment of construct validity in management research: A structured guideline. Journal of Management Research, 9, 59–71.
33. Brogden H. E., (1949). When testing pays off. Personnel Psychology, 2, 171–183.
34. Busemeyer J. R., Jones L. E., (1983). Analysis of multiplicative combination rules When the causal variables are measured with error. Psychological Bulletin, 93, 549–562.
35. Carte T. A., Russell C. J., (2003). In pursuit of moderation: Nine common errors and their solution. MIS Quarterly, 27, 1–22.
36. Cleary T. A., (1966). An individual differences model for multiple regression. Psychometrika, 31, 215–224.
37. Cleary T. A., (1968). Test bias: Prediction of grades of Negro and White students in integrated colleges. Journal of Educational Measurement, 5, 115–124.
38. Cohen J., (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
39. Cortina J. M., (1993). Interaction, nonlinearity, and multicolinearity: Implications for multiple regression. Journal of Management, 19, 915–922.
40. Cortina J. M., DeShon R. P., (1998). Determining relative importance of predictors with the observational design. Journal of Applied Psychology, 83, 798–804.
41. Cortina J. M., Landis R. S., (2009). When small effect sizes tell a big story, and when large effect sizes don’t. In Lance C. E., Vandenberg R. J., (Eds.), Statistical and methodological myths and urban legends: Doctrine, verity, and fable in the organizational and social sciences (pp. 289–310). New York, NY: Routledge.
42. Cronbach L. J., (1987). Statistical tests for moderator variables: Flaws in analyses recently proposed. Psychological Bulletin, 102, 414–417.
43. Cronbach L. J., Snow R., (1977). Aptitudes and instructional methods: A handbook for research on interactions. New York, NY: Irvington.
44. Dalton D. R., Aguinis H., (2013). Measurement malaise in strategic management studies: The case of corporate governance research. Organizational Research Methods, 16, 88–99.
45. Dawes R. M., (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34, 571–582.
46. Dawes R. M., Corrigan B., (1974). Linear models in decision making. American Psychologist, 81, 95–106.
47. Dubin R., (1978). Theory building (Rev. ed.). New York, NY: Free Press.
48. Dweck C. S., (1999). Self-theories: Their role in motivation, personality, and development. Philadelphia, PA: Psychology Press.
49. Eden D., (2002). Replication, meta-analysis, scientific progress and AMJ’s publication policy. Academy of Management Journal, 45, 841–846.
50. Eden D., (2004). Reflections on the AMJ associate editor role. Academy of Management Journal, 47, 167–173.
51. Edwards J. R., (2009). Seven deadly myths of testing moderation in organizational research. In Lance C. E., Vandenberg R. J., (Eds.), Statistical and methodological myths and urban legends: Doctrine, verity, and fable in the organizational and social sciences (pp. 143–164). New York, NY: Routledge.
52. Edwards J. R., (2010). Reconsidering theoretical progress in organizational and management research. Organizational Research Methods, 13, 615–619.
53. Edwards J. R., Scully J. A., Bartek M. D., (2000). The nature and outcomes of work: A replication and extension of interdisciplinary work design research. Journal of Applied Psychology, 85, 860–868.
54. Efron B., Tibshirani R., (1993). An introduction to the bootstrap. London, UK: Chapman & Hall.
55. Efron B., Tibshirani R., (1997). Improvements on cross-validation: The.632 plus bootstrap method. Journal of the American Statistical Association, 92, 548–560.
56. Evans M. G., Kiggundu M. N., House R. J., (1979). A partial test and extension of the job characteristics model of motivation. Organizational Behavior and Human Performance, 24, 354–381.
57. Foreman P., Murphy G., (1996). Work values and expectancies in occupational rehabilitation: The role of cognitive variables in the return-to-work process. Journal of Rehabilitation, 62, 44–49.
58. Fraas J. W., Newman I., (1997). The use of the Johnson–Neyman confidence bands and multiple regression models to investigate interaction effects: Important tools for educational researchers and program evaluators. Multiple Linear Regression Viewpoints, 24, 14–24.
59. Grant A. M., Wall T. D., (2009). The neglected science and art of quasi-experimentation: Why-to, when-to, and how-to article for organizational researchers. Organizational Research Methods, 12, 653–686.
60. Hackman J. R., Lawler E. E., (1971). Employee reactions to job characteristics. Journal of Applied Psychology Monograph, 55, 259–286.
61. Hall J. A., Rosenthal R., (1991). Testing for moderator variables in meta-analysis: Issues and methods. Communication Monographs, 58, 437–448.
62. Hambrick D. C., (2007). The field of management’s devotion to theory: Too much of a good thing? Academy of Management Journal, 50, 1346–1352.
63. Hoffman P. J., (1960). The paramorphic representation of clinical judgment. Psychological Bulletin, 57, 116–131.
64. Huxley T. H., (1901). Memoirs of Thomas Henry Huxley. London, UK: Macmillan.
65. Johnson P. O., Fay L. C., (1950). The Johnson–Neyman technique: Its theory and application. Psychometrika, 15, 349–367.
66. Kacmar K. M., Whitfield J. M., (2000). An additional rating method for journal articles in the field of management. Organizational Research Methods, 3, 392–406.
67. Kanfer R., Ackerman P. L., (1989). Motivation and cognitive abilities: An integrative/aptitude-treatment interaction approach to skill acquisition. Journal of Applied Psychology, 74, 657–690.
68. Ketchen D. J., Ireland R. D., Baker L. T., (2013). The use of archival proxies in strategic management studies: Castles made of sand? Organizational Research Methods, 16, 32–42.
69. Landy F. J., (2006). The long, frustrating and fruitless search for social intelligence. In Murphy K., (Ed.), A critique of emotional intelligence: What are the problems and how can they be fixed? (pp. 81–124). Mahwah, NJ: Lawrence Erlbaum.
70. Latham G. P., Kinne S. B., (1974). Improving job performance through training in goal setting. Journal of Applied Psychology, 59, 187–191.
71. Latham G. P., Pinder C. C., (2005). Work motivation theory and research at the dawn of the twenty-first century. Annual Review of Psychology, 56, 485–516.
72. Mathieu J. E., Aguinis H., Culpepper S. A., Chen G., (2012). Understanding and estimating the power to detect cross-level interaction effects in multilevel modeling. Journal of Applied Psychology, 97, 951–966.
73. McClelland G. H., Judd C. M., (1993). Statistical difficulties of detecting interactions and moderator effects. Psychological Bulletin, 114, 376–390.
74. McKinley W., (2010). Organizational theory development: Displacement of ends? Organization Studies, 31, 47–68.
75. Mitchell T. R., (1974). Expectancy models of job satisfaction, occupational preference and effort: A theoretical, methodological, and empirical appraisal. Psychological Bulletin, 81, 1053–1077.
76. Mitchell T. R., James L. R., (2001). Building better theory: Time and the specification of when things happen. Academy of Management Review, 26, 530–547.
77. Murphy K., Myors B., Wolach A., (2014). Statistical power analysis: A simple and general model for traditional and modern hypothesis tests (4th ed.). New York, NY: Taylor & Francis.
78. Naylor J. C., Pritchard R. D., Ilgen D. R., (1980). A theory of behavior in organizations. New York, NY: Academic Press.
79. O’Boyle E., Aguinis H., (2012). The best and the rest: Revisiting the norm of normality of individual performance. Personnel Psychology, 65, 79–119.
80. Pierce J. R., Aguinis H., (2013). The too-much-of-a-good-thing effect in management. Journal of Management, 39, 313–338.
81. Raju N. S., Burke M. J., Normand J., Langlois G. M., (1991). A new meta-analytic approach. Journal of Applied Psychology, 76, 432–446.
82. Rodgers J. L., (2010). The epistemology of mathematical and statistical modeling: A quiet methodological revolution. American Psychologist, 65, 1–12.
83. Rogers W. M., (2002). Theoretical and mathematical constraints of interactive regression models. Organizational Research Methods, 5, 221–230.
84. Russell C. J., (1985). Individual decision processes in an assessment center. Journal of Applied Psychology, 70, 737–746.
85. Russell C. J., (2009). Establishing the usefulness of strategic management research: On inverted Lewinians and naked strategy scholars. In Ketchen D., Bergh D., (Eds.), Research methodology in strategy and management (Vol. 5, pp. 55–74). Bingley, UK: JAI.
86. Russell C. J., (2011). Evidence-based human resources management. San Diego, CA: Cognella/University Readers.
87. Russell C. J., (2013). Is it time to voluntarily turnover theories of voluntary turnover? Industrial and Organizational Psychology: Perspectives on Science and Practice, 6, 156–173.
88. Russell C. J., Bobko P., (1992). Moderated regression and Likert scales: Too coarse for comfort. Journal of Applied Psychology, 77, 336–342.
89. Russell C. J., Dean M. A., (2000). To log or not to log: Bootstrap as an alternative to parametric estimation of moderation effects in the presence of skewed dependent variables. Organizational Research Methods, 3, 167–185.
90. Russell C. J., Pinto J. K., Bobko P., (1991). Appropriate moderated regression and inappropriate research strategy: A demonstration of information loss due to scale coarseness. Applied Psychological Measurement, 15, 257–266.
91. Sackett P. R., Harris M. M., Orr J. M., (1986). On seeking moderators in meta-analysis of correlational data: A Monte-Carlo investigation of statistical power and resistance to Type I error. Journal of Applied Psychology, 71, 302–310.
92. Schepanski A., (1983). The predictive ability criterion in experimental judgment research in accounting. Decision Sciences, 14, 503–512.
93. Schmidt F. L., (1973). Implications of a measurement problem for expectancy theory research. Organizational Behavior and Human Performance, 10, 243–251.
94. Schmidt F. L., Hunter J. E., (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124, 262–274.
95. Shanteau J., (1977). Correlation as a deceiving measure of fit. Bulletin of the Psychonomic Society, 10, 134–136.
96. Shapiro K. V., (1975). An overview of problems encountered in aptitude-treatment interaction research. AV Communication Review, 23, 227–241.
97. Shieh G., (2009). Detecting interaction effects in moderated multiple regression with continuous variables: Power and sample size considerations. Organizational Research Methods, 12, 510–528.
98. Stone-Romero E. F., Alliger G. F., Aguinis H. F., (1994). Type II error problems in the use of moderated multiple regression procedures for the detection of moderating effects of dichotomous variables. Journal of Management, 20, 167–178.
99. Swanson H. L., (2013). Linking children and adolescent assessment to effective instruction: An evidence-based perspective from the experimental literature. In Saklofske D., Reynolds C., Schwean V., (Eds.), The Oxford handbook of child psychological assessment (pp. 822–839). New York, NY: Oxford University Press.
100. Taguchi G., Clausing D., Tung L. W., (1987). System of experimental design: Engineering methods to optimize quality and minimize costs. White Plains, NY: UNIPUB/Kraus International.
101. Tett R. P., Burnett D. D., (2003). A personality trait–based interactionist model of job performance. Journal of Applied Psychology, 88, 500–517.
102. Equal Employment Opportunity Commission. (1978). Uniform guidelines on employment selection procedures. Federal Register, 43(166), 38296–38309.
103. Vroom V. H., (1964). Work and motivation. New York, NY: John Wiley.
104. Weick K. E., (1989). Theory construction as disciplined imagination. Academy of Management Review, 14, 516–531.
105. Weick K. E., (1996). What theory is not, theorizing is. Administrative Science Quarterly, 40, 385–390.
106. Yntema D. B., Torgerson W. S., (1961). Man-machine cooperation in decisions requiring common sense. IEE Transactions on Human Factors in Electronics, HFE-2, 20–26.