Effect size and power in assessing moderating effects of categorical variables using multiple regression: A 30-year review

Journal of Applied Psychology

Volumn 90, Issue 1, 2005, Pages 94-107

  Aguinis, Herman ^a   Beaty, James C ^b,d   Boik, Robert J ^c   Pierce, Charles A ^c,e  

^a UNIVERSITY OF COLORADO   (United States)

^b ePredix ^*

^c MONTANA STATE UNIVERSITY   (United States)

^d ePredix   (United States)

^e UNIVERSITY OF MEMPHIS   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

HUMAN; PSYCHOLOGY; REGRESSION ANALYSIS; REPRODUCIBILITY; REVIEW; SAMPLE SIZE; STATISTICS;

HUMANS; PSYCHOLOGY; REGRESSION ANALYSIS; REPRODUCIBILITY OF RESULTS; SAMPLE SIZE;

EID: 12944283033     PISSN: 00219010     EISSN: None     Source Type: Journal    
DOI: 10.1037/0021-9010.90.1.94     Document Type: Review

References (64)

1. Aguinis, H. (1995). Statistical power problems with moderated multiple regression in management research, Journal of Management, 21, 1141-1158.
2. Aguinis, H. (2004). Regression analysis for categorical moderators. New York: Guilford Press.
3. Aguinis, H., Boik, R. J., & Pierce, C. A. (2001). A generalized solution for approximating the power to detect effects of categorical moderator variables using multiple regression. Organizational Research Methods, 4, 291-323.
4. Aguinis, H., Petersen, S. A., & Pierce, C. A. (1999). Appraisal of the homogeneity of error variance assumption and alternatives to multiple regression for estimating moderating effects of categorical variables. Organizational Research Methods, 2, 315-339.
5. Aguinis, H., & Pierce, C. A. (1998a). Heterogeneity of error variance and the assessment of moderating effects of categorical variables: A conceptual review, Organizational Research Methods, 1, 296-314.
6. Aguinis, H., & Pierce, C. A. (1998b). Statistical power computations for detecting dichotomous moderator variables with moderated multiple regression. Educational and Psychological Measurement, 58, 668-676.
7. Aguinis, H., Pierce, C. A., & Stone-Romero, E. F. (1994). Estimating the power to detect dichotomous moderators with moderated multiple regression. Educational and Psychological Measurement, 54, 690-692.
8. Aguinis, H., & Stone-Romero, E. F. (1997). Methodological artifacts in moderated multiple regression and their effects on statistical power. Journal of Applied Psychology, 82, 192-206.
9. Aiken, L. S., & West, S. G. (1991). Multiple regression: Testing and interpreting interactions. Newbury Park, CA: Sage.
10. Aiken, L. S., West, S. G., Sechrest, L., Reno, R., Roediger, H. L., Scarr, S., et al. (1990). Graduate training in statistics, methodology, and measurement in psychology: A survey of PhD programs in North America. American Psychologist, 45, 721-734.
11. Alexander, R. A., & DeShon, R. P. (1994). Effect of error variance heterogeneity on the power of tests for regression slope differences. Psychological Bulletin, 115, 308-314.
12. Battlett, C. J., Bobko, P., Mosier, S. B., & Hannan, R. (1978). Testing for fairness with a moderated multiple regression strategy: An alternative to differential analyses. Personnel Psychology, 31, 233-242.
13. Bartol, K. M., & Manhardt, P. J. (1979). Sex differences in job outcome preferences: Trends among newly hired college graduates. Journal of Applied Psychology, 64, 477-482.
14. Bloom, M., & Milkovich, G. T. (1998). Relationships among risk, incentive pay, and organizational performance. Academy of Management Journal, 41, 283-297.
15. Bobko, P., & Rieck, A. (1980). Large sample estimators for standard errors of functions of correlation coefficients. Applied Psychological Measurement, 4, 385-398.
16. Bobko, P., & Russell, C. J. (1994). On theory, statistics, and the search for interactions in the organizational sciences. Journal of Management, 20, 193-200.
17. Box, G. E. P., & Cox, D. R. (1964). An analysis of transformations (with discussion). Journal of the Royal Statistical Society Series B, 26, 211-246.
18. Breaugh, J. A. (2003). Effect size estimation: Factors to consider and mistakes to avoid. Journal of Management, 29, 79-97.
19. Brock, J. K.-U. (2003). The "power" of international business research. Journal of International Business Studies, 34, 90-99.
20. Campion, M. A., Pursell, E. D., & Brown, B. K. (1988). Structured interviewing: Raising the psychometric properties of the employment interview. Personnel Psychology, 41, 25-42.
21. Chaplin, W. F. (1997). Personality, interactive relations, and applied psychology. In R. Hogan, J. Johnson, & S. Briggs (Eds.), Handbook of personality psychology (pp. 873-890). San Diego, CA: Academic Press.
22. Cohen, J. (1962). The statistical power of abnormal-social psychological research: A review. Journal of Abnormal and Social Psychology, 65, 145-153.
23. Cohen, J. (1977). Statistical power analysis for the behavioral sciences (Revised Ed.). New York: Academic Press.
24. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
25. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Erlbaum.
26. Eden, D. (2002). Replication, meta-analysis, scientific progress, and AMJ's publication policy. Academy of Management Journal, 45, 841-846.
27. Eden, D., & Zuk, Y. (1995). Seasickness as a self-fulfilling prophecy: Raising self-efficacy to boost performance at sea. Journal of Applied Psychology, 80, 628-635.
28. Fichman, M. (1999). Variance explained: Why size does not (always) matter. Research in Organizational Behavior, 21, 295-331.
29. Fisicaro, S. A., & Lautenschlager, G. J. (1992). Power and reliability: The case of homogeneous true score regression across treatments. Educational and Psychological Measurement, 52, 505-511.
30. Ford, J. K., & Noe, R. A. (1987). Self-assessed training needs: The effects of attitudes toward training, management level, and function. Personnel Psychology, 40, 25-42.
31. Gerard, P. D., Smith, D. R., & Weerakkody, G. (1998). Limits of retrospective power analysis. Journal of Wildlife Management, 62, 801-807.
32. Goodman, S. N., & Berlin, J. A. (1994). The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Annals of Internal Medicine, 121, 200-206.
33. Hoenig, J. M., & Heisey, D. M. (2001). The abuse of power: The pervasive fallacy of power calculations for data analysis. American Statistician, 55, 19-24.
34. Ibarra, H. (1995). Race, opportunity, and diversity of social circles in managerial networks. Academy of Management Journal, 38, 673-703.
35. Khatri, C. G. (1966). A note on a MANOVA model applied to problems in growth curve. Annals of the Institute of Statistical Mathematics, 18, 75-86.
36. Latack, J. C., Josephs, S. L., Roach, B. L., & Levine, M. D. (1987). Carpenter apprentices: Comparison of career transitions for men and women. Journal of Applied Psychology, 72, 393-400.
37. Martell, R. F., Lane, D. M., & Emrich, C. (1996). Male-female differences: A computer simulation. American Psychologist, 51, 157-158.
38. Mazen, A. M., Graf, L. A., Kellogg, C. E., & Hemmasi, M. (1987). Statistical power in contemporary management research. Academy of Management Journal, 30, 369-380.
39. Mazen, A. M., Hemmasi, M., & Lewis, M. F. (1987). Assessment of statistical power in contemporary strategy research. Strategic Management Journal, 8, 403-410.
40. Mone, M. A., Mueller, G. C., & Mauland, W. (1996). The perceptions and usage of statistical power in applied psychology and management research. Personnel Psychology, 49, 103-120.
41. Olejnik, S., & Algina, J. (2000). Measures of effect size for comparative studies: Applications, interpretations, and limitations. Contemporary Educational Psychology, 25, 241-286.
42. Peters, L. H., Fisher, C. D., & O'Connor, E. J. (1982). The moderating effect of situational control of performance variance on the relationship between individual differences and performance. Personnel Psychology, 35, 609-621.
43. Raju, N. S., & Brand, P. A. (2003). Determining the significance of correlations corrected for unreliability and range restriction. Applied Psychological Measurement, 27, 52-71.
44. Russell, C. J., & Bobko, P. (1992). Moderated regression analysis and Likert scales: Too coarse for comfort. Journal of Applied Psychology, 77, 336-342.
45. Sackett, P. R., & Wilk, S. L. (1994). Within-group norming and other forms of source adjustment in preemployment testing. American Psychologist, 49, 929-954.
46. Salgado, J. F. (1998). Sample size in validity studies of personnel selection. Journal of Occupational and Organizational Psychology, 71, 161-164.
47. Schaubroeck, J., May, D. R., & Brown, F. W. (1994). Procedural justice explanations and employee reactions to economic hardship: A field experiment. Journal of Applied Psychology, 79, 455-460.
48. Schmidt, F. L. (1988). The problem of group differences in ability scores in employment selection. Journal of Vocational Behavior, 33, 272-292.
49. Schmidt, F. L. (2002). The role of general cognitive ability in job performance: Why there cannot be a debate. Human Performance, 15, 187-210.
50. Schmidt, F. L., & Hunter, J. E. (1981). Employment testing: Old theories and new research findings. American Psychologist, 36, 1128-1137.
51. Sedlmeier, P., & Gigerenzer, G. (1989). Do studies of statistical power have an effect on the power of studies? Psychological Bulletin, 105, 309-316.
52. Society for Industrial and Organizational Psychology, Inc. (1999). SIOP annual membership directory. Bowling Green, OH: Author.
53. Starbuck, B., & Mezias, J. (1996). Journal impact ratings. The Industrial-Organizational Psychologist, 33(4), 101-105.
54. Stone-Romero, E. F., Alliger, G. M., & Aguinis, H. (1994). Type II error problems in the use of moderated multiple regression for the detection of moderating effects of dichotomous variables. Journal of Management, 20, 167-178.
55. Stone-Romero, E. F., & Liakhovitski, D. (2002). Strategies for detecting moderator variables: A review of conceptual and empirical issues. Research in Personnel and Human Resources Management, 21, 333-372.
56. Tepper, B. J. (1994). Investigation of general and program specific attitudes toward corporate drug testing policies. Journal of Applied Psychology, 79, 392-401.
57. Tubbs, M. E. (1993). Commitment as a moderator of the goal-performance relation: A case for clearer construct definition. Journal of Applied Psychology, 78, 86-97.
58. Uhl-Bien, M., & Graen, G. B. (1998). Individual self-management: Analysis of professionals' self-managing activities in functional and cross-functional work teams. Academy of Management Journal, 41, 340-350.
59. Van Dyne, L., & Ang, S. (1998). Organizational citizenship behavior of contingent workers in Singapore. Academy of Management Journal, 41, 692-703.
60. Vecchio, R. P. (1980). A test of a moderator of the job satisfaction-job quality relationship: The case of religious affiliation. Journal of Applied Psychology, 65, 195-201.
61. West, S. G., Aiken, L. S., & Krull, J. L. (1996). Experimental personality designs: Analyzing categorical by continuous variable interactions. Journal of Personality, 64, 1-48.
62. Wigdor, A. K., & Garner W. R. (Eds.). (1982). Ability testing: Uses, consequences, and controversies. (Report of the National Research Council Committee on Ability Testing). Washington, DC: National Academy of Sciences Press.
63. Williams, K. J., Suis, J., Alliger, G. M., & Learner, S. M. (1991). Multiple role juggling and daily mood states in working mothers: An experience sampling study. Journal of Applied Psychology, 76, 664-674.
64. Zedeck, S. (1971). Problems with the use of "moderator" variables. Psychological Bulletin, 76, 295-310.