Current misuses of multiple regression for investigating bivariate hypotheses: An example from the organizational domain

Behavior Research Methods

Volumn 46, Issue 3, 2014, Pages 798-807

  O'Neill, Thomas A ^a   McLarnon, Matthew J W ^b   Schneider, Travis J ^b   Gardner, Robert C ^b  

^a UNIVERSITY OF CALGARY   (Canada)

^b UNIVERSITY OF WESTERN ONTARIO   (Canada)

Author keywords

Beta; Correlation; Multiple regression; Organizational science; Research methods

Indexed keywords

ALGORITHM; ARTICLE; BIOMETRY; DECISION MAKING; HUMAN; METHODOLOGY; MULTIVARIATE ANALYSIS; NONBIOLOGICAL MODEL; PSYCHOLOGY;

ALGORITHMS; BIOMETRY; DECISION MAKING; HUMANS; MODELS, ORGANIZATIONAL; MULTIVARIATE ANALYSIS; PSYCHOLOGY; RESEARCH DESIGN;

EID: 84904740158     PISSN: 1554351X     EISSN: 15543528     Source Type: Journal    
DOI: 10.3758/s13428-013-0407-1     Document Type: Article

References (67)

1. Atinc, G., Simmering, M. J., & Kroll, M. J. (2012). Control variable use and reporting in macro and micro management research. Organizational Research Methods, 15, 57-75. doi: 10. 1177/1094428110397773.
2. Bandura, A. (1997). Self-efficacy: The exercise of control. New York, NY: W. H. Freeman.
3. Bauer, T. N., Erdogan, B., Liden, R. C., & Wayne, S. J. (2006). A longitudinal study of the moderating role of extraversion: Leader-member exchange, performance, and turnover during new executive development. Journal of Applied Psychology, 91, 298-310.
4. Becker, T. E. (2005). Potential problems in the statistical control of variables in organizational research: A qualitative analysis with recommendations. Organizational Research Methods, 8, 274-289. doi: 10. 1177/1094428105278021.
5. Bernstein, J. (2004). Evidence-based medicine. American Academy of Orthopaedic Surgeons, 12, 80-88.
6. Biemann, T. (2013). What if we were Texas sharpshooters? Predictor reporting bias in regression analysis. Organizational Research Methods, 16, 335-363. doi: 10. 1177/1094428113485135.
7. Bobko, P. (1990). Multivariate correlational analysis. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial and organizational psychology (Vol. 1, pp. 637-686). Palo Alto, CA: Consulting Psychologists Press.
8. Bobko, P. (2001). Correlation and regression: Applications for industrial and organizational psychology and management. Thousand Oaks, CA: Sage.
9. Braun, M. T., & Oswald, F. L. (2011). Exploratory regression analysis: A tool for selecting models and determining predictor importance. Behavior Research Methods, 43, 331-339. doi: 10. 3758/s13428-010-0046-8.
10. Breaugh, J. A. (2006). Rethinking the control of nuisance variables in theory testing. Journal of Business and Psychology, 20, 429-443. doi: 10. 1007/s10869-005-9009-y.
11. Breaugh, J. A. (2008). Important considerations in using statistical procedures to control for nuisance variables in non-experimental studies. Human Resource Management Review, 18, 282-293. doi: 10. 1016/j. hrmr. 2008. 03. 001.
12. Budescu, D. V. (1993). Dominance analysis: A new approach to the problem of relative importance of predictors in multiple regression. Psychological Bulletin, 114, 542-551. doi: 10. 1037/0033-2909. 114. 3. 542.
13. Campion, M. A. (1993). Article review checklist: A criterion checklist for reviewing research articles in applied psychology. Personnel Psychology, 46, 705-718. doi: 10. 1111/j. 1744-6570. 1993. tb00896. x.
14. Carlson, K. D., & Wu, J. (2012). The illusion of statistical control: Control variable practice in management research. Organizational Research Methods, 15, 413-435. doi: 10. 1177/1094428111428817.
15. Chan, D. (2011). Advances in analytic strategies. In S. Zedeck (Ed.), APA handbook of industrial and organizational psychology (Vol. 1, pp. 85-113). Washington, DC: American Psychological Association.
16. Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences. Mahwah, NJ: Erlbaum.
17. Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Erlbaum.
18. Courville, T., & Thompson, B. (2001). Use of structure coefficient in published multiple regression articles: β is not enough. Educational and Psychological Measurement, 61, 229-248. doi: 10. 1177/0013164401612006.
19. Darlington, R. B. (1968). Multiple regression in psychological research and practice. Psychological Bulletin, 69, 161-182. doi: 10. 1037/h0025471.
20. Darlington, R. B. (1990). Regression and linear models. New York, NY: McGraw-Hill.
21. Dickson, J. M., Moberly, N. J., Hannon, E. M., & Bates, G. W. (2009). Are repressors so special after all? Specificity of negative personal events as a function of anxiety and defensiveness. Journal of Research in Personality, 43, 386-391. doi: 10. 1016/j. jrp. 2009. 01. 002.
22. Dineen, B. R., Lewicki, R. J., & Tomlinson, E. C. (2006). Supervisory guidance and behavioral integrity: Relationships with employee citizenship and deviant behaviors. Journal of Applied Psychology, 91, 622-635. doi: 10. 1037/0021-9010. 91. 3. 622.
23. Francis, G. (2012). Too good to be true: Publication bias in two prominent studies from experimental psychology. Psychonomic Bulletin & Review, 19, 151-156. doi: 10. 3758/s13423-012-0227-9.
24. Gardner, R. C. (2001). Psychological statistics using SPSS for Windows. Upper Saddle River, NJ: Prentice-Hall.
25. Gordon, R. A. (1968). Issues in multiple regression. American Journal of Sociology, 73, 592-616.
26. Hinkin, T. R., & Schriesheim, C. A. (2008). An examination of "nonleadership": From laissez-faire leadership to leader reward omission and punishment omission. Journal of Applied Psychology, 93, 1234-1248. doi: 10. 1037/a0012875.
27. Hirschfeld, R. R., & Bernerth, J. B. (2008). Mental efficacy and physical efficacy at the team level: Inputs and outcomes among newly formed action teams. Journal of Applied Psychology, 93, 1429-1437. doi: 10. 1037/a0012574.
28. Hoffman, B. J., Bynum, B. H., Piccolo, R. F., & Sutton, A. W. (2011). Person-organization value congruence: How transformational leaders influence work. Academy of Management Journal, 54, 779-796. doi: 10. 5465/AMJ. 2011. 64870139.
29. Homan, A. C., Hollenbeck, J. R., Humphrey, S. E., van Knippenberg, D., Ilgen, D. R., & van Kleef, G. (2008). Facing differences with an open mind: Openness to experience, salience of intragroup differences, and performance of diverse work groups. Academy of Management Journal, 51, 1204-1222. doi: 10. 5465/AMJ. 2008. 35732995.
30. Horst, P. (1941). The role of the predictor variables which are independent of the criterion. Social Science Research Council, 48, 431-436.
31. Hunter, J. E., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings. Thousand Oaks, CA: Sage.
32. Johnson, J. W. (2000). A heuristic method for estimating the relative weight of predictor variables in multiple regression. Multivariate Behavioral Research, 35, 1-19. doi: 10. 1207/S15327906MBR3501_1.
33. Johnson, J. W., & LeBreton, J. M. (2004). History and use of relative importance indices in organizational research. Organizational Research Methods, 7, 238-257. doi: 10. 1177/1094428104266510.
34. Keller, R. T. (2006). Transformational leadership, initiating structure, and substitutes for leadership: A longitudinal study of research and development project team performance. Journal of Applied Psychology, 91, 202-210. doi: 10. 1037/0021-9010. 91. 1. 202.
35. Kraha, A., Turner, H., Nimon, K., Zientek, L., & Henson, R. K. (2012). The multifaceted concept of predictor importance: Tools to support interpreting multiple regression. Frontiers in Psychology, 3, 1-16. doi: 10. 3389/fpsyg. 2012. 00044.
36. LeBreton, J. M., Hargis, M. B., Griepentrog, B., Oswald, F. L., & Ployhart, R. E. (2007). A multidimensional approach for evaluating variables in organizational research and practice. Personnel Psychology, 60, 475-498. doi: 10. 1111/j. 1744-6570. 2007. 00080. x.
37. Lehman, W. E. K., & Simpson, D. D. (1992). Employee substance use and on-the-job behaviors. Journal of Applied Psychology, 77, 309-321. doi: 10. 1037/0021-9010. 77. 3. 309.
38. Lindeman, R., Merenda, P., & Gold, R. (1980). Introduction to bivariate and multivariate analysis. Glenview, IL: Scott, Foresman.
39. Liu, C., Spector, P. E., & Jex, S. M. (2005). The relation of job control with job strains: A comparison of multiple data sources. Journal of Occupational and Organizational Psychology, 78, 325-336. doi: 10. 1348/096317905X26002.
40. Lorenzo-Seva, U., & Ferrando, P. J. (2011). FIRE: an SPSS program for variable selection in multiple linear regression via the relative importance of predictors. Behavior Research Methods, 43, 1-7. doi: 10. 3758/s13428-010-0043-y.
41. Lorenzo-Seva, U., Ferrando, P. J., & Chico, E. (2010). Two SPSS programs for interpreting multiple regression results. Behavior Research Methods, 42, 29-35. doi: 10. 3758/BRM. 42. 1. 29.
42. MacKinnon, D. P., Krull, J. L., & Lockwood, C. M. (2000). Equivalence of the mediation, confounding, and suppression effect. Prevention Science, 4, 173-181. doi: 10. 1023/A: 1026595011371.
43. Meade, A. W., Behrend, T. S., & Lance, C. E. (2009). Dr. StrangeLOVE, or how I learned to stop worrying and love omitted variables. In C. E. Lance & R. J. Vandenberg (Eds.), Statistical and methodological myths and urban legends (pp. 247-265). New York, NY: Routledge.
44. Miller, G. A., & Chapman, J. P. (2001). Misunderstanding analysis of covariance. Journal of Abnormal Psychology, 110, 40-48. doi: 10. 1037/0021-843X. 110. 1. 40.
45. Newcombe, N. S. (2003). Some controls control too much. Child Development, 74, 1050-1052. doi: 10. 1111/1467-8624. 00588.
46. Nimon, K. F., & Oswald, F. L. (2013). Understanding the results of multiple linear regression: Beyond standardized regression coefficients. Organizational Research Methods, 16, 650-674. doi: 10. 1177/1094428113493929.
47. Park, H., & Antonioni, D. (2007). Personality, reciprocity, and strength of conflict resolution strategy. Journal of Research in Personality, 41, 110-125. doi: 10. 1016/j. jrp. 2006. 03. 003.
48. Pea, R., Nass, C., Meheula, L., Rance, M., Kumar, A., Bamford, H., & Zhou, M. (2012). Media use, face-to-face communication, media multitasking, and social well-being among 8- to 12-year-old girls. Developmental Psychology, 48, 327-336. doi: 10. 1037/a0027030.
49. Pedhazur, E. J. (1982). Multiple regression in behavioral research: Explanation and prediction (2nd ed.). New York, NY: Holt, Rinehart, & Winston.
50. Pedhazur, E. J. (1997). Multiple regression in behavioral research: Explanation and prediction (3rd ed.). Forth Worth, TX: Harcourt.
51. Peterson, R. A., & Brown, S. P. (2005). On the use of beta coefficients in meta-analysis. Journal of Applied Psychology, 90, 175-181. doi: 10. 1037/0021-9010. 90. 1. 175.
52. Porath, C. L., & Bateman, T. S. (2006). Self-regulation: From goal orientation to job performance. Journal of Applied Psychology, 91, 185-192. doi: 10. 1037/0021-9010. 91. 1. 185.
53. Ramirez, M., Roth, L., Young, T., & Peek-Asa, C. (2013). Rural roadway safety perceptions among rural teen drivers living in and outside of towns. Journal of Rural Health, 29, 46-54. doi: 10. 1111/j. 1748-0361. 2012. 00435. x.
54. Rousseau, D. M. (Ed.). (2012). The Oxford handbook of evidence-based management. New York, NY: Oxford University Press.
55. Rowe, C. L., La Greca, A. M., & Alexandersson, A. (2010). Family and individual factors associated with substance involvement and PTS symptoms among adolescents in Greater New Orleans after Hurricane Katrina. Journal of Consulting and Clinical Psychology, 78, 806-817. doi: 10. 1037/a0020808.
56. Schmitt, N., Keeny, J., & Oswald, F. L. (2009). Prediction of 4-year college student performance using cognitive and noncognitive predictors and the impact on demographic status of admitted students. Journal of Applied Psychology, 94, 1479-1497. doi: 10. 1037/a0016810.
57. Schooler, J. (2011). Unpublished results hide the decline effect. Nature, 470, 437. doi: 10. 1038/470437a.
58. Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359-1366. doi: 10. 1177/0956797611417632.
59. Spector, P. E., & Brannick, M. T. (2011). Methodological urban legends: The misuse of statistical control variables. Organizational Research Methods, 14, 287-305. doi: 10. 1177/1094428110369842.
60. Spector, P. E., Dwyer, D., & Jex, S. M. (1988). Relation of job stressors to affective, health, and performance outcomes: A comparison of multiple data sources. Journal of Applied Psychology, 73, 11-19.
61. Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics. Boston, MA: Pearson/Allyn & Bacon.
62. Tepper, B. J., Moss, S. E., Lockhart, D. E., & Carr, J. C. (2007). Abusive supervision, upward maintenance communication, and subordinates' psychological distress. Academy of Management Journal, 50, 1169-1180. doi: 10. 2307/20159918.
63. Tonidandel, S., LeBreton, J. M., & Johnson, J. W. (2009). Determining the statistical significance of relative weights. Psychological Methods, 14, 387-399. doi: 10. 1037/a0017735.
64. Wanberg, C. R., Zhu, J., Kanfer, R., & Zhang, Z. (2012). After the pink slip: Applying dynamic motivation frameworks to the job search experience. Academy of Management Journal, 55, 261-284. doi: 10. 5465/amj. 2010. 0157.
65. Wang, S., Tomlinson, E. C., & Noe, R. A. (2010). The role of mentor trust and protégé internal locus of control in formal mentoring relationships. Journal of Applied Psychology, 95, 358-367. doi: 10. 1037/a0017663.
66. Watson, D., Pennebaker, J. W., & Folger, R. (1987). Beyond negative affectivity: Measuring stress and satisfaction in the workplace. Journal of Organizational Behavior Management, 8, 141-158. doi: 10. 1300/J075v08n02_09.
67. Winne, P. H. (1983). Distortions of construct validity in multiple regression analysis. Canadian Journal of Behavioral Science, 15, 187-202. doi: 10. 1037/h0080736.