Sources of method bias in social science research and recommendations on how to control it

Annual Review of Psychology

Volumn 63, Issue , 2012, Pages 539-569

  Podsakoff, Philip M ^a   MacKenzie, Scott B ^a   Podsakoff, Nathan P ^b  

^a INDIANA UNIVERSITY   (United States)

^b UNIVERSITY OF ARIZONA   (United States)

Author keywords

Common method variance; Instrumental variable technique; Marker variable technique; Response style biases; Unmeasured latent variable technique

Indexed keywords

EPIDEMIOLOGY; HUMAN; METHODOLOGY; REVIEW; SOCIOLOGY; STANDARD;

BIAS (EPIDEMIOLOGY); HUMANS; RESEARCH DESIGN; SOCIAL SCIENCES;

EID: 82955240674     PISSN: 00664308     EISSN: 15452085     Source Type: Book Series    
DOI: 10.1146/annurev-psych-120710-100452     Document Type: Review

References (95)

1. Alessandri G, Vecchione M, Fagnani C, Bentler PM, Barbaranelli C, et al. 2010. Much more than model fitting? Evidence for the heritability of method effect associated with positively worded items of the life orientation test revised. Struct. Equ. Model. 17:642-53
2. Antonakis J, Bendahan S, Jacquart P, Lalive R. 2010. On making causal claims: a review and recommendations. Leadersh. Q. 6:1086-20
3. Aronson E, Wilson TD, Brewer MB. 1998. Experimentation in social psychology. In The Handbook of Social Psychology, ed. DT Gilbert, ST Fiske, G Lindzey, Vol. 1, pp. 99-142. Boston, MA: McGraw-Hill. 4th ed.
4. Arora R. 1982. Validation of an S-O-R model for situation, enduring, and response components of involvement. J. Mark. Res. 19:505-16
5. Bagozzi RP. 1984. A prospectus for theory construction in marketing. J. Mark. 48:11-29
6. Bagozzi RP. 1993. Assessing construct-validity in personality research: applications to measures of self-esteem. J. Res. Personal. 27:49-87
7. Bagozzi RP, Phillips LW. 1982. Representing and testing organizational theories-a holistic construal. Admin. Sci. Q. 27:459-89
8. Bagozzi RP, Yi Y. 1990. Assessing method variance in multitrait- multimethod matrices: the case of self-reported affect and perceptions at work. J. Appl. Psychol. 75:547-60
9. Baumgartner H,Steenkamp JBEM. 2001. Response stylesinmarketing research:across-national investigation. J. Mark. Res. 38:143-56
10. Billiet JB, McClendon MJ. 2000. Modeling acquiescence in measurement models for two balanced sets of items. Struct. Equ. Model. 7:608-28
11. Bodner TE. 2006. Designs, participants, and measurement methods in psychological research. Can. Psychol. 47:263-72
12. Bollen KA. 1989. Structural Equations with Latent Variables. New York: Wiley
13. Brannick MT, Chan D, Conway JM, Lance CE, Spector PE. 2010. What is method variance and how can we cope with it? A panel discussion. Organ. Res. Methods 13:407-20
14. Brannick MT, Spector PE. 1990. Estimation problems in the block-diagonal model of the multitrait-multimethod matrix. Appl. Psychol. Meas. 14:325-39
15. Buckley MR, Cote JA, Comstock SM. 1990. Measurement errors in the behavioral sciences: the case of personality attitude research. Educ. Psychol. Meas. 50:447-74
16. Campbell DT, Fiske D. 1959. Convergent and discriminant validation by the multitrait-multimethod matrix. Psychol. Bull. 56:81-105
17. Campbell DT, O'Connell EJ. 1967. Methods factors in multitrait- multimethod matrices: multiplicative rather than additive? Multivar. Behav. Res. 2:409-26
18. Chang S-J, van Wittleloostuijn A, Eden L. 2010. From the editors: common method variance in international business research. J. Int.Bus. Stud. 41:178-84
19. Chen PY, Spector PE. 1991. Negative affectivity as the underlying cause of correlations between stressors and strains. J. Appl. Psychol. 76:398-407
20. Cote JA, Buckley R. 1987. Estimating trait, method, and error variance: generalizing across 70 construct validation studies. J. Mark. Res. 24:315-18
21. Cote JA, Buckley R. 1988. Measurement error and theory testing in consumer research: an illustration of the importance of construct validation. J. Consum. Res. 14:579-82
22. Cronbach LJ. 1946. Response sets and test validity. Educ. Psychol. Meas. 6:475-94
23. Cronbach LJ. 1950. Further evidence on response sets and test validity. Educ. Psychol. Meas. 10:3-31
24. DeBeuckelaer A, Weijters B, Rutten A. 2010. Using ad hoc measures for response styles: a cautionary note. Qual. Quant. 44:761-75
25. Doty DH, Glick WH. 1998. Common methods bias: Does common methods variance really bias results? Organ. Res. Methods 1:374-406 (Pubitemid 128543457)
26. Edwards AL. 1970. The Measurement of Personality Traits by Scales and Inventories. New York: Holt, Rinehart & Winston
27. Edwards JR. 2008. To prosper, organizational psychology should overcome methodological barriers to progress. J. Organ. Behav. 29:469-91
28. Ericsson KA, Simon HA. 1980. Verbal reports as data. Psychol. Rev. 87:215-57
29. Evans MG. 1985. A Monte Carlo study of the effects of correlated method variance in moderated multiple regression analysis. Organ. Behav. Hum. Decis. Process. 36:305-23
30. Feldman JM, Lynch JG. 1988. Self-generated validity and other effects of measurement on belief, attitude, intention, and behavior. J. Appl. Psychol. 73:421-35
31. Fiske DW. 1982. Convergent-discriminant validation in measurements and research strategies. In Forms of Validity in Research, ed. D Brinbirg, LH Kidder, pp. 77-92. San Francisco, CA: Jossey-Bass
32. Flamer S. 1983. Assessment of the multitrait-multimethod matrix validity of Likert scales via confirmatory factor analysis. Multivar. Behav. Res. 18:275-308
33. Harris MM, Bladen A. 1994. Wording effects in the measurement of role conflict and role ambiguity: a multitrait-multimethod analysis. J. Manage. 20:887-901
34. Harrison DA, McLaughlin ME, Coalter TM. 1996. Context, cognition, and common method variance: psychometric and verbal protocol evidence. Organ. Behav. Hum. Decis. Process. 68:246-61 (Pubitemid 126162629)
35. Higgins ET, Rholes WS, Jones CR. 1977. Category accessibility and impression formation. J. Exp. Soc. Psychol. 13:141-54
36. Hulsheger UR, Anderson N, Salgado JF. 2009. Team-level predictors of innovation at work: a comprehensive meta-analysis spanning three decades of research. J. Appl. Psychol. 94:1128-45
37. Jackson DN. 1967. Acquiescence response styles: problems of identification and control. In Response Set in Personality Assessment, ed. IA Berg, pp. 71-114. Chicago: Aldine
38. Johnson JA. 2004. The impact of item characteristics on item and scale validity. Multivar. Behav. Res. 39:273-302 (Pubitemid 39234289)
39. Johnson RE, Rosen CC, Djurdevic E. 2011. Assessing the impact of common method variance on higher-order multidimensional constructs. J. Appl. Psychol. 96:744-61
40. Jöreskög KG. 1998. Interaction and nonlinear modeling: issues and approaches. In Interaction and Nonlinear Effects in Structural Equation Modeling, ed. RE Schumacker, GA Marcoulides, pp. 239-50. Mahwah, NJ: Erlbaum
41. Kennedy P. 2008. A Guide to Econometrics. Malden, MA: Blackwell. 6th ed.
42. Kozlowski S. 2009. Editorial. J. Appl. Psychol. 94:1-4
43. Kothandapani V. 1971. Validation of feeling, belief, and intention to act as three components of attitude and their contribution to prediction of contraceptive behavior. J. Personal. Soc. Psychol. 19:321-33
44. Krosnick JA. 1991. Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl. Cogn. Psychol. 5:213-36
45. Krosnick JA. 1999. Survey research. Annu. Rev. Psychol. 50:537-67
46. Kuncel NR, Tellegen A. 2009. A conceptual and empirical reexamination of the measurement of the social desirability of items: implications for detecting desirable response style and scale development. Pers. Psychol. 62:201-228
47. Lance CE, Baranik LE, Lau AR, Scharlau EA. 2009. If it ain't trait it must be method: (mis)application of the multitrait-multimethod design in organizational research. In Statistical and Methodological Myths and Urban Legends: Doctrine, Verity, and Fable in the Organizational and Social Sciences, ed. CE Lance, RL Vandenberg, pp. 337-60. New York: Routledge
48. Lance CE, Dawson B, Birkelbach D, Hoffman BJ. 2010. Method effects, measurement error, and substantive conclusions. Organ. Res. Methods 13:407-20
49. Le H, Schmidt FL, Putka DJ. 2009. The multifaceted nature of measurement artifacts and its implications for estimating construct-level relationships. Organ. Res. Methods 12:165-200
50. Lindell MK, Whitney DJ. 2001. Accounting for common method variance in cross-sectional designs. J. Appl. Psychol. 86:114-21 (Pubitemid 33404733)
51. Lowe KB, Kroeck KG, Sivasubramaniam N. 1996. Effectiveness correlates of transformational and transac-tional leadership: a meta-analytic review of the MLQ literature. Leadersh. Q. 7:385-425 (Pubitemid 126400794)
52. MacKenzie SB, Podsakoff PM, Podsakoff NP. 2011. Construct measurement and validity assessment in behavioral research: integrating new and existing techniques. MIS Q. 35: 293-334
53. Messick S. 1991. Psychology and methodology of response styles. In Improving the Inquiry in Social Science: A Volume in Honor of Lee J. Cronbach, ed. RE Snow, DE Wiley, pp. 161-200. Hillsdale, NJ: Erlbaum
54. Mirowsky J, Ross CE. 1991. Eliminating defense and agreement bias from measures of the sense of control: a 2 × 2 index. Soc. Psychol. Q. 54:127-45
55. Nisbett RE, Wilson TD. 1977. Telling more than we can know: verbal reports on mental processes. Psychol. Rev. 84:231-59
56. Nunally JC, Bernstein IH. 1994. Psychometric Theory. New York: McGraw-Hill. 3rd ed.
57. Ostroff C, Kinicki AJ, Clark MA. 2002. Substantive and operational issues of response bias across levels of analysis: an example of climate-satisfaction relationships. J. Appl. Psychol. 87:355-68
58. Paulhus DL. 1984. Two-component models of socially desirable responding. J. Personal. Soc. Psychol. 46:598-609
59. Phillips JS, Lord RG. 1986. Notes on the theoretical and practical consequences of implicit leadership theories for the future of leadership measurement. J. Manage. 12:31-41
60. Podsakoff PM, MacKenzie SB, Lee J-Y, Podsakoff NP. 2003. Common method biases in behavioral research: a critical review of the literature and recommended remedies. J. Appl. Psychol. 88:879-903 (Pubitemid 37239703)
61. Podsakoff PM, Organ DW. 1986. Self-reports in organizational research-problems and prospects. J. Manage. 12:531-44
62. Rafferty AE, Griffin MA. 2004. Dimensions of transformational leadership: conceptual and empirical extensions. Leadersh. Q. 15:329-54 (Pubitemid 38597405)
63. Rafferty AE, Griffin MA. 2006. Refining individualized consideration: distinguishing developmental leadership and supportive leadership. J. Occup. Organ. Psychol. 79:37-61 (Pubitemid 43528625)
64. Richardson HA, Simmering MJ, Sturman MC. 2009. A tale of three perspectives: examining post hoc statistical techniques for detection and correction of common method variance. Organ. Res. Methods 12:762-800
65. Rindfleisch A, Malter AJ, Ganesan S, Moorman C. 2008. Cross-sectional versus longitudinal research: concepts, findings, and guidelines. J. Mark. Res. 45:261-79
66. Rosenberg MJ. 1965. When dissonance fails: on eliminating evaluation apprehension from attitude measurement. J. Personal. Soc. Psychol. 1:28-42
67. Scherpenzeel A, Saris W. 1997. The validity and reliability of survey questions: a meta-analysis of MTMM studies. Soc. Methods Res. 25:341-83
68. Schaubroeck J, Ganster DC, Fox ML. 1992. Dispositional affect and work-related stress. J. Appl. Psychol. 77:322-35
69. Schmitt N. 1994. Method bias: the importance of theory and measurement. J. Organ. Behav. 15:393-98
70. Schwarz N, Hippler HJ, Noelle-Neumann E. 1992. A cognitive model of response-order effects in survey measurement. In Context Effects in Social and Psychological Research, ed. N Schwarz, S Sudman, pp. 187-201. New York: Springer-Verlag
71. Sechrest L, Davis MF, Stickle TR, McKnight PE. 2000. Understanding method variance. In Research Design: David Campbell's Legacy, ed. L Bickman, Vol. 2, pp. 63-88. Thousand Oaks, CA: Sage
72. Sharma R, Yetton P, Crawford J. 2009. Estimating the effect of common method variance: the method-method pair technique with an illustration from TAM research. MIS Q. 33:473-90
73. Siemsen E, Roth A, Oliveira P. 2010. Common method bias in regression models with linear, quadratic, and interaction effects. Organ. Res. Methods 13:456-76
74. Spector PE. 1987. Method variance as an artifact in self-reported affect and perceptions at work: myth or significant problem. J. Appl. Psychol. 72:438-43
75. Spector PE. 2006. Method variance in organizational research: truth or urban legend? Organ. Res. Methods 9:221-32 (Pubitemid 43907721)
76. Spector PE, Brannick MT. 2009. Common method variance or measurement bias? The problem and possible solutions. In The Sage Handbook of Organizational Research Methods, ed. DA Buchanan, A Bryman, pp. 346-62. Los Angeles, CA: Sage
77. Steenkamp JBEM, DeJong MG, Baumgartner H. 2010. Socially desirable response tendencies in survey research. J. Mark. Res. 47:199-214
78. Straub DW. 2009. Creating blue oceans of thought via highly citable articles. MIS Q. 33:iii-vii
79. Sudman S, Bradburn NM, Schwarz N. 1996. Thinking About Answers: The Application of Cognitive Processes to Survey Methodology. San Francisco, CA: Jossey-Bass
80. Tourangeau R, Rips LJ, Rasinski KA. 2000. The Psychology of Survey Response. London: Cambridge Univ. Press
81. Tourangeau R, Singer E, Presser S. 2003. Context effects in attitude surveys-effects on remote items and impact on predictive validity. Soc. Methods Res. 31:486-513
82. Weijters B, Cabooter E, Schillewaert N. 2010c. The effect of rating scale format on response styles: the number of response categories and response category labels. Int. J. Mark. 27:236-47
83. Weijters B, Geuens M, Schillewaert N. 2009. The proximity effect: the role of inter-item distance on reverse-item bias. Int. J. Res. Market. 26:2-12
84. Weijters B, Geuens M, Schillewaert N. 2010a. The stability of individual response styles. Psychol. Methods 15:96-110
85. Weijters B, Geuens M, Schillewaert N. 2010b. The individual consistency of acquiescence and extreme response style in self-report questionnaires. Appl. Psychol. Meas. 34:105-21
86. Weijters B, Schillewaert N, Geuens M. 2008. Assessing response styles across modes of data collection. J. Acad. Mark. Sci. 36:409-22
87. Werts CE, Linn RL, Joreskog KG. 1974. Intraclass reliability estimates-testing structural assumptions. Educ. Psychol. Meas. 34:25-33
88. Widaman KF. 1985. Hierarchically nested covariance structural models for multitrait-multimethod data. Appl. Psychol. Meas. 9:1-26
89. Williams LJ, Anderson SE. 1994. An alternative approach to method effects by using latent-variable models: applications in organizational behavior research. J. Appl. Psychol. 79:323-31
90. Williams LJ, Cote JA, Buckley MR. 1989. Lack of method variance in self-reported affect and perceptions at work: reality or artifact? J. Appl. Psychol. 74:462-68
91. Williams LJ, Gavin MB, Williams ML. 1996. Measurement and nonmeasurement processes with negative affectivity and employee attitudes. J. Appl. Psychol. 81:88-101
92. Williams LJ, Hartman N, Cavazotte F. 2010. Method variance and marker variables: a review and comprehensive CFA marker technique. Organ. Res. Methods 13:477-514
93. Woszczynski AB, Whitman ME. 2004. The problem of common method variance in IS research. In Handbook of Information Systems Research, ed. ME Whitman, AB Woszczynski, pp. 66-77. Hershey, PA: Idea Group
94. Wothke W, Browne MW. 1990. The direct product model for the MTMM matrix parameterized as a second order factor analysis model. Psychometrika 55:255-62
95. Zinkhan GM. 2006. Research traditions and patterns in marketing scholarship. J. Acad. Mark. Sci. 34:281-83 (Pubitemid 43877107)