Minimizing method bias through programmatic research

MIS Quarterly: Management Information Systems

Volumn 33, Issue 3, 2009, Pages 445-471

  Burton Jones, Andrew ^a  

^a UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Construct; Measurement; Method bias; Method variance; Qualitative; Quantitative; Validity

Indexed keywords

MEASUREMENT;

CONSTRUCT; METHOD BIAS; METHOD VARIANCES; QUALITATIVE; QUANTITATIVE; VALIDITY;

MANAGEMENT INFORMATION SYSTEMS;

EID: 69549134152     PISSN: 02767783     EISSN: None     Source Type: Journal    
DOI: 10.2307/20650304     Document Type: Article

References (88)

1. Agarwal, R., and Karahanna, E. 2000. "Time Flies When You're Having Fun: Cognitive Absorption and Beliefs About Information Technology Usage," MIS Quarterly (24:4), pp. 665-694.
2. rd ed.), Upper Saddle River, NJ: Prentice Hall.
3. Bagozzi, R. P. and Yi, Y. 1991. "Multitrak-Multimethod Matrices in Consumer Research," Journal of Consumer Research ( 17), pp. 426-439.
4. Barki, H., Titah, R., and Boffo, C. 2007. "Information System UseRelated Activity: An Expanded Behavioral Conceptualization of Individual-Level Information System Use," Information Systems Research (18:2), pp. 173-192.
5. Bern, D. J. 1972. "Self-Perception Theory," in Advances in Experimental Social Psychology (Volume 6), L. Berkowitz. (ed.), New York: Academic Press, pp. 1-62.
6. Berthon, P., Pitt, L., Ewing, M.,Carr, C. L. 2002. Potential Research Space in MIS: A Framework for Envisioning and Evaluating Research Replication, Extension, and Generation, Information Systems Research (13:4), 416-427. (Pubitemid 36039706)
7. Blair, E., and Burton, S. 1987. "Cognitive Processes Used by Survey Respondents to Answer Behavioral Frequency Questions," Journal of Consumer Research (14), pp. 280-288.
8. Bolger, F., and Onkal-Atay, D. 2004. The Effects of Feedback on Judgmental Interval Predictions, International Journal of Forecasting (20:1), 29-39. (Pubitemid 38148292)
9. Bollen, K. A., and Lennox, R. 1991. "Conventional Wisdom on Measurement: A Structural Equation Perspective," Psychological Bulletin (110:2), pp. 305-314.
10. Borsboom, D. 2005. Measuring the Mind, Cambridge, UK: Cambridge University Press.
11. Brewer, M. B., and Collins, B. E. (eds.). 1981. Scientific Inquiry and the Social Sciences: A Volume in Honor of Donald T. Campbell, San Francisco: Jossey-Bass.
12. Burton-Jones, A. 2005. New Perspectives on the System Usage Construct, unpublished doctoral dissertation, Georgia State University available at: http://etd.gsu.edu/theses/available/ etd-07222005-162348/).
13. Burton-Jones, A., and Straub, D. W. 2006. Reconceptualizing System Usage: An Approach and Empirical Test, Information Systems Research (17:3), pp. 228-246. (Pubitemid 44371787)
14. Campbell, D. T., and Fiske, D. W. 1959. "Convergent and Discriminant Validity by the Multitrait-Multimethod Matrix," Psychological Bulletin (56), pp. 81-105.
15. Campbell, D. T., and O'Connell, E. J. 1982. "Methods as Diluting Trait Relationships Rather than Adding Irrelevant Systematic Variance," New Directions for Methodology of Social and Behavioral Science: Forms of Validity in Research, D. Brinberg and L. H. Kidder (eds.), San Francisco: Jossey-Bass, CA, pp. 93-111.
16. Campbell, D. T., and Russo, M. J. (eds.). 2001. Social Measurement, Thousand Oaks, CA: Sage Publications.
17. Chidambaram, L., and Tung, L. L. 2005. Is Out of Sight, Out of Mind? An Empirical Study of Social Loafing in Technology-Supported Groups, Information Systems Research (16:2), 149-168. (Pubitemid 43057220)
18. Chin, W. W. 1996. "The Measurement and Meaning of IT Usage: Reconciling Recent Discrepancies Between Self Reported and Computer Recorded Usage," in Proceedings of the Administrative Sciences Association of Canada, B. A. Aubert and W. W. Chin (eds.), Montreal, Canada, pp. 65-74.
19. Chin, W. W., Gopal, A., and Salisbury, W. D. 1997. Advancing the Theory of Adaptive Structuration: The Development of a Scale to Measure Faithfulness of Appropriation, Information Systems Research (8:4), pp. 342-367. (Pubitemid 127468617)
20. Cote, J. A., and Buckley, R. 1987. "Estimating Trait, Method, and Error Variance: Generalizing Across 70 Construct Validation Studies," Journal of Marketing Research (24), pp. 315-318.
21. Crampton, S., and Wagner, J. A. 1994. Percept-Percept Inflation in Microorganizational Research: An Investigation of Prevalence and Effect, Journal of Applied Psychology (79), 67-76. (Pubitemid 24769287)
22. Crano, W. D. 1981. "Triangulation and Cross-Cultural Research," in Scientific Inquiry and the Social Sciences: A Volume in Honor of Donald T. Campbell, M. B. Brewer and B. E. Collins (eds.), San Francisco: Jossey-Bass, pp. 317-344.
23. Cronbach, L. J. 1995. "Giving Method Variance Its Due," in Personality Research, Methods, and Theory: A Festschrift Honoring Donald W. Fiske, P. E. Shrout and S. T. Fiske (eds.), Hillsdale, NJ: Lawrence Erlbaum, pp. 145-157.
24. Csikszentmihalyi, M. 1990. Flow: The Psychology of Optimal Experience, San Francisco: Jossey-Bass.
25. Davis, F. 1989. "Perceived Usefulness, Perceived Ease of Use, and End User Acceptance of Information Technology," MIS Quarterly (13:3), pp. 318-339.
26. DeSanctis, G., and Poole, M. S. 1994. Capturing the Complexity in Advanced Technology Use: Adaptive Structuration Theory, Organization Science (5:2), 121-147. (Pubitemid 24803029)
27. Diamantopoulos, A., and Winklhofer, H. M. 2001. Index Construction with Formative Indicators: An Alternative to Scale Development,JournalojMarketing Research (38), 269-277. (Pubitemid 33280372)
28. Donaldson, S. I., Thomas, C. W., Graham, J. W., Au, J. G., and Hansen, W. B. 2000. "Verifying Drug Abuse Prevention Program Effects Using Reciprocal Best Friend Reports," Journal of Behavioral Medicine (23:6), pp. 585-601.
29. Doty, D. H., and Glick, W. H. 1998. "Common Methods Bias: Does Common Methods Variance Really Bias Results?," Organizational Research Methods (1), pp. 374-406.
30. Fazio, R. H., and Olson, M. A. 2003. "Implicit Measures in Social Cognition Research: Their Meaning and Use," Annual Review of Psychology (54), pp. 297-327.
31. Fiske, D. W. 1982. "Convergent-Discriminant Validation in Measurements and Research Strategies," in New Directions for Methodology of Social and Behavioral Science: Forms of Validity in Research, D. Brinbirg and L. H. Kidder (eds.), San Francisco: Jossey Bass, pp. 77-92.
32. Fiske, D. W., and Campbell, D. T. 1992. "Citations Do Not Solve Problems," Psychological Bulletin (112:3), pp. 393-395.
33. Glick, W. H., Jenkins Jr., G. D., and Gupta, N. 1986. "Method Versus Substance: How Strong Are Underlying Relationships Between Job Characteristics and Attitudinal Outcomes?," Academy of Management Journal (29:3), pp. 441-464.
34. th Hawaii International Conference on Information Systems, Los Alamitos, CA: IEEE Computer Society Press.
35. Harrison, D. A., McLaughlin, M. E., and Coalter, T. M. 1996. "Context, Cognition, and Common Method Variance: Psychometric and Verbal Protocol Evidence," Organizational Behavior and Human Decision Processes (68), pp. 246-261.
36. Hoyt,W. T. 2000. "Rater Bias in Psychological Research: When Is it a Problem and What Can We Do About It?," Psychological Methods (5:1), pp. 64-86.
37. Hufhagel, E. M., and Conca, C. 1994. "User Response Data: The Potential for Errors and Biases," Information Systems Research (5:1), pp. 48-73.
38. James, L. R. 1998. "Measurement of Personality via Conditional Reasoning," Organizational Research Methods (1:2), pp. 131-163.
39. Jiang, J. J., Klein, G., and Carr, C. L. 2002. "Measuring Information System Service Quality: SERVQUAL from the Other Side," MIS Quarterly (26:2), pp. 145-166.
40. Kim, J., Hahn, J., and Hahn, H. 2000. How Do We Understand a System with (So) Many Diagrams? Cognitive Integration Processes in Diagrammatic Reasoning, Information Systems Research (11:3), 284-303. (Pubitemid 33214330)
41. Kim, S. S., and Malhotra, N. K. 2005. "Predicting System Usage from Intention and Past Use: Scale Issues in the Predictors," Decision Sciences (36:1), pp. 187-196.
42. Kim, S. S., Malhotra, N. K., and Narasimhan, S. 2005. Two Theoretical Perspectives on Automatic Use: A Theoretical and Empirical Comparison, Information Systems Research (16:4), 418-432. (Pubitemid 43011650)
43. King, W. R., Liu, C. Z., and Haney, M. H. 2007. "Method Effects in IS Survey Research: An Assessment and Recommendations," Communications of the Associationfor Information Systems (20), pp. 457-482.
44. Lee, A. S. 1991. "Integrating Positivist and Interpretive Approaches to Organizational Research," Organization Science (2), pp. 342-365.
45. Levins, R. 1966. "The Strategy of Model Building in Population Biology," American Scientist (54), pp. 421-431.
46. Lewis, B. R., Templeton, G. F., and Byrd, T. A. 2005. "A Methodology for Construct Development in MIS Research," European Journal of Information Systems (14:4), pp. 388-400
47. Liang, H., Saraf, N., Hu, Q., and Xue, Y. 2007. Assimilation of Enterprise Systems: The Effect of Institutional Pressures and the Mediating Role of Top Management, MIS Quarterly (31:1), 59-87. (Pubitemid 46382518)
48. Lindell, M. K., and Whitney, D. J. 2001. "Accounting for Common Method Variance in Cross-Sectional Research Designs," Journal of Applied Psychology (86:1), pp. 114-121.
49. Mackay, J. M., and Elam, J. J. 1992. "A Comparative Study of how Experts and Novices Use a Decision Aid to Solve Problems in Complex Knowledge Domains," Information Systems Research (3:2), pp. 150-172.
50. Malhotra, N. K., Kim, S. S., and Patil, A. 2006. "Common Method Variance in IS Research: A Comparison of Alternative Approaches and a Reanalysis of Past Research, Management Science (52:12), 1865-1883. (Pubitemid 44927001)
51. Marcolin, B. L., Compeau, D. R., Munro, M. C., and Huff, S. L. 2000. "Assessing User Competence: Conceptualization and Measurement, "Information Systems Research (11:1), pp. 37-60.
52. Meister, D. (ed.). 1986. Human Factors Testing and Evaluation. Amsterdam: Elsevier.
53. rd ed.), R. Linn (ed.), New York: Macmillan, pp. 13-103.
54. Miles, M. B., and Huberman, M. A. 1994. Qualitative Data Analysis (2nd ed.), Thousand Oaks, CA: Sage Publications.
55. Millsap, R.E. 1995. "The Statistical Analysis of Method Variance in Multitrait-Multimethod Data: A Review," in Personality Research, Methods, and Theory: A Festschrift Honoring Donald W. Fiske, P. E. Shrout and S. T. Fiske (eds.), Hillsdale, NJ: Lawrence Erlbaum, pp. 93-109.
56. Murphy, K. R., Cleveland, J. N., Skattebo, A. L. and Kinney, T. B. 2004. "Raters Who Pursue Different Goals Give Different Ratings," Journal of Applied Psychology (89:1), pp. 158-164.
57. rd ed.), New York: McGraw-Hill.
58. Orne, M. T. 1962. "On the Social Psychology of the Psychological Experiment: With Particular Reference to Demand Characteristics and their Implications," American Psychologist (IT), pp. 776-783.
59. Payne, S., Finch, J. F., and Tremble Jr., T. R. 2003. Validating Surrogate Measures of Psychological Constructs: The Application of Construct Equivalence to Archival Data, Organizational Research Methods (6:3), pp. 363-382. (Pubitemid 36810050)
60. Pitt, L. F., Watson, R. T., and Kavan, C. B. 1995. "Service Quality: A Measure of Information Systems Effectiveness," MIS Quarterly (19:2), pp. 173-187.
61. Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y., and Podsakoff, N. P. "Common Method Bias in Behavioral Research: A Critical Review of the Literature and Recommended Remedies," Journal of Applied Psychology (88:5), pp. 879-903.
62. Richardson, H. A., Simmering, M. J., and Sturman, M. C. 2009. "A Tale of Three Perspectives: Examining Post Hoc Statistical Techniques for Detection and Correction of Common Method Variance," Organizational Research Methods, preprint, pp. 1 -39.
63. Rossiter, J. R. 2002. "The C-OAR-SE Procedure for Scale Development in Marketing," International Journal of Research in Marketing (19), pp. 305-335.
64. Schaeffer, N. C., and Presser, S. 2003. "The Science of Asking Questions," Annual Review of Sociology (29), pp. 65-88.
65. Schmidt, F. L. 1992. "What Do Data Really Mean? Research Findings, Meta-Analysis, and Cumulative Knowledge in Psychology," American Psychologist (47:10), pp. 1173-1181.
66. Searle, J. R. 1995. The Construction of Social Reality, New York: The Free Press.
67. Sechrest, L., Davis, M. F., Stickle, T. R., and McKnight, P. E. 2000. "Understanding "Method" Variance," in Research Design: Donald Campbell's Legacy (Volume 2), L. Bickman (ed.), Thousand Oaks, CA: Sage Publications, pp. 63-87.
68. Sharma, R., Yetton, P., and Crawford, J. 2009. "Estimating the Effect of Common Method Variance: The Method-Method Pair Technique with an Illustration from TAM Research," MIS Quarterly (33:3), pp. 473-490.
69. Shrout.P. E., and Fleiss, J. L. 1979. "Intraclass Correlations: Uses in Assessing Rater Reliability," Psychological Bulletin (86:2), pp. 420-428.
70. Sivo, S., Saunders, C., Chang, Q., and Jiang, J. 2006. "How Low Should You Go? Low Response Rate and the Validity of Inference in IS Questionnake Research" Journal of the Association for Information Systems (7:6), pp. 351-414.
71. Smallwood, J., and Schooler, J. W. 2006. "The Restless Mind," Psychological Bulletin (132:6), pp. 946-958.
72. Spector, P. E. 1994. "Using Self-Report Questionnakes in OB Research: A Comment on the Use of a Controversial Method," Journal of Organizational Behavior (15), pp. 385-392.
73. Spector, P. E. 2006. "Method Variance in Organizational Research: Truth or Urban Legend," Organizational Research Methods (9:2), pp. 221-232.
74. Stewart, K. J., and Gosain, S. 2006. The Impact of Ideology on Effectiveness in Open Source Software Development Teams, MIS Quarterly (30:2), pp. 291-314. (Pubitemid 46050493)
75. Straub, D., Limayem, M., and Karahanna-Evaristo, E. 1995. "Measuring System Usage: Implications for IS Theory Testing," Management Science (4:8), pp. 1328-1342.
76. Talion, P. P., and Kraemer, K. L. 2007. "Fact or Fiction? A Sensemaking Perspective on the Reality Behind Executives' Perceptions of IT Business Value," Journal of Management Information Systems (24:1), pp. 13-54.
77. Tam, K. Y., Kwan, S. S. K., and So, M. K. P. 2009. "Applying Randomized Response Technique to Elicit Truthful Responses to Sensitive Questions in IS Research: The Case of Software Piracy Behavior," Information Systems Research (forthcoming).
78. van Bruggen, G. H., Lilien, G. L., and Kacker, M. 2002. "Informants in Organizational Marketing Research: Why Use Multiple Informants and How to Aggregate Responses," Journal of Marketing Research (34), pp. 469-478.
79. Vandenberg, R. J. 2002. "Toward a Further Understanding of and Improvement in Measurement Invariance Methods and Procedures," Organizational Research Methods (5:2), pp. 139-158.
80. Venkatesh, V., Morris, M. G., Davis, G. B., and Davis, F. D. 2003. "User Acceptance of Information Technology: Toward a Unified View," MIS Quarterly (27:3), pp. 425-478.
81. Vessey, I., and Conger, S. A. 1994. "Requkements Specification: Learning Object, Process, and Data Methodologies," Communications of the ACM (31:5), pp. 102-113.
82. Viswanathan, M. 2005. Measurement Error and Research Design, Thousand Oaks, CA: Sage Publications.
83. Webb, E., Campbell, D. T., Schwartz, R. D., and Sechrest, L. 2000. Unobtrusive Measures (Revised Edition), Thousand Oaks, CA: Sage Publications.
84. Westen, D., and Rosenthal, R. 2003. Quantifying Construct Validity: Two Simple Measures, Journal of Personality and Social Psychology (84:3), 608-618. (Pubitemid 38397275)
85. Williams, L. J., and Brown, B. K. 1994. "Method Variance in Organizational Behavior and Human Resources Research: Effects on Correlations, Path Coefficients, and Hypothesis Testing," Organizational Behavior and Human Decision Processes (57), pp. 185-209.
86. Wilson, D. B., and Lipsey, M. W. 2001. "The Role of Method in Treatment Effectiveness Research: Evidence from MetaAnalysis," Psychological Methods (6:4), pp. 413-429.
87. Wilson, T. D., and Dunn, E. W. 2004. Self-Knowledge: Its Limits, Value, and Potential for Improvement, Annual Review of Psychology (55), 493-518. (Pubitemid 40505736)
88. Woszczynski, A. B., and Whitman, M. E. 2004. "The Problem of Common Method Variance in IS Research," The Handbook of Information Systems Research, M. E. Whitman and A. B. Woszczynski (eds.), Hershey, PA: Idea Group Inc., pp. 66-77.