Countering method bias in questionnaire-based user studies

Journal of Documentation

Volumn 67, Issue 3, 2011, Pages 507-524

  Gorrell, Genevieve ^a   Ford, Nigeln ^a   Madden, Andrew ^a   Holdridge, Peter ^a   Eaglestone, Barry ^a  

^a UNIVERSITY OF SHEFFIELD   (United Kingdom)

Author keywords

Research methods; Tests and testing; Variance

Indexed keywords

EID: 79955628271     PISSN: 00220418     EISSN: None     Source Type: Journal    
DOI: 10.1108/00220411111124569     Document Type: Review

References (53)

1. Alexander, P.A. and Judy, J.E. (1988), "The interaction of domain-specific and strategic knowledge in academic performance" in Review of Educational Research, Vol. 58, No. 4, pp. 375-404.
2. Bernerth, J.B., Armenakis, A.A., Feild, H.S., Giles, W.F. and Walker, H.J. (2008), "The influence of personality differences between subordinates and supervisors on perceptions of LMX: an empirical investigation" in Group Organization Management, Vol. 33, No. 2, pp. 216-40.
3. Brewer, M.B., Campbell, D.T. and Crano, W.D. (1970), "Testing a single factor model as an alternative to the misuse of partial correlations in hypothesis-testing research" in Sociometry, Vol. 33, pp. 1-8.
4. Brown, A. (1987), "Metacognition, executive control, self-regulation, and other more mysterious mechanisms", Metacognition, Motivation, and Understanding, Lawrence Earlbaum Associates, Hillsdale, NJ.
5. Brown, A.L., Bransford, J.D., Ferrara, R.A. and Campione, J.C. (1983), "Learning, remembering, and understanding" in Flavell, J.H. and Markman, E.M. (Eds.), Handbook of Child Psychology, 3, Cognitive Development, 4th ed., Wiley, New York, NY, pp. 77-166.
6. Bstieler, L. and Hemmert, M. (2008), "Influence of tie strength and behavioural factors on effective knowledge acquisition: a study of Korean new product alliances" in Asian Business and Management, Vol. 7, pp. 75-94.
7. Campbell, D.T. and Fiske, D.W. (1959), "Convergent and discriminant validation by the multitrait-multimethod matrix" in Psychological Bulletin, Vol. 58, pp. 56-81.
8. Carr, A.S. and Kaynak, H. (2007), "Communication methods, information sharing, supplier development and performance" in International Journal of Operations & Production Management, Vol. 27, No. 4, pp. 346-70.
9. Carr, A. and Muthusamy, S.K. (2008), "The cross-functional coordination between operations, marketing, purchasing and engineering and the impact on performance" in International Journal of Manufacturing Technology and Management, Vol. 13, No. 1, pp. 55-77.
10. Çetinkaya, P. and Ertkin, E. (2002), "Assessment of metacognition and its relationship with reading comprehension, achievement, and aptitude" in Boǧaziçi University Journal of Education, Vol. 19, No. 1, pp. 1-11.
11. Chi, L., Jones, K.G., Lederer, A.L., Li, P., Newkirk, H.E. and Sethi, V. (2004), "Environmental assessment in strategic information systems planning" in International Journal of Information Management, Vol. 25, No. 3, pp. 253-69.
12. Chungtai, A.A. (2008), "Impact of job involvement on in-role job performance and organizational citizenship behaviour" in Journal of the Institute of Behavioral and Applied Management, Vol. 9, No. 2, pp. 169-83.
13. Cote, J.A. and Buckley, M.R. (1987), "Estimating trait, method, and error variance: generalizing across 70 construct validation studies" in Journal of Marketing Research, Vol. 24, No. 3, pp. 315-18.
14. Crampton, S.A. and Wagner, J.A. (1994), "Percept-percept inflation in micro-organizational research: an investigation of prevalence and effect" in Journal of Applied Psychology, Vol. 79, pp. 67-76.
15. Cronbach, L.J. (1951), "Coefficient alpha and the internal structure of tests" in Psychometrika, Vol. 16, No. 3, pp. 297-334.
16. Darnall, N., Jolley, G.J. and Handfield, R. (2008), "Environmental management systems and green supply chain management: complements for sustainability?" in Business Strategy and the Environment, Vol. 18, pp. 30-45.
17. De Cuyper, N., Bernhard-Oettel, C., Berntson, E., De Witte, H. and Alarco, B. (2008), "Employability and employees' wellbeing: mediation by job insecurity" in Applied Psychology, Vol. 57, No. 3, pp. 488-509.
18. Doty, D.H. and Glick, W.H. (1998), "Common methods bias: does common methods variance really bias results?" in Organizational Research Methods, Vol. 1, pp. 374-406.
19. Flavell, J. (1979), "Metacognition and cognitive monitoring: a new area of cognitive-developmental inquiry" in American Psychologist, Vol. 34, pp. 906-11.
20. Ford, N. (2004), "Modeling cognitive processes in information seeking: from Popper to Pask" in Journal of the American Society for Information Science and Technology, Vol. 55, No. 9, pp. 769-82.
21. Gorrell, G., Eaglestone, B., Ford, N., Holdridge, P. and Madden, A. (2008), "The role of meta-cognition in web searching to support inquiry-based learning" in Proceedings of the ECEL, 2008, pp. 443-50.
22. Gorrell, G., Madden, A., Holdridge, P., Ford, N. and Eaglestone, B. (2009), "Towards 'metacognitively aware' IR systems: an initial user study" in Journal of Documentation, Vol. 65, No. 3, pp. 446-69.
23. Greene, C.N. and Organ, D.W. (1973), "An evaluation of causal models linking the received role with job satisfaction" in Administrative Science Quarterly, Vol. 18, No. 1, pp. 95-103.
24. Hannafin, M.J., Land, S. and Oliver, K. (1999), "Open learning environments: foundations, methods, and models" in Reigeluth, C.M. (Ed.), Instructional-Design Theories and Models: A New Paradigm of Instruction Theory, Erlbaum, Hillsdale, NJ, pp. 115-40.
25. Harman, H.H. (1967), Modern Factor Analysis, University of Chicago Press, Chicago, IL.
26. Hill, J.R. and Hannafin, M.J. (1997), "Cognitive strategies and learning from the world wide web" in Educational Technology Research and Development, Vol. 45, No. 4, pp. 37-64.
27. Jimmieson, N.L., Peach, M. and White, K.M. (2008), "Utilizing the theory of planned behaviour to inform change management: an investigation of employee intentions to support organizational change" in Journal of Applied Behavioural Science, Vol. 44, No. 2, pp. 237-62.
28. Karatepe, O.M. and Magaji, A. (2008), "Work-family conflict and facilitation in the hotel industry: a study in Nigeria" in Cornell Hospitality Quarterly, Vol. 49, No. 4, pp. 395-412.
29. Kemery, E.R. and Dunlap, W.P. (1986), "Partialling factor scores does not control method variance: a reply to Podsakoff and Todor" in Journal of Management, Vol. 12, No. 4, pp. 525-30.
30. Kim, Y.J., Song, J. and Koo, C. (2008), "Exploring the effect of strategic positioning on firm performance in the e-business context" in Journal of Information Management, Vol. 28, No. 3, pp. 203-14.
31. Kourteli, L. (2005), "Scanning the business external environment for information: evidence from Greece", Information Research, Vol. 11, No. 1.
32. Lau, A.Y.S. and Coiera, E.W. (2008), "Impact of web searching and social feedback on consumer decision making: a prospective online experiment" in Journal of Medical Internet Research, Vol. 10, No. 1, p. e2.
33. Lindell, M.K. and Whitney, D.J. (2001), "Accounting for common method variance in cross-sectional research designs" in Journal of Applied Psychology, Vol. 86, No. 1, pp. 114-21.
34. Malhotra, N.K., Kim, S. and Patil, A. (2006), "Common method variance in IS research: a comparison of alternative approaches and a reanalysis of past research" in Management Science, Vol. 52, No. 12, pp. 1865-83.
35. Mansourian, Y. (2008), "Contextual elements and conceptual components of information visibility on the web" in Library Hi Tech, Vol. 26, No. 3, pp. 440-53.
36. Metcalfe, J. and Shimamura, A.P. (1994), Metacognition: Knowing about Knowing, MIT Press, Cambridge, MA.
37. Muis, K.R., Winne, P.H. and Jamieson-Noel, D. (2007), "Using a multitrait-multimethod analysis to examine conceptual similarities of three self-regulated learning inventories" in British Journal of Educational Psychology, Vol. 77, No. 1, pp. 177-95.
38. Newkirk, H.E., Lederer, A.L. and Johnson, A.M. (2008), "Rapid business and IT change: drivers for strategic information systems planning?" in European Journal of Information Systems, Vol. 17, pp. 198-218.
39. O'Neil, H.F. and Abedi, J. (1996), Reliability and Validity of a State Metacognitive Inventory: Potential for Alternative Assessment, National Center for Research on Evaluation, Standards, and Student Testing (CRESST), University of California, Los Angeles, CA, CSE Tech. Rep. 469.
40. Paulraj, A., Chen, I.J. and Flynn, J. (2006), "Levels of strategic purchasing: impact on supply integration and performance" in Journal of Purchasing and Supply Management, Vol. 12, No. 3, pp. 107-22.
41. Podsakoff, P.M. and Organ, D.W. (1986), "Self-reports in organizational research: problems and prospects" in Journal of Management, Vol. 12, No. 2, pp. 531-44.
42. Podsakoff, P.M. and Todor, W.D. (1985), "Relationships between leader reward and punishment behavior and group processes and productivity" in Journal of Management, Vol. 11, No. 1, pp. 55-73.
43. Podsakoff, P.M., MacKenzie, S.B., Lee, J.Y. and Podsakoff, N.P. (2003), "Common method biases in behavioral research: a critical review of the literature and recommended remedies" in Journal of Applied Psychology, Vol. 88, No. 5, pp. 879-903.
44. Podsakoff, P.M., Todor, W.D., Grover, R.A. and Huber, V.L. (1984), "Situational moderators of leader reward and punishment behaviors: fact or fiction?" in Organizational Behavior and Human Performance, Vol. 34, pp. 21-63.
45. Rego, A., Sousa, F., Pina e Cunha, M., Correia, A. and Saur-Amarai, I. (2007), "Leader self-reported emotional intelligence and perceived employee creativity: an exploratory study" in Creativity and Innovation Management, Vol. 16, No. 3, pp. 250-64.
46. Schraw, G. and Dennison, R.S. (1994), "Assessing metacognitive awareness" in Contemporary Educational Psychology, Vol. 19, pp. 460-75.
47. Schriesheim, C.A. (1979), "The similarity of individual-directed and group-directed leader behavior descriptions" in Academy of Management Journal, Vol. 22, No. 2, pp. 345-55.
48. Schriesheim, J.F. (1980), "The social context of leader-subordinate relations: an investigation of the effects of group cohesiveness" in Journal of Applied Psychology, Vol. 65, No. 2, pp. 183-94.
49. Song, M., Dyer, B. and Thieme, R.J. (2006), "Conflict management and innovation performance: an integrated contingency perspective" in Journal of the Academy of Marketing Science, Vol. 34, No. 3, pp. 341-56.
50. Tang, T.L.P., Sutarso, T., Akande, A. and Allen, M.W. (2006), "The love of money and pay level satisfaction: measurement and functional equivalence in 29 geopolitical entities around the world" in Management and Organizational Review, Vol. 2, No. 3, pp. 423-52.
51. Thacker, R.A. and Wayne, S.J. (1995), "An examination of the relationship between upward influence tactics and assessments of promotability" in Journal of Management, Vol. 21, No. 4, pp. 739-56.
52. Woszczynski, A.B. and Whitman, M.E. (2004), "The problem of common method variance in IS research" in Woszczynski, A.B. and Whitman, M.E. (Eds.), The Handbook of Information Systems Research, Idea Group, Hershey, PA, pp. 66-77.
53. Zhang, X. and Chen, R. (2008), "Examining the mechanism of the value co-creation with customers" in International Journal of Production Economics, Vol. 116, No. 2, pp. 242-50.