Differential effects of web-based and paper-based administration of questionnaire research instruments in authentic contexts-of-use

Journal of Educational Computing Research

Volumn 42, Issue 1, 2010, Pages 103-133

  Hardré, Patricia ^a   Crowson, H Michael ^a   Xie, Kui ^b  

^a UNIVERSITY OF OKLAHOMA   (United States)

^b MISSISSIPPI STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 76149137413     PISSN: 07356331     EISSN: None     Source Type: Journal    
DOI: 10.2190/EC.42.1.e     Document Type: Article

References (80)

1. Alessi, S. M., & Trollip, S. R. (2001). Multimedia for learning: Methods and development (3rd ed. ). Needham Heights, MA: Pearson.
2. Altemeyer, B. (2002). Dogmatic behavior among students: Testing a new measure of dogmatism. Journal of Social Psychology, 142, 713-721.
3. Bälter, K. A., Bälter, O., Fondell, E., & Lagerros, Y. T. (2005). Web-based and mailed questionnaires: A comparison of response rates and compliance. Epidemiology, 16(4), 577-579.
4. Bandilla, W., Bosnjak, M., & Altdorfer, P. (2003). Survey administration effects?: A comparison of web-based and traditional written self-administered surveys using the ISSP environment module. Social Science Computer Review, 21(2), 235-243.
5. Berge, Z. L., & Collins, M. P. (1996). IPT Journal readership survey. Journal of the American Society for Information Science, 47(9), 701-710.
6. Bertot, J. C., & McClure, C. R. (1996). Electronic surveys: Methodological implications for using the worldwide web to collect survey data. In S. Hardin (Ed. ), Proceedings of the 59th Annual Meeting of the American Society for Information Science (pp. 173-185). Medford, NJ: Information Today.
7. Booth-Kewley, S., Edwards, J. E., & Rosenfeld, P. (1992). Impression management, social desirability, and computer administration of attitude questionnaires: Does the computer make a difference? Journal of Applied Psychology, 77, 562-566.
8. Bosnjak, M., Tuten, T. L., & Wittmann, W. W. (2005). Unit (non)response in webbased access panel surveys: An extended planned-behavior approach. Psychology & Marketing, 22(6), 489-505.
9. Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42, 116-131.
10. Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 48, 306-307.
11. Carini, R. M., Hayek, J. C., Kuh, G. D., Kennedy, J. M., & Ouimet, J. A. (2003). College student responses to web and paper surveys: Does mode matter? Research in Higher Education, 44(1), 1-19.
12. Chua, S. L., Chen, D. T., & Wong, A. F. L. (1999). Computer anxiety and its correlates: A meta-analysis. Computers in Human Behavior, 15, 609-623.
13. Ciolek, T. M. (1998). The scholarly uses of the Internet: 1998 online survey. Available at: http://www. ciolek. com/PAPERS/InternetSurvey-98. html
14. Clariana, R., & Wallace, P. (2002). Paper-based versus computer-based assessment: Key factors associated with the test mode effect. British Journal of Educational Technology, 33(5), 593-602.
15. Cobanoglu, C., & Cobanoglu, N. (2003). The effect of incentives in web surveys: application and ethical considerations. International Journal of Market Research, 45(4), 475-488.
16. Comley, P. (1996). Internet surveys: The use of the internet as a data collection method, ESOMAR/EMAC: Research methodologies for the new marketing symposium. Esomar Public Services, 20, 335-346.
17. Cornelis, I., & Van Hiel, A. (2006). The impact of cognitive styles on the development of author authoritarianism based conservatism and racism. Basic and Applied Social Psychology, 28, 32-50.
18. Crowson, H. M. (2009). Are all conservatives alike? A study of psychological correlates of cultural and economic conservatism within a U. S. population sample. Journal of Psychology: Interdisciplinary and Applied, 143, 449-463.
19. DeBacker, T. K., Miller, R. B., Walker, C. O., & Mansell, R. (2004). Perceptions of classroom climate, student motivation, and achievement: Changes and interrelationships across an academic year. Paper presented at the Ninth International Conference on Motivation, Lisbon, Portugal.
20. Denscombe, M. (2008). The length of responses to open-ended questions: A comparison of online and paper questionnaires in terms of a mode effect. Social Science Computer Review, 26(3), 359-368.
21. De Witte, H. (1990). Conformism, radicalism and powerlessness: Examining Flemish workers' socio-cultural and socio-economic opinions. Leuven, Belgium: HIVA, K. U, Leuven.
22. Dillman, D. A. (2000). Navigating the rapids of change: Some observations on survey methodology in the early 21st century. Public Opinion Quarterly, 66(3), 473-494.
23. Dillman, D. A., & Bowker, D. K. (2001). The web questionnaire challenge to survey methodologists. In U. Reips & M. Bosnjak (Eds. ), Dimensions in internet science. Lengerich, Germany: Pabst Science Publishers.
24. Dillman, D. A., & Christian, L. M. (2005). Survey mode as a source of instability in responses across surveys. Field Methods, 17(1), 30-52.
25. Dodge, M. (2005). The role of maps in virtual research methods. In C. Hine (Ed. ), Virtual methods: Issues in social research on the internet (pp. 113-127). New York: Berg.
26. Dodge, M., & Kitchin, R. (2001). Mapping cyperspace. New York: Routledge.
27. Dolenko, M. (1998). Making user-based assessments work: Evaluation statistic Canada's Web site. In M. E. Williams (Ed. ), Proceedings of the 1998 National Online Meeting (pp. 105-114). New York: Information Today.
28. Ferriter, M. (1993). Computer-aided interviewing and the psychiatric social history. Social Work and Social Sciences Review, 4, 255-263.
29. Fletcher, L. A., Erickson, D. J., Toomey, T. L., & Wagenaar, A. C. (2003). Handheld computers: A feasible alternative to paper forms for field data collection. Evaluation Review, 27(2), 165-178.
30. Fox, S., & Schwartz, D. (2002). Social desirability and controllability in computerized and paper-and-pencil personality questionnaires. Computers in Human Behavior, 18, 389-410.
31. Goree, D. T., & Marszalek, J. F. III. (1995). Electronic surveys: Ethical issues for researchers. College Student Affairs Journal, 15(1), 75-79.
32. Granello, D. H., & Wheaton, J. E. (2004). Online data collection: Strategies for research. Journal of Counseling and Development, 82(4), 387-393.
33. Greene, B. A., Dillon, C., & Crynes, B. (2003). Distributive learning in introductory chemical engineering: University students' learning, motivation, and attitudes using a CD-ROM. Journal of Educational Computing Research, 29, 189-207.
34. Ha, T. S., Marsh, J., & Jones, J. (1998). A web-based system for teaching evaluation. Paper presented at the Lingnam College 30th Anniversary Conference on New Challenges and Innovations in Teaching and Training into the 21st Century (NCITT '98), May 1998, Hong Kong.
35. Hardré, P. L. (2003). The effects of instructional design training on university teaching assistants. Performance Improvement Quarterly, 16(4), 23-39.
36. Hardré, P. L., Crowson, H. M., Ly, C., & Xie, K. (2007). Testing differential effects of computer-based, web-based, and paper-based administration of questionnaire research instruments. British Journal of Educational Technology, 38(1), 5-22.
37. Hardré, P., & Reeve, J. (2003). A motivational model of rural students' intentions to persist in, versus drop out, of high school. Journal of Educational Psychology, 95(2), 347-356.
38. Hardré, P. L., & Sullivan, D. (2008). Teachers' perceptions and individual differences: How they influence teachers' motivating strategies. Journal of Teaching and Teacher Education, 24(8), 2059-2075.
39. Hardré, P., Xie, K., & Ly, C. (2005). Production and data management issues of digital administration of research questionnaires. Performance Improvement, 44(5), 33-39.
40. Hine, C. (2005). Research-relationships and online relationships (Introduction). In C. Hine (Ed. ), Virtual methods: Issues in social research on the internet (pp. 17-20). New York: Berg.
41. Honaker, L. M. (1988). The equivalency of computerized and conventional MMPI administration: A critical review. Clinical Psychology Review, 8, 561-577.
42. Joinson, A. N. (2005). Internet behavior and the design of virtual methods. In C. Hine (Ed. ), Virtual methods: Issues in social research on the internet (pp. 21-34). New York: Berg.
43. Kiesler, S., & Sproull, L. (1986). Response effects in the electronic survey. Public Opinion Quarterly, 50, 243-253.
44. Kline, R. B. (2005). Principles and practice of structural equation modeling. New York: Guilford.
45. Leece, P., Bhandari, M., Sprague, S., Swiontkowski, M. F., Schemitsch, E. H., Tornetta, P., et al. (2004). Internet versus mailed questionnaires: A randomized comparison. Journal of Medical Internet Research, 6(3), e30.
46. MacEachren, A. M. (1995). How maps work: Representation, visualization and design. New York: Guilford Press.
47. MacIsaac, D. L. (1999). Comparing web and paper-based administration of standardized introductory physics conceptual and attitudinal instruments. Final Program and Abstracts: 72nd NARST Annual Meeting 1999, 204.
48. MacIsaac, D. L., Cole, R. P., Cole, D. M., McCullough, L., & Maxka, J. (2002). Standardized testing in physics via the world wide web. Electronic Journal of Science Education, 6, 3.
49. MacKay, H. (2005). New connections, familiar settings: Issues in the ethnographic study of new media use at home. In C. Hine (Ed. ), Virtual methods: Issues in social research on the internet (pp. 129-140). New York: Berg.
50. McCabe, S. E. (2004). Comparison of web and mail surveys in collecting illicit drug use data: A randomized experiment. Journal of Drug Education, 34(1), 61-72.
51. Mehta, R., & Sivadas, E. (1995). Comparing response rates and response content in mail versus electronic mail surveys. Journal of the Market Research Society, 37(4), 429-439.
52. Miller, R. B., Greene, B. A., Montalvo, G. P., Ravindran, B., & Nichols, J. D. (1996). Engagement in academic work: The role of learning goals, future consequences, pleasing others, and perceived ability. Contemporary Educational Psychology, 21, 388-422.
53. Miller, D., & Slater, D. (2000) The internet: An ethnographic approach. London: Berg.
54. Moon, Y. (2000). Intimate exchanges: Using computers to elicit self-disclosure from consumers. Journal of Consumer Research, 26, 323-339.
55. Nass, C., Fogg, B. J. & Moon, Y. (1996). Can computers be teammates? International Journal of Human-Computer Studies, 45(6), 669-678.
56. Nichols, E., & Sedivi, B. (1998). Economic data collection via the web: A census bureau case study. Paper presented at the meeting of the American Statistical Association, Dallas, Texas.
57. Noyes, J., Garland, K., & Robbins, L. (2004). Colloquium. British Journal of Educational Technology, 35(1), 111-113.
58. Porter, S. R., & Whitcomb, M. E. (2003). The impact of contact type on web survey response rates. Public Opinion Quarterly, 67, 579-588.
59. Ritter, P., Lorig, K., Laurent, D., & Matthews, K. (2004). Internet versus mailed questionnaires: A randomized comparison. Journal of Medical Internet Research, 6(3), e29.
60. Robinson, R., & West, R. (1992). A comparison of computer and questionnaire methods of history-taking in a genito-urinary clinic. Psychology and Health, 6, 77-84.
61. Sarrazin, M. S. V., Hall, J. A., Richards, C., & Carswell, C. (2002). A comparison of computer-based versus pencil-and-paper assessment of drug use. Research on Social Work Practice, 12, (5), 669-683.
62. Sax, L., Gilmartin, S., & Bryant, A. (2003). Assessing response rates and nonresponse bias in web and paper surveys. Research in Higher Education, 44(4), 409-432.
63. Schonlan, M., Fricker, R. D., Jr., & Elliott, M. N. (2002). Conducting research surveys via email and the web. Santa Monica, CA: Rand.
64. Schulenberg, S. E., & Yurtzenka, B. A. (1999). The equivalence of computerized and paper-and-pencil psychological instruments: Implications for measures of negative affect. Behavior Research Methods, Instruments and Computers, 31(2), 315-321.
65. Sears, G. J., Prakash, S., &Chiocchio, F. (2003). The world wide web: An added dimension in 360° feedback? Report of the Public Service Commission of Canada. Available at: http://www/psc-cfp.gc.ca/ppc/world-wide-web- e.htm
66. Spink, A., Bateman, J., & Jansen, B. J. (1998). Users' searching behavior on the Excite web search engine. In M. E. Williams (Ed. ), Proceedings of the 1998 National Online Meeting (pp. 375-386). New York: Information Today.
67. Stevens, J. P. (2002). Applied multivariate statistics for the social sciences (4th ed. ). Mahwah, NJ: Erlbaum.
68. Thompson, L. F., Surface, E. A., Martin, D. L., & Sanders, M. G. (2003). From paper to pixels: Moving personnel surveys to the web. Personnel Psychology, 56, 197-227.
69. Tourangeau, R. (2004). Survey research and societal change. Annual Review of Psychology, 55, 775-801.
70. Truman, J., Robinson, K., Evans, A. L., Smith, D., Cunningham, L., Millward, R., et al. (2003). The strengths and difficulties questionnaire: A pilot study of a new computer version of the self-report scale. European Child and Adolescent Psychiatry, 12, 9-14.
71. Tschannen-Moran, M., Woolfolk-Hoy, A., & Hoy, W. (1998). Teacher efficacy: Its meaning and measure. Review of Educational Research, 68, 202-248.
72. Tseng, H., Tiplady, B., MacLeod, H. A., & Wright, P. (1998). Computer anxiety: A comparison of pen-based personal digital assistants, conventional computer, and paper assessment of mood and performance. British Journal of Psychology, 89, 599-610.
73. Vallerand, R. J., Fortier, M. S., & Guay, F. (1997). Self-determination and persistence in a real-life setting: Toward a motivational model of high school dropout. Journal of Personality and Social Psychology, 72(5), 1161-1176.
74. Van Hiel, A., & Kossowsky, M. (2007). Contemporary attitudes and their ideological representation in Flanders (Belgium), Poland, and the Ukraine. International Journal of Psychiatry, 42, 16-26.
75. Wallace, P. (1999). The psychology of the internet. Cambridge: Cambridge University Press.
76. Webster, J., & Compeau, D. (1996). Computer-assisted versus paper-and-pencil administration of questionnaires. Behavior Research Methods, Instruments & Computers, 28(4), 567-576.
77. Whittier, D. K., Seeley, S., & St. Lawrence, J. S. (2004). A comparison of web with paper-based surveys of gay and bisexual men who vacationed in a gay resort community. AIDS Education and Prevention, 16(5), 476-485.
78. Williams, G. C., & Deci, E. L. (1996). Internalization of biopsychological values by medical students: A test of self-determination theory. Journal of Personality and Social Psychology, 70, 767-779.
79. Yook, S. H., Jeong, H., & Barabási A. -L. (2002). Modeling the internet's large-scale topology. Proceedings of the National Academy of Sciences, 99, 13382-13386.
80. Zhang, Y. (1999). Using the internet for survey research: A case study. Journal of American Society for Information Science, 51(1), 57-68.