A New Twist on an Old Method: A Guide to the Applicability and Use of Web Experiments in Information Systems Research

Data Base for Advances in Information Systems

Volumn 36, Issue 3, 2005, Pages 69-88

  Wade, Michael R ^a   Tingling, Peter ^b  

^a YORK UNIVERSITY   (Canada)

^b SIMON FRASER UNIVERSITY   (Canada)

Author keywords

experimental research; Internet; research methodology; Web experiments

Indexed keywords

EID: 33751159605     PISSN: 00950033     EISSN: None     Source Type: Journal    
DOI: 10.1145/1080390.1080396     Document Type: Article

References (71)

1. Barnes, S., and Vidgen, R. (2001). “An evaluation of cyber-bookshops: The WebQual method,” International Journal of Electronic Commerce, Vol. 6, No. 3, pp. 11–30.
2. Batagelj, Z., Lozar, K., and Vehovar, V. (1998). “Who are Non-respondents in Web Surveys?,” Procedings of the 9th International Workshop on Household Survey Non-response, Bled, Slovenia, pp. 22–35.
3. Bauman, S., and Airey, J. (2000). “Effective Use of the Internet for Survey Research,” CASRO Journal Wirthlin Worldwide.(http://www.wirthlin.webaxxs.com/wintellitech/ca sro.htm)
4. Bayan, R. (2001). “Privacy means knowing your cookies,” Link-up, Vol. 18, No. 2, pp. 22–23.
5. Behling, O. (1980). “The Case for the Natural Science Model for Research in Organizational Behavior and Organizational Theory,” Academy of Management Review, Vol. 5, No. 4, pp. 483 490.
6. Benbasat, I., and Lim, L. (1993). “The effects of groups, task, context, and technology variables on the usefulness of group support systems: A meta-analysis of Experimental Studies,” Small Group Research, Vol. November, pp. 430–462.
7. Benbasat, I., and Zmud, B. (2003). “The identity crisis within the IS discipline: Defining and communicating the discipline's core properties,” MIS Quarterly, Vol. 29, No. 2, pp. 183–194.
8. Bikhchandani, S., Hirshleifer, D., and Welch, I. (1998). “Learning from the Behavior of Others: Conformity, Fads and Informational Cascades,” Journal of Economic Perspectives, Vol. 12, No. 3, pp. 151–170.
9. Birnbaum, M.H. (1999). “Testing critical properties of decision making on the Internet,” Psychological Science, Vol. 10, pp. 399–407.
10. Bolton, G.E. (1979). “Bargaining and Dilemma Games: From Laboratory Data Towards Theoretical Synthesis,” Experimental Economics, Vol. 1, No. 3, pp. 257–281.
11. Cappel, J., and Windsor, J. (1998). “A comparative investigation of ethical decision making: Information systems professionals versus students,” The DATA BASE for Advances in Information Systems, Vol. 29, No. 2, pp. 20–34.
12. Chua, C., Cao, L., Cousins, K., and Straub, D.W. (2003). “Assessing researcher production in Information Systems,” Journal of AIS, Vol. 3, No. 6, pp. 145–215.
13. Cobanoglu, C., Warde, B., and Moreo, P. (2001). “A comparison of mail, fax and Web-based survey methods,” International Journal of Market Research, Vol. 43, No. 4, pp. 441–452.
14. Comley, P. (1998). “On-line Research: Some methods, some problems, some case studies,” Association for Survey Computing Conference, Southampton, England, pp. 52–59.
15. Cook, T.D., and Campbell, D.T. (1979). Quasi experimentation: Design and analysis issues for field settings, Chicago: Rand McNally.
16. Couper, M. (2000). “Web surveys: A review of issues and approaches,” Public Opinion Quarterly, Vol. 64, pp. 464–494.
17. Dahan, E., and Srinivasan, V. (1998). “The predictive power of Internet-based product concept testing using visual depiction and animation,” Working Paper: 1502, Stanford Graduate School of Business.
18. Dennis, A., and Valacich, J. (2001). “Conducting Research in Information Systems,” Communications of the AIS, Vol. 7, No. 5, pp. 1 41.
19. Dickson, G.W., Senn, J.A., and Chervany, N.L. (1977). “Research in Management Information Systems: The Minnesota Experiments,” Management Science, Vol. 23, No. 9, pp. 913 923.
20. Fromkin, H., and Streufert, S. (1976). “Laboratory Experimentation,” in Dunnette, M. (Ed.), Handbook of Industrial and Organizational Psychology, Chicago, IL: Rand McNally, pp. 415–465.
21. Gefen, D., Karahanna, E., and Straub, D. (2003). “Trust and TAM in Online Shopping: An Integrated Model,” MIS Quarterly, Vol. 27, No. 1, pp. 51–90.
22. Gordon, M., Slade, L.A., and Schmitt, N. (1986). “The 'science of the Sophomore’ Revisited: From Conjecture to Empiricism,” Academy of Management Review, Vol. 11, No. 1, pp. 191 207.
23. Greenberg, J., Gordon, M., Slade, L.A., and Schmitt, N. (1987). “The College Sophomore as Guinea Pig: Setting the Record Straight/Student Guinea Pigs: Porcine Predictors and Particularistic Phenomena,” Academy of Management Review, Vol. 12, No. 1, pp. 157–163.
24. Hunter, J., Schmidt, F., and Jackson, G. (1982). Meta Analysis: Cumulating Research Findings Across Studies, Beverly Hills, CA: Sage Publications.
25. Hunton, J. E., and Beeler, J. D. (1997). “Effects of User Participation in Systems Development: A Longitudinal Field Experiment,” MIS Quarterly, Vol. 21, No. 4, pp. 359–388.
26. Jarvenpaa, S.L., Dickson, G.W., and DeSanctis, G. (1985). “Methodological issues in Experimental IS Research: Experiences and recommendations,” MIS Quarterly, Vol. 9, No. 2, pp. 141–156.
27. Judd, C., Smith, E., and Kidder, L. (1991). Research methods in social relations, 6th Ed., Fort Worth, TX: Harcourt Brace Jovanovich.
28. Kalfs, N. (1993). “Hour by hour: Effects of the data collection mode in time use research,” Amsterdam: NIMMO.
29. Kim, B., Barua, A., and Whinston, A. (2002). “Virtual field experiments for a digital economy: A new research methodology for exploring an information economy,” Decision Support Systems, Vol. 32, pp. 215–231.
30. Krantz, J., and Dalal, R. (2000). “Validity of Web-based psychological research,” in Birnbaum, M. (Ed.), Psychological Experiments on the Internet, Orlando, FL: Academic Press, pp. 33 60.
31. Kraut, A. (2001). “An e-mail letter to a friend,” The Industrial-Organizational Psychologist, Vol. 38, No. 4, pp. 37–39.
32. Latham, G.P., Erez, M., and Locke, E.A. (1988). “Resolving scientific disputes by the joint design of crucial experiments by the antagonists: Application to the Erez-Latham dispute regarding participation in goal setting,” Journal of Applied Psychology, Vol. 73, pp. 753–772.
33. Lee, B., Barua, A., and Whinston, A. (1997). “Discovery and representation of causal relationships in MIS research: A methodological framework,” MIS Quarterly, Vol. 21, No. 1, pp. 109–136.
34. Lynch, J. (1982). “On the external validity of experiments in consumer research: The concept of external validity,” Journal of Consumer Research, Vol. 9, No. 3, pp. 225–244.
35. Lynch, J. (1983). “The role of external validity in theoretical research/response/comment,” Journal of Consumer Research, Vol. 10, No. 1, pp. 109–124.
36. MacElroy, B. (2000). “Variables influencing dropout rates in Web-based surveys,” Quirk's Marketing Research Review, Vol. July. (http://www.quirks.com/articles/article.asp?arg_articleid=605)
37. McGraw, K., Tew, M., and Williams, J. (2000). “The integrity of Web-delivered experiments: Can you trust the data?,” Psychological Science, Vol. 11, No. 6, pp. 502–506.
38. Mechling, J. (1999). “Taking Intel Inside online: Extending an existing brand into a new channel,” Proceedings of the EXPLOR Forum, Madison, Madison, WI, pp. 245–253.
39. Miller, T., and Dickson, P. (2001). “On-line market research,” International Journal of Electronic Commerce, Vol. 5, No. 3, pp. 139–167.
40. Musch, J., and Reips, U. (2000). “A brief history of Web experimenting,” in Birnbaum, M.H. (Ed.), Psychological Experiments on the Internet, San Diego, CA: Academic Press, pp. 61–87.
41. Oppermann, M. (1995). “E-mail surveys: Potential and pitfalls,” Marketing Research, Vol. 7, No. 3, pp. 29–33.
42. Parker, L. (1992). “Collecting data the e-mail way,” Training and Development, Vol. 46, pp. 52–54.
43. Peng, W., and Cisna, J. (2000). “HTTP cookies - a promising technology,” Online Information Review, Vol. 24, No. 2, pp. 150–153.
44. Pitkow, J. E., and Recker, M. M. (1995). “Using the Web as a Survey Tool: Results from the Second WWW User Survey,” Computer Networks & ISDN Systems, Vol. 27, No. 6, pp. 809–822.
45. Pitta, D. (2001). “Internet Currency,” The Journal of Consumer Marketing, Vol. 18, No. 4/5, pp. 382 383.
46. Ray, N., and Tabor, S. (2003). “Smart marketers are overcoming the challenges of online research,” Marketing Research, Vol. Spring, pp. 32–37.
47. Reips, U. (2000). “The Web experiment method: Advantages, disadvantages, and solutions,” in Birnbaum, M.H. (Ed.), Psychological Experiments on the Internet, San Diego, CA: Academic Press, pp. 89–117.
48. Rose, G., Evaristo, R., and Straub, D. (2003). “Culture and consumer responses to Web download time: A four-continent study of mono and polychronism,” IEEE Transactions on Engineering Management, Vol. 50, No. 1, pp. 31–44.
49. Rose, G., and Straub, D. (2001). “The effect of download time on consumer attitude toward the e-service retailer,” e-Service Journal, Vol. 1, No. 1, pp. 55–76.
50. Rosenthal, R., and Rosnow, R.L. (1969). “The volunteer subject,” in Rosenthal, R., and Rosnow, R.L. (Eds.), Artifact in Behavioral Research, New York: Academic Press, pp. 59 118.
51. Saris, W. (1998). “Ten years of interviewing without interviewers: The telepanel,” in Couper, M. (Ed.), Computer-Assisted Survey Information Collection, New York: Wiley, pp. 409–429.
52. Schaefer, D.R., and Dillman, D.A. (1998). “Development of a standard e-mail methodology: Results of an experiment,” Public Opinion Quarterly, Vol. 62, No. 3, pp. 378–393.
53. Sheehan, K. B., and Hoy, M. G. (1999). “Using E mail To Survey Internet Users In The United States: Methodology And Assessment,” Journal of Computer-Mediated Communication, Vol. 4, No. 3. (http://www.ascusc.org/jcmc/vol4/issue3/sheeha n.html)
54. Smith, M., and Leigh, B. (1997). “Virtual subjects: Using the Internet as an alternative source of subjects and research environment,” Behavior Research Methods, Instruments and Computers, Vol. 29, No. 4, pp. 496–505.
55. Solso, R.L., Johnson, H.H., and Beal, M.L. (1998). Experimental Psychology (6th Ed), New York: Addison-Wesley.
56. Stone, Eugene F. (1978). Research Methods in Organizational Behavior, Glenview, Il: Scott, Foresman.
57. Straub, D.W., and Watson, R.T. (2001). “Transformational issues is researching IS and Net-enabled organizations,” Information Systems Research, Vol. 12, No. 4, pp. 337–345.
58. Thompson, L., Surface, E., Martin, D., and Sanders, M. (2003). “From paper to pixels: Moving Personnel surveys to the Web,” Personnel Psychology, Vol. 56, pp. 197–227.
59. Tingling, P., and Parent, M. (2002). “Mimetic Isomorphism and Technology Evaluation: Does Imitation Transcend Judgment?,” Journal of the AIS, Vol. 3, No. 5, pp. 113–143.
60. Tingling, P., Parent, M., and Wade, M. (2003). “Extending the capabilities of Internet-based research: Lessons from the field,” Internet Research, Vol. 13, No. 3, pp. 223–233.
61. Tourangeau, R., and Smith, T. (1998). “Collecting sensitive information with different modes of data collection,” in Couper, M. (Ed.), Computer- Assisted Survey Information Collection, New York: Wiley, pp. 431–453.
62. Vehovar, V., Manfreda, K.L., and Batagelj, Z. (2001). “Sensitivity of electronic commerce measurement to the survey instrument,” International Journal of Electronic Commerce, Vol. 6, No. 1, pp. 31–51.
63. Volkov, V. (1999). “Personalized Content Aggregation at Web Portals: Tailoring the Content Bundle Based on Content Provider's Credibility,” Unpublished Dissertation, Georgia State University.
64. Wade, M., and Parent, M. (2002). “Relationships between job skills and performance: A study of Webmasters,” Journal of Management Information Systems, Vol. 18, No. 3, pp. 71–96.
65. Ward, S.G., and Clark, J.M. (2002). “Bidding behavior in on-line auctions: An examination of the eBay Pokemon card market,” International Journal of Electronic Commerce, Vol. 6, No. 4, pp. 139.
66. Weick, K.E. (1965). “Laboratory Experimentation With Organizations,” in March, J. G. (Ed.), Handbook of Organizations, Chicago: Rand- McNally, pp. 194–260.
67. White, J., Carey, L., and Dailey, K. (2001). “Web-based instrumentation in educational survey research,” WebNet Journal: Internet Technologies, Applications & Issues Vol. 3, No. 1, pp. 46–50.
68. Wilke, J. (2000). “A landmark study of the differences between mall-intercept and Internet interviewing from bases,” Proceedings of the Eleventh Annual Advanced Research Techniques Forum, Monterey, CA.
69. Witt, K. J. (1998). “Best Practices in Interviewing via the Internet,” Proceedings of the Sawtooth Software Conference, Sequim, Washington, pp. 15–37.
70. Yavas, U. (1994), “Research Note: Students as Subjects in Advertising and Marketing Research,” International Marketing Review, Vol. 11, No. 4, pp. 35–43.
71. Zigurs, I., Buckland, B., Connelly, J., and Wilson, E.V. (1999). “A Test of Task-Technology Fit Theory for Group Support Systems,” The DATA BASE for Advances in Information Systems, Vol. 30, No. 3&4, pp. 34–50.