Online panel research: History, concepts, applications and a look at the future

Online Panel Research: A Data Quality Perspective

Volumn , Issue , 2014, Pages 1-22

  Callegaro, Mario ^a   Baker, Reg ^b   Bethlehem, Jelke ^c,d   Göritz, Anja S ^e   Krosnick, Jon A ^f   Lavrakas, Paul J ^g  

^a GOOGLE   (United Kingdom)

^b Market Strategies International ^*  (United States)

^c Division of Process Development IT and Methodology Statistics Netherlands   (Netherlands)

^d LEIDEN UNIVERSITY   (Netherlands)

^e UNIVERSITY OF FREIBURG   (Germany)

^f STANFORD UNIVERSITY   (United States)

^g NORTHERN ARIZONA UNIVERSITY   (United States)

Author keywords

Attrition; Future of online panels; History of online panels; Incentives; Industry standards; Non probability based panels; Panel types; Probability based panels; Recruitment; Sampling and weighting

Indexed keywords

EID: 84927800439     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781118763520.ch1     Document Type: Chapter

References (78)

1. American Association for Public Opinion Research. (2011). Final dispositions of case codes and outcomes rates for surveys (7th ed.). Deerfield, IL: AAPOR.
2. AMSRO. (2011). Background: QSOAP. Retrieved January 1, 2013, from: http://www.amsro.com.au/background-qsoap.
3. ARF. (2011). Online survey research: Findings, best practices, and future research. Paper presented at the Research Quality Forum, New York. Retrieved January 1, 2013, from: http://my.thearf.org/source/custom/downloads/2011-04-07_ARF_RQ_Presentation.pdf.
4. Baker, R., Blumberg, S. J., Brick, J. M., Couper, M. P., Courtright, M., Dennis, J. M., Dillman, D. A., et al. (2010). Research synthesis: AAPOR report on online panels. Public Opinion Quarterly, 74, 711-781.
5. Baker, R., Brick,M. J., Bates, N., Battaglia,M. P., Couper, M. P., Deever, J. A., Gile, K. J., et al. (2013). Non-probability sampling: AAPOR task force report. Deerfield, IL: AAPOR.
6. Bethlehem, J., & Biffignandi, S. (2012). Handbook of web surveys. Hoboken, NJ: John Wiley & Sons, Inc.
7. Bethlehem, J., Cobben, F., & Schouten, B. (2011). Handbook of nonresponse in household surveys. Hoboken, NJ: John Wiley & Sons, Inc.
8. Börsch-Supan, A., Elsner, D., Faβbender, H., Kiefer, R., McFadden, D., & Winter, J. (2007). How to make internet surveys representative: A case study of a two-step weighting procedure. In CenterData workshop: Measurement and experimentation with Internet panels. The state of the art of Internet interviewing. Tilburg University: CenterData. Retrieved January 1, 2013, from: http://www.mea.mpisoc.mpg.de/uploads/user_mea_discussionpapers/loil50ozz320r55b_pd1_040330%20geschuetzt.pdf.
9. Callegaro, M., & DiSogra, C. (2008). Computing response metrics for online panels. Public Opinion Quarterly, 72, 1008-1032.
10. Clarke, H. D., Sanders, D., Stewart, M. C., & Whiteley, P. (2008). Internet surveys and national election studies: A symposium. Journal of Elections, Public Opinion & Parties, 18, 327-330.
11. Comley, P. (2007). Online market research. In M. van Hamersveld & C. de Bont (Eds.), Market research handbook (5th ed., pp. 401-419). Chichester: John Wiley & Sons, Ltd.
12. Couper, M. P. (2000). Web surveys: A review of issues and approaches. Public Opinion Quarterly, 64, 464-494.
13. Couper, M. P. (2007). Issues of representation in eHealth research (with a focus on web surveys). American Journal of Preventive Medicine, 32(5S), S83-S89.
14. Das, M. (2012). Innovation in online data collection for scientific research: The Dutch MESS project. Methodological Innovations Online, 7, 7-24.
15. Delmas, D., & Levy, D. (1998). Consumer panels. In C. McDonald, & P. Vangelder (Eds.), ESOMAR handbook of market and opinion research (4th ed., pp. 273-317). Amsterdam: ESOMAR.
16. DiSogra, C., & Callegaro, M. (2010). Computing response rates for probability-based online panels. In AMSTAT (Ed.), Proceedings of the Joint Statistical Meeting, Survey Research Methods section (pp. 5309-5320). Alexandria, VA: AMSTAT.
17. DiSogra, C., Callegaro, M., & Hendarwan, E. (2010). Recruiting probability-based web panel members using an Address-Based Sample frame: Results from a pilot study conducted by Knowledge Networks. In Proceedings of the Annual Meeting of the American Statistical Association (pp. 5270-5283). Paper presented at the 64th Annual conference of the American Association for Public Opinion Research, Hollywood, FL: AMSTAT.
18. Duffy, B., Smith, K. Terhanian, G., & Bremer, J. (2005). Comparing data from online and face-to-face surveys. International Journal of Market Research, 47, 615-639.
19. Eggers,M., & Drake, E. (2011). Blend, balance, and stabilize respondent sources. Paper presented at the 75th Annual Conference of the Advertising Research Foundation, New York: ARF. Retrieved January 1, 2013, from: http://thearf-org-aux-assets.s3.amazonaws.com/annual/presentations/kif/04_D1_KIF _Eggers_Drake_v04.pdf.
20. EKOS. (2011). What is Probit? Retrieved from: http://www.probit.ca/?page_id=.7
21. ESOMAR. (2011). ESOMAR guideline for online research. Retrieved July 1, 2011, from: http://www .esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Guideline -for-online-research.pdf.
22. ESOMAR. (2012). 28 Questions to help research buyers of online samples. Retrieved December 12, 2012, from: http://www.esomar.org/knowledge-and-standards/research-resources/28-questions-on -online-sampling.php.
23. Faasse, J. (2005). Panel proliferation and quality concerns. Paper presented at the ESOMAR panel research conference, Budapest.
24. Gittelman, S., & Trimarchi, E. (2010). Online research... and all that jazz! The practical adaption of old tunes to make new music. ESOMAr Online Research 2010. Amsterdam: ESOMAR.
25. Gittelman, S., Trimarchi, E., & Fawson, B. (2011). A new representative standard for online research:Conquering the challenge of the dirty little "r" word. Presented at the ARF Key Issue Forum, Re:Think conference, New York. Retrieved January 1, 2013, from: http://www.mktginc.com/pdf /Opinionology%20and%20Mktg%20Inc%20%20ARF%202011_March.pdf.
26. GMI. (2012). GMI Pinnacle. Retrieved January 1, 2013, from: http://www.gmi-mr.com/uploads/file /PDFs/GMI_Pinnacle_10.7.10.pdf.
27. Göritz, A. S. (2014). Incentive effects. Chapter 28 In U. Engel, B. Jann, P. Lynn, A. Scherpenzeel, & P. Sturgis (Eds.), Improving survey method. New York: Taylor & Francis.
28. Göritz, A. S. (2007). Using online panels in psychological research. In A. N. Joinson, K. Y. A. McKenna, T. Postmes, & U.-D. Reips (Eds.), The Oxford handbook of Internet psychology (pp. 473-485). Oxford: Oxford University Press.
29. Göritz, A. S. (2010). Web panels: replacement technology for market research. In T. L. Tuten (Ed.), Enterprise 2.0: How technology, eCommerce, and Web 2.0 are transforming business virtually (Vols. 1, 2, Vol. 1, pp. 221-236). Santa Barbara, CA: ABC-CLIO.
30. Göritz, A. S., Reinhold, N., & Batinic, B. (2002). Online panels. In B. Batinic, U.-D. Reips, & M. Bosnjak (Eds.), Online social sciences (pp. 27-47). Seattle, WA: Hogrefe & Huber.
31. Hansen, J. (2008). Panel surveys. In M.W. Traugott&W. Donsbach (Eds.), The Sage handbook of public opinion research (pp. 330-339). Thousand Oaks, CA: Sage.
32. Harding, D., & Jackson, P. (2012). Quality in market research: From theory to practice (2nd ed.). London:British Standards Institution.
33. Horrigan, J. B. (2010). Broadband adoption and use in America (No. OBI working papers series No 1). Federal Communication Commission. Retrieved from: http://hraunfoss.fcc.gov/edocs_public /attachmatch/DOC-296442A1.pdf.
34. Horvitz, D. G., & Thompson, D. J. (1952). A generalization of sampling without replacement from a finite universe. Journal of the American Statistical Association, 47, 663-685.
35. Inside Research. (2012). Worldwide online research spending. Inside Research, March, 5.
36. International Organization for Standardization. (2009). ISO 26362 Access panels in market, opinion, and social research: Vocabulary and service requirements. Geneva: ISO.
37. International Organization for Standardization. (2012). ISO 20252 Market, opinion and social research:Vocabulary and service requirements (2nd ed.). Geneva: ISO.
38. Knowledge Networks. (2011). Knowledge panel design summary. Retrieved August 6, 2011, from: http://www.knowledgenetworks.com/knpanel/KNPanel-Design-Summary.html.
39. Krosnick, J. A., Ackermann, A., Malka, A., Sakshaug, J., Tourangeau, R., De Bell, M., & Turakhia, C. (2009). Creating the Face-to-Face Recruited Internet Survey Platform (FFRISP). Paper presented at the Third Annual Workshop on Measurement and Experimentation with Internet Panels, Santpoort, the Netherlands.
40. Kruse, Y., Callegaro, M., Dennis, M. J., DiSogra, C., Subias, T., Lawrence, M., & Tompson, T. (2010). Panel conditioning and attrition in the AP-Yahoo news election panel study. In Proceedings of the Joint Statistical Meeting, American Association for Public Opinion Research Conference (pp. 5742-5756). Washington, DC: AMSTAT.
41. Lavrakas, P. J., Dennis, M. J., Peugh, J., Shand-Lubbers, J., Lee, E., & Charlebois, O. (2012). Experimentingwith noncontingent and contingent incentives in a mediameasurement panel. Paper presented at the 67thAnnual conference of theAmerican Association for Public Opinion Research,Orlando, FL.
42. Lee, S., & Valliant, R. (2009). Estimation for volunteer panel web surveys using propensity score adjustment and calibration adjustment. Sociological Methods Research, 37, 319-343.
43. Lozar Manfreda, K., & Vehovar, V. (2008). Internet surveys. International handbook of survey methodology (pp. 264-284). New York: Lawrence Erlbaum.
44. MarketTools. (2009). MarketTools TrueSample. Retrieved January 1, 2013, from: http://www .truesample.net/marketing/DataSheetTrueSample.pdf.
45. Miller, J. (2010). The state of online research in the U.S. Paper presented at the MRIA Net Gain 4.0, Toronto, Ontario.
46. MRIA. (2013). Ten questions to ask your online survey provider. Retrieved January 1, 2013, from: http://www.mria-arim.ca/STANDARDS/TenQuestions.asp.
47. MRS. (2012, January). MRS Guidelines for online reseach. Retrieved January 1, 2013, from: http://www.mrs.org.uk/pdf/2012-02-16%20Online%20Research%20Guidelines.pdf.
48. Napoli, P. M. (2011). Audience evolution: New technologies and the transformation of media audiences. New York: Columbia University Press.
49. National Telecommunication and Information Administration. (2013). Exploring the digital nation:America's emerging online experience. Washington, DC: U.S. Department of Commerce. Retrieved from: http://www.ntia.doc.gov/files/ntia/publications/exploring_the_digital_nation_-_americas _emerging_online_experience.pdf.
50. Nunan, D., & Knox, S. (2011). Can search engine advertising help access rare samples? International Journal of Market Research, 53, 523-540.
51. Postoaca, A. (2006). The anonymous elect: Market research through online access panels. Berlin:Springer.
52. Poynter, R. (2010). The handbook of online and social media research: Tools and techniques for market researchers. Chichester: John Wiley & Sons, Ltd.
53. PR Newswire Associaton. (2000). DMS/AOL's Opinion Place expands research services to offer broadest online representation available. Retrieved January 1, 2013, from: http://www.thefreelibrary .com/DMS%2FAOL%27s+Opinion+Place+Expands+Research+Services+to+Offer+Broadest... -a066296354.
54. Public Works and Government Services Canada. (2008). The advisory panel on online public opinion survey quality: Final report June 4, 2008. Ottawa: Public Works and Government Services Canada. Retrieved from: http://www.tpsgc-pwgsc.gc.ca/rop-por/rapports-reports/comiteenligne -panelonline/tdm-toc-eng.html.
55. Ragnedda, M., & Muschert, G. (Eds.). (2013). The digital divide: The internet and social inequality in international perspective. New York: Routledge.
56. Research Live. (2006, November 29). Greenfield unveils real-time sampling. Retrieved January 1, 2013, from: http://www.research-live.com/news/greenfield-unveils-real-time-sampling/3002563.article.
57. Rivers, D. (2006). Understanding people: Sample matching. Retrieved January 1, 2013, from: http://psfaculty.ucdavis.edu/bsjjones/rivers.pdf.
58. Rivers, D. (2007). Sampling for web surveys. Joint Statistical Meeting, section on Survey Research Methods. Paper presented at the 2007 Joint Statistical Meeting, Salt Lake City: AMSTAT. Retrieved January 1, 2013, from: http://www.laits.utexas.edu/txp_media/html/poll/files/Rivers_matching.pdf.
59. Rookey, B. D., Hanway, S., & Dillman, D. A. (2008). Does a probability-based household panel benefit from assignment to postal response as an alternative to Internet-only? Public Opinion Quarterly, 72, 962-984.
60. Saris,W. E. (1998). Ten years of interviewing without interviewers: The telepanel. InM. P. Couper, R. P. Baker, J. Bethlehem, C. Z. F. Clark, J. Martin, W. L. Nicholls II, & J. M. O'Reilly (Eds.), Computer assisted survey information collection (pp. 409-429). New York: John Wiley & Sons, Inc.
61. Särndal, C.-E.,&Lundström, S. (2005). Estimation in surveys with nonresponse. Chichester: JohnWiley & Sons, Ltd.
62. Scherpenzeel, A. (2011). Data collection in a probability-based Internet panel: How the LISS panel was built and how it can be used. Bulletin of Sociological Methodology/Bullétin de Méthodologie Sociologique, 109, 56-61.
63. Scherpenzeel, A. C., & Das, M. (2010). True longitudinal and probability-based Internet panels: Evidence from the Netherlands. Social and behavioral research and the internet: Advances in applied methods and research strategies (pp. 77-104). New York: Routledge.
64. Schonlau, M., van Soest, A., & Kapteyn, A. (2007). Are "Webographic" or attitudinal questions useful for adjusting estimates from Web surveys using propensity scoring? Survey Research Methods, 1, 155-163.
65. Schonlau, M., Zapert, K., Payne Simon, L., Hayness Sanstand, K., Marcus, S. M., Adams, J., Spranka, M., et al. (2004). A comparison between responses from a propensity-weighted web survey and an identical RDD survey. Social Science Computer Review, 22, 128-138.
66. Seybert, H., & Reinecke, P. (2013). Internet use in households and by individual in 2013. Eurostat Statistics in Focus 29/2013. Eurostat. Retrieved from: http://epp.eurostat.ec.europa.eu /statistics_explained/index.php/Internet_use_statistics_-_individuals.
67. Sikkel, D., & Hoogendoorn, A. (2008). Panel surveys. In E. De Leeuw, J. Hox, & D. A. Dillman (Eds.), International handbook of survey methodology (pp. 479-499). New York: Lawrence Erlbaum Associates.
68. Sudman, S.,&Wansink, B. (2002). Consumer panels (2nd ed.). Chicago, IL: American Marketing Association.
69. Terhanian, G., & Bremer, J. (2012). A smarter way to select respondents for surveys? International Journal of Market Research, 54, 751-780.
70. Terhanian, G., Smith, R., Bremer, J., & Thomas, R. K. (2001). Exploiting analytical advances: Minimizing the biases associated with non-random samples of internet users. Proceedings from the 2001 ESOMAR/ARF Worldwide Measurement Conference (pp. 247-272). Athens.
71. Tortora, R. (2008). Recruitment and retention for a consumer panel. In P. Lynn (Ed.), Methodology of longitudinal surveys (pp. 235-249). Hoboken, NJ: John Wiley & Sons. Inc.
72. U.S. Department of Commerce, & National Telecommunications and Information Administration. (2010). Digital Nation: 21st century America's progress toward universal broadband Internet access. Washington, DC: National Telecommunications and Information Administration.
73. Valliant, R., & Dever, J. A. (2011). Estimating propensity adjustments for volunteer web surveys. Sociological Methods & Research, 40, 105-137.
74. Vannette, D. L. (2011). Online survey research: Findings, best practices, and future research. Report prepared for the Advertising Research Foundation. New York: Advertising Research Foundation.
75. Vaygelt, M. (2006). Emerging from the shadow of consumer panels: B2B challenges and best practices. Panel Research 2006. ESOMAR.
76. Visioncritical University. (2013). Insight communities. Retrieved July 16, 2013, from: http://vcu .visioncritical.com/community-panel/.
77. Walker, R., Pettit, R.,&Rubinson, J. (2009). Aspecial report from the Advertising Research Foundation. The foundations of quality initiative:Afive-part immersion into the quality of online research. Journal of Advertising Research, 49, 464-485.
78. YouGov. (2011). YouGov's record. Public polling results compared to other pollsters and actual outcomes. Retrieved January 1, 2013, from: http://cdn.yougov.com/today_uk_import/yg-archives-pol -trackers-record2011.pdf