Professional respondents in nonprobability online panels

Online Panel Research: A Data Quality Perspective

Volumn , Issue , 2014, Pages 219-237

  Hillygus, D Sunshine ^a   Jackson, Natalie ^b   Young, McKenzie ^c  

^a DUKE UNIVERSITY   (United States)

^b MARIST COLLEGE   (United States)

^c Global Strategy Group in the Research Team   (United States)

Author keywords

Attrition; CCES; Data quality; Frequent survey takers; Political attitudes; Professional respondents; Satisficing; Survey experience; Voting; YouGov

Indexed keywords

EID: 84927800502     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/9781118763520.ch10     Document Type: Chapter

References (58)

1. Adams, A., Atkeson, L., & Karp, J. (2012). Panel conditioning in online survey panels: Problems of increased sophistication and decreased engagement. In Proceedings of the 2012 Annual Conference of the American Political Science Association. New Orleans.
2. Ansolabehere, S. (2012, August 10). Guide to the 2010 cooperative congressional election survey. Cambridge, MA: Harvard University Press.
3. Ansolabehere, S., & Schaffner, B. F. (2011). Re-examining the validity of different survey modes for measuring public opinion in the U.S.: Findings from a 2010 multi-mode comparison. Amherst, MA:University of Massachusetts Press.
4. Baker, R., Blumberg, S., Brick, J., Couper, M., Courtright, M., Dennis, J.,... Zahs, D. (2010). American association of public opinion researchers report on online panels. Public Opinion Quarterly, 74, 711-781.
5. Baker, R., & Downes-LeGuin, T. (2007). Separating the wheat from the chaff: Ensuring data quality in internet panel samples. In Proceedings of the Fifth International Conference of the Association of Survey Computing (pp. 157-166). Berkeley, UK: ASC.
6. Bartels, L. (1999). Panel effects in the American National Election Studies. Political Analysis, 8, 1-20.
7. Battaglia, M., Zell, E., & Ching, P. (1996). Can participating in a panel sample introduce bias into trend estimates? In Proceedings of the Section on Survey ResearchMethods at the 1996 Annual Conference of the American Statistical Association (pp. 1010-1013). Alexandria, VA.
8. Binswanger, J., Schunk, D., & Toepoel, V. (2006). Panel conditioning in difficult attitudinal questions. Dutch Online Panel Study NOPVO 2006.
9. Casdas, D., Fine, B., & Menictas, C. (2006). Attitudinal differences: Comparing people who belong to multiple versus single panels. In Proceedings of ESOMAR Panel Research 2006. Orlando.
10. Chang, L., & Krosnick, J. (2009). National surveys via RDD telephone interviewing versus the internet comparing sample representativeness and response quality. Public Opinion Quarterly, 73, 641-678.
11. Coen, T., Lorch, J., & Piekarski, L. (2005). The effects of survey frequency on panelists' responses. White Paper, Survey Sampling International.
12. Comley, P. (2005). Understanding the online panelist. In Proceedings of ESOMARWorld Research Conference, Barcelona.
13. Conrad, F., Couper, M., Tourangeau, R., Galesic, M., & Yan, T. (2005, May). Interactive feedback can improve the quality of responses in web surveys. In Proceedings of the 2005 annual meeting of the American Association for Public Opinion Research (pp. 3835-3840). Miami Beach.
14. Das, M., Toepoel, V., & van Soest, A. (2011). Nonparametric tests of panel conditioning and attrition bias in panel surveys. Sociological Methods & Research, 40, 32-56.
15. Dennis, J. (2001). Are internet panels creating professional respondents? Marketing Research, 13, 34-39.
16. De Wulf, K., & Berteloot, S. (2007). Duplication and multi-source online panel recruitment: real quality differences or idle rumours? In Proceedings of ESOMAR World Research Conference on Panel Research (pp. 49-62).
17. Dillman, D., Smyth, J., & Christian, L. (2009). Internet, mail, and mixed-mode surveys: the total design method. Hoboken, NJ: John Wiley & Sons.
18. Downes-Le Guin, T., Mechling, J., & Baker, R. (2006). Great results from ambiguous sources: Cleaning internet panel data. In Proceedings of ESOMAR Panel Research 2006 (pp. 1-10).
19. Frede, S., Markowitz, L., & Coffery, S. (2006). Heavier Responders in Online Survey Research. White Paper, Ipsos, NPD Group, and TNS.
20. Fulgoni, G. M. (2005). The "professional respondent" problem in online survey panels today. In Proceedings of the 2005 Marketing Research Association Annual Conference.
21. Garland, P., Santus, D., & Uppal, R. (2012). Survey lockouts: Are we too cautious? White Paper, Survey Sampling International.
22. Gittelman, S.,&Trimarchi, E. (2009). Variance between purchasing behavior profiles in a wide spectrum of online sample sources. White Paper, Marketing. Inc.
23. Gittelman, S., & Trimarchi, E. (2010). Are hyperactive respondents different? Quirks Marketing Research Review, November.
24. Goyder, J. (1987). The silent minority: Nonrespondents on sample surveys. Boulder, CO: Westview Press.
25. Groves, R., Presser, S., & Dipko, S. (2004). The role of topic interest in survey participation decisions. Public Opinion Quarterly, 68, 2-31.
26. Groves, R., Singer, E., & Corning, A. (2000). Leverage-saliency theory of survey participation: Description and an illustration. Public Opinion Quarterly, 64, 299-308.
27. Grover, R., & Vriens,M. (Eds.). (2006). The handbook of marketing research: Uses, misuses, and future advances. Thousand Oaks, CA: Sage.
28. Harlow, A. (2010). Online surveys - possibilities, pitfalls and practicalities: The experience of the TELA evaluation. Faculty of Education, University of Waikato, New Zealand.
29. Honda, N., & Motokawa, A. (2004). Can internet surveys be used for social surveys?: Results of an experimental study. The Japan Institute for Labour Policy and Training Research Report, No. 17.
30. Kapelner, A., & Chandler, D. (2010). Preventing satisficing in online surveys: A "kapcha" to ensure higher quality data. In Proceedings of CrowdConf ACM.
31. King, G. (1990). When not to use R-squared. The Political Methodologist, 3, 11-12.
32. Knapton, K., & Garlick, R. (2007). Catch me if you can. Quirks, November, 58-63.
33. Krosnick, J. (1991). Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5, 213-236.
34. Krosnick, J., Nie, N.,&Rivers, D. (2005).Web survey methodologies: A comparison of survey accuracy. American Association for Public Opinion Research, 60th Annual Conference, Miami.
35. Kruse, Y., Callegaro, M., Dennis, J., Subias, S., Lawrence, M., DiSogra, C., & Tompson, T. (2010). Panel conditioning and attrition in the AP-Yahoo! news election panel study. In Proceedings of the 2010 Annual Conference of the American Association for Public Opinion Research (pp. 5742-5756). Washington, DC: American Statistical Association.
36. Malhotra, N., & Krosnick, J. (2007). The effect of survey mode and sampling on inferences about political attitudes and behavior: Comparing the 2000 and 2004 ANES to Internet surveys with nonprobability samples. Political Analysis, 15, 286-323.
37. Mason, W., & Watts, D. (2010). Financial incentives and the performance of crowds. ACM SIGKDD Explorations Newsletter, 11, 100-108.
38. Mathiowetz, N., & Lair, T. (1994). Getting better? Change or error in the measurement of functional limitations. Journal of Economic and Social Measurement, 20, 237-262.
39. Matthijsse, S., Leo, E., & Hox, J. (2006). Professional respondents in online panels: A threat to data quality? In Proceedings of ESOMAR Panel Research.
40. Meurs, H., Wissen, L., & Visser, J. (1989). Measurement biases in panel data. Transportation, 16, 175-194.
41. Miller, J. (2007). Burke panel quality research and development summary. White Paper, Advertising Research Foundation Online Research Quality Council.
42. Nancarrow, C., & Cartwright, T. (2007). Online access panels and tracking research: The conditioning issue. International Journal of Market Research (formerly Journal of the Market Research Society), 49, 573-594.
43. Oppenheimer, D., Meyvis, T., & Davidenko, N. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45, 867-872.
44. Paolacci, G., Chandler, J., & Ipeirotis, P. (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision Making, 5, 411-419.
45. Peruzzi, N. (2010, July 8). Online survey sample is not clean enough - clean it yourself. Retrieved from:www.researchaccess.com.
46. Pineau, V., Nukulkij, P., & Tang, X. (2005). Assessing panel bias in the knowledge networks panel:Updated results from 2005 research. In Joint Statistical Meeting Proceedings (pp. 3480-3486).
47. Prior, M., & Lupia, A. (2008). Money, time and political knowledge: Distinguishing quick recall and political learning skills. American Journal of Political Science, 52, 169-183.
48. Rogers, F., & Richarme, M. (2009). The honesty of online survey respondents: Lessons learned and prescriptive remedies. White Paper, Decision Analyst.
49. Schlackman, W. (1984). A discussion of the use of sensitivity panels in market research: The use of trained respondents in qualitative studies. Journal of the Market Research Society, 26, 191-208.
50. Smith, R., & Brown, H. H. (2006). Data and panel quality: Comparing metrics and assessing claims. In Proceedings of the ESOMAR Panel Research Conference.
51. Sparrow, N. (2007). Quality issues in online research. JOURNAL OF ADVERTISING RESEARCH, 47, 179.
52. Toepoel, V., Das, M., & van Soest, A. (2008). Effects of design in web surveys: Comparing trained and fresh respondents. Public Opinion Quarterly, 72, 985-1007.
53. Toepoel, V., Das, M., & Van Soest, A. (2009). Relating question type to panel conditioning: A comparison between trained and fresh respondents. Survey Research Methods, 3, 73-80.
54. Walker, R., Pettit, R., & Rubinson, J. (2009). The foundations of online research quality executive summary 3: Interstudy comparability and benchmark analysis. White Paper, Advertising Research Foundation.
55. Waterton, J., & Lievesley, D. (1989). Evidence of conditioning effects in the British Attitudes Panel Survey. In G. Duncan, G. Kalton, D. Kasprzyk, & M. P. Singh (Eds.), Panel surveys. New York:Wiley.
56. Willems, P., van Ossenbruggen, R., & Vonk, T. (2006a). The effects of panel recruitment and management on research results. In Proceedings of the conference of the European Society for Opinion and Marketing Research. Barcelona.
57. Willems, P., Vonk, T., & van Ossenbruggen, R. (2006b). The effects of panel recruitment and management on research results: A study across 19 online panels. In Proceedings of ESOMAR Panel Research.
58. Yan, T., & Tourangeau, R. (2008). Fast times and easy questions: The effects of age, experience and question complexity on web survey response times. Applied Cognitive Psychology, 22, 51-68.