When will nonprobability surveys mirror probability surveys? Considering types of inference and weighting strategies as criteria for correspondence

International Journal of Public Opinion Research

Volumn 28, Issue 2, 2016, Pages 269-291

  Pasek, Josh ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84978993846     PISSN: 09542892     EISSN: 14716909     Source Type: Journal    
DOI: 10.1093/ijpor/edv016     Document Type: Review

References (55)

1. Alvarez, R. M., Sherman, R. P., & Van Besalaere, C. (2003). Subject acquisition for web-based surveys. Political Analysis, 11, 23-43. http://doi.org/10.1093/pan/11.1. 23
2. Ansolabehere, S. D., & Schaffner, B. F. (2014). Does survey mode still matter? Findings from a 2010 multi-mode comparison. Political Analysis, 22, 285-303. http://doi.org/10.1093/pan/mpt025
3. Atkeson, L. R., Adams, A. N., & Alvarez, R. M. (2014). Nonresponse and mode effects in self- and interviewer-administered surveys. Political Analysis, 22, 304-320. http://doi.org/10.1093/pan/mpt049
4. Baker, R., Blumberg, S. J., Brick, J. M., Couper, M. P., Courtright, M., Dennis, J. M.,... & Lavrakas, P. J. (2010). AAPOR report on online panels. Public Opinion Quarterly, 74, 711-781. http://doi.org/10.1093/poq/nfq048
5. Baker, R., Brick, J. M., Bates, N. A., Battaglia, M. P., Couper, M. P., Dever, J. A.,... & Tourangeau, R. (2013). Summary report of the AAPOR task force on nonprobability sampling. Journal of Survey Statistics and Methodology, 1, 90-143. doi: 10.1093/jssam/smt008
6. Berrens, R. P., Bokhara, A. K., Jenkins-Smith, H., Silva, C., & Weimar, D. L. (2003). The advent of internet surveys for political research: A comparison of telephone and internet samples. Political Analysis, 11, 1-22. http://doi.org/10.1093/pan/11. 1.1
7. Blair, E., & Zinkhan, G. M. (2006). From the Editor: Nonresponse and generalizability in academic research. Journal of the Academy of Marketing Science, 34, 4-7. http://doi.org/10.1177/0092070305283778
8. Callegaro, M., Baker, R., Bethlehem, J., Goritz, A. S., Krosnick, J. A., & Lavrakas, P. J. (2014). Online panel research: History, concepts, applications and a look at the future. In M. Callegaro, R. Baker, J. Bethlehem, A. S. Goritz, J. A. Krosnick, & P. J. Lavrakas (Eds.), Online Panel Research A Data Quality Perspective (pp. 1-22). Chichester, UK: Blackwell Publishing.
9. Chang, L., & Krosnick, J. A. (2010). Comparing oral interviewing with self-administered computerized questionnaires: An experiment. Public Opinion Quarterly, 74, 154-167. http://doi.org/10.1093/poq/nfp090
10. Couper, M. P. (2000). Review: Web surveys: A review of issues and approaches. Public Opinion Quarterly, 64, 464-494. http://doi.org/10.2307/3078739
11. DeBell, M., & Krosnick, J. A. (2009). Computing weights for American national election study survey data (Report no. nes012427). Ann Arbor, MI; Palo Alto, CA: ANES Technical Report series. Retrieved from http://www.electionstudies.org/resources/papers/nes012427.pdf
12. Denscombe, M. (2006). Web-based questionnaires and the mode effect: An evaluation based on completion rates and data contents of near-identical questionnaires delivered in different modes. Social Science Computer Review, 24, 246-254. http://doi.org/10.1177/0894439305284522
13. Deville, J.-C., & Sarndal, C.-E. (1992). Calibration estimators in survey sampling. Journal of the American Statistical Association, 87, 376-382. http://doi.org/10.2307/2290268
14. Deville, J.-C., Sarndal, C.-E., & Sautory, O. (1993). Generalized raking procedures in survey sampling. Journal of the American Statistical Association, 88, 1013-1020. http://doi.org/10.2307/2290793
15. Dillman, D. A., & Christian, L. M. (2005). Survey mode as a source of instability in responses across surveys. Field Methods, 17, 30-52. http://doi.org/10.1177/1525822X04269550
16. Duffy, B., Smith, K., Terhanian, G., & Bremer, J. (2005). Comparing data from online and face-to-face surveys. International Journal of Market Research, 47, 615-639.
17. Fricker, R. D., Jr., & Schonlau, M. (2002). Advantages and disadvantages of internet research surveys: Evidence from the literature. Field Methods, 14, 347-367. http://doi.org/10.1177/152582202237725
18. Fricker, S. S., Galesic, M., Tourangeau, R., & Yan, T. (2005). An experimental comparison of web and telephone surveys. Public Opinion Quarterly, 69, 370-392. http://doi.org/10.1093/poq/nfi027
19. Gelman, A. (2007). Struggles with survey weighting and regression modeling. Statistical Science, 22, 153-164. http://doi.org/10.1214/088342306000000691
20. Glenn, N. D. (1987). Social trends in the United States: Evidence from sample surveys. Public Opinion Quarterly, 51, S109-S126. http://doi.org/10.2307/2749192
21. Greenlaw, C., & Brown-Welty, S. (2009). A comparison of web-based and paperbased survey methods: Testing assumptions of survey mode and response cost. Evaluation Review, 33, 464-480. http://doi.org/10.1177/0193841X09340214
22. Groves, R. M., & Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias: A meta-analysis. Public Opinion Quarterly, 72, 167-189. http://doi.org/10.1093/poq/nfn011
23. Heckman, J. J. (1979). Sample selection bias as a specification error. Econometrica, 47, 153-161. http://doi.org/10.2307/1912352
24. Holt, D., & Smith, T. M. F. (1979). Post stratification. Journal of the Royal Statistical Society. Series A (General), 142, 33-46. http://doi.org/10.2307/2344652
25. Iyengar, S. (2011). Laboratory experiments in political science. In J. N. Druckman, D. P. Green, J. H. Kuklinski, & A. Lupia, Cambridge handbook of experimental political science (pp. 73-88). New York, NY: Cambridge University Press.
26. Keeter, S., Kennedy, C., Dimock, M., Best, J., & Craighill, P. (2006). Gauging the impact of growing nonresponse on estimates from a national RDD telephone survey. Public Opinion Quarterly, 70, 759-779.
27. Keeter, S., Miller, C., Kohut, A., Groves, R. M., & Presser, S. (2000). Consequences of reducing nonresponse in a national telephone survey. Public Opinion Quarterly, 64, 125-148. http://doi.org/10.1086/317759
28. Kreuter, F. (2013). Facing the nonresponse challenge. The Annals of the American Academy of Political and Social Science, 645, 23-35. http://doi.org/10.1177/0002716212456815
29. Lee, S. (2006). Propensity score adjustment as a weighting scheme for volunteer panel web surveys. Journal of Official Statistics, 22, 329-349.
30. Lee, S., & Valliant, R. (2009). Estimation for volunteer panel web surveys using propensity score adjustment and calibration adjustment. Sociological Methods & Research, 37, 319-343. http://doi.org/10.1177/0049124108329643
31. Lohr, S. (2009). Sampling: Design and analysis (2nd ed.) Boston, MA: Cengage Learning.
32. Loosveldt, G., & Sonck, N. (2008). An evaluation of the weighting procedures for an online access panel survey. Survey Research Methods, 2, 93-105. Retrieved from http://w4.ub.uni-konstanz.de/srm/article/viewDownloadInterstitial/82/1657
33. Malhotra, N., & Krosnick, J. A. (2006). The effect of survey mode and sampling on inferences about political attitudes and behavior: Comparing the 2000 and 2004 ANES to internet surveys with nonprobability samples. Political Analysis, 15, 286-323. http://doi.org/10.1093/pan/mpm003
34. Mook, D. G. (1983). In defense of external invalidity. American Psychologist, 38, 379-387. http://doi.org/10.1037/0003-066X.38.4.379
35. Owen, L., McNeill, A., & Callum, C. (1998). Trends in smoking during pregnancy in England, 1992-7: Quota sampling surveys, 317, 728-730. http://doi.org/10.1136/bmj.317.7160.728
36. Page, B. I., & Shapiro, R. Y. (1992). The rational public. Chicago, IL: University of Chicago Press.
37. Pasek, J., Jang, S. M., Cobb, C. L., III., Dennis, J. M., & Disogra, C. (2014). Can marketing data aid survey research? Examining accuracy and completeness in consumer-file data. Public Opinion Quarterly, 78, 889-916. http://doi.org/10. 1093/poq/nfu043
38. Pasek, J., & Krosnick, J. A. (2010). Measuring intent to participate and participation in the 2010 census and their correlates and trends: Comparisons of RDD telephone and nonprobability sample internet survey data (Report no. Survey Methodology # 2010-15). Washington, DC: Statistical Research Division, U. S. Census Bureau, Research Report Series. Retrieved from https://www.census.gov/srd/papers/pdf/ssm2010-15.pdf
39. Pew Research Center. (n. d.). Internet use over time. Retrieved from http://www.pewinternet.org/data-trend/internet-use/internet-use-over-time/. Retrieved April 1, 2014.
40. Putnam, R. (2000). Bowling alone. New York, NY: Simon and Schuster.
41. Romer, D., Kenski, K., Winneg, K., Adasiewicz, C., & Jamieson, K. H. (2006). Capturing campaign dynamics, 2000 and 2004. Philadelphia, PA: University of Pennsylvania Press.
42. Sanchez, M. E., & Morchio, G. (1992). Probing "don't know" answers: Effects on survey estimates and variable relationships. Public Opinion Quarterly, 56, 454-474. http://doi.org/10.1086/269337
43. Sanders, D., Clarke, H. D., Stewart, M. C., & Whiteley, P. (2006). Does mode matter for modeling political choice? Evidence from the 2005 British election study. Political Analysis, 15, 257-285. http://doi.org/10.1093/pan/mpl010
44. Schonlau, M., Soest, A. V., Kapteyn, A., & Couper, M. P. (2009). Selection bias in web surveys and the use of propensity scores. Sociological Methods & Research, 37, 291-318. http://doi.org/10.1177/0049124108327128
45. Schonlau, M., Zapert, K., Simon, L. P., Sanstad, K. H., Marcus, S. M., Adams, J.,... & Berry, S. H. (2004). A comparison between responses from a propensityweighted web survey and an identical RDD survey. Social Science Computer Review, 22, 128-138. http://doi.org/10.1177/0894439303256551
46. Schreuder, H. T., Gregoire, T. G., & Weyer, J. P. (2001). For what applications can probability and non-probability sampling be used? Environmental Monitoring and Assessment, 66, 281-291. http://doi.org/10.1023/A:1006316418865
47. Sears, D. O. (1986). College sophomores in the laboratory: Influences of a narrow data base on social psychology's view of human nature. Journal of Personality and Social Psychology, 51, 515-530. http://doi.org/10.1037/0022-3514.51.3.515
48. Skitka, L. J., & Sargis, E. G. (2006). The internet as psychological laboratory. Annual Review of Psychology, 57, 529-555. http://doi.org/10.1146/annurev. psych.57. 102904.190048
49. Terhanian, G., Bremer, J., Smith, R., & Thomas, R. (2000). Correcting data from online surveys for the effects of nonrandom selection and nonrandom assignment. Harris Interactive White Paper. Retrieved from http://www.academia.edu/403721/Correcting-Data-From-Online-Surveys-for-the-Effects-of-Nonrandom-Selection-and-Nonrandom-Assignment
50. Vavreck, L., & Rivers, D. (2008). The 2006 cooperative congressional election study. Journal of Elections, Public Opinion and Parties, 18, 355-366. http://doi.org/10. 1080/17457280802305177
51. Voss, D. S., Gelman, A., & King, G. (1995). A review: Pre-election survey methodology: Details from eight polling organizations, 1988 and 1992. Public Opinion Quarterly, 59, 98-132. http://doi.org/10.2307/2749653
52. Winship, C., & Mare, R. D. (1992). Models for sample selection bias. Annual Review of Sociology, 18, 327-350. http://doi.org/10.2307/2083457
53. Wlezien, C. (1995). The public as thermostat: Dynamics of preferences for spending. American Journal of Political Science, 39, 981-1000. http://doi.org/10.2307/2111666
54. Wright, K. B. (2005). Researching internet-based populations: Advantages and disadvantages of online survey research, online questionnaire authoring software packages, and web survey services. Journal of Computer-Mediated Communication, 10. http://doi.org/10.1111/j.1083-6101.2005.tb00259.x
55. Zaller, J. (2003). A new standard of news quality: Burglar alarms for the monitorial citizen. Political Communication, 20, 109-130. http://doi.org/10.1080/10584600390211136