Yes–no answers versus check-all in self‑administered modes: A systematic review and analyses

International Journal of Market Research

Volumn 57, Issue 2, 2015, Pages 1-22

  Callegaro, Mario ^a   Murakami, Michael H ^b   Tepman, Ziv ^c   Henderson, Vani ^a  

^a GOOGLE   (United Kingdom)

^b Latinum Network   (United Kingdom)

^c Staffing Industry Analysts   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84925734142     PISSN: 14707853     EISSN: 25152173     Source Type: Journal    
DOI: 10.2501/IJMR-2015-013     Document Type: Article

References (39)

1. Ares, G., Barreiro, C., Deliza, R., Giménez, A. & Gámbaro, A. (2010) Application of a checkall-that-apply question to the development of chocolate milk desserts. Journal of Sensory Studies, 25, s1, pp. 67–86.
2. Birnbaum, M.H. (2001) Introduction to Behavioral Research on the Internet. Upper Saddle River, NJ: Prentice Hall.
3. Borenstein, M., Hedges, L.V., Higgins, J.P.T. & Rothstein, H.R. (2010) A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods, 1, 2, pp. 97–111.
4. Borenstein, M., Hedges, L.V., Higgins, J.P.T. & Rothstein, H.R. (2011) Comprehensive Meta Analysis. Englewood, NJ: Biostat.
5. Bradburn, N.M., Sudman, S. & Wansink, B. (2004) Asking Questions. The Definitive Guide to Questionnaire Design for Market Research, Political Polls, and Social and Health Questionnaires, revised edn. San Francisco, CA: Jossey Bass.
6. Callegaro, M., Murakami, M.H., Henderson, V., Tepman, Z. & Dong, Q. (2012) Yes–no vs options in web surveys: what form is closer to benchmarks? Paper presented at the 64th Annual Conference of the American Association for Public Opinion Research, Orlando, FL.
7. Dickersin, K. (2005) Publication bias: recognizing the problem, understanding its origins and scope, and preventing harm, in Rothstein, H.R., Sutton, A.J. & Borenstein, M. (eds) Publication Bias in Meta-analysis. Prevention, Assessment and Adjustments. Chichester: Wiley, pp. 11–33.
8. Dyck, B.W. & Moore, D.L. (2008) Validating a check-all and forced-choice question in a paper survey of provincial park campground users. Paper presented at the 63rd Annual Conference of the American Association for Public Opinion Research, New Orleans, LA.
9. Dykema, J., Schaeffer, N.C., Beach, J., Lein, V. & Day, B. (2011) Designing questions for web surveys: effects of check-list, check-all, and stand-alone response formats on survey reports and data quality. Paper presented at the 66th Annual Conference of the American Association for Public Opinion Research Phoenix, AZ. Available online at: http://uwsc.wisc.edu/2011fd/DykemaEtAl_2011_AAPOR.pdf (accessed 15 August 2014).
10. Ericson, L. & Nelson, C.T. (2007) A comparison of forced-choice and mark-all-that-apply formats for gathering information on health insurance in the 2006 American Community Survey Content Test. Paper presented at the Federal Committee on Statistical Methodology Research Conference, Arlington, VA. Available online at: https://fcsm.sites.usa.gov/files/2014/05/2007FCSM_Ericson-VI-A.pdf (accessed 15 August 2014).
11. Feindt, P., Schreiner, I. & Bushery, J. (1997) Reinterview: a tool for survey quality improvement. Proceedings of the Joint Statistical Meeting, Survey Research Methods Section. Washington, DC: American Statistical Association, pp. 105–110.
12. Forsman, G. & Schreiner, I. (1991) The design and analysis of reinterview: an overview, in Biemer, P.P., Groves, R.M., Lyberg, L.E., Mathiowetz, N.A. & Sudman, S. (eds) Measurement Errors in Surveys. New York: Wiley, pp. 279–301.
13. Groves, R.M., Fowler, F.J. Jr, Couper, M.P., Lepkowski, J.M., Singer, V. & Tourangeau, R. (2009) Survey Methodology, 2nd edn. Hoboken, NJ: Wiley.
14. Holbrook, A. (2008) Acquiescence response bias, in Lavrakas, P.J. (ed.) Encyclopedia of Survey Research Methods. Vols. 1–2. Los Angeles, CA: Sage, Vol. 1, pp. 3–4.
15. Krosnick, J.A. (1991) Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology, 5, 3, pp. 213–236.
16. Krosnick, J.A. (1999) Survey research. Annual Review of Psychology, 50, 1, pp. 537–567.
17. Krosnick, J.A. & Presser, S. (2010) Question and questionnaire design, in Marsden, P.V. & Wright, J.D. (eds) Handbook of Survey Research, 2nd edn. Howard House, UK: Emerald, pp. 263–313.
18. Krosnick, J.A., Narayan, S. & Smith, W.R. (1996) Satisficing in surveys: initial evidence. New Directions for Evaluation, 70, pp. 29–44.
19. Lado, J., Vicente, E., Manzzioni, A. & Ares, G. (2010) Application of a check-all-that-apply question for the evaluation of strawberry cultivars from a breeding program. Journal of the Science of Food and Agriculture 90, 13, pp. 2268–2275.
20. Larkins, A.G. & Shaver, J.P. (1967) Matched-pair scoring technique used on a first-grade yes–no type economics achievement test. Utah Academy of Science, Arts and Letters: Proceedings, 44, pp. 229–242.
21. Martin, W.C., Engelland, B.T. & Collier, J.E. (2011) Assessing the impact of acquiescence response bias on marketing data. Marketing Management Journal, 21, 1, pp. 31–46.
22. Mitofsky, W.J. & Edelman, M. (1995) A review of the 1992 VRS exit poll, in Lavrakas, P.J. & Traugott, M.W. (eds) Presidential Polls and the News Media. Boulder, CO: Westview Press, pp. 81–100.
23. Mooney, G.M. & Carlson, B.L. (1996) Reducing mode effects in ‘mark all that apply’ questions. Proceedings of the Joint Statistical Meeting, Survey Research Methods Section. Washington, DC: American Statistical Association, pp. 614–619.
24. Nicolaas, G., Campanelli, P., Hope, S., Jäckle, A. & Lynn, P. (2011) Is it a good idea to optimise question format for mode of data collection? Results from a mixed modes experiment. ISER working paper 2011–31. Available online at: www.iser.essex.ac.uk/publications/working-papers/iser/2011-31.pdf (accessed 15 August 2014).
25. Parente, M.E., Manzoni, A.V. & Ares, G. (2011) External preference mapping of commercial antiaging creams based on consumers’ responses to a check-all-that-apply question. Journal of Sensory Studies, 26, 2, pp. 158–166.
26. Peytchev, A. (2009) Survey breakoff. Public Opinion Quarterly, 73, 1, pp. 74–97.
27. Rasinski, K.A., Mingay, D. & Bradburn, N.M. (1994) Do respondents really ‘mark all that apply’ on self-administered questions? Public Opinion Quarterly, 58, 3, pp. 400–408.
28. Schuman, H. & Presser, S. (1981) Question & Answers in Attitude Surveys. Experiment of Question Form, Wording, and Context. San Diego, CA: Academic Press.
29. Smyth, J.D., Christian, L.M. & Dillman, D.A. (2008) Does ‘yes or no’ on the telephone mean the same as ‘check-all-that-apply’ on the web? Public Opinion Quarterly, 72, 1, pp. 103–113.
30. Smyth, J.D., Dillman, D.A., Christian, L.M. & Stern, M.J. (2005) Comparing check-all and forced-choice question formats in web surveys. The role of satisficing, depth of processing, and acquiescence in explaining differences. Social and Economic Sciences Research Center, Pullman, Washington. Available online at: www.sesrc.wsu.edu/dillman/papers/2005/comparingcheckall.pdf (accessed 15 August 2014).
31. Smyth, J.D., Dillman, D.A., Christian, L.M. & Stern, M.J. (2006) Comparing check-all and forced-choice question formats in web surveys. Public Opinion Quarterly, 70, 1, pp. 66–77.
32. Stern, M.J., Dillman, D. & Smyth, J.D. (2007) Visual design, order effects, and respondent characteristics in a self-administered survey. Survey Research Methods, 1, 3, pp. 121–138.
33. Stern, M.J., Smyth, J.D. & Mendez, J. (2012) The effects of item saliency and question design on measurement error in a self-administered survey. Field Methods, 24, 1, pp. 3–27.
34. Sudman, S. & Bradburn, N.M. (1982) Asking Questions: A Practical Guide to Questionnaire Design. San Francisco, CA: Jossey-Bass.
35. Thomas, R.K. (2011) Effects of response format on requalification for recontact studies. Paper presented at the 66th Annual Conference of the American Association for Public Opinion Research, Phoenix, AZ.
36. Thomas, R.K. & Klein, J.D. (2006) Merely incidental? Effect on response format on selfreported behavior. Journal of Official Statistics, 22, 2, pp. 221–244.
37. Thomas, R.K., Dillman, D.A. & Smyth, J.D. (2009) Response format effects on measurement of employment. Paper presented at the Federal Committee on Statistical Methodology Research Conference, Washington, DC.
38. Tsuchiya, T. & Hirai, Y. (2010) Elaborate item count questioning: why do people underreport in item count response? Survey Research Methods, 4, 3, pp. 139–149.
39. Wallace, B., Dahabreh, I.J., Trikalinos, T., Lau, J., Trow, P. & Schmid, C.H. (2012) Closing the gap between methodologists and end-users: R as a computational back-end. Journal of Statistical Software, 49, 2, pp. 1–15.