Research into questionnaire design: A summary of the literature

International Journal of Market Research

Volumn 52, Issue 2, 2010, Pages 249-272

  Lietz, Petra ^a  

^a Australian Council for Education Research   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 77951039992     PISSN: 14707853     EISSN: None     Source Type: Journal    
DOI: 10.2501/S147078530920120X     Document Type: Article

References (118)

1. Aday, L. (2006) Designing and Conducting Health Surveys: A Comprehensive Guide (3rd edn). San Francisco, CA: Jossey-Bass.
2. Allport, G. (1935) Attitudes. In C.M. Murchison (ed.) Handbook of Social Psychology. Worchester, MA: Clark University Press, pp. 798-844.
3. Alwin, D.F. (1992) Information transmission in the survey interview: number of response categories and the reliability of attitude measurement. Sociological Methodology, 22, pp. 83-118. doi:10.2307/270993
4. Alwin, D.F. (1997) Feeling thermometers vs 7-point scales: which are better? Sociological Methods and Research, 25, pp. 318-340. doi:10.1177/0049124197025003003
5. Andrews, F. (1984) Construct validity and error components of survey measures: a structural modeling approach. Public Opinion Quarterly, 48, 2, pp. 409-442. doi:10.1086/268840
6. Andrich, D. (1978) Scaling attitude items constructed and scored in the Likert tradition. Educational and Psychological Measurement, 38, pp. 665-680. doi:10.1177/001316447803800308
7. Baker M.J. (2003) Data collection - questionnaire design. Marketing Review, 3, 3, pp. 343-370. doi:10.1362/146934703322383507
8. Bartram, P. & Yelding, D. (1973) The development of an empirical method of selecting phrases used in verbal rating scales: a report on a recent experiment. Journal of the Market Research Society, 15, pp. 151-156.
9. Bates, N., Martin, E., DeMaio, T.J. & de la Puente, M. (2006). Questionnaire Effects on Measurements of Race and Spanish Origin. Research Report Series. Washington, DC: Statistical Research Division, US Census Bureau.
10. Belson, W.A. (1981) The Design and Understanding of Survey Questions. Aldershot: Gower.
11. Bergstrom, B.A. & Lunz, M.E. (1998) Rating Scale Analysis: Gauging the Impact of Positively and Negatively Worded Items. Paper presented at the Annual Meeting of the American Educational Research Association, 13-17 April.
12. Bhandari, A. & Wagner, T. (2006) Self-reported utilization of health care services: measurement and accuracy. Medical Care Research and Review, 63, 2, pp. 217-135. doi:10.1177/1077558705285298
13. Bishop, G.F., Oldendick, R.W., Tuchfarber, A.J. & Bennett, S.E. (1979) Effects of opinion filtering and opinion floating: evidence from a secondary analysis. Political Methodology, 6, pp. 293-309.
14. Blair, E., Sudman, S., Bradburn, N. & Stocking, C. (1977) How to ask questions about drinking and sex: response effects in measuring consumer behavior. Journal of Marketing Research, 14, pp. 316-321. doi:10.2307/3150769
15. Bortz, J. & Döring, N. (2003) Forschungsmethoden und Evaluation für Sozialwissenschaftler. Berlin, Heidelberg: Springer.
16. Bowling, A. (2005) Quantitative social science: the survey. In A. Bowling & S. Ebrahim (eds) Handbook of Health Research Methods: Investigation, Measurement and Analysis. New York: McGraw-Hill, pp. 190-214.
17. Brace, I. (2004) Questionnaire Design. How to Plan, Structure and Write Survey Material for Effective Market Research. London: Kogan Page.
18. Bradburn, N. & Miles, C. (1979) Vague quantifiers. Public Opinion Quarterly, 43, 1, pp. 92-101. doi:10.1086/268494
19. Bradburn, N. & Sudman, S. (1979) Improving Interview Method and Questionnaire Design. San Francisco, CA: Jossey-Bass.
20. Bradburn, N. & Sudman, S. (2003) The current status of questionnaire design. In P.P. Biemer, R.M. Groves, L.E. Lyberg, N.A. Mathiowetz & S. Sudman (eds) Measurement Errors in Surveys. New York: Wiley, pp. 29-40.
21. Bradburn, N., Sudman, S. & Wansink, B. (2004) Asking Questions. The Definitive Guide to Questionnaire Design - for Market Research, Political Polls, and Social and Health Questionnaires. San Francisco, CA: Jossey-Bass.
22. Brislin, R.W. (1986) The wording and translation of research instruments. In W.J. Lonner & J.W. Berry (eds) Field Methods in Cross-Cultural Research. Newbury Park, CA: Sage, pp. 137-164.
23. Cannell, C.F., Miller, P.V. & Oksenberg, L. (1981) Research on interviewing techniques. In S. Leinhardt (ed.) Sociological Methodology. San Francisco, CA: Jossey-Bass.
24. Cliff, N. (1959) Adverbs as multipliers. Psychological Review, 66, pp. 27-44. doi:10.1037/h0045660
25. Coelho, P.S. & Esteves, S.P. (2007) The choice between a five-point and a ten-point scale in the framework of customer satisfaction measurement. International Journal of Market Research, 49, 3, pp. 313-339.
26. Converse, J. & Presser, S. (1986) Survey Questions. Handcrafting the Standard Questionnaire. London: Sage.
27. Cox III, E.P. (1980) The optimal number of response alternatives for a scale: a review. Journal of Marketing Research, 17, pp. 407-422. doi:10.2307/3150495
28. Cronbach, L.J. (1951) Coefficient alpha and the internal structure of tests. Psychometrica, 16, pp. 93-96. doi:10.1007/BF02310555
29. Crowne, D. & Marlowe, D. (1960) A new scale of social desirability independent of psychopathology. Journal of Consulting Psychology, 24, pp. 963-968. doi:10.1037/h0047358
30. Dawes, J. (2008) Do data characteristics change according to the number of scale points used? An experiment using 5-point, 7-point and 10-point scales. International Journal of Market Research, 50, 1, pp. 61-77.
31. deLeeuw, E. & deHeer, W. (2002) Trends in household survey nonresponse: a longitudinal and international comparison. In R. Groves, D. Dillman, J. Eltinge & R. Little (eds) Survey Non-Response. New York: Wiley, pp. 41-54.
32. DeVellis, R.F. (2003) Scale Development. Theory and Application (2nd edn). Thousand Oaks, CA: Sage.
33. Diekmann, A. (2003) Empirische Sozialforschung. Grundlagen, Methoden, Anwendungen. Reinbeck bei Hamburg: Rowohlt.
34. Dillman, D. (2000) Mail and Internet Surveys. The Tailored Design Method. New York: John Wiley & Sons, Inc.
35. DiStefano, C. & Motl, R.W. (2006) Further investigating method effects associated with negatively worded items on self-report surveys. Structural Equation Modeling: A Multidisciplinary Journal, 13, 3, pp. 440-464
36. Dörnyei, Z. (2003) Questionnaires in Second Language Research. Mahwah, NJ: Lawrence Erlbaum.
37. Drennan, J. (2003) Cognitive interviewing: verbal data in the design and pretesting of questionnaires. Journal of Advanced Nursing, 41, 1, pp. 57-63. doi:10.1046/j.1365-2648.2003.02579.x
38. Dudycha, A.L. & Carpenter, J.B. (1973) Effects of item format on item discrimination and difficulty. Journal of Applied Psychology, 58, pp. 116-121. doi:10.1037/h0035197
39. Edwards, A. (1957) The Social Desirability Variable in Personality Assessment and Research. New York: Dryden Press.
40. Eifermann, R.R. (1961) Negation: a linguistic variable. Acta Psychologica, 18, pp. 258-273. doi:10.1016/0001-6918(61)90016-6
41. Fife-Schaw, C. (2006) Questionnaire design. In G.M. Breakwell, S. Hammond, C. Fife-Schaw, & J.A. Smith (eds) Research Methods in Psychology (3rd edn). Thousand Oaks, CA: Sage, pp. 210-231.
42. Fink, A. (2003) How to Ask Survey Questions. Thousand Oaks, CA: Sage.
43. Finn, R. (1972) Effects of some variations in rating scale characteristics on the means and reliabilities of ratings. Educational and Psychological Measurement, 32, 2, pp. 255-265. doi:10.1177/001316447203200203
44. Fitzgerald, R. & Widdop, S. (eds) (2004) European Social Survey Round 3. Measuring Social and Political Change in Europe. Publishable final activity report. Available online at: http://www.europeansocialsurvey.org/index.php?option=com_content&view=article&id=233:r3-far&catid=22:news&Itemid=48 (accessed 1 December 2008).
45. Foddy, W. (1993) Constructing Questions for Interviews and Questionnaires. Theory and Practice in Social Research. Cambridge, UK: Cambridge University Press. doi:10.1017/CBO9780511518201
46. Fowler, F.J. (1992) How unclear terms affect survey data. Public Opinion Quarterly, 56, 2, pp. 218-231. doi:10.1086/269312
47. Fowler, F.J. (1995) Improving Survey Questions. Design and Evaluation. London: Sage.
48. Fowler, F.J. (2004) The case for more split-sample experiments in developing survey instruments. In S. Presser, J.M. Rothgeb, M.P. Couper, J.T. Lessler, E. Martin, J. Martin & E. Singer (eds) Methods for Testing and Evaluating Survey Questionnaires. New York: Wiley. doi:10.1002/0471654728.ch9
49. Garland, R. (1991) The mid-point on a rating scale: is it desirable? Marketing Bulletin, 2, pp. 66-70.
50. Gaskell, G.D., O'Muircheartaigh, C.A. & Wright, D.B. (1994) Survey questions about the frequency of vaguely defined events: the effects of response alternative. Public Opinion Quarterly, 58, 2, pp. 241-254. doi:10.1086/269420
51. Hippler, H.-J. & Schwarz, N. (1986) Not forbidding isn't allowing: the cognitive basis of the forbid-allow asymmetry. Public Opinion Quarterly, 50, 1, pp. 87-96. doi:10.1086/268961
52. Hippler, H.-J., Schwarz, N. & Sudman, S. (1987) (eds) Social Information Processing and Survey Methodology. New York: Springer.
53. Holbrook, A., Cho, Y.I. & Johnson, T. (2006) The impact of question and respondent characteristics on comprehension and mapping difficulties. Public Opinion Quarterly, 70, 4, pp. 565-595. doi:10.1093/poq/nfl027
54. Holtgraves, T. (2004) Social desirability and self-reports: testing models of socially desirable responding. Personality and Social Psychology Bulletin, 30, 2, pp. 161-172. doi:10.1177/0146167203259930 PMid:15030631
55. Hunt, W.H., Sparkman, R.D. Jr & Wilcox, J.B. (1982) The pretest in survey research: issues and preliminary findings. Journal of Marketing Research, 19, 2, pp. 269-273. doi:10.2307/3151627
56. Jabine, T.B. (1987) Reporting chronic conditions in the National Health Interview Survey: a review of tendencies from evaluation studies and methodological test. Vital and Health Statistics, Series 2, 105. Washington, DC: Government Printing Office.
57. Jobe, J. & Mingay, D. (1989) Cognitive research improves questionnaires. American Journal of Public Health, 79, 8, pp. 1053-1055. doi:10.2105/AJPH.79.8.1053 PMid:2751028 PMCid:1349911
58. Kalton, G., Robert, J. & Holt, D. (1980) The effects of offering a middle response option with opinion questions. The Statistician, 29, pp. 65-79.
59. Krosnick J.A. (1991) Response strategies for coping with the cognitive demands of attitude measurement in surveys. Applied Cognitive Psychology, 5, 2, pp. 213-236. doi:10.1002/acp.2350050305
60. Krumpal, I., Rauhut, H., Böhr, D. & Naumann, E. (2008) Wie wahrscheinlich ist 'wahrscheinlich'? Methoden - Daten - Analysen, 2, 1, pp. 3-27.
61. Leite, W. & Beretvas, N. (2005) Validation of scores on the Marlowe-Crowne Social Desirability Scale and the Balanced Inventory of Desirable Responding. Educational and Psychological Measurement, 65, 1, pp. 140-154. doi:10.1177/0013164404267285
62. Lenski, G.E. & Leggett, J.C. (1960) Caste, class and deference in the research interview. American Journal of Sociology, 65, pp. 463-467. doi:10.1086/222750
63. Leverkus-Brüning, R. (1966) Die Meinungslosen. Die Bedeutung der Restkategorie in der empirischen Sozialforschung. Berlin: Duncker & Humbolt.
64. Liechtenstein, S. & Newman, J.R. (1967) Empirical scaling of common verbal phrases associated with numerical probabilities. Psychonomic Science, 9, 10, pp. 563-564.
65. Litwin, M.S. (2003) How to Assess and Interpret Survey Psychometrics (2nd edn). Thousand Oaks, CA: Sage.
66. Martin, E. (2002) The effects of questionnaire design and reporting of detailed Hispanic origin in census 2000 mail questionnaires. Public Opinion Quarterly, 66, 4, pp. 582-593. doi:10.1086/343863
67. Masters, J.R. (1974) The relationship between number of response categories and reliability of Likert-type questionnaires. Journal of Educational Measurement, 11, 1, pp. 49-53. doi:10.1111/j.1745-3984.1974.tb00970.x
68. Matell, M. & Jacoby, J. (1971) Is there an optimal number of alternatives for Likert scale items? Study I: reliability and validity. Educational and Psychological Measurement, 31, pp. 657-74. doi:10.1177/001316447103100307
69. Mayer, H. (2002) Interview und schriftliche Befragung. Entwicklung, Durchführung und Auswertung. Munich: Oldenbourg Wissenschaftsverlag.
70. Mittelstaedt, R. (1971) Semantic properties of selected evaluative adjectives: other evidence. Journal of Marketing Research, 8, 2, pp. 236-237. doi:10.2307/3149768
71. Moorman, R.H. & Podsakoff, P.M. (1992) A meta-analytic review and empirical test of the potential confounding effects of social desirability response sets in organizational behavior research. Journal of Occupational and Organizational Psychology, 65, pp. 131-149.
72. Mosier, C.I. (1941) A psychometric study of meaning. Journal of Social Psychology, 13, pp. 123-140.
73. Mullis, I.V.S., Martin, M.O., Kennedy, A.M. & Foy, P. (2007) PIRLS 2006 International Report: IEA's Progress in International Reading Literacy Study in Primary Schools in 40 Countries. Chestnut Hill, MA: International Association for the Evaluation of Educational Achievement (IEA).
74. Myers, J. & Warner, G. (1968) Semantic properties of selected evaluation adjectives. Journal of Marketing Research, 5, pp. 409-412. doi:10.2307/3150266
75. Nunnally, J.C. & Bernstein, I. (1994) Psychometric Theory (3rd edn). New York: McGraw-Hill.
76. O'Muircheartaigh, C., Gaskell, G. & Wright, D. (1993) Intensifiers in behavioral frequency questions. Public Opinion Quarterly, 57, 4, pp. 552-565. doi:10.1086/269395
77. O'Muircheartaigh, C., Gaskell, G. & Wright, D. (1995) Weighing anchors: verbal and numeric labels for response scales. Journal of Official Statistics, 11, 3, pp. 295-307.
78. O'Muircheartaigh, C., Krosnick, J. & Helic, A. (2000) Middle Alternatives, Acquiescence, and the Quality of Questionnaire Data. Unpublished manuscript. Retrieved 19 May 2008 from http://harrisschool.uchicago.edu/about/publications/working-papers/pdf/wp_01_3.pdf
79. Oksenberg, L. & Cannell, C. (1977) Some factors underlying the validity of response in self report. Bulletin of the International Statistical Institute, 47, pp. 325-346.
80. Oppenheim, A.N. (1992) Questionnaire Design, Interviewing and Attitude Measurement. London: Pinter.
81. Paulhus, D.L. (1984) Two-component models of socially desirable responding. Journal of Personality and Social Psychology, 46, pp. 598-609. doi:10.1037/0022-3514.46.3.598
82. Paulhus, D.L. (1991) Measurement and control of response bias. In J. Robinson, P. Shaver & L. Wrightsman (eds) Measures of Personality and Social Psychological Attitudes. San Diego, CA: Academic Press, pp. 17-59.
83. Paulhus, D.L. & Van Selst, M. (1990) The spheres of control scale: 10 years of research. Personality and Individual Differences, 11, 10, pp. 1029-1036. doi:10.1016/0191-8869(90)90130-J
84. Porst, R. (2000) Praxis der Umfrageforschung. Stuttgart: Teubner.
85. Presser, S. & Blair, J. (1994) Survey pretesting: do different methods produce different results? Sociological Methodology, 24, pp. 73-104. doi:10.2307/270979
86. Rammstedt, B. & Krebs, D. (2007). Response scale format and answering of personality scales. European Journal of Psychological Assessment, 23, 1, pp. 32-38. doi:10.1027/1015-5759.23.1.32
87. Rockwood, T., Sangster, R. & Dillman, D. (1997) The effects of response categories on questionnaire answers: context and mode effects. Sociological Methods and Research, 26, 1, pp. 118-140. doi:10.1177/0049124197026001004
88. Rodgers, W., Andrews, F. & Herzog, R. (1992) Quality of survey measures: a structural modeling approach. Journal of Official Statistics, 8, 3, pp. 251-275.
89. Saris, W.E. & Gallhofer, I. (2007) Estimation of the effects of measurement characteristics on the quality of survey questions. Survey Research Methods, 1, 1, pp. 29-43.
90. Saris, W.E., Satorra, A. & Coenders, G. (2004) A new approach for evaluating quality of measurement instruments. Sociological Methodology, 34, 1, pp. 311-347. doi:10.1111/j.0081-1750.2004.00155.x
91. Schaeffer, N.C. & Presser, S. (2006) The science of asking questions. Annual Review of Sociology, 29, pp. 65-88. doi:10.1146/annurev.soc.29.110702.110112
92. Scholl, A. (2003) Die Befragung. Sozialwissenschaftliche Methode und kommunikationswissenschaftliche Anwendung. Konstanz: UVK Verlagsgesellschaft.
93. Schuman, H. & Presser, S. (1977) Question wording as an independent variable in survey analysis. Sociological Methods and Research, 6, pp. 151-176. doi:10.1177/004912417700600202
94. Schuman, H. & Presser, S. (1978) Attitude measurement and the gun control paradox. Public Opinion Quarterly, 41, pp. 427-439. doi:10.1086/268405
95. Schuman, H. & Presser, S. (1996) Questions and Answers in Attitude Surveys. London: Sage.
96. Schwarz, N. & Hippler, H. (1991) Response alternatives: the impact of their choice and presentation order. In P.P. Biemer, R.M. Groves, L.E. Lyberg, N.A. Mathiowetz & S. Sudman (eds) Measurement Errors in Surveys. New York: Wiley Interscience, pp. 41-56.
97. Schwarz, N. & Sudman, S. (1992) Context Effects in Social and Psychological Research. New York: Springer.
98. Schwarz, N., Grayson, C.E. & Knäuper, B. (1998) Formal features of rating scales and the interpretation of question meaning. International Journal of Public Opinion Research, 10, 2, pp. 177-183.
99. Schwarz, N., Hippler, H., Deutsch, B. & Strack, F. (1985) Response scales: effects of category range on reported behavior and comparative judgments. Public Opinion Quarterly, 49, 3, pp. 388-395. doi:10.1086/268936
100. Schwarz, N., Knäuper, B., Hippler, H., Noelle-Neumann, E. & Clark, L. (1991) Rating scales: numeric values may change the meaning of scale labels. Public Opinion Quarterly, 55, pp. 570-582. doi:10.1086/269282
101. Seitz, W. (1977) Persönlichkeitsbeurteilung durch Fragebogen. Eine Einführung in die diagnostische Praxis und ihre theoretischen Grundlagen für Psychologen, Pädagogen, Heilpädagogen und Mediziner. Braunschweig: Westermann.
102. Simpson, R.H. (1944) The specific meanings of certain terms indicating differing degrees of frequency. Quarterly Journal of Speech, 21, 3, pp. 328-330. doi:10.1080/00335634409381009
103. Smith, P.B. (2004) Acquiescent response bias as an aspect of cultural communications style. Journal of Cross-Cultural Psychology, 35, pp. 50-61. doi:10.1177/0022022103260380
104. Smith, T.W. (2003) Developing comparable questions in cross-national surveys. In J.A. Harkness, F.J.R. van de Vijver & P.P. Mohler (eds) Cross-Cultural Survey Methods. Hoboken, NJ: Wiley Interscience, pp. 69-92.
105. Stocké, V. & Hunkler, C. (2007) Measures of desirability beliefs and their validity as indicators for socially desirable responding. Field Methods, 19, 3, pp. 313-336. doi:10.1177/1525822X07302102
106. Sudman, S. & Bradburn, N.M. (1974) Response Effects in Surveys. Chicago, IL: Aldine.
107. Tourangeau, R., Rips, L.J. & Rasinski, K. (2000) The Psychology of Survey Response. New York: Cambridge University Press.
108. Trometer, R. (1996) Warum sind Befragte 'meinungslos'? Kognitive und kommunikative Prozesse im Interview. Inauguraldissertation zur Erlangung des akademischen Grades eines Doktors der Philosophie der Universität zu Mannheim. Mannheim: Universität Mannheim.
109. van der Zouwen, J. (2000) An assessment of the difficulty of questions used in the ISSP-questionnaires, the clarity of their wording, and the comparability of the responses. ZA-Information, 46, pp. 96-114.
110. Vidali, J.J. (1975) Context effects on scaled evaluatory adjective meaning. Journal of the Market Research Society, 17, 1, pp. 21-25.
111. Vikat, A., Speder, Z., Beets, G., Billari, F.C. & Buhler, C. (2007) Generations and gender survey (GGS): towards a better understanding of relationships and processes. Demographic Research, 17, 14, pp. 389-440. doi:10.4054/DemRes.2007.17.14
112. von der Linden, W.J. & Hambleton, R.K. (1997) Handbook of Modern Item Response Theory. New York: Springer.
113. Weems, G.H., Onwugbuzie, A.J. & Lustig, D. (2002) Profiles of respondents who respond inconsistently to positively and negatively worded items on rating scales. Paper presented at the Annual Meeting of the Mid-South Educational Research Association, Chattanooga, TN, 6-8 November.
114. White, P.C., Jennings, N.V., Renwick, A.R. & Barker, N.H.L. (2005) Questionnaires in ecology: a review of past use and recommendations for best practice. Journal of Applied Ecology, 42, 3, pp. 421-430. doi:10.1111/j.1365-2664.2005.01032.x
115. Wildt, A.R. & Mazis, M.B. (1978) Determinants of scale response: label versus position. Journal of Marketing Research, 15, pp. 261-267. doi:10.2307/3151256
116. Willis, G.B. (2005) Cognitive Interviewing. A Tool for Improving Questionnaire Design. Thousand Oaks, CA: Sage.
117. Worcester, R. & Burns, T. (1975) A statistical examination of the relative precision of verbal scales. Journal of the Market Research Society, 17, pp. 181-197.
118. Wright, B.D. & Masters, G.N. (1982) Rating Scale Analysis. Chicago, IL: MESA Press.