Non-response and measurement error

Handbook of Survey Methodology for the Social Sciences

Volumn , Issue , 2012, Pages 149-178

  Billiet, Jaak ^a   Matsuo, Hideko ^a  

^a UNIVERSITY OF LEUVEN   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84994895496     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-1-4614-3876-2_10     Document Type: Chapter

References (121)

1. AAPOR. (2006). Standard Definitions. Final Dispositions of case Codes and Outcome rates for Surveys (pp. 46).
2. American Association for Public Opinion Research: http://www.aapor.org/pdfs/standarddefs_4.pdf)
3. Alwin, D. F. (1989). Problems in the estimation and interpretation of the reliability of survey data. Quality & Quantity, 23(3-4), 277-331.
4. Alwin, D. F. (1992). Information transmission in the survey interview: Number of response categories and the reliability of attitude measurement. In P. V. Marsden (Ed.), Sociological Methodology 1992 (pp. 83-118). San Francisco: Jossey Bass.
5. Alwin, D. F. (1997). Feeling thermometers versus 7-point scales. Which are better? Sociological Methods & Research, 25(3), 318-340.
6. Andrews, F. M. (1990). Some observations on metaanalysis of MTMM studies. In W. E. Saris & A. Van Meurs (Eds.), Evaluation of measurement instruments by meta-analysis of multitrait-multimethod studies (pp. 15-51). North Holland: Amsterdam.
7. Andrews, F. M. (1984). Construct validity and error components of survey measures:Astructural modeling approach. Public Opinion Quarterly, 48(2), 409-442.
8. Andrews, F. M., & Crandall, R. (1976). The validity of measures of self-reported well-being. Social Indicators Research, 3(1), 1-19.
9. Andrews, F. M., & Withey, S. B. (1976). Social indicators of well-being. Americans' perceptions of life quality. New York: Plenum Press.
10. Baumgartner, H., & Steenkamp, J. -B. E. M. (2001). Response styles in marketing research:Across national investigation. Journal of Marketing Research, 38(2), 143-156.
11. Belson, W. A. (1981). Design and understanding of survey questions. Aldershot: Gower.
12. Beckers, M., & Billiet, J. (2010). Uphold or revoke? A study of question wording in twelve municipal plebiscites in Flanders. World Political Science Review, 6(1), article 5, p. 29: http://www.bepress.com/wpsr/vol6/iss1/art5/
13. Biderman, A. (1980). Report of a workshop on applying cognitive psychology to recall problems of the national crime survey. Washington, DC: Bureau of Social Research.
14. Biemer, P., & Caspar, R. (1994). Continuous quality improvement for survey operations: Some general principles and applications. Journal of official statistics, 10(3), 307-326.
15. Biemer, P. P. (2010). Total survey error, design, implementation and evaluation. Public Opinion Quarterly, 74(5), 817-848.
16. Biemer, P. P., Groves, R. M., Lyberg, L. E., Mathiowetz, N. A., & Sudman, S. (Eds.). (1991). Measurement errors in survey. New York: Wiley.
17. Billiet, J. (2011). Godsdienstige betrokkenheid en de houding tegenover vreemden: Een verdwijnend verband? In K. Abts, K. Dobbelaere & L. Voyé (Eds.), Nieuwe tijden, nieuwe mensen. Belgen over arbeid, gezin, ethiek, religie en politiek, Chapt. X (pp. 215-243). Leuven: Lannoo campus & Koning Boudewijnstichting.
18. Billiet, J., Waterplas, L., & Loosveldt, G. (1992). Context effects as substantive data in social surveys. In N. Schwarz & S. Sudman (Eds.), Context effects in social and psychological research (pp. 131-147). New York: Springer.
19. Billiet, J., & McClendon, J. M. (2000). Modeling acquiescence in measurement models for two balanced sets of items. Structural Equation Modeling: A Multidisciplinary Journal, 7(4), 608-628.
20. Billiet, J., Cambré, B., & Welkenhuysen-Gybels, J. (2002). Equivalence of Measurement Instruments for Attitude Variables in Comparative Surveys taking Method Effects into Account: the Case of Ethnocentrism. In A. Ferligoj & A. Mrvar (Eds.), Developments in Social Science Methodology (pp. 73-96). Ljubljana: FDV.
21. Billiet, J., Swyngedouw, M., & Waege, H. (2004). Attitude Strength and Response Stability of a Quasi-Balanced Political Alienation Scale in a Panel Study. In W.E. Saris & P. Sniderman (Eds.), Studies in Public Opinion: Attitudes, Nonattitudes, Measurement Error and Change (pp. 268-292). Princeton: Princeton University.
22. Billiet, J., Koch, A., & Philippens, M. (2007a). Understanding and improving response rates. In R. Jowell, C. Roberts, R. Fitzgerald, & G. Eva (Eds.), Measuring attitudes cross-nationally: Lessons from the European Social Survey (pp. 113-137). London: Sage.
23. Billiet, J., Philippens, M., Fitzgerald, R., & Stoop, I. (2007b). Estimation of nonresponse bias in the European social survey: Using information from reluctant respondents. Journal of Official Statistics, 23(2), 135-162.
24. Billiet, J., Matsuo, H., Beullens, K., & Vehovar, V. (2009). Non-response bias in cross-national surveys: Designs for detection and adjustment in the ESS. ASK. Research and Methods, 18, 3-43.
25. Blom, A., Jäckle, A., & Lynn, P. (2010). The use of contact data in understanding cross-national differences in unit-nonresponse. In J. A. Harkness, M. Braun, B. Edwards, T. P. Johnson, T. L. E. Lyberg, P. P. Mohler, B.-E. Pennell, & T. W. Smith (Eds.), Survey methods in multicultural, multinational, and multiregional contexts (pp. 335-354). Hoboken: Wiley
26. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.
27. Bradburn, N. M. (1992). What have we learned? In N. Schwarz & S. Sudman (Eds.), Context effects in social and psychological research (pp. 315-324). New York: Springer.
28. Borhnstedt, G. W. (1983). Measurement. In P. H. Rossie, J. D. Wright, & A. B. Anderson (Eds.), Handbook of survey research: Quantitative studies in social relations (pp. 70-122). New York: Academic Press.
29. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81-105.
30. Cantril, H. (Ed.). (1944). Gauging public opinion. Princeton: Princeton University Press.
31. Converse, J. M. (1964). The nature of belief systems in mass publics. In D. Apter (Ed.), Ideology and discontent (pp. 206-261). New York: The Free Press.
32. Converse, J. M., & Presser, S. (1986). Survey Questions: Handcrafting the Standardized Questionnaire, Sage University, Series on Quantitative Applications in the Social Sciences. Beverly Hills: Sage Publications.
33. Couper, M. P., & De Leeuw, E. (2003). Nonresponse in cross-cultural and cross-national surveys. In J. A. Harkness, F. J. R. van de Vijver, & P. Ph. Mohler (Eds.), Cross-cultural survey methods (pp. 157-178). New Jersey: Wiley.
34. Couper, M. P. (2001). Measurement error in web surveys. Methodology & Statistics.
35. Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 281-302.
36. Davidov, E., Schmidt, P., & Billiet, J. (Eds.). (2011). Cross-cultural analysis: Methods and applications (p. 503). New York: Francis & Taylor.
37. De Wit, H. (1994). Cijfers en hun achterliggende realiteit. De MTMM-kwaliteitsparameters op hun kwaliteit onderzocht. Leuven: K.U.Leuven, Departement Sociologie.
38. De Wit, H., & Billiet, J. (1995). The MTMM design: Back to the founding fathers. In W. E. Saris & A. Münnich (Eds.), The multitrait-mutlimethod approach to evaluate measurement instruments (pp. 39-60). Budapest: Eötvös University Press.
39. de Heer, W. (1999). International response trends: Results of an international survey. Journal of Official Statistics, 15(2), 129-142.
40. Dillman, D. A. (1978). Mail and telephone surveys: The total design method. New York: Wiley.
41. Dillman, D. A. (2000). Mail and Internet surveys: The tailored design method. New York: Wiley.
42. Dillman, D. A., Phelps, G., Tortora, R., Swift, K, Kohrell, J., Berck, J., & Messer, B. L. (2009). Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet. Social Science Research, 38(1), 1-18.
43. Foddy, W. (1993). Constructing questions for interviews and questionnaires. Theory and practice in social research. Cambridge: Cambridge University Press.
44. Forsyth, B. H.,&Lessler, J. T. (1991). Cognitive laboratory methods: A taxonomy. In P. P. Biemer, et al. (Eds.), Measurement errors in surveys (pp. 393-418). New York:Wiley.
45. Greenleaf, E. A. (1992a). Improving rating scale measures by detecting and correcting bias components in some response styles. Journal of Marketing Research, 29(2), 176-188.
46. Greenleaf, E. A. (1992b). Measuring extreme response style. Public Opinion Quarterly, 56(3), 328-351.
47. Groves, R. M. (1989). Survey errors and survey costs. New York: Wiley.
48. Groves, R. M. (2006). Non response rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646-675.
49. Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2004). Survey methodology. New York: Wiley.
50. Groves, R., & Herringa, S. G. (2006). Responsive design for household surveys: Tools for actively controlling survey errors and costs. Journal of the Royal Statistical Society Series A, 169(3), 439-457.
51. Groves, R., & Lyberg, L. (2010). Total survey error: Past, present, and future. Public Opinion Quarterly, 74(5), 849-879.
52. Heise, D. R. (1969). Separating reliability and stability in test-retest-correlation. American Sociological Review, 34(1), 93-101.
53. Heerwegh, D. (2003). Explaining response latencies and changing answers using client-side paradata from a web survey. Social Science Computer Review, 21(3), 360-373.
54. Hippler, H.-J., Schwarz, N., & Sudman, S. (Eds.). (1987). Social information processing and survey methodology. New York: Springer.
55. Holleman, B. (2000). The forbid/allow assymetrie. On the cognitive mechanisms underlying wording effects in surveys. Amsterdam: Ed. Rodopi.
56. Jabine, T. B, Straf, M. L, Tanur, J. M.,&Rourangeau, R. T. (Eds.). (1984). Cognitive aspects of survey methodology: Building a bridge between disciplines. Washington, DC: National Academy Press.
57. Jowell, R., Roberts, C., Fitzgerald, R., & Eva, G. (Eds.). (2007). Measuring attitudes cross-nationally: Lessons from the European social survey. London: Sage.
58. Jöreskog, K. G. (1971). Statistical analysis of sets of congeneric tests. Psychometrica, 36(2), 109-133.
59. Kallenberg, A.L., & Kluegel, J.R. (1975). Analysis of the multitrait multimethod matrix: some limitations and an alternative. Journal of Applied Psychology, 60, 1-9.
60. Kaminska, O., McCutcheon, A., & Billiet, J. (2011). Satisficing among reluctant respondents in a crossnational context. Public Opinion Quarterly, 74(5), 956-984.
61. Kalton, G., & Schuman, H. (1982). The effect of the question on survey responses: A Review. Journal of the Royal Statistical Society, 145(1), 42-73.
62. Kreuter, F., & Kohler, U., (2009). Analysing contact sequences in call record data. Potential and limitations of sequence indicators for nonresponse adjustment in the European Social Survey. Journal of Official Statistics, 25, 203-226.
63. Kreuter, F., Olson, K., Wagner, J., Yan, T., Ezzati-Rice, T., Casas-Cordero, C., et al. (2010). Using proxy measures and other correlates of survey outcomes to adjust for nonresponse: Examples from multiple surveys. Journal of the Royal Statistical Society Series A, 173(2), 389-407.
64. Krosnick, J. A., Narayan, S., & Smith, W. R. (1996). Satisficingin surveys: Initial evidence. In M.T.Braverman & J. K. Slater (Eds.), Advances in survey research (pp. 29-44). San Francisco: Josey-Bass.
65. Loosveldt, G., Carton, A., & Billiet, J. (2004). Assessment of survey data quality: A pragmatic approach focused on interviewer tasks. International Journal of Market Research, 46(1), 65-82.
66. Loosveldt, G., & Beullens, K. (2009). Fieldwork monitoring. Work package 6, Deliverable 5. (Version 2): http://www.risq-project.eu
67. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading: Addison-Wesley.
68. Lyberg,L., Biemer, P., Collins,M., deLeeuw,E., Dippo, C., Schwarz, N., & Trewin, D. (1997) (Eds.), Survey measurement and process quality. New York: Wiley.
69. Lynn, P. J., Beerten, R., Laiho, J., & Martin, J. (2001) Recommended standard final outcome categories and standard definitions of response rate for social surveys. Working Paper 2001, 23 October.
70. Lynn, P. (2003). PEDAKSI: Methodology for collecting data about survey non-respondents. Quality & Quantity, 37, 239-261.
71. Lynn, P. & Kaminska, O. (2011). The impact of mobile phones on survey measurement error (p. 45). Working Paper Series. Essex: Institute for Social & Economic Research, 2011-2007
72. Matsuo, H., Billiet, J., Loosveldt, G., & Malnar, B. (2010a). Response-based quality assessment of ESS Round 4: Results for 30 countries based on contact files (pp. 80). CeSO: Working paper, CeSO/SM/2010-2.
73. Matsuo, H., Billiet, J., Loosveldt, G., Berglund, F., & Kleven, Ø. (2010b). Measurement and adjustment of non-response bias based on non-response surveys: The case of Belgium and Norway in European Social Survey Round 3. Survey Research Methods, 4(3): 165-178.
74. McClendon, M. J. (1991). Acquiescence and recency response-order effect in interview surveys. Sociological Methods and Research, 20(1), 60-103.
75. McClendon, M.J. (1992). On the measurement and control of acquiescence in latent variable models. Paper presented at the annual meeting of the American Sociological Association. (pp. 24-29). Pittsburg: Augustus.
76. Mirowsky, J., & Ross, C. E. (1991). Eliminating defense and agreement bias from measures of the sense of control: A 292 Index. Social Psychology Quarterly, 54(2), 127-145.
77. Molenaar, N. J. (1986). Formuleringseffecten in surveyinterviews: een nonexperimenteel onderzoek. Amsterdam: Vrije Universiteit.
78. Moors, G. (2004). Facts and artefacts in the comparison of attitudes among ethnic minorities. A multigroup latent class structure model with adjustment for response style behavior. European Sociological Review, 20(4), 303-320.
79. Olson, K. (2006). Survey participation, nonresponse bias, measurement error bias, and total bias. Public Opinion Quarterly, 70(5), 737-758.
80. O'Muircheartaigh, C., Gaskell, G., & Wright, D. B. (1995). Weighing anchors: Verbal and numeric labels for response scales. Journal of Official Statistics, 11(3), 295-307.
81. Paulhus, D. L. (1991). Measurement and control of response bias. In J. P. Robinson, P. R. Shaver, & L. S. Wrightsman (Eds.), Measures of personality and social psychological attitudes (pp. 17-59). San Diego: Academic Press.
82. Payne, S. L. (1951). The art of asking questions. Princeton: Princeton University Press.
83. Rorer, L. G. (1965). The great response style myth. Psychological Bulletin, 63(3), 129-156.
84. Rossi, P. H., Wright, J. D., & Anderson, A. B. (1983). Handbook of survey research: Quantitative Studies in Social Relations. New York: Academic Press.
85. Rugg, D. (1941). Experiments in wording questions: II. Public Opinion Quarterly, 5(1), 91-92.
86. Saris, W.E. (1990a). Models or evaluation of measurement instruments. In W. E. Saris. & A. van Meurs. (Eds.), Evaluation of measurement instruments by meta-analysis of multitrait-multimethod studies (pp. 52-80). Amsterdam: North Holland.
87. Saris, W. E. (1990b). The choice of a model fore evaluation of measurement instruments. In W. E. Saris & A. van Meurs (Eds.), Evaluation of measurement instruments by meta-analysis of multitrait-multimethod studies (pp. 118-132). Amsterdam: North Holland.
88. Saris, W. (1995). Designs and models for quality assessment of survey measures. In W. Saris & A. Munnich (Eds.), The multitrait-Multimethod Approach to Evaluate measuremnt Instruments. Eötvös University press: Budapest.
89. Saris, W. E., & Andrews, F. M. (1991). Evaluation of measurement instruments using a structural modeling approach. In P. P. Biemer, R. M. Groves, L. E. Lyberg, N. A. Mathiowetz, & S. Sudman (Eds.), Measurement errors in surveys (pp. 575-598). New York: Wiley.
90. Saris, W. E., Satorra, A., & Coenders, G. (2004). A new approach to evaluating quality of measurement instruments: The split-ballot MTMM Design. Sociological Methodology, 34(1), 311-347.
91. Saris, W. E., & Gallhofer, I. N. (2007). Design, evaluation, and analysis of questionnaires for survey research. Hoboken: Wiley.
92. Saris, W. E. & Sniderman, P. M. (Eds.) (2004). Studies in public opinion: Attitudes, nonattitudes, measurement error and change. Princeton: Princeton University.
93. Scherpenzeel, A. (1995). Misspecification effects. In W. E. Saris & A. Münnich (Eds.), The multitraitmutlimethod approach to evaluate measurement instruments (pp. 61-70). Budapest: Eötvös University Press.
94. Scherpenzeel, A. C., & Saris, W. E. (1997). The validity and reliability of survey questions: A meta-analysis of MTMM studies. Sociological Methods & Research, 25(3), 341-383.
95. Schmitt, N., & Stults, D. N. (1986). Methodology review. Analysis of multitrait-multimethod matrices. Applied Psychological Measurement, 10, 1-22.
96. Schouten, B., Cobben, F., & Bethlehem, J. (2009). Indicators for the representativeness of survey response. Survey Methodology, 35(1), 101-113.
97. Schuman, H., & Presser, S. (1979). The open and closed question. American Sociological Review, 44, 692-712.
98. Schuman, H. (1982). Artifacts are in the mind of the beholder. The American Sociologist, 17, 21-28.
99. Schuman, H. (1984). Response effects with subjective survey questions. In C. F. Turner & E. Martin (Eds.), Surveying subjective phenomena (Vol. Vol. I, pp. 129-147). New York: Russell Sage Foundation.
100. Schuman, H., & Presser, S. (1981). Questions and answers in attitude surveys: Experiments on question form, wording and context. New York: Academic Press.
101. Schuman, H., & Ludwig, J. (1983). The norm of even handedness in surveys as in life. American Sociological Review, 48(1), 112-120.
102. Schwarz, N., & Hippler, H. J. (1987). What response scales may tell your respondents. In H. Hippler, N. Schwarz, & S. Sudman (Eds.), Social information processing and survey methodology (pp. 163-178). New York: Springer.
103. Schwarz, N., Strack, F., Mueller, G., & Chassein, B. (1988). The range of response alternatives may determine the meaning of questions: Further evidence on informative functions of response alternatives. Social Cognition, 6(2), 107-117.
104. Schwarz, N., & Hippler, H. J. (1991). Response alternatives: The impact of their choice and presentation order. In P. P. Biemer, R. M. Groves, L. E. Lyberg, N. A. Mathiowetz, & S. Sudman (Eds.), Measurement errors in surveys (pp. 163-178). New York: Wiley.
105. Schwarz, N., & Sudman, S. (Eds.) (1992). Context effects in social and psychological research. New York: Springer.
106. Schwarz, N., Knäper, B., Oyserman, D., & Stich, C. (2008). The psychology of asking questions. In E. de Leeuw, J.J. Hox & D.A. Dillman (Eds.), International Handbook of Survey Methodology. (pp. 18-34). New York: Lawrence Erlbaum Associates.
107. Smith, T. W. (1984). Nonattitudes: A review and evaluation. In C. F. Turner & E. Martin (Eds.), Surveying subjective phenomena (Vol. 2, pp. 215-255). New York: Russell Sage Foundation.
108. Sniderman, P. M., & Grob, D. B. (1996). Innovations in experimental design in attitude surveys. Annual Review of Sociology, 22, 377-399.
109. Sniderman, P. M., & Hagendoorn, L. (2009). When ways of life collide: Multiculturalism and its discontents in The Netherlands. Princeton: Princeton University Press.
110. Strack, F., & Martin, L. L. (1987). Thinking, judging, and communicating: A processs account of context effects in attitude surveys. In H. Hippler, N. Schwarz & S. Sudman (Eds.) Social Information Processing and Survey Methodology (pp. 123-148). New York: Springer.
111. Strack, F., Schwarz, N., & Wänke, M. (1991). Semantic and pragmatic aspects of context effects in social and psychological research. Social Cognition, 9(1), 111-125.
112. Sudman, S., & Bradburn, N. M. (1974). Response effects in surveys: A review and synthesis. Chicago: Aldine.
113. Sudman, S., & Bradburn, N. M. (1982). Asking questions. A practical guide to questionnaire design. San Francisco: Jossey Bass.
114. Sudman, S., Bradburn, N. M., & Schwarz, N. (1996). Thinking about answers. The application of cognitive processes to survey methodology. San Francisco: Jossey-Bass Publishers.
115. Stoop, I., Billiet, J., Koch, A., & Fitzgerald, R. (2010). Improving survey response: Lessons learned from the European Social Survey. Chichester: Wiley.
116. Tourangeau, R. (1992). Context effects on responses to attitude questions: attitudes as memory structures. In N. Schwarz & S. Sudman (Eds.), Context effects in social and psychological research (pp. 35-48). New York: Springer.
117. Tourangeau, R., & Rasinski, K. A. (1988). Cognitive processes underlying context effects in attitude measurement. Psychological Bulletin, 103(3), 299-314.
118. Tourangeau, R., Rips, L. J., & Rasinski, K. A. (2000). The psychology of survey response. Cambridge: Cambridge University Press.
119. Vehovar, V. (2007). Non-response bias in the European Social Survey. In G. Loosveldt, M. Swyngedouw & B. Cambré (Eds.), Measuring meaningful data in social research (pp. 335-356). Leuven: Acco publisher.
120. Weijters, B. (2006). Response styles in consumer research. Belgium: Vlerick Leuven Gent Management School.
121. Zaller, J. R. (1992). The nature and origins of mass opinion. Cambridge: Cambridge University Press.