Unit nonresponse and weighting adjustments: A critical review

Journal of Official Statistics

Volumn 29, Issue 3, 2013, Pages 329-353

  Michael Brick, J ^a  

^a WESTAT   (United States)

Author keywords

Bias; Calibration; Data collection; Response propensity

Indexed keywords

EID: 84883400884     PISSN: 0282423X     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (98)

1. Andridge, R.H. and Little, R.J. (2011). Proxy Pattern-Mixture Analysis for Survey Nonresponse. Journal of Official Statistics, 27, 153-180.
2. Atrostic, B.K., Bates, N., Burt, G., and Silberstein, A. (2001). Nonresponse in U.S. Government Household Surveys: Consistent Measures, Recent Trends, and New Insights. Journal of Official Statistics, 17, 209-226.
3. Bartholomew, D.J. (1961). A Method of Allowing for 'Not-at-Home' Bias in Sample Surveys. Applied Statistics, 10, 52-59.
4. Bates, N., Dahlhamer, J., and Singer, E. (2008). Privacy Concerns, too Busy, or Just not Interested: Using Doorstep Concerns to Predict Survey Nonresponse. Journal of Official Statistics, 24, 591-612.
5. Beaumont, J.F. (2005). On the Use of Data Collection Process Information for the Treatment of Unit Nonresponse Through Weight Adjustment. Survey Methodology, 31, 227-231.
6. Bethlehem, J.G. (1988). Reduction of Nonresponse Bias Through Regression Estimation. Journal of Official Statistics, 4, 251-260.
7. Bethlehem, J.G. (2002). Weighting Nonresponse Adjustments Based on Auxiliary Information. Survey Nonresponse, R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little (eds). New York: Wiley.
8. Bethlehem, J., Cobben, F., and Schouten, B. (2011). Handbook in Nonresponse in Household Surveys. New York: Wiley.
9. Brehm, J. (1993). The Phantom Respondents: Opinion Surveys and Political Representation. Ann Arbor: University of Michigan Press.
10. Brick, J.M. and Jones, M.E. (2008). Propensity to Respond and Nonresponse Bias. Metron-International Journal of Statistics, LXVI, 51-73.
11. Brick, J.M. and Kalton, G. (1996). Handling Missing Data in Survey Research. Statistical Methods in Medical Research, 5, 215-238.
12. Brick, J.M. and Montaquila, J.M. (2009). Nonresponse and Weighting. Handbook of Statistics. Sample Surveys: Design, Methods, and Applications, D. Pfeffermann and C.R. Rao (eds). Vol. 29A. Amsterdam: Elsevier-North Holland, 163-186.
13. Brick, J.M., Montaquila, J., Han, D., and Williams, D. (2012). Improving Response Rates for Spanish-Speakers in Two-Phase Mail Surveys. Public Opinion Quarterly, 76, 721-732.
14. Brick, J.M. and Williams, D. (2013). Explaining Rising Nonresponse Rates in Cross-Sectional Surveys. The ANNALS of the American Academy of Political and Social Science, 645, 36-59.
15. Cassel, C., Särndal, C.-E., and Wretman, J. (1983). Some Uses of Statistical Models in Connection With the Nonresponse Problem. Incomplete Data in Sample Surveys, W.G. Madow and I. Olkin (eds). Vol. 3. New York: Academic Press.
16. Chang, T. and Kott, P.S. (2008). Using Calibration Weighting to Adjust for Nonresponse Under a Plausible Model. Biometrika, 95, 557-571.
17. Cochran, W. (1977). Sampling Techniques, (3rd edition). New York: Wiley.
18. Colley, R.H. (1945). Don't Look Down Your Nose at Mail Questionnaires. Printers' Ink, March, 16, 21-108.
19. Curtin, R., Presser, S., and Singer, E. (2000). The Effects of Response Rate Changes on the Index of Consumer Sentiment. Public Opinion Quarterly, 64, 413-428.
20. Curtin, R., Presser, S., and Singer, E. (2005). Changes in Telephone Survey Nonresponse Error Over the Past Quarter Century. Public Opinion Quarterly, 69, 87-98.
21. Da Silva, D.N. and Opsomer, J.D. (2004). Properties of the Weighting Cell Estimator Under a Nonparametric Response Mechanism. Survey Methodology, 30, 45-55.
22. Da Silva, D.N. and Opsomer, J.D. (2009). Nonparametric Propensity Weighting for Survey Nonresponse Through Local Polynomial Regression. Survey Methodology, 35, 165-176.
23. Dalenius, T. (1983). Some Reflections on the Problem of Missing Data. Incomplete Data in Sample Surveys, W.G. Madow and I. Olkin (eds). Vol. 3. New York: Academic Press, 411-413.
24. David, M., Little, R., Samuhel, M., and Triest, R. (1983). Nonrandom Nonresponse Models Based on the Propensity to Respond. Proceedings of the Business and Economic Statistics Section of the American Statistical Association, 168-173.
25. David, M., Little, R.J.A., Samuhel, M., and Triest, R. (1986). Alternative Methods for CPS Income Imputation. Journal of the American Statistical Association, 81, 29-41.
26. De Leeuw, E. and De Heer, W. (2002). Trends in Household Survey Nonresponse: A Longitudinal and International Comparison. Survey Nonresponse, R.M. Groves, D.A. Dillman, J.L. Eltinge, and R.J.A. Little (eds). New York: Wiley, 41-54.
27. Deming, W. (1953). On a Probability Mechanism to Attain an Economic Balance Between Resultant Error of Response and the Bias of Nonresponse. Journal of the American Statistical Association, 48, 743-772.
28. Deville, J.C. and Särndal, C.-E. (1992). Calibration Estimators in Survey Sampling. Journal of the American Statistical Association, 87, 376-382.
29. Dillman, D. (1978). Mail and Telephone Surveys: The Total Design Method. New York: Wiley.
30. Dillman, D., Smyth, J., and Christian, L. (2009). Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method, (3rd edition). New York: Wiley.
31. Dunkelburg, W. and Day, G. (1973). Nonresponse Bias and Callbacks in Sample Surveys. Journal of Marketing Research, 10, 160-168.
32. Ferber, R. (1949). The Problem of Bias in Mail Returns: A Solution. Public Opinion Quarterly, 12, 669-676.
33. Feskins, R., Hoop, J., Lensvelt-Mulders, G., and Schmeets, H. (2011). Collecting Data Among Ethnic Minorities in an International Perspective. Field Methods, 18, 284-304.
34. Fuller, W.A., Loughin, M.M., and Baker, H.D. (1994). Regression Weighting for the 1987-88 National Food Consumption Survey. Survey Methodology, 20, 75-85.
35. Goyder, J. (1987). The Silent Minority: Nonrespondents on Sample Surveys. Boulder, CO: Westview Press.
36. Greenlees, J., Reece, W., and Zieschang, K. (1982). Imputation of Missing Values When the Probability of Response Depends on the Variable Being Imputed. Journal of the American Statistical Association, 77, 251-261.
37. Groves, R.M. (2006). Nonresponse Rates and Nonresponse Bias in Household Surveys. Public Opinion Quarterly, 70, 646-675.
38. Groves, R.M. and Couper, M.P. (1998). Nonresponse in Household Interview Surveys. New York: Wiley.
39. Groves, R.M., Couper, M., Presser, S., Singer, E., Tourangeau, R., Acosta, G.P., and Nelson, L. (2006). Experiments in Producing Nonresponse Bias. Public Opinion Quarterly, 70, 720-736.
40. Groves, R., Dillman, D., Eltinge, J., and Little, R. (2002). Survey Nonresponse. New York: Wiley, 41-54.
41. Groves, R.M. and Heeringa, S.G. (2006). Responsive Design for Household Surveys: Tools for Actively Controlling Survey Errors and Costs. Journal of the Royal Statistical Society, Series A, 169, 439-457.
42. Hansen, M.H. and Hurwitz, W.N. (1946). The Problem of Non-Response in Sample Surveys. Journal of the American Statistical Association, 41, 517-529.
43. Haring, R., Alte, D., Völzkea, H., Sauer, S., Wallaschofski, H., John, U., and Schmidt, C. (2009). Extended Recruitment Efforts Minimize Attrition but not Necessarily Bias. Journal of Clinical Epidemiology, 62, 252-260.
44. Hartley, H.O. (1946). Discussion of "A Review of Recent Statistical Developments in Sampling and Sample surveys. ". Journal of the Royal Statistical Society, 109, 37-38.
45. Heckman, J. (1979). Sample Selection Bias as a Specification Error. Econometrica, 47, 153-162.
46. Holt, D. and Smith, T.M.F. (1979). Post-Stratification. Journal of the Royal Statistical Society, Series A, 142, 33-46.
47. Ingen, E., Stoop, I., and Breedveld, K. (2009). Nonresponse in the Dutch Time Use Survey: Strategies for Response Enhancement and Bias Reduction. Field Methods, 21, 69-90.
48. Kalton, G. (1983). Compensating for Missing Survey Data. Ann Arbor: University of Michigan Press.
49. Kalton, G. and Flores-Cervantes, I. (2003). Weighting Methods. Journal of Official Statistics, 18, 81-97.
50. Kalton, G. and Kasprzyk, D. (1986). The Treatment of Missing Survey Data. Survey Methodology, 12, 1-16.
51. Keeter, S., Miller, C., Kohut, A., Groves, R.M., and Presser, S. (2000). Consequences of Reducing Nonresponse in a Large National Telephone Survey. Public Opinion Quarterly, 64, 125-148.
52. Kreuter, F., Olson, K., Wagner, J., Yan, T., Ezzati-Rice, T.M., Casas-Cordero, C., Lemay, M., Peytchev, A., Groves, R.M., and Raghunathan, T.E. (2010). Using Proxy Measures and Other Correlates of Survey Outcomes to Adjust for Non-Response: Examples from Multiple Surveys. Journal of the Royal Statistical Society, Series A, 173, 389-407.
53. Lin, I.-F. and Schaeffer, N.C. (1995). Using Survey Participants to Estimate the Impact of Nonparticipation. Public Opinion Quarterly, 59, 236-258.
54. Little, R.J.A. (1986). Survey Nonresponse Adjustments for Estimates of Means. International Statistical Review, 54, 139-157.
55. Little, R.J.A. (1993). Pattern-Mixture Models for Multivariate Incomplete Data. Journal of the American Statistical Association, 88, 125-134.
56. Little, R.J.A. and Rubin, D.B. (2002). Statistical Analysis With Missing data, (2nd edition). New York: Wiley.
57. Lumley, T., Shaw, P., and Dai, J. (2011). Connections Between Survey Calibration Estimators and Semiparametric Models for Incomplete Data. International Statistical Review, 79, 200-220.
58. Lundström, S. and Särndal, C.-E. (1999). Calibration as a Standard Method for Treatment of Nonresponse. Journal of Official Statistics, 15, 305-327.
59. Madow, W.G., Nisselson, H., and Olkin, I. (1983). Incomplete Data in Sample Surveys, Vol. 1. New York: Academic Press.
60. Madow, W.G. and Olkin, I. (1983). Incomplete Data in Sample Surveys, Vol. 3. New York: Academic Press.
61. Madow, W.G., Olkin, I., and Rubin, D.B. (1983). Incomplete Data in Sample Surveys, Vol. 2. New York: Academic Press.
62. Merkle, D., Edelman, M., Dykeman, K., and Brogan, C. (1998). An Experimental Study of Ways to Increase Exit Poll Response Rates and Reduce Survey Error. Paper presented at the Annual Conference of the American Association for Public Opinion Research, St. Louis, MO.
63. Micklewright, J., Schnepf, S., and Skinner, C. (2012). Non-Response Biases in Surveys of Schoolchildren: The Case of the English Programme for International Student Assessment (PISA) samples. Journal of the Royal Statistical Society, Series A, 175, 915-938.
64. Mohadjer, L., Berlin, M., Rieger, S., Waksberg, J., Rock, D., Yamamoto, K., Kirsch, I., and Kolstad, A. (1997). The Role of Incentives in Literacy Survey Research. Adult Basic Skills: Innovations in Measurement and Policy Analysis, A. Tuijnman, I. Kirsch, and D. Wagner (eds). Creskill, NJ: Hampton Press.
65. Molenberghs, G., Beunckens, C., Sotto, C., and Kenward, M.G. (2008). Every Missingness not at Random Model has a Missingness at Random Counterpart With Equal Fit. Journal of the Royal Statistical Society: Series B, 70, 371-388.
66. Oh, H.L. and Scheuren, F.J. (1983). Weighting Adjustments for Unit Nonresponse. Incomplete Data in Sample Surveys, W.G. Madow, I. Olkin, and D.B. Rubin (eds). Vol. 2. New York: Academic Press, 143-184.
67. Olsen, K. and Groves, R.M. (2012). An Examination of Within-Person Variation in Response Propensity over the Data Collection Field Period. Journal of Official Statistics, 28, 29-51.
68. Peytcheva, E. and Groves, R.M. (2009). Using Variation in Response Rates of Demographic Subgroups as Evidence of Nonresponse Bias in Survey Estimates. Journal of Official Statistics, 25, 193-201.
69. Phipps, P. and Toth, D. (2012). Analyzing Establishment Nonresponse Using an Interpretable Regression Tree Model with Linked Administrative Data. Annals of Applied Statistics, 6, 772-794.
70. Politz, A. and Simmons, W. (1949). An Attempt to Get "Not at Homes" Into the Sample Without Callbacks. Journal of the American Statistical Association, 44, 9-31.
71. Rosenbaum, P.R. and Rubin, D.B. (1983). The Central Role of the Propensity Score in Observational Studies for Causal Effects. Biometrika, 70, 41-55.
72. Rubin, D.B. (1976). Inference and Missing Data (with discussion). Biometrika, 63, 581-592.
73. Särndal, C.-E. (2011a). Morris Hansen Lecture: Dealing With Survey Nonresponse in Data Collection, in Estimation. Journal of Official Statistics, 27, 1-21.
74. Särndal, C.-E. (2011b). Three Factors to Signal Non-Response Bias with Applications to Categorical Auxiliary Variables. International Statistical Review, 79, 233-254.
75. Särndal, C.-E. and Lundström, S. (2005). Estimation in Surveys with Nonresponse. Chichester, UK: Wiley.
76. Särndal, C.-E. and Lundström, S. (2008). Assessing Auxiliary Vectors for Control of Nonresponse Bias in the Calibration Estimator. Journal of Official Statistics, 4, 251-260.
77. Särndal, C.-E. and Lundström, S. (2010). Design for Estimation: Identifying Auxiliary vectors to reduce nonresponse bias. Survey Methodology, 36, 131-144.
78. Särndal, C.-E., Swensson, B., and Wretman, J. (1992). Model Assisted Survey Sampling. New York: Springer-Verlag.
79. Schmeets, H. (2010). Increasing Response Rates and the Consequences in the Dutch Parliamentary Election Study 2006. Field Methods, 22, 391-412.
80. Schouten, B. (2007). A Selection Strategy for Weighting Variables Under a Not-Missingat-Random Assumption. Journal of Official Statistics, 23, 51-68.
81. Schouten, B., Calinescu, M., and Luiten, A. (2011a). Optimizing Quality of Response Through Adaptive Survey Designs. The Hague: Statistics Netherlands, Available at: http://www.cbs.nl/NR/rdonlyres/2D62BF4A-6783-4AC4-8E4512EF20C6675C/0/ 2011x1018.pdf. (Accessed May 24, 2013).
82. Schouten, B., Cobben, F., and Bethlehem, J. (2009). Measures for the Representativeness of Survey Response. Survey Methodology, 35, 101-113.
83. Schouten, B., Schlomo, N., and Skinner, C. (2011b). Indicators for Monitoring and Improving Representativeness of Response. Journal of Official Statistics, 27, 231-253.
84. Singer, E. (2002). Use of Incentives to Reduce Nonresponse in Household Surveys. Survey Nonresponse, R. Groves, D. Dillman, J. Eltinge, and R. Little (eds). New York: Wiley, 163-177.
85. Singer, E. and Ye, C. (2013). The Use and Effects of Incentives in Surveys. The ANNALS of the American Academy of Political and Social Science, 645, 112-141.
86. Skinner, C.J. and D'Arrigo, J. (2011). Inverse Probability Weighting for Clustered Nonresponse. Biometrika, 98, 953-966.
87. Smith, T.W. (1995). Trends in Non-Response Rates. International Journal of Public Opinion Research, 7, 157-171.
88. Steeh, C., Kirgis, N., Cannon, B., and DeWitt, J. (2001). Are They Really as Bad as They Seem? Nonresponse Rates at the End of the Twentieth Century. Journal of Official Statistics, 17, 227-247.
89. Steele, F. and Durrant, G.B. (2011). Alternative Approaches to Multilevel Modelling of Survey Non-Contact and Refusal. International Statistical Review, 79, 70-91.
90. Stoop, I.A.L. (2005). The Hunt for the Last Respondent: Nonresponse in Sample Surveys. The Hague: Social and Cultural Planning Office.
91. Stoop, I., Billiet, J., Koch, A., and Fitzgerald, R. (2010). Improving Survey Response: Lessons Learned from the European Social Survey. Chichester: Wiley.
92. Synodinos, N.E. and Yamada, S. (2000). Response Rate Trends in Japanese Surveys. International Journal of Public Opinion Research, 12, 48-72.
93. Tanur, J. (1999). Looking Backwards and Forwards at the CASM Movement. Cognition and Survey Research, M. Sirken, D. Hermann, S. Schechter, N. Schwarz, J. Tanur, and R. Tourangeau (eds). New York: Wiley, 13-20.
94. Thomsen, I. (1973). A Note on the Efficiency of Weighting Subclass Means to Reduce the Effects of Nonresponse When Analyzing Survey Data. Statistisk Tidskrift, 11, 278-285.
95. Tourangeau, R., Rips, L.J., and Rasinski, K. (2000). The Psychology of Survey Response. New York: Cambridge University Press.
96. Wagner, J. (2010). The Fraction of Missing Information as a Tool for Monitoring the Quality of Survey Data. Public Opinion Quarterly, 74, 223-243.
97. Wetzels, W., Schmeets, H., Van den Brakel, J., and Feskens, R. (2008). Impact of Prepaid Incentives in Face-to-Face Surveys: A Large-Scale Experiment With Postage Stamps. International Journal of Public Opinion Research, 20, 507-516.
98. Yates, F. (1946). A Review of Recent Statistical Developments in Sampling and Sample Surveys. Journal of the Royal Statistical Society, 109, 12-43.