An examination of the quality and utility of interviewer observations in the National Survey of Family Growth

Journal of the Royal Statistical Society. Series A: Statistics in Society

Volumn 176, Issue 1, 2013, Pages 211-225

  West, Brady T ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

Author keywords

Error in auxiliary variables; Interviewer observations; Non response adjustment; Paradata; Survey non response

Indexed keywords

EID: 84871645368     PISSN: 09641998     EISSN: 1467985X     Source Type: Journal    
DOI: 10.1111/j.1467-985X.2012.01038.x     Document Type: Article

References (52)

1. Alwin, D. F. (2008) Margins of Error: a Study of Reliability in Survey Measurement. Hoboken: Wiley.
2. Baruch, Y. and Holtom, B. C. (2008) Survey response rate levels and trends in organizational research. Hum. Relns, 61, 1139-1160.
3. Beaumont, J.-F. (2005) On the use of data collection process information for the treatment of unit nonresponse through weight adjustment. Surv. Methodol., 31, 227-231.
4. Bethlehem, J. (2002) Weighting nonresponse adjustments based on auxiliary information. In Survey Nonresponse (eds R. Groves, D. Dillman, J. Eltinge and R. Little ), pp. 275-287. New York: Wiley.
5. Biemer, P. P., Chen, P. and Wang, K. (2011) Errors in the recorded number of call attempts and their effect on nonresponse adjustments using callback models. To be published.
6. Biemer, P. P., Wang, K. and Chen, P. (2010) Using call-back data to adjust for nonignorable nonresponse: results of an empirical study. Jt Statist. Meet., Vancouver, May .
7. Biener, L., Garrett, C. A., Gilpin, E. A., Roman, A. M. and Currivan, D. B. (2004) Consequences of declining survey response rates for smoking prevalence estimates. Am. J. Preventv. Med., 27, 254-257.
8. Blom, A. G., de Leeuw, E. D. and Hox, J. J. (2011) Interviewer effects on nonresponse in the European Social Survey. J. Off. Statist., 27, 359-377.
9. Campanelli, P., Sturgis, P. and Purdon, S. (1997) Can You Hear Me Knocking: an Investigation into the Impact of Interviewers on Survey Response Rates. London: Social and Community Planning Research.
10. Couper, M. P. (1998) Measuring survey quality in a CASIC environment. In Proc. Jt Statist. Meet., Dallas. Alexandria: American Statistical Association.
11. Couper, M. P. and Lyberg, L. (2005) The use of paradata in survey research. In Proc. 55th Sess. International Statistical Institute, Sydney. Voorburg: International Statistical Institute.
12. Cull, W. L., O'Connor, K. G., Sharp, S. and Tang, S. S. (2005) Response rates and response bias for 50 surveys of pediatricians. Hlth Serv. Res., 40, 213-226.
13. Curtin, R., Presser, S. and Singer, E. (2005) Changes in telephone survey nonresponse over the past quarter century. Publ. Opin. Q., 69, 87-98.
14. Drury, T. F., Moy, C. S. and Poe, G. S. (1980) Going beyond interviewer observations of race in the National Health Interview Survey. In Classification Issues in Measuring the Health Status of Minorities. Hyattsville: National Center for Health Statistics.
15. Durrant, G. B., D'Arrigo, J. and Steele, F. (2011) Using paradata to predict best times of contact, conditioning on household and interviewer influences. J. R. Statist. Soc. A, 174, 1029-1049.
16. Durrant, G. B., D'Arrigo, J. and Steele, F. (2012) Analysing interviewer call record data using a multilevel discrete time event history modelling approach. Submitted to J. R. Statist. Soc. A.
17. Durrant, G. B., Groves, R. M., Staetsky, L. and Steele, F. (2010) Effects of interviewer attitudes and behaviors on refusal in household surveys. Publ. Opin. Q., 74, 1-36.
18. Durrant, G. B. and Steele, F. (2009) Multilevel modelling of refusal and non-contact non-response in household surveys: evidence from six UK Government surveys. J. R. Statist. Soc. A, 172, 361-381.
19. Faraway, J. J. (2005) Linear Models with R. Boca Raton: Chapman and Hall-CRC.
20. Feldman, J. J., Hyman, H. and Hart, C. W. (1951) A field study of interviewer effects on the quality of survey data. Publ. Opin. Q., 15, 734-761.
21. Fuller, W. (1987) A single explanatory variable. In Measurement Error Models, ch. 1. New York: Wiley.
22. Groves, R. M. (2006) Nonresponse rates and nonresponse bias in household surveys. Publ. Opin. Q., 70, 646-675.
23. Groves, R. M. and Heeringa, S. G. (2006) Responsive design for household surveys: tools for actively controlling survey errors and costs. J. R. Statist. Soc. A, 169, 439-457.
24. Groves, R. M., Mosher, W. D., Lepkowski, J. and Kirgis, N. G. (2009) Planning and development of the continuous National Survey of Family Growth. In Vital Health Statistics, vol. 1, no. 48. National Center for Health Statistics.
25. Hahn, R. A., Truman, B. T. and Barker, N. D. (1996) Identifying ancestry: the reliability of ancestral identi fication in the United States by self, proxy, interviewer, and funeral director. Epidemiology, 7, 75-80.
26. Hill, M. E. (2002) Race of the interviewer and perception of skin color: evidence from the Multi-city Study of Urban Inequality. Am. Sociol. Rev., 67, 99-108.
27. Kott, P. S. (2006) Using calibration weighting to adjust for nonresponse and coverage errors. Surv. Methodol., 32, 133-142.
28. Kreuter, F., Olson, K., Wagner, J., Yan, T., Ezzati-Rice, T. M., Casas-Cordero, C., Lemay, M., Peytchev, A., Groves, R. M. and Raghunathan, T. E. (2010) Using proxy measures and other correlates of survey outcomes to adjust for non-response: examples from multiple surveys. J. R. Statist. Soc. A, 173, 389-407.
29. Landis, J. R. and Koch, G. G. (1977) The measurement of observer agreement for categorical data. Biometrics, 33, 159-174.
30. de Leeuw, E. and de Heer, W. (2002) Trends in household survey nonresponse: a longitudinal and international comparison. In Survey Nonresponse (eds R. M. Groves, D. A. Dillman, J. L. Eltinge and R. J. A. Little), ch. 3. New York: Wiley.
31. Lepkowski, J. M., Mosher, W. D., Davis, K. E., Groves, R. M. and Van Hoewyk, J. (2010) The 2006-2010 National Survey of Family Growth: sample design and analysis of a continuous survey. In Vital Health Statistics, vol. 2, no. 150. National Center for Health Statistics.
32. Lessler, J. and Kalsbeek, W. (1992) Nonresponse: dealing with the problem. In Nonsampling Errors in Surveys, ch. 8. New York: Wiley-Interscience.
33. Little, R. J. A. (1986) Survey nonresponse adjustments for estimates of means. Int. Statist. Rev., 54, 139-157.
34. Little, R. J. A. and Rubin, D. B. (2002) Statistical Analysis with Missing Data, 2nd edn. Hoboken: Wiley-Interscience.
35. Little, R. J. A. and Vartivarian, S. (2005) Does weighting for nonresponse increase the variance of survey means?Surv. Methodol., 31, 161-168.
36. Lynn, P. (2003) PEDAKSI: methodology for collecting data about survey non-respondents. Qual. Quant., 37, 239-261.
37. Matsuo, H., Billiet, J., Loosveldt, G. and Malnar, B. (2010) Response-based Quality Assessment of ESS Round 4: Results for 30 Countries based on Contact Files. Leuven: European Social Survey.
38. Mosteller, F. (1944) The reliability of interviewers' ratings. In Gauging Public Opinion, vol. 2, ch. 7, pp. 98-106. Princeton: Princeton University Press.
39. Nagelkerke, N. (1991) A note on a general definition of the coefficient of determination. Biometrika, 78, 691-692.
40. Pickering, K., Thomas, R. and Lynn, P. (2003) Testing the shadow sample approach for the English House Condition survey. Report. Office of the Deputy Prime Minister, London.
41. Särndal, C.-E. and Lundström, S. (2010) Design for estimation: identifying auxiliary vectors to reduce nonresponse bias. Surv. Methodol., 36, 131-144.
42. SAS Institute (2011) SAS 9.2 Online Help and Documentation: the GLIMMIX Procedure (Default Output). Cary: SAS Institute.
43. Stähli, M. E. (2011) Examples and experiences from the Swiss interviewer training on observable data (neighborhood characteristics) for ESS 2010 (R5). National CoordinatorsMeet. Mannheim, Mar. 31st-Apr. 1st .
44. StataCorp (2009) Stata Statistical Software: Release 11. College Station: StataCorp.
45. Stefanski, L. A. and Carroll, R. J. (1985) Covariate measurement error in logistic regression. Ann. Statist., 13, 1335-1351.
46. Steiner, P. M., Cook, T. D. and Shadish, W. R. (2011) On the importance of reliable covariate measurement in selection bias adjustments using propensity scores. J. Educ. Behav. Statist., 36, 213-236.
47. Tolonen, H., Helakorpi, S., Talala, K., Helasoja, V., Martelin, T. and Prattala, R. (2006) 25-year trends and socio-demographic differences in response rates: Finnish adult health behaviour survey. Eur. J. Epidem., 21, 409-415.
48. Valliant, R. (2004) The effect of multiple weighting steps on variance estimation. J. Off. Statist., 20, 1-18.
49. West, B. T. (2010) A practical technique for improving the accuracy of interviewer observations: evidence from the National Survey of Family Growth. Technical Report 10-013 . Population Studies Center, Institute for Social Research, University of Michigan, Ann Arbor. (Available from http://www.psc.isr.umich.edu/pubs/abs/6705.)
50. West, B. T. and Little, R. J. A. (2011) Nonresponse adjustment of survey estimates based on auxiliary variables subject to error. Submitted to Appl. Statist.
51. Wolter, K. M. (2007) Introduction to Variance Estimation, 2nd edn. New York: Springer.
52. Zhang, D. and Lin, X. (2008) Variance component testing in generalized linear mixed models for longitudinal/clustered data and other related topics. Random Effect and Latent Variable Model Selection (ed. D. B. Dunson ), ch. 2. New York: Springer.