Factors affecting the accuracy of interviewer observations

Public Opinion Quarterly

Volumn 77, Issue 2, 2013, Pages 522-548

  West, Brady T ^a   Kreuter, Frauke ^b  

^a UNIVERSITY OF MICHIGAN   (United States)

^b UNIVERSITY OF MARYLAND   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84879627627     PISSN: 0033362X     EISSN: 15375331     Source Type: Journal    
DOI: 10.1093/poq/nft016     Document Type: Article

References (57)

1. Agresti, Alan. 2012. Categorical Data Analysis. 3rd ed. New York: Wiley.
2. Alwin, Duane F. 2008. Margins of Error: A Study of Reliability in Survey Measurement. Hoboken, NJ: John Wiley and Sons.
3. Ambady, Nalini, Mark Hallahan, and Brett Conner. 1999. "Accuracy of Judgments of Sexual Orientation from Thin Slices of Behavior." Journal of Personality and Social Psychology 77:538-47.
4. Ambady, Nalini, Mark Hallahan, and Robert Rosenthal. 1995. "On Judging and Being Judged Accurately in Zero-Acquaintance Situations." Journal of Personality and Social Psychology 69:518-29.
5. American Association for Public Opinion Research (AAPOR). 2008. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. 5th ed. Lenexa, KS: AAPOR.
6. Beaumont, Jean-Francois. 2005. "On the Use of Data-Collection Process Information for the Treatment of Unit Nonresponse through Weight Adjustment." Survey Methodology 31:227-31.
7. Bethlehem, Jelke. 2002. "Weighting Nonresponse Adjustments Based on Auxiliary Information." In Survey Nonresponse, edited by Robert Groves, Don Dillman, John Eltinge, and Roderick Little, 275-88. New York: John Wiley and Sons.
8. Biemer, Paul P., Patrick Chen, and Kevin Wang. 2013. "Using Level of Effort Paradata in Nonresponse Adjustments with Application to Field Surveys." Journal of the Royal Statistical Society: Series A 176:147-68.
9. Biemer, Paul P., and Michael W. Link. 2008. "Evaluating and Modeling Early Cooperator Effects in RDD Surveys." In Advances in Telephone Survey Methodology, edited by James Lepkowski, Clyde Tucker, J. Michael Brick, Edith de Leeuw, Lilli Japec, Paul Lavrakas, Michael Link, and Roberta Sangster, 587-618. Hoboken, NJ: John Wiley and Sons.
10. Blohm, Michael, Joop Hox, and Achim Koch. 2006. "The Influence of Interviewers' Contact Behavior on the Contact and Cooperation Rate in Face-to-Face Household Surveys." International Journal of Public Opinion Research 19:97-111.
11. Blom, Annalies G., Edith D. de Leeuw, and Joop J. Hox. 2011. "Interviewer Effects on Nonresponse in the European Social Survey." Journal of Official Statistics 27:359-77.
12. Blom, Annalies G., Annette Jäckle, and Peter Lynn. 2010. "The Use of Contact Data in Understanding Cross-National Differences in Unit Nonresponse." In Survey Methods in Multinational, Multiregional, and Multicultural Contexts, edited by Janet Harkness, Michael Braun, Brad Edwards, Timothy Johnson, Lars Lyberg, Peter Mohler, Beth-Ellen Pennell, and Tom Smith, 335-54. New York: John Wiley and Sons.
13. Campanelli, Pamela, Patrick Sturgis, and Susan Purdon. 1997. Can You Hear Me Knocking: An Investigation into the Impact of Interviewers on Survey Response Rates. London: SCPR.
14. Casas-Cordero, Carolina. 2010. Testing Neighborhood Mechanisms Influencing Participation in Household Surveys. Ph.D. dissertation, University of Maryland.
15. Casas-Cordero, Carolina, Frauke Kreuter, Yueyan Wang, and Sue Babey. 2013. "Assessing the Measurement Error Properties of Interviewer Observations of Neighborhood Characteristics." Journal of the Royal Statistical Society: Series A 176:227-49.
16. Durrant, Gabriele B., and Fiona Steele. 2009. "Multilevel Modeling of Refusal and Noncontact Nonresponse in Household Surveys: Evidence from Six UK Government Surveys." Journal of the Royal Statistical Society: Series A 172:361-81.
17. Feldman, Jacob J., Herbert Hyman, and Clyde W. Hart. 1951. "A Field Study of Interviewer Effects on the Quality of Survey Data." Public Opinion Quarterly 15:734-61.
18. Goldstein, Harvey. 1995. Multilevel Statistical Models. 2nd ed. Kendall's Library of Statistics 3. London: Edward Arnold.
19. Grant, Catherine, Keith Bolling, and Matthew Sexton. 2006. 2005-2006 British Crime Survey (England and Wales): Technical Report Volume 1. London: Research Development and Statistics, Home Office.
20. Groves, Robert M., and Mick P. Couper. 1998. Nonresponse in Household Interview Surveys. New York: John Wiley and Sons.
21. Groves, Robert M., and Steven G. Heeringa. 2006. "Responsive Design for Household Surveys: Tools for Actively Controlling Survey Errors and Costs." Journal of the Royal Statistical Society: Series A 169:439-57.
22. Groves, Robert M., William D. Mosher, James Lepkowski, and Nicole G. Kirgis. 2009. "Planning and Development of the Continuous National Survey of Family Growth." Vital Health Statistics 1:1-64.
23. Hahn, Robert A., Benedict T. Truman, and Nancy D. Barker. 1996. "Identifying Ancestry: The Reliability of Ancestral Identification in the United States by Self, Proxy, Interviewer, and Funeral Director." Epidemiology 7:75-80.
24. Hill, Mark E. 2002. "Race of the Interviewer and Perception of Skin Color: Evidence from the Multi-City Study of Urban Inequality." American Sociological Review 67:99-108.
25. Hosmer, David W., and Stanley Lemeshow. 2000. Applied Logistic Regression. 2nd ed. New York: John Wiley and Sons.
26. Hox, Joop J. 1998. "Multilevel Modeling: When and Why." In Classification, Data Analysis, and Data Highways, edited by Ingo Balderjahn, Rudolf Mathar, and Martin Schader, 147-54. New York: Springer-Verlag.
27. Jones, Malia, Anne R. R. Pebley, and Narayan Sastry. 2011. "Eyes on the Block: Measuring Urban Physical Disorder through In-Person Observation." Social Science Research 40:523-37.
28. Kennickell, Arthur B. 2003. "Reordering the Darkness: Application of Effort and Unit Nonresponse in the Survey of Consumer Finances." Proceedings of the Survey Research Methods Section of the American Statistical Association, Joint Statistical Meetings.
29. Kreuter, Frauke, Kristen Olson, James Wagner, Ting Yan, Trena Ezzati-Rice, Carolina Casas-Cordero, Michael Lemay, Andy Peytchev, Robert M. Groves, and Trivellore E. Raghunathan. 2010. "Using Proxy Measures and Other Correlates of Survey Outcomes to Adjust for Nonresponse: Examples from Multiple Surveys." Journal of the Royal Statistical Society: Series A 173:1-21.
30. Lepkowski, James M., William D. Mosher, Karen E. Davis, Robert M. Groves, and John Van Hoewyk. 2010. "The 2006-2010 National Survey of Family Growth: Sample Design and Analysis of a Continuous Survey." National Center for Health Statistics, Vital and Health Statistics 2(150).
31. Lessler, Judith, and William Kalsbeek. 1992. "Nonresponse: Dealing with the Problem." In Nonsampling Errors in Surveys. New York: Wiley-Interscience.
32. Lin, I-Fen, and Nora Cate Schaeffer. 1995. "Using Survey Participants to Estimate the Impact of Nonparticipation." Public Opinion Quarterly 59:236-58.
33. Little, Roderick J. A., and Sonya Vartivarian. 2005. "Does Weighting for Nonresponse Increase the Variance of Survey Means?" Survey Methodology 31:161-68.
34. Lynn, Peter. 2003. "PEDAKSI: Methodology for Collecting Data about Survey Non-Respondents." Quality and Quantity 37:239-61.
35. Moriarity, Chris, and James Dahlhamer. 2012. "Adjustment for Unit Nonresponse in the National Health Interview Survey." Paper presented at the 2012 Joint Statistical Meetings, Session 671, San Diego, CA, USA.
36. Most, Steven B., Brian J. Scholl, Erin R. Clifford, and Daniel J. Simons. 2005. "What You See Is What You Get: Sustained Inattentional Blindness and the Capture of Awareness." Psychological Review 112:217-42.
37. Mosteller, Frederick. 1944. "The Reliability of Interviewers' Ratings." In Gauging Public Opinion, edited by Hadley Cantril and Research Associates in the Office of Public Opinion Research, 98-106. Princeton, NJ: Princeton University Press.
38. O'Hare, Barbara. 2012. "The Use of Paradata to Improve Survey Quality: Census Bureau." Paper presented at the Plenary Panel of the 2012 FedCASIC Conference, Washington, D.C.
39. O'Muircheartaigh, Colm, and Pamela Campanelli. 1999. "A Multilevel Exploration of the Role of Interviewers in Survey Nonresponse." Journal of the Royal Statistical Society: Series A 162:437-46.
40. Pickering, Kevin, Roger Thomas, and Peter Lynn. 2003. "Testing the Shadow Sample Approach for the English House Condition Survey." Prepared for the Office of the Deputy Prime Minister by the National Centre for Social Research, London.
41. Raudenbush, Stephen W., and Anthony S. Bryk. 2002. Hierarchical Linear Models: Applications and Data Analysis Methods. Newbury Park, CA: Sage Publications.
42. Research Triangle Institute. 2004. 2005 National Survey on Drug Use and Health: Field Interviewer Manual. Contract No. 283-2004-00022, Project No. 9009. Available at http://www.samhsa.gov/data/nsduh/2k5mrb/2k5fimanualexternal/ 2k5FIManualFrontMatter.pdf.
43. Schnell, Rainer, and Frauke Kreuter. 2000. "Das DEFECT Projekt: Sampling Errors und Nonsampling Errors in komplexen Bevö lkerungsstichproben." ZUMA-Nachrichten 24:89-102.
44. Scholes, Shaun, Heather Wardle, Kerry Sproston, Bob Erens, Mark Griffiths, and Jim Orford. 2008. "Understanding Nonresponse to the British Gambling Prevalence Survey 2007." Prepared for the Gambling Commission by the National Centre for Social Research. Available at http://www. gamblingcommission.gov.uk/pdf/understanding%20non- responses%20to%20bgps%202007%20-%20oct%202008.pdf.
45. Simons, Daniel J., and Melinda S. Jensen. 2009. "The Effects of Individual Differences and Task Difficulty on Inattentional Blindness." Psychonomic Bulletin and Review 16:398-403.
46. Sinibaldi, Jennifer, Gabriele B. Durrant, and Frauke Kreuter. 2013. "Evaluating the Measurement Error of Interviewer Observed Paradata." Public Opinion Quarterly 77:173-93.
47. Steiner, Peter M., Thomas D. Cook, and William R. Shadish. 2011. "On the Importance of Reliable Covariate Measurement in Selection Bias Adjustments Using Propensity Scores." Journal of Educational and Behavioral Statistics 36:213-36.
48. Stoop, Ineke A. 2005. The Hunt for the Last Respondent. The Hague: Social and Cultural Planning Office of the Netherlands.
49. Tipping, Sarah, and Jennifer Sinibaldi. 2010. "Examining the Trade-Off between Sampling and Targeted Nonresponse Error in a Targeted Nonresponse Follow-Up." Paper presented at the 2010 International Total Survey Error Workshop, Stowe, VT, USA.
50. Tversky, Amos, and Daniel Kahneman. 1974. "Judgment under Uncertainty: Heuristics and Biases." Science 185:1124-31.
51. Wagner, James, Brady T. West, Nicole G. Kirgis, James M. Lepkowski, William G. Axinn, and Shonda Kruger-Ndiaye. 2012. "Use of Paradata in a Responsive Design Framework to Manage a Field Data Collection." Journal of Official Statistics 28:477-99.
52. West, Brady T. 2010. "A Practical Technique for Improving the Accuracy of Interviewer Observations: Evidence from the National Survey of Family Growth." NSFG Paper No. 10-013. http://www.psc.isr.umich.edu/pubs/ abs/6705.
53. -. 2013a. "An Examination of the Quality and Utility of Interviewer Observations in the National Survey of Family Growth." Journal of the Royal Statistical Society: Series A (General) 176:211-25.
54. -. 2013b. "The Effects of Error in Paradata on Weighting Class Adjustments: A Simulation Study." In Improving Surveys with Paradata: Analytic Uses of Process Information, edited by Frauke Kreuter. New York: Wiley Publishing.
55. West, Brady T., and Frauke Kreuter. 2011. "Observational Strategies Associated with Increased Accuracy of Interviewer Observations: Evidence from the National Survey of Family Growth." In JSM Proceedings, AAPOR 2011, 5646-58. Alexandria, VA: American Statistical Association.
56. West, Brady T., Frauke Kreuter, and Mark Trappmann. 2012. "Observational Strategies Associated with Increased Accuracy of Interviewer Observations in Employment Research." Paper presented at the 2012 Annual Conference of the American Association for Public Opinion Research (AAPOR), Orlando, FL, USA.
57. West, Brady T., Kathleen B. Welch, and Andrzej T. Galecki. 2006. Linear Mixed Models: A Practical Guide Using Statistical Software. Boca Raton, FL: Chapman Hall/CRC Press.