Measurement equivalence in mixed mode surveys

Frontiers in Psychology

Volumn 6, Issue FEB, 2015, Pages

  Hox, Joop J ^a   De Leeuw, Edith D ^a   Zijlmans, Eva A O ^b  

^a UTRECHT UNIVERSITY   (Netherlands)

^b TILBURG UNIVERSITY   (Netherlands)

Author keywords

Measurement equivalence; Measurement invariance; Mixed mode survey; Mode effect; Propensity score adjustment; Selection bias

Indexed keywords

EID: 84926434468     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2015.00087     Document Type: Article

References (57)

1. Alwin, D. F. (2007). Margins of Error: a Study of Reliability in Survey Measurement. New York, NY: Wiley.
2. Berzelak, J. (2014). Mode Effects in Web Surveys. Unpublished Ph.D. thesis, University of Ljubjana, Slovenia. Available online at http://dk.fdv.uni-lj.si/doktorska_dela/pdfs/dr_berzelak-jernej.pdf (Accessed January 25, 2015).
3. Biemer, P. P., and Lyberg, L. E. (2003). Introduction to Survey Quality. New York, NY: Wiley.
4. Blyth, B. (2008). Mixed mode: the only 'fitness' regime? Int. J. Market Res. 50, 241-266.
5. Bollen, K. A., and Lennox, R. (1991). Conventional wisdom in measurement: a structural equation perspective. Psychol. Bull. 110, 305-314. doi: 10.1037/0033-2909.110.2.305.
6. Byrne, B. M., Shavelson, R. J., and Muthén, B. O. (1989). Testing for the equivalence of factor and mean structures: the issue of partial measurement invariance. Psychol. Bull. 105, 456-466. doi: 10.1037/0033-2909.105.3.456.
7. Chang, L., and Krosnick, J. A. (2009). National surveys via RDD telephone interviewing versus the internet. Comparing sample representativeness and response quality. Public Opin. Q. 73, 641-678. doi: 10.1093/poq/nfp075.
8. Chang, L., and Krosnick, J. A. (2010). Comparing oral interviewing with self-administered computerized questionnaires. an experiment. Public Opin. Q. 74, 154-167. doi: 10.1093/poq/nfp090.
9. Christian, L. M., Dillman, D. A., and Smyth, J. D. (2007). Helping respondents get it right the first time: the influence of words, symbols, and graphics in web surveys. Public Opin. Q. 71, 113-125. doi: 10.1093/poq/nfl039.
10. Christian, L. M., Dillman, D. A., and Smyth, J. D. (2008). "The effects of mode and format on answers to scalar questions in telephone and web surveys," in Advances in Telephone Survey Methodology, eds J. Lepkowski, C. Tucker, M. Brick, E. D. De Leeuw, L. Japec, P. Lavrakas, et al. (New York, NY: Wiley), 250-275.
11. Couper, M. P. (2000). Web surveys: a review of issues and approaches. Public Opin. Q. 64, 464-494. doi: 10.1086/318641.
12. Couper, M. P. (2011). The future of modes of data collection. Public Opin. Q. 75, 889-908. doi: 10.1093/poq/nfr046.
13. De Leeuw, E. D. (1992). Data Quality in Mail, Telephone and Face to Face Surveys. Amsterdam: TT-Publikaties. Available online at: http://edithl.home.xs4all.nl/pubs/disseddl.pdf (Accessed January 25, 2015).
14. De Leeuw, E. D. (2005). To mix or not to mix data collections in surveys. J. Off. Stat. 21, 233-255.
15. De Leeuw, E. D., and Berzelak, J. (2014). "Survey mode or survey modes?," in The Sage Book of Survey Methodology, eds C. Wolf, D. Joye, T. W. Smith, and Y.-C. Fu (Thousand Oaks, CA: Sage).
16. De Leeuw, E. D., and Hox, J. J. (2011). "Internet surveys as part of a mixed-mode design," in Social and Behavioral Research and the Internet, eds M. Das, P. Ester, and L. Kaczmirek (New York, NY: Routledge), 45-76.
17. De Leeuw, E. D., Hox, J. J., and Dillman, D. A. (2008). "Mixed-mode surveys: when and why," in International Handbook of Survey Methodology, eds E. D. De Leeuw, J. J. Hox, and D. A. Dillman (New York, NY: Erlbaum/Taylor and Francis), 299-316.
18. De Leeuw, E. D., Mellenbergh, G. J., and Hox, J. J. (1996). The influence of data collection method on structural models: a comparison of a mail, a telephone, and a face to face survey. Soc. Methods Res. 24, 443-472. doi: 10.1177/0049124196024004002.
19. Dex, S., and Gumy, J. (2011). "On the experience and evidence about mixing modes of data collection in large-scale surveys where the web is used as one of the modes in data collection," in National Centre for Research Methods Review Paper (London: National Centre for Research Methods/Economic and Social Research Council). Available online at: https://kar.kent.ac.uk/39197/1/mixing_modes_of_data_collection_in_large_surveys.pdf (Accessed January 25, 2015).
20. Dillman, D. A., and Christian, L. M. (2005). Survey mode as a source of instability across surveys. Field Methods 17, 30-52. doi: 10.1177/1525822X04269550.
21. Dillman, D. A., Smyth, J. D., and Christian, L. M. (2014). Internet, Phone, Mail, and Mixed-Mode Surveys. New York, NY: Wiley.
22. Finney, S. J., and DiStefano, C. (2006). "Nonnormal and categorical data in structural equation modeling," in Structural Equation Modeling. A Second Course, eds G. R. Hancock and R. O. Mueller (Greenwich, CT: Information Age Publishing), 269-314.
23. Glockner-Rist, A., and Hoijtink, H. J. A. (2003). The best of both worlds: factor analysis of dichotomous data using item response theory and structural equation modeling. Struct. Equ. Modeling 10, 544-565. doi: 10.1207/S15328007SEM1004_4.
24. Gordoni, G., Schmidt, P., and Gordoni, Y. (2012). Measurement invariance across face-to-face and telephone modes: the case of minority-status collectivistic-oriented groups. Int. J. Public Opin. Res. 24, 185-207. doi: 10.1093/ijpor/edq054.
25. Groves, R. M. (1989). Survey Errors and Survey Costs. New York, NY: Wiley.
26. Guo, S., and Fraser, M. W. (2010). Propensity Score Analysis. Statistical Methods and Applications. Los Angeles, CA: Sage.
27. Hair, J. F., Black, W. C., Babin, B. J., and Anderson, R. E. (2010). Multivariate Data Analysis. London: Pearson.
28. Heerwegh, D., and Loosveldt, G. (2011). Assessing mode effects in a national crime victimization survey using structural equation models: social desirability bias and acquiescence. J. Off. Stat. 27, 49-63.
29. Hox, J. J., and De Leeuw, E. D. (1994). A comparison of nonresponse in mail, telephone, and face-to-face surveys. Applying multilevel modeling to meta-analysis. Qual. Quant. 329-344. doi: 10.1007/BF01097014.
30. Jäckle, A., Roberts, C., and Lynn, P. (2010). Assessing the effect of data collection mode on measurement. Int. Stat. Rev. 78, 3-20. doi: 10.1111/j.1751-5823.2010.00102.x.
31. Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika 36, 409-426. doi: 10.1007/BF02291366.
32. Klausch, L. T., Hox, J. J., and Schouten, B. (2013). Measurement effects of survey mode on the equivalence of attitudinal rating scale questions. Soc. Methods Res. 42, 227-263. doi: 10.1177/0049124113500480.
33. Kreuter, F. (2015). "The use of paradata," in Improving Survey Methods: Lessons From Recent Research, eds U. Engel, B. Jann, P. Lynn, A. Scherpenzeel, and P. Sturgis (New York, NY: Routledge), 303-315.
34. Lawley, D. N., and Maxwell, A. E. (1963). Factor Analysis as a Statistical Method. London: Butterworths.
35. Lee, S. (2006). Propensity score adjustment as a weighting scheme for volunteer panel Web surveys. J. Off. Stat. 22, 329-249.
36. Link, M. W., and Mokdad, A. H. (2005). Effects of survey mode on self-reports of adult alcohol consumption. A comparison of mail, web, and telephone approached. J. Stud. Alcohol 66, 239-245.
37. Lozar Manfreda, K., Bosnjak, M., Berzelak, J., Haas, I., and Vehovar, V. (2008). Web surveys versus other survey modes-A meta-analysis comparing response rates. Int. J. Market Res. 50, 79-104.
38. Mellenbergh, G. J. (1989). Item bias and item response theory. Int. J. Educ. Res. 13, 127-143. doi: 10.1016/0883-0355(89)90002-5.
39. Meredith, W. (1993). Measurement invariance, factor-analysis and factorial invariance. Psychometrika 58, 525-543. doi: 10.1007/BF02294825.
40. Meredith, W. E. (1964). Notes on factorial invariance. Psychometrika 29, 177-185. doi: 10.1007/BF02289699.
41. Millsap, R. E., and Meredith, W. (2007). "Factorial invariance: historical perspectives and new problems," in Factor Analysis at 100: Historical Developments and Future Directions, eds R. Cudeck and R. C. MacCallum (Mahwah, NJ: Erlbaum), 131-152.
42. Mohorko, A., De Leeuw, E., and Hox, J. (2013). Internet coverage and coverage bias in Europe: developments across countries and over time. J. Off. Stat. 29, 1-15. doi: 10.2478/jos-2013-0042.
43. Muthén, B. O. (1989). Factor structure in groups selected on observed scores. Br. J. Math. Stat. Psychol. 42, 81-90. doi: 10.1111/j.2044-8317.1989.tb01116.x.
44. Muthén, B. O. (2002). Beyond SEM: general latent variable modeling. Behaviormetrika 29, 81-117. doi: 10.2333/bhmk.29.81.
45. Muthén, L. K., and Muthén, B. O. (1998-2012). Mplus User's Guide, 7th Edition. Los Angeles, CA: Muthén and Muthén.
46. Revilla, M. (2013). Measurement invariance and quality of composite scores in a face-to-face and a web survey. Surv. Res. Methods 7, 17-28.
47. Roberts, C. (2007). "Mixing modes of data collection in surveys: a methodological review," in ESRC National Centre for Research Methods Briefing Paper. Available online at: http://eprints.ncrm.ac.uk/418/1/MethodsReviewPaperNCRM-008.pdf.
48. Rosenbaum, P. R., and Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika 70, 41-55. doi: 10.1093/biomet/70.1.41.
49. Saris, W. E., and Revilla, M. A. (2013). A comparison of the quality of questions in a face-to-face and a web survey. Int. J. Public Opin. Res. 25, 242-253. doi: 10.1093/ijpor/eds007.
50. Steenkamp, J.-B. E. M., and Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. J. Consum. Res. 25, 78-90. doi: 10.1086/209528.
51. Tabachnick, B. G., and Fidell, L. S. (2013). Using Multivariate Statistics. New York, NY: Pearson.
52. Tourangeau, R., Conrad, F. R., and Couper, M. P. (2013). The Science of Web Surveys. New York, NY: Oxford University Press.
53. Tourangeau, R., Rips, L. J., and Rasinski, K. (2000). The Psychology of Survey Response. New York, NY: Cambridge University Press.
54. Vandenberg, R., and Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Org. Res. Methods 3, 4-70. doi: 10.1177/109442810031002.
55. van de Schoot, R., Kluytmans, A., Tummers, L., Lugtig, P., Hox, J., and Muthén, B. (2013). Facing off with Scylla and Charybdis: a comparison of scalar, partial, and the novel possibility of approximate measurement invariance. Front. Psychol. 4:770. doi: 10.3389/fpsyg.2013.00770.
56. Vannieuwenhuyze, J., and Loosveldt, G. (2013). Evaluating relative mode effects in mixed-mode surveys: three methods to disentangle selection and measurement effects. Soc. Methods Res. 42, 82-104. doi: 10.1177/0049124112464868.
57. Vannieuwenhuyze, J., Loosveldt, G., and Molenberghs, G. (2011). A method for evaluating mode effects in mixed-mode surveys. Public Opin. Q. 74, 1027-1045. doi: 10.1093/poq/nfq059.