The comparability of the universalism value over time and across countries in the European Social Survey: Exact vs. approximate measurement invariance

Frontiers in Psychology

Volumn 6, Issue JUN, 2015, Pages

  Zercher, Florian ^a   Schmidt, Peter ^a   Cieciuch, Jan ^b,c   Davidov, Eldad ^d  

^a JUSTUS LIEBIG UNIVERSITY   (Germany)

^b UNIVERSITY OF ZURICH   (Switzerland)

^c CARDINAL STEFAN WYSZY SKI UNIVERSITY   (Poland)

^d UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

Approximate vs. exact measurement invariance; Bayesian estimation; Cross national research; European Social Survey; Portrait Value Questionnaire; Repeated cross sections; Universalism

Indexed keywords

EID: 84940564922     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2015.00733     Document Type: Article

References (67)

1. Beckers, T., Siegers, P., and Kuntz, A. (2012). Congruence and performance of value concepts in social research. Surv. Res. Method 6, 13-24.
2. Beierlein, C., Davidov, E., Schmidt, P., Schwartz, S., and Rammstedt, B. (2012). Testing the discriminant validity of Schwartz' Portrait Value Questionnaire items-a replication and extension of Knoppen and Saris (2009). Surv. Res. Method 6, 25-36.
3. Billiet, J., Maddens, B., and Beerten, R. (2003). National identity and attitude toward foreigners in a multinational state: a replication. Polit. Psychol. 24, 241-257. doi: 10.1111/0162-895X.00327.
4. Bollen, K. A. (1989). Structural Equations with Latent Variables. New York, NY: Wiley. doi: 10.1002/9781118619179.
5. Braun, M., Behr, D., and Kaczmirek, L. (2013). Assessing cross-national equivalence of measures of Xenophobia: evidence from probing in web surveys. Int. J. Public Opin. R. 25, 383-395. doi: 10.1093/ijpor/eds034.
6. Brown, T. A. (2006). Confirmatory Factor Analysis for Applied Research. New York, NY: Guilford Press.
7. Byrne, B. M., Shavelson, R. J., and Muthén, B. O. (1989). Testing for the equivalence of factor covariance and mean structures-the issue of partial measurement invariance. Psychol. Bull. 105, 456-466. doi: 10.1037/0033-2909.105.3.456.
8. Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct. Equ. Modeling 14, 464-504. doi: 10.1080/10705510701301834.
9. Cheung, G. W., and Rensvold, R. B. (2002). Evaluating goodness of fit indexes to lack of measurement invariance. Struct. Equ. Modeling 9, 233-255. doi: 10.1207/S15328007SEM0902_5.
10. Cieciuch, J., Davidov, E., Schmidt, P., Algesheimer, R., and Schwartz, S. H. (2014a). Comparing results of an exact versus an approximate (Bayesian) measurement invariance test: a cross-country illustration with a scale to measure 19 human values. Front. Psychol. 5:982. doi: 10.3389/fpsyg.2014.00982.
11. Cieciuch, J., Davidov, E., Vecchione, M., Beierlein, C., and Schwartz, S. H. (2014b). The cross-national invariance properties of a new scale to measure 19 basic human values a test across eight countries. J. Cross Cult. Psychol. 45, 764-776. doi: 10.1177/0022022114527348.
12. Datler, G., Jagodzinski, W., and Schmidt, P. (2013). Two theories on the test bench: internal and external validity of the theories of Ronald Inglehart and Shalom Schwartz. Soc. Sci. Res. 42, 906-925. doi: 10.1016/j.ssresearch.2012.12.009.
13. Davidov, E. (2008). A cross-country and cross-time comparison of the human values measurements with the second round of the European Social Survey. Surv. Res. Method 2, 33-46.
14. Davidov, E. (2010). Testing for comparability of human values across countries and time with the third round of the European Social Survey. Int. J. Comp. Sociol. 51, 171-191. doi: 10.1177/0020715210363534.
15. Davidov, E., Cieciuch, J., Schmidt, P., Meuleman, B., and Algesheimer, R. (2015). The comparability of measurements of attitudes toward immigration in the European Social Survey: exact versus approximate measurement equivalence. Public Opin. Qurterly 79, 244-266. doi: 10.1093/poq/nfv008.
16. Davidov, E., Dülmer, H., Schlüter, E., Schmidt, P., and Meuleman, B. (2012). Using a multilevel structural equation modeling approach to explain cross-cultural measurement noninvariance. J. Cross Cult. Psychol. 43, 558-575. doi: 10.1177/0022022112438397.
17. Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., and Billiet, J. (2014a). Measurement equivalence in cross-national research. Annu. Rev. Sociol. 40, 55-75. doi: 10.1146/annurev-soc-071913-043137.
18. Davidov, E., Meuleman, B., Schwartz, S. H., and Schmidt, P. (2014b). Individual values, cultural embeddedness, and anti-immigration sentiments: explaining differences in the effect of values on attitudes toward immigration across Europe. Kölner Z. Soz. Sozpsychol. 66, 263-285. doi: 10.1007/s11577-014-0274-5.
19. Davidov, E., Schmidt, P., and Schwartz, S. (2008). Bringing values back in. The adequacy of the European Socialg Survey to measure values in 20 countries. Public Opin. Q. 72, 420-445. doi: 10.1093/poq/nfn035.
20. Davidov, E., and Siegers, P. (2010). "Comparing basic human values in East and West Germany", in Komparative Empirische Sozialforschung [Comprative Empirical Social Research], eds T. Beckers, K. Birkelbach, J. Hagenah, and U. Rosar (Wiesbaden: Verlag), 43-63. doi: 10.1007/978-3-531-92472-4_2.
21. Gelman, A. (2003). A Bayesian formulation of explanatory data analysis and goodness of fit testing. Int. Stat. Rev. 71, 369-382. doi: 10.1111/j.1751-5823.2003.tb00203.x.
22. Gelman, A. (2013). Two simple examples for understanding posterior p-values whose distributions are far from unform. Electron. J. Statist. 7, 2595-2602. doi: 10.1214/13-EJS854.
23. Harkness, J. A., Braun, M., Edwards, B., Johnson, T. P., Lyberg, L., Mohler, P. P., et al. (eds.) (2010). Survey Methods in Multinational, Multiregional, and Multicultural Contexts. Hoboken, NJ: John Wiley & Sons. doi: 10.1002/9780470609927.
24. Hitlin, S., and Piliavin, J. A. (2004). Values: reviving a dormant concept. Annu. Rev. Sociol. 30, 359-393. doi: 10.1146/annurev.soc.30.012703.110640.
25. Horn, J. L., and McArdle, J. J. (1992). A practical and theoretical guide to measurement invariance in aging research. Exp. Aging Res. 18, 117-144. doi: 10.1080/03610739208253916.
26. Inglehart, R. (1977). The Silent Revolution: Changing Values and Political Styles among Western Publics. Princeton, NJ: Princeton University Press.
27. Inglehart, R., and Welzel, C. (2005). Modernization, Cultural Change, and Democracy. The Human Development Sequence. Cambridge: Cambridge University Press.
28. Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika 36, 409-426. doi: 10.1007/BF02291366.
29. Jöreskog, K. G. (1978). Structural analysis of covariance and correlation matrices. Psychometrika 43, 443-477. doi: 10.1007/BF02293808.
30. Jöreskog, K. G. (1993). "Testing structural equation models," in Testing Structural Equation Models, eds K. A. Bollen and J. S. Long (Newbury Park, CA: Sage), 294-316.
31. Jouha, J., and Moustaki, I. (2013). Non-Equivalence of Measurement in Latent Variable Modelling of Multigroup Data: a Sensitivity Analysis. Available online at: http://ssrn.com/abstract=2332071.
32. Jowell, R., Roberts, C., Fitzgerald, R., and Eva, G. (2007). Measuring Attitudes Cross-Nationally. Lessons from the European Social Survey. London: Sage.
33. Kaplan, D. (2014). Bayesian Statistics for the Social Sciences. New York, NY: Guilford Press.
34. Kass, R. E., and Raftery, A. E. (1995). Bayes factors. J. Am. Stat. Assoc. 90, 773-795. doi: 10.1080/01621459.1995.10476572.
35. Kruschke, J. K., Aguinis, H., and Joo, H. (2012). The time has come! Bayesian methods for data analysis in the organizational sciences. Organ. Res. Methods 15, 722-752. doi: 10.1177/1094428112457829.
36. Kuntz, A., Davidov, E., Schwartz, S. H., and Schmidt, P. (2015). Human values, legal regulation and approval of homosexuality in Europe: a cross-country comparison. Eur. J. Soc. Psychol. 45, 120-134. doi: 10.1002/ejsp.2068.
37. Latcheva, R. (2011). Cognitive interviewing and factor-analytic techniques: a mixed method approach to validity of survey items measuring national identity. Qual. Quant. 45, 1175-1199. doi: 10.1007/s11135-009-9285-0.
38. Levy, R. (2011). Bayesian data-model fit assessment for structural equation modeling. Struct. Equ. Modeling 18, 663-685. doi: 10.1080/10705511.2011.607723.
39. Levy, R., and Choi, J. (2013). "Bayesian structural equation modeling," in Structural Equation Modeling. A Second Course, 2nd Edn., eds G. R. Hancock and R. O. Mueller (Charlottesville, NC: Information Age Publishing), 563-624.
40. Little, T. D., Slegers, D. W., and Card, N. A. (2006). A non-arbitrary method of identifying and scaling latent variables in SEM and MACS models. Struct. Equ. Modeling 13, 59-72. doi: 10.1207/s15328007sem1301_3.
41. Meade, A. W., Johnson, E. C., and Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. J. Appl. Psychol. 93, 568-592. doi: 10.1037/0021-9010.93.3.568.
42. Meredith, W. (1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika 58, 525-543. doi: 10.1007/BF02294825.
43. Meuleman, B. (2012). "When are intercept differences substantively relevant in measurement invariance testing," in Methods, Theories, and Empirical Applications in the Social Sciences: Festschrift for Peter Schmidt, eds S. Salzborn, E. Davidov, and J. Reinecke (Wiesbaden: Springer), 97-104.
44. Meuleman, B., Davidov, E., Schmidt, P., and Billiet, J. (2012). "Social location and value priorities: a European-wide comparison of the relation between social-structural variables and human values," in Society and Democracy in Europe, eds O. Gabriel and S. I. Keil (London: Routledge), 45-67.
45. Millsap, R. E. (2011). Statistical Approaches to Measurement Invariance. New York, NY: Routledge.
46. Muthén, B. O., and Asparouhov, T. (2012). Bayesian structural equation modeling: a more flexible representation of substantive theory. Psychol. Methods 17, 313-335. doi: 10.1037/a0026802.
47. Muthén, B. O., and Asparouhov, T. (2013). BSEM Measurement Invariance Analysis. Mplus Web Notes: No. 17. Available online at: http://www.statmodel.com.
48. Muthén, L., and Muthén, B. O. (1998-2014). Mplus User's Guide, 7th Edn. Los Angeles, CA: Muthén and Muthén.
49. Oberski, D. L. (2014). Evaluating sensitivity of parameters of interest to measurement invariance in latent variable models. Polit. Anal. 22, 45-60. doi: 10.1093/pan/mpt014.
50. Oberski, D. L., Weber, W., and Revilla, M. (2012). "The effect of individual characteristics on reports of socially desirable attitudes towards immigration," in Methods, Theories, and Empirical Applications in the Social Sciences: Festschrift for Peter Schmidt, eds S. Salzborn, E. Davidov, and J. Reinecke (Wiesbaden: Springer), 151-158.
51. Opp, K.-D. (2011). Modeling micro macro relationships: problems and solutions. J. Math. Sociol. 35, 209-234. doi: 10.1080/0022250X.2010.532257.
52. Piurko, Y., Schwartz, S. H., and Davidov, E. (2011). Basic personal values and the meaning of left-right political orientations in 20 countries. Polit. Psychol. 32, 537-561. doi: 10.1111/j.1467-9221.2011.00828.x.
53. Popper, K. R. (2005). The Logic of Scientific Discovery. London: Routledge.
54. Saris, W., Knoppen, D., and Schwartz, S. (2013). Operationalizing the theory of human values: balancing homogeneity of reflective items and theoretical coverage. Surv. Res. Method 7, 29-44.
55. Saris, W., Satorra, A., and van der Veld, W. M. (2009). Testing structural equation models or detection of misspecifications? Struct. Equ. Modeling 16, 561-582. doi: 10.1080/10705510903203433.
56. Sarrasin, O., Green, E. G. T., Berchtold, A., and Davidov, E. (2012). Measurement equivalence across subnational groups: an analysis of the conception of nationhood in Switzerland. Int. J. Public Opin. R. 25, 522-534. doi: 10.1093/ijpor/eds033.
57. Schafer, J., and Graham, J. (2002). Missing data: our view of the state of the art. Psychol. Methods 7, 147-177. doi: 10.1037/1082-989X.7.2.147.
58. Schwartz, S. H. (2003). "A proposal for measuring value orientations across nations," in Questionnaire Development Package of the European Social Survey, Chapter 7. Available online at: www.europeansocialsurvey.org.
59. Schwartz, S. H., Cieciuch, J., Vecchione, M., Davidov, E., Fischer, R., Beierlein, C., et al. (2012). Refining the theory of basic individual values. J. Pers. Soc. Psychol. 103, 663-688. doi: 10.1037/a0029393.
60. Schwarz, G. (1978). Estimating the dimension of a model. Ann. Statist. 6, 461-464. doi: 10.1214/aos/1176344136.
61. Spiegelhalter, D. J., Best, N. G., Carlin, B. P., and van der Linde, A. (2002). Bayesian measure of model complexity and fit. J. R. Statist. Soc. B. 64, 583-639. doi: 10.1111/1467-9868.00353.
62. Steenkamp, J.-B. E. M., and Baumgartner, H. (1998). Assessing measurement invariance in cross-national consumer research. J. Consum. Res. 25, 78-90. doi: 10.1086/209528.
63. Steinmetz, H. (2013). Analyzing observed composite differences across groups: is partial measurement invariance enough? Methodology 9, 1-12. doi: 10.1027/1614-2241/a000049.
64. Thompson, M. S., and Green, S. B. (2013). "Evaluating between group differences in latent variable means," in Structural Equation Modeling. A Second Course, 2nd Edn., eds G. R. Hancock and R. O. Mueller (Charlottesville, VA: Information Age Publishing), 163-218.
65. van de Schoot, R., Kluytmans, A., Tummers, L., Lugtig, P., Hox, J., and Muthén, B. O. (2013). Facing off with Scylla and Charybdis: a comparison of scalar, partial, and the novel possibility of approximate measurement invariance. Front. Psychol. 4:770. doi: 10.3389/fpsyg.2013.00770.
66. van de Schoot, R., Lugtig, P., and Hox, J. (2012). A checklist for testing measurement invariance. Eur. J. Dev. Psychol. 9, 486-492. doi: 10.1080/17405629.2012.686740.
67. Vandenberg, R. J., and Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: suggestions, practices, and recommendations for organizational research. Organ. Res. Methods 3, 4-70. doi: 10.1177/109442810031002.