Assessing statistical aspects of test fairness with structural equation modelling

Educational Research and Evaluation

Volumn 19, Issue 2-3, 2013, Pages 204-222

  Kline, Rex B ^a  

^a Gina Cody School of Engineering and Computer Science   (Canada)

Author keywords

measurement invariance; multiple group confirmatory factor analysis; structural equation modelling; test bias; test fairness

Indexed keywords

EID: 84875836611     PISSN: 13803611     EISSN: 17444187     Source Type: Journal    
DOI: 10.1080/13803611.2013.767624     Document Type: Article

References (51)

1. Aguinis, H. 2004. Regression analysis for categorical moderators., New York, NY: Guilford Press.
2. Blanton, H. and Jaccard, J. 2006. Arbitrary metrics in psychology. American Psychologist, 61: 27-41. doi:10.1037/0003-066X.61.1.27
3. Borsboom, D., Romeijn, J. W. and Wicherts, J. M. 2008. Measurement invariance versus selection invariance: Is fair selection possible?. Psychological Methods, 13: 75-98. doi:10.1037/1082-989X.13.2.75
4. Bovaird, J. A. and Koziol, N. A. 2012. "Measurement models for categorical indicators". In Handbook of structural equation modeling, Edited by: Hoyle, R. H. 495-511. New York, NY: Guilford Press.
5. Brown, T. A. 2006. Confirmatory factor analysis for applied research, New York, NY: Guilford Press.
6. Chen, F. F., Sousa, K. H. and West, S. G. 2005. Teacher's corner: Testing measurement invariance of second-order factor models. Structural Equation Modeling, 12: 471-492. doi:10.1207/s15328007sem1203_7
7. Cheung, G. W. and Lau, R. S. 2012. A direct comparison approach for testing measurement invariance. Organizational Research Methods, 15: 167-198. doi:10.1177/1094428111421987
8. Cheung, G. W. and Rensvold, R. B. 2002. Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9: 233-255. doi:10.1207/S15328007SEM0902_5
9. DeShon, R. P. 2004. Measures are not invariant across groups without error variance homogeneity. Psychology Science, 46: 137-149. Retrieved fromhttp://www.pabst-publishers.de/psychology-science/
10. Ellis, P. D. 2010. The essential guide to effect sizes: Statistical power, meta-analysis, and the interpretation of research results, New York, NY: Cambridge University Press.
11. Forer, B. and Zumbo, B. D. 2011. Validation of multilevel constructs: Validation methods and empirical findings for the EDI. Social Indicators Research, 103: 231-265. doi:10.1007/s11205-011-9844-3
12. Gregorich, S. E. 2006. Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Medical Care, 44(Suppl. 3): S78-S94. doi:10.1097/01.mlr.0000245454.12228.8f
13. Grissom, R. J. and Kim, J. J. 2011. Effect sizes for research: Univariate and multivariate applications, 2, New York, NY: Routledge.
14. Gunnell, K. E., Wilson, P. M., Zumbo, B. D., Mack, D. E. and Crocker, P. R.E. 2012. Assessing psychological need satisfaction in exercise contexts: Issues of score invariance, item modification, and context. Measurement in Physical Education and Exercise Science, 16: 219-236. doi:10.1080/1091367X.2012.693340
15. Harrington, G. M. 1984. "An experimental model of bias in mental testing". In Perspectives on bias in mental testing, Edited by: Reynolds, C. R. and Brown, R. T. 101-138. New York, NY: Plenum.
16. Hayduk, L., Cummings, G., Boadu, K., Pazderka-Robinson, H. and Boulianne, S. 2007. Testing! testing! one, two, three-Testing the theory in structural equation models!. Personality and Individual Differences, 42: 841-850. doi:10.1016/j.paid.2006.10.001
17. Hoyle, R. H. 2012. "Model specification in structural equation modeling". In Handbook of structural equation modeling, Edited by: Hoyle, R. H. 126-144. New York, NY: Guilford Press.
18. Jensen, A. R. 1980. Bias in mental testing, New York, NY: Free Press.
19. Kazdin, A. 2006. Arbitrary metrics: Implications for identifying evidence-based treatments. American Psychologist, 61: 42-49. doi:10.1037/0003-066X.61.1.42
20. Kim, E. S. and Yoon, M. 2011. Testing measurement invariance: A comparison of multiple-group categorical CFA and IRT. Structural Equation Modeling, 18: 212-228. doi:10.1080/10705511.2011.557337
21. Kline, R. B. 2010. Principles and practice of structural equation modeling, 3, New York, NY: Guilford Press.
22. Kline, R. B. 2011. "Convergence of structural equation modeling and multilevel modeling". In Handbook of methodological innovation, Edited by: Williams, M. and Vogt, W. P. 562-589. Thousand Oaks, CA: Sage.
23. Kline, R, B. 2012. "Assumptions of structural equation modeling". In Handbook of structural equation modeling, Edited by: Hoyle, R. 111-125. New York, NY: Guilford Press.
24. Kline, R. B. 2013a. Beyond significance testing: Statistics reform in the behavioral sciences, Washington, DC: American Psychological Association.
25. Kline, R. B. 2013b. "Exploratory and confirmatory factor analysis". In Applied quantitative analysis in the social sciences (pp. 171-207), Edited by: Petscher, Y. and Schatschneider, C. New York, NY: Routledge.
26. Koh, K. and Zumbo, B. D. 2008. Multi-group confirmatory factor analysis for testing measurement invariance in mixed item format data. Journal of Modern Applied Statistical Methods, 7: 471-477. Retrieved fromhttp://www.jmasm.com/
27. Lambdin, C. 2012. Significance tests as sorcery: Science is empirical-significance tests are not. Theory & Psychology, 22: 67-90. doi:10.1177/0959354311429854
28. Larry P. v. Riles, 343 F. Supp. 1306 (N.D. Cal. 1972) (order granting preliminary injunction), aff'd, 502 F.2d 963 (9th Cir. 1974), 495 F. Supp. 926 (N. D. Cal. 1979) (decision on merits), aff'd, No. 80-427 (9th Cir. Jan. 23, 1984), No. C-71-2270 RFP (Sept. 25, 1986) (order modifying judgment), 793 F.2d 969 (9th Cir. 1994).
29. Little, T. D., Slegers, D. W. and Card, N. A. 2006. A non-arbitrary method of identifying and scaling latent variables in SEM and MACS models. Structural Equation Modeling, 13: 59-72. doi:10.1207/s15328007sem1301_3
30. Lynam, D. R., Moffitt, T. and Stouthamer-Loeber, M. 1993. Explaining the relation between IQ and delinquency: Class, race, test motivation, or self-control?. Journal of Abnormal Psychology, 102: 187-196. doi:10.1037//0021-843X.102.2.187
31. Matsueda, R. L. 2012. "Key advances in the history of structural equation modeling". In Handbook of structural equation modeling, Edited by: Hoyle, R. H. 17-42. New York, NY: Guilford Press.
32. Meade, A. W., Johnson, E. C. and Braddy, P. W. 2008. Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93: 568-592. doi:10.1037/0021-9010.93.3.568
33. Meade, A. W. and Lautenschlager, G. J. 2004. A comparison of item response theory and confirmatory factor analytic methodologies for establishing measurement equivalence/invariance. Organizational Research Methods, 7: 361-388. doi:10.1177/1094428104268027
34. Messick, S. 1998. Test validity: A matter of consequence. Social Indicators Research, 45: 35-44. doi:10.1023/A:1006964925094
35. Millsap, R. E. and Olivera-Aguilar, M. 2012. "Investigating measurement invariance using confirmatory factor analysis". In Handbook of structural equation modeling, Edited by: Hoyle, R. H. 380-392. New York, NY: Guilford Press.
36. Millsap, R. E. and Yun-Tein, J. 2004. Assessing factorial invariance in ordered-categorical measures. Multivariate Behavioral Research, 39: 479-515. doi:10.1207/S15327906MBR3903_4
37. Moons, K. G.M., Van Es, G. A., Deckers, J. W., Habbema, J. D.F. and Grobbee, D. E. 1997. Limitations of sensitivity, specificity, likelihood ratio, and Bayes' theorem in assessing diagnostic probabilities: A clinical example. Epidemiology, 8: 12-17. doi:10.1097/00001648-199701000-00002
38. Morison, P., White, S. H. and Feuer, M. J. 1996. The use of IQ tests in special education decision making and planning: Summary of two workshops, Edited by: Morison, P., White, S. H. and Feuer, M. J. Washington, DC: National Academy Press.
39. Muthén, L. K. and Muthén, B. O. 1998-2010. Mplus user's guide, 6, Los Angeles, CA: Authors.
40. Nimon, K. and Reio, T., Jr. 2011. Measurement invariance: A foundational principle for quantitative theory building. Human Resource Development Review, 10: 198-214. doi:10.1177/1534484311399731
41. Oliveri, M. E., Olson, B. D., Ercikan, K. and Zumbo, B. D. 2012. Methodologies for investigating item- and test-level measurement equivalence in international large-scale assessments. International Journal of Testing, 12: 203-223. doi:10.1080/15305058.2011.617475
42. Preacher, K. J., Curran, P. J. and Bauer, D. J. 2006. Computational tools for probing interactions in multiple linear regression, multilevel modeling, and latent curve analysis. Journal of Educational and Behavioral Statistics, 31: 437-448. doi:10.3102/10769986031004437
43. Radloff, L. S. 1977. The CES-D scale: A self-report depression scale for research in the general populations. Applied Psychological Measurement, 1: 385-401. Retrieved fromhttp://apm.sagepub.com/, (doi:10.1177/014662167700100306)
44. Reynolds, C. R. 1982. "Methods for detecting construct and predictive bias". In Handbook of methods for detecting test bias, Edited by: Berk, R. A. 199-227. Baltimore, MD: John Hopkins University Press.
45. Reynolds, C. R. and Kaiser, S. M. 1990. "Bias in assessment of aptitude". In Handbook of psychological and educational assessment of children, Edited by: Reynolds, C. R. and Kamphaus, R. W. 611-653. New York, NY: Guilford Press.
46. Schmitt, N. and Kuljanin, G. 2008. Measurement invariance: Review of practice and limitations. Human Resource Management Review, 18: 210-222. doi:10.1016/j.hrmr.2008.03.003
47. Steenkamp, J.-B. E. M. and Baumgartner, H. 1998. Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research, 25: 78-90. doi:10.1086/209528
48. Whitaker, B. G. and McKinney, J. L. 2007. Assessing the measurement invariance of latent job satisfaction ratings across survey administration modes for respondent subgroups: A MIMIC modeling approach. Behavior Research Methods, 39: 502-509. doi:10.3758/BF03193019
49. Wirth, R. J. and Edwards, M. C. 2007. Item factor analysis: Current approaches and future directions. Psychological Methods, 12: 58-79. doi:10.1037/1082-989X.12.1.58
50. Ziliak, S. and McCloskey, D. N. 2008. The cult of statistical significance: How the standard error costs us jobs, justice, and lives, Ann Arbor, MI: University of Michigan Press.
51. Zumbo, B. D. 2007. Three generations of DIF analyses: Considering where it has been, where it is now, and where it is going. Language Assessment Quarterly, 4: 223-233. Retrieved fromhttp://www.tandfonline.com/loi/hlaq20