Using multi-group confirmatory factor analysis to evaluate cross-cultural research: identifying and understanding non-invariance

International Journal of Research and Method in Education

Volumn 40, Issue 1, 2017, Pages 66-90

  Brown, Gavin T L ^a   Harris, Lois R ^b   O'Quin, Chrissie ^c   Lane, Kenneth E ^d  

^a UNIVERSITY OF AUCKLAND   (New Zealand)

^b CENTRAL QUEENSLAND UNIVERSITY   (Australia)

^c O'Quin Consulting LLC   (United States)

^d SOUTHEASTERN LOUISIANA UNIVERSITY   (United States)

Author keywords

confirmatory factor analysis; cross cultural research; invariance testing; survey research

Indexed keywords

EID: 84937915425     PISSN: 1743727X     EISSN: 17437288     Source Type: Journal    
DOI: 10.1080/1743727X.2015.1070823     Document Type: Article

References (62)

1. Boomsma, Anne, and Jeffrey J., Hoogland. 2001. “The Robustness of LISREL Modeling Revisited.” In Structural Equation Modeling:Present and Future, edited by R., Cudeck, Stephen, Du Toit, and Dag, Sorbom, 139–168. Lincolnwood, IL:Scientific Software International.
2. Borsboom, Denny. 2005. Measuring the Mind:Conceptual Issues in Contemporary Psychometrics. Cambridge:Cambridge University Press.
3. Brown, G. T. L., 2004. “Measuring Attitude with Positively Packed Self-Report Ratings:Comparison of Agreement and Frequency Scales.” Psychological Reports 94 (3):1015–1024. doi:10.2466/pr0.94.3.1015-1024.
4. Brown, G. T. L., 2006. “Teachers’ Conceptions of Assessment:Validation of an Abridged Instrument.” Psychological Reports 99 (1):166–170. doi:10.2466/pr0.99.1.166-170.
5. Brown, G. T. L., 2011. “Teachers’ Conceptions of Assessment:Comparing Primary and Secondary Teachers in New Zealand.” Assessment Matters 3:45–70.
6. Brown, G. T. L., and L. R., Harris. 2009. “Unintended Consequences of Using Tests to Improve Learning:How Improvement-Oriented Resources Heighten Conceptions of Assessment as School Accountability.” Journal of MultiDisciplinary Evaluation 6 (12):68–91.
7. Brown, G. T. L., L. R., Harris, and J., Harnett. 2012. “Teacher Beliefs about Feedback within an Assessment for Learning Environment:Endorsement of Improved Learning Over Student Well-Being.” Teaching and Teacher Education 28 (7):968–978. doi:10.1016/j.tate.2012.05.003.
8. Burnham, Kenneth P., and David R., Anderson. 2004. “Multimodel Inference:Understanding AIC and BIC in Model Selection.” Sociological Methods & Research 33 (2):261–304. doi:10.1177/0049124104268644.
9. Byrne, Barbara M., 2001. Structural Equation Modeling with AMOS:Basic Concepts, Applications, and Programming. Mahwah, NJ:LEA.
10. Chen, Feinian, Kenneth A., Bollen, Pamela, Paxton, Patrick J., Curran, and James B., Kirby. 2001. “Improper Solutions in Structural Equation Models:Causes, Consequences, and Strategies.” Sociological Methods & Research 29 (4):468–508. doi:10.1177/0049124101029004003
11. Cheung, Gordon W., and Roger B., Rensvold. 2002. “Evaluating Goodness-of-fit Indexes for Testing Measurement Invariance.” Structural Equation Modeling:A Multidisciplinary Journal 9 (2):233–255. doi:10.1207/S15328007SEM0902_5
12. Chou, C.-P., and P. M., Bentler. 1995. “Estimates and Tests in Structural Equation Modeling.” In Structural Equation Modeling:Concepts, Issues, and Applications, edited by R. H., Hoyle, 37–55. Thousand Oaks, CA:Sage.
13. Cohen, Jacob. 1992. “A Power Primer.” Psychological Bulletin 112 (1):155–159. doi:10.1037/0033-2909.112.1.155
14. Cooper, Beverley, and B., Cowie. 2010. “Collaborative Research for Assessment for Learning.” Teaching & Teacher Education 26:979–986. doi:10.1016/j.tate.2009.10.040.
15. Council for a Better Louisiana. 2006. “Fighting Poverty, Building Community.” Accessed February 17, 2007. http://cabl.org/PDFs/Poverty_UPDATE_2006.pdf.
16. Council for a Better Louisiana. 2009. “2009 Louisiana Report Card on Major Education Indicators.” Accessed July 9, 2010. http://www.cabl.org/pdfs/reportcard_09.pdf.
17. Cowie, B. 2009. “My Teacher and My Friends Helped Me Learn:Student Perceptions and Experiences of Classroom Assessment.” In Student Perspectives on Assessment:What Students can Tell Us about Assessment for Learning, edited by D. M., McInerney, Gavin T. L., Brown, and G. A. D., Liem, 85–105. Charlotte, NC:Information Age Publishing.
18. Crooks, T. J., 2010. “Classroom Assessment in Policy Context (New Zealand).” In The International Encyclopedia of Education, edited by Barry, McGraw, P., Peterson, and Eva L., Baker, 443–448. Oxford:Elsevier.
19. Crossley, M., and P., Broadfoot. 1992. “Comparative and International Research in Education:Scope, Problems and Potential.” British Educational Research Journal 18 (2):99–112. doi:10.1080/0141192920180201.
20. Curran, P. J., S. G., West, and J. F., Finch. 1996. “The Robustness of Test Statistics to Nonnormality and Specification Error in Confirmatory Factor Analysis.” Psychological Methods 1 (1):16–29. doi:10.1037/1082-989X.1.1.16
21. Deary, I. J., 2001. Intelligence:A Very Short Introduction. Oxford:OUP.
22. Dench, Olivia. 2010. Education Statistics of New Zealand:2009. Wellington, NZ:Ministry of Education.
23. Fan, Xitao, and Stephen A., Sivo. 2005. “Sensitivity of fit Indexes to Misspecified Structural or Measurement Model Components:Rationale of two-Index Strategy Revisited.” Structural Equation Modeling 12 (3):343–367. doi:10.1207/s15328007sem1203_1
24. Fan, Xitao, and Stephen A., Sivo. 2007. “Sensitivity of fit Indices to Model Misspecification and Model Types.” Multivariate Behavioral Research 42 (3):509–529. doi:10.1080/00273170701382864
25. Finney, Sara J., and Christine, DiStefano. 2006. “Non-normal and Categorical Data in Structural Equation Modeling.” In Structural Equation Modeling:A Second Course, edited by Gregory R., Hancock and Ralph D., Mueller, 269–314. Greenwich, CT:Information Age Publishing.
26. Foldnes, Njal, Ulf Henning, Olsson, and Tron, Foss. 2012. “The Effect of Kurtosis on the Power of two Test Statistics in Covariance Structure Analysis.” British Journal of Mathematical and Statistical Psychology 65:1–18. doi:10.1111/j.2044-8317.2010.02010.x.
27. Harris, L. R., and G. T., Brown. 2008. Teachers’ Conceptions of Feedback Inventory (Unpublished test). Auckland, NZ:University of Auckland.
28. Harris, L. R., and G. T. L., Brown. 2009. “The Complexity of Teachers’ Conceptions of Assessment:Tensions Between the Needs of Schools and Students.” Assessment in Education:Principles, Policy & Practice 16 (3):365–381. doi:10.1080/09695940903319745.
29. Harris, L. R., and G. T. L., Brown. 2010. “Mixing Interview and Questionnaire Methods:Practical Problems in Aligning Data.” Practical Assessment Research & Evaluation 15 (1). http://pareonline.net/pdf/v15n1.pdf.
30. Harris, L. R., and G. T. L., Brown. 2013. “Opportunities and Obstacles to Consider When Using Peer- and Self-assessment to Improve Student Learning:Case Studies Into Teachers’ Implementation.” Teaching and Teacher Education 36:101–111. doi:10.1016/j.tate.2013.07.008.
31. Hattie, J. 2009. Visible Learning:A Synthesis of Meta-Analyses in Education. London:Routledge.
32. Hattie, John, and H., Timperley. 2007. “The Power of Feedback.” Review of Educational Research 77 (1):81–112. doi:10.3102/003465430298487
33. Hoyle, R. H., 1995. “The Structural Equation Modeling Approach:Basic Concepts and Fundamental Issues.” In Structural Equation Modeling:Concepts, Issues, and Applications, edited by R. H., Hoyle, 1–15. Thousand Oaks, CA:Sage.
34. Hoyle, R. H., and Jamieson L., Duvall. 2004. “Determining the Number of Factors in Exploratory and Confirmatory Factor Analysis.” In The SAGE Handbook of Quantitative Methodology for Social Sciences, edited by David, Kaplan, 301–315. Thousand Oaks, CA:Sage.
35. Hoyle, R. H., and G. T., Smith. 1994. “Formulating Clinical Research Hypotheses as Structural Equation Models–A Conceptual Overview.” Journal of Consulting and Clinical Psychology 62 (3):429–440. doi:10.1037/0022-006X.62.3.429
36. Hu, Li-Tze, and P. M., Bentler. 1999. “Cutoff Criteria for fit Indexes in Covariance Structure Analysis:Conventional Criteria Versus new Alternatives.” Structural Equation Modeling 6 (1):1–55. doi:10.1080/10705519909540118
37. IBM. 2011. Amos [Computer Program] Version 20, Build 817. Meadville, PA:Amos Development Corporation.
38. Irving, S., Lois, Harris, and Elizabeth, Peterson. 2011. “One Assessment Doesn't Serve all the Purposes’ or Does it? New Zealand Teachers Describe Assessment and Feedback.” Asia Pacific Education Review 12 (3):413–426. doi:10.1007/s12564-011-9145-1.
39. Klem, Laura. 2000. “Structural Equation Modeling.” In Reading and Understanding More Multivariate Statistics, edited by Laurence G., Grimm and Paul R., Yarnold, 227–260. Washington, DC:APA.
40. Kline, Paul. 1994. An Easy Guide to Factor Analysis. London:Routledge.
41. Kluger, A. N., and A., DeNisi. 1996. “The Effects of Feedback Interventions on Performance:A Historical Review, a Meta-Analysis, and A Preliminary Feedback Intervention Theory.” Psychological Bulletin 119 (2):254–284. doi:10.1037/0033-2909.119.2.254
42. Louisiana Department of Education. 2006. “2004–2005 Louisiana State Education Progress Report.” Accessed October 18, 2006. http://www.louisianaschools.net.
43. Louisiana Department of Education. 2008, August. “Ten Schools Overcome Unacceptable Label:School Performance Scores Show Overall Improvement.” Accessed August 25, 2008. http://www.louisianaschools.net/lde/comm/pressrelease.aspx?PR=1123.
44. Louisiana Educator. 2010, May 18. “The Title of Education Legislation ‘Misleading’.” Accessed July 9, 2010. http://louisianaeducator.blogspot.com/2010_05_16_archive.html.
45. Lussier, C., 2010, July 5. “Some Charters Fall Short.” Accessed July 9, 2010. http://www.2theadvocate.com/news/97781354.html?index=1andc=y.
46. Marsh, H. W., Kit-Tai, Hau, and Zhonglin, Wen. 2004. “In Search of Golden Rules:Comment on Hypothesis-Testing Approaches to Setting Cutoff Values for fit Indexes and Dangers in Overgeneralizing Hu and Bentler's (1999) Findings.” Structural Equation Modeling 11 (3):320–341. doi:10.1207/s15328007sem1103_2
47. Marsh, H. W., Ian D., Smith, and Jennifer, Barnes. 1983. “Multi-trait Multi-method Analyses of the Self-Description Questionnaire:Student–Teacher Agreement on Multidimensional Ratings of Student Self-concept.” American Educational Research Journal 20 (3):333–357. doi:10.3102/00028312020003333
48. Murnane, Richard J., and John B., Willett. 2011. Methods Matter:Improving Causal Inference in Educational and Social Science Research. Oxford:Oxford University Press.
49. New Zealand Ministry of Education. 2005. “Report on the Findings of the 2004 Teacher Census.” Accessed December 10, 2010. http://www.educationcounts.govt.nz/publications/schooling/teacher_census.
50. New Zealand Ministry of Education. 2010. “Ministry of Education Position Paper:Assessment [Schooling Sector]:Ko te Wharangi Takotoranga Arunga, a te Tauhuhu o te Matauranga, te matekitenga.” Accessed November 20, 2010. http://www.minedu.govt.nz/theMinistry/PublicationsAndResources/AssessmentPositionPaper.aspx.
51. O'Quin, Christel Rogerie. 2009. “Feedback for students:What do teachers believe?” EdD in Educational Leadership unpublished dissertation. The Consortium of Southeastern Louisiana University and University of Louisiana Lafayette.
52. Osborne, Jason W., and Amy, Overbay. 2004. “The Power of Outliers (and why Researchers Should Always Check for Them).” Practical Assessment, Research & Evaluation 9 (6). http://PAREonline.net/getvn.asp?v=9&n=6.
53. Ravitch, Diane. 2013. Reign of Error:The Hoax of the Privatization Movement and the Danger to America's Public Schools. New York:A. E. Knopf.
54. Raykov, Tenko, and George A., Marcoulides. 2007. A First Course in Structural Equation Modeling. New York:Psychology Press.
55. Schildkamp, K., and A., Visscher. 2010. “The Use of Performance Feedback in School Improvement in Louisiana.” Teaching and Teacher Education 26:1389–1403. doi:10.1016/j.tate.2010.04.004
56. Stiggins, Richard J., 2002. “Assessment Crisis:The Absence of Assessment for Learning.” Phi Delta Kappan 83 (10):758–765. doi:10.1177/003172170208301010
57. Thompson, Bruce. 2000. “Ten Commandments of Structural Equation Modeling.” In Reading and Understanding More Multivariate Statistics, edited by Laurence G., Grimm and Paul R., Yarnold, 261–283. Washington, DC:APA.
58. Tran, Thanh V., 2009. Developing Cross-cultural Measurement. Oxford:Oxford University Press.
59. Ullman, Jodie B., 2006. “Structural Equation Modeling:Reviewing the Basics and Moving Forward.” Journal of Personality Assessment 87 (1):35–50. doi:10.1207/s15327752jpa8701_03
60. Vandenberg, Robert J., and Charles E., Lance. 2000. “A Review and Synthesis of the Measurement Invariance Literature:Suggestions, Practices, and Recommendations for Organizational Research.” Organizational Research Methods 3 (4):4–70. doi:10.1177/109442810031002
61. Vollmeyer, R., and F., Rheinberg. 2005. “A Surprising Effect of Feedback on Learning.” Learning and Instruction 15 (6):589–602. doi:10.1016/j.learninstruc.2005.08.001
62. Wu, A. D., Z., Li, and B. D., Zumbo. 2007. “Decoding the Meaning of Factorial Invariance and Updating the Practice of Multi-Group Confirmatory Factor Analysis:A Demonstration with TIMSS Data.” Practical Assessment, Research & Evaluation 12 (3). http://pareonline.net/getvn.asp?v=12&n=3.