Structural equation modeling in language testing and learning research: A review

Language Assessment Quarterly

Volumn 8, Issue 3, 2011, Pages 250-276

  In'nami, Yo ^a   Koizumi, Rie ^b  

^a TOYOHASHI UNIVERSITY OF TECHNOLOGY   (Japan)

^b TOKIWA UNIVERSITY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 80051985920     PISSN: 15434303     EISSN: 15434311     Source Type: Journal    
DOI: 10.1080/15434303.2011.582203     Document Type: Review

References (112)

1. Allison. P. D. (2003). Missing data techniques for structural equation modeling. Journal of Abnormal Psychology, 112, 545-557.
2. Arbuckle, J. L. (1994-2011). Amos [Computer software]. Chicago, IL: SPSS.
3. Arbuckle, J. L. (1999). Amos (Version 4.01) [Computer software]. Chicago, IL: Smallwaters.
4. Bachman, L. F. (1982). The trait structure of cloze test scores. TESOL Quarterly, 16, 61-70.
5. Bachman, L. F. (1998). Appendix: Language testing-SLA research interfaces. In L. F. Bachman & A. D. Cohen (Eds.), Interfaces between second language acquisition and language testing research (pp. 177-195). New York, NY: Cambridge University Press.
6. Bachman, L. F., & Eignor, D. R. (1997). Recent advances in quantitative test analysis. In C. Clapham & D. Corson (Eds.), Encyclopedia of language and education (Vol. 7): Language testing and assessment (pp. 227-242). Dordrecht, the Netherlands: Kluwer.
7. Bachman, L. F., & Palmer, A. S. (1981). The construct validation of the FSI oral interview. Language Learning, 31, 67-86.
8. Bachman, L. F., & Palmer, A. (1982). The construct validation of some components of communicative proficiency. TESOL Quarterly, 16, 449-465.
9. Bachman, L. F., & Palmer, A. (1989). The construct validation of self-ratings of communicative language ability. Language Testing, 6, 14-29.
10. Bae, J., & Bachman, L. F. (1998). A latent variable approach to listening and reading: Testing factorial invari-ance across two groups of children in the Korean/English Two-Way Immersion Program. Language Testing, 15, 380-414.
11. Bandalos, D. L., & Finney, S. J. (2010). Factor analysis: Exploratory and confirmatory. In G. R. Hancock & R. O. Mueller (Eds.), The reviewer's guide to quantitative methods in the social sciences (pp. 93-114). New York, NY: Routledge.
12. Barkaoui, K. (2010). Explaining ESL essay holistic scores: A multilevel modeling approach. Language Testing, 27, 515-535.
13. Bentler, P. M. (2006). EQS 6 structural equations program manual. Encino, CA: Multivariate Software.
14. Bentler, P. M. (1994-2011). EQS for Windows [Computer software]. Encino, CA: Multivariate Software.
15. Blais, J. G., & Laurier, M. D. (1995). The dimensionality of a placement test from several analytical perspectives. Language Testing, 12, 72-98.
16. Bollen, K. A. (1989a). A new incremental fit index for general structural equation models. Sociological Methods and Research, 17, 303-316.
17. Bollen, K. A. (1989b). Structural equations with latent variables. New York, NY: Wiley.
18. Bollen, K. A., & Long, J. S. (1993). Introduction. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 1-9). Newbury Park, CA: Sage.
19. Boomsma, A. (2000). Reporting analyses of covariance structures. Structural Equation Modeling, 7, 461-483.
20. Breckler, S. J. (1990). Applications of covariance structure modeling in psychology: Cause for concern? Psychological Bulletin, 107, 260-273.
21. Brown, T. A. (2006). Confirmatory factor analysis for applied research. New York, NY: Guilford.
22. Byrne, B. M. (2001). Structural equation modeling with AMOS: Basic concepts, applications, and programming. Mahwah, NJ: Erlbaum.
23. Byrne, B. M. (2006). Structural equation modeling with EQS: Basic concepts, applications, and programming (2nd ed.). Mahwah, NJ: Erlbaum.
24. Carr, N. T. (2006). The factor structure of test task characteristics and examinee performance. Language Testing, 23, 269-289.
25. Chen, J. F., Warden, C. A., & Chang, H.-T. (2005). Motivators that do not motivate: The case of Chinese EFL learners and the influence of culture on motivation. TESOL Quarterly, 39, 609-633.
26. Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255.
27. Choi, I.-C., Kim, K. S., & Boo, J. (2003). Comparability of a paper-based language test and a computer-based language test. Language Testing, 20, 295-320.
28. Csizér, K., & Dörnyei, Z. (2005). The internal structure of language learning motivation and its relationship with language choice and learning effort. Modern Language Journal, 89, 19-36.
29. Csizér, K., & Kormos, J. (2009). Modelling the role of inter-cultural contact in the motivation of learning English as a foreign language. Applied Linguistics, 30, 166-185.
30. Ding, L., Velicer, W. F., & Harlow, L. L. (1995). Effects of estimation methods, number of indicators per factor and improper solutions on structural equation modeling fit indices. Structural Equation Modeling, 2, 119-143.
31. Dörnyei, Z. (2007). Research methods in applied linguistics. Oxford, UK: Oxford University Press.
32. Eckes, T., & Grotjahn, R. (2006). A closer look at the construct validity of C-tests. Language Testing, 23, 290-325.
33. Ellis, R., & Loewen, S. (2007). Confirming the operational definitions of explicit and implicit knowledge in Ellis (2005): Responding to Isemonger. Studies in Second Language Acquisition, 29, 119-126.
34. Enders, C. K. (2001). A primer on maximum likelihood algorithms available for use with missing data. Structural Equation Modeling, 8, 128-141.
35. Fan, X., & Sivo, S. A. (2005). Sensitivity of fit indexes to misspecified structural or measurement model components: Rationale of two-index strategy revisited. Structural Equation Modeling, 12, 343-367.
36. Finney, S. J., & DiStefano, C. (2006). Non-normal and categorical data in structural equation modeling. In G. R. Hancock & R. O. Mueller (Eds.), Structural equation modeling: A second course (pp. 269-314). Greenwich, CT: Information Age.
37. Fornell, C. (1982). A second generation of multivariate analysis: An overview. In C. Fornell (Ed.), A second generation of multivariate analysis (Vol. 1, pp. 1-21). New York, NY: Praeger.
38. Gardner, R. C., Tremblay, P. F., & Masgoret, A. M. (1997). Towards a full model of second language learning: An empirical investigation. Modern Language Journal, 81, 344-362.
39. Garson, D. (2009). Structural equation modeling. Retrieved from http://faculty.chass.ncsu.edu/garson/PA765/structur.htm
40. Ginther, A., & Stevens, J. (1998). Language background, ethnicity, and the internal construct validity of the Advanced Placement Spanish. In A. J. Kunnan (Ed.), Validation in language assessment (pp. 169-194). Mahwah, NJ: Erlbaum.
41. Gorsuch, G. J. (2000). EFL educational policies and educational cultures: Influences on teachers' approval of communicative activities. TESOL Quarterly, 34, 675-710.
42. Hoyle, R. A., & Panter, A. T. (1995). Writing about structural equation models. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 158-176). Thousand Oaks, CA: Sage.
43. Hsiao, T.-Y., & Oxford, R. L. (2002). Comparing theories of language learning strategies: A confirmatory factor analysis. Modern Language Journal, 86, 368-383.
44. Hu, L.-T., & Bentler, P. M. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 76-99). Thousand Oaks, CA: Sage.
45. Hu, L.-T., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424-453.
46. Hu, L.-T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
47. Hulland, J., Chow, Y.-H., & Lam, S. (1996). Use of causal models in marketing research: A review. International Journal of Research in Marketing, 13, 181-197.
48. In'nami, Y. (2006). The effects of test anxiety on listening test performance, System, 34, 317-340.
49. In'nami, Y., & Koizumi, R. (2010). Can structural equation models in second language testing and learning research be successfully replicated? International Journal of Testing, 10, 262-273.
50. Isemonger, I., & Watanabe, K. (2007). The construct validity of scores on a Japanese version of the perceptual component of the Style Analysis Survey. System, 35, 134-147.
51. Jackson, D. L., Gillaspy Jr., J. A., & Purc-Stephenson, R. (2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods, 14, 6-23.
52. Jöreskog, K. G., & Sörbom, D. (1974-2011). LISREL for Windows [Computer software]. Lincolnwood, IL: Scientific Software International.
53. Kahn, J. H. (2006). Factor analysis in counseling psychology research, training, and practice: Principles, advances, and applications. The Counseling Psychology, 34, 684-718.
54. Kashy, D. A., Donnellan, M. B., Ackerman, R. A., & Russell, D. W. (2009). Reporting and interpreting research in PSPB: Practices, principles, and pragmatics. Personality and Social Psychology Bulletin, 35, 1131-1142.
55. Kline, R. B. (2005). Principles and practice of structural equation modeling (2nd ed.). New York, NY: Guilford.
56. Kunnan, A. J. (1994). Modelling relationships among some test-taker characteristics and performance on EFL tests: An approach to construct validation. Language Testing, 11, 225-250.
57. Kunnan, A. J. (1995). Test taker characteristics and test performance: A structural equation modeling approach. Cambridge, UK: Cambridge University Press.
58. Kunnan, A. J. (1998). An introduction to structural equation modelling for language assessment research. Language Testing, 15, 295-332.
59. Lee, S.-y. (2005). Facilitating and inhibiting factors in English as a foreign language writing performance: A model testing with structural equation modeling. Language Learning, 55, 335-374.
60. Liao, Y.-F. (2009). A construct validation study of the GEPT reading and listening sections: Re-examining the models of L2 reading and listening abilities and their relations to lexico-grammatical knowledge (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3348356)
61. Llosa, L. (2007). Validating a standards-based classroom assessment of English proficiency: A multitrait-multimethod approach. Language Testing, 24, 489-515.
62. Lomax, R. G. (2010). Structural equation modeling: Multisample covariance and mean structures. In G. R. Hancock & R. O. Mueller (Eds.), The reviewer's guide to quantitative methods in the social sciences (pp. 385-395). New York, NY: Routledge.
63. Lumley, T., & Brown, A. (2005). Research methods in language testing. In E. Hinkel (Ed.), Handbook of research in second language teaching and learning (pp. 833-855). Mahwah, NJ: Erlbaum.
64. MacCallum, R. C., & Austin, J. T. (2000). Application of structural equation modeling in psychological research. Annual Review of Psychology, 51, 201-226.
65. Martens, M. P. (2005). The use of structural equation modeling in counseling psychology research. The Counseling Psychologist, 33, 269-298.
66. Matsumura, S. (2003). Modelling the relationships among interlanguage pragmatic development, L2 proficiency, and exposure to L2. Applied Linguistics, 24, 465-491.
67. McDonald, R. P. (1989). An index of goodness-of-fit based on noncentrality. Journal of Classification, 6, 97-103.
68. McDonald, R. P., & Ho, M-H. R. (2002). Principles and practice in reporting structural equation analyses. Psychological Methods, 7, 64-82.
69. Mueller, R. O., & Hancock, G. R. (2008). Best practices in structural equation modeling. In J. W. Osborne (Ed.), Best practices in quantitative methods (pp. 488-508). Thousand Oaks, CA: Sage.
70. Mueller, R. O., & Hancock, G. R. (2010). Structural equation modeling. In G. R. Hancock & R. O. Mueller (Eds.), The reviewer's guide to quantitative methods in the social sciences (pp. 371-383). New York, NY: Routledge.
71. Muthén, B., Kaplan, D., & Hollis, M. (1987). On structural equation modeling with data that are not missing completely at random. Psychometrika, 52, 431-462.
72. Muthén, L. K., & Muthén, B. O. (1998-2011). Mplus [Computer software]. Los Angeles, CA: Muthén & Muthén.
73. Muthén, L. K., & Muthén, B. O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9, 599-620.
74. Onwuegbuzie, A. J., Bailey, P., & Daley, C. E. (2000). The validation of three scales measuring anxiety at different stages of the foreign language learning process: The input anxiety scale, the processing anxiety scale, and the output anxiety scale. Language Learning, 50, 87-117.
75. Ortega, L. & Byrnes, H. (2008). The longitudinal study of advanced L2 capacities. New York, NY: Routledge.
76. Pae, T.-I., & Park, G.-P. (2006). Examining the relationship between differential item functioning and differential test functioning. Language Testing, 23, 475-496.
77. Phakiti, A. (2008a). Construct validation of Bachman and Palmer's (1996) strategic competence model over time in EFL reading tests, Language Testing, 25, 237-272.
78. Phakiti, A. (2008b). Strategic competence as a fourth-order factor model: A structural equation modeling approach. Language Assessment Quarterly, 5, 20-42.
79. Purpura, J. E. (1997). An analysis of the relationships between test takers' cognitive and metacognitive strategy use and second language test performance. Language Learning, 47, 289-325.
80. Purpura, J. E. (1998). Investigating the effects of strategy use and second language test performance with high-and low-ability test takers: A structural equation modelling approach. Language Testing, 15, 333-379.
81. Raykov, T., & Marcoulides, G. A. (2006). A first course in structural equation modeling (2nd ed.). Mahwah, NJ: Erlbaum.
82. Ross, S. J. (2005). The impact of assessment method on foreign language proficiency growth. Applied Linguistics, 26, 317-342.
83. Russell, D. W. (2002). In search of underlying dimensions: The use (and abuse) of factor analysis in Personality and Social Psychology Bulletin. Personality and Social Psychology Bulletin, 28, 1629-1646.
84. Sang, F., Schmitz, B., Vollmer, H. J., Baumert, J., & Roeder, P. M. (1986). Models of second language competence: A structural equation approach. Language Testing, 3, 54-79.
85. Sasaki, M. (1993). Relationships among second language proficiency, foreign language aptitude, and intelligence: A structural equation modeling approach. Language Learning, 43, 313-344.
86. Sasaki, M. (1996). Second language proficiency, foreign language aptitude, and intelligence: Quantitative and qualitative analyses. New York, NY: Peter Lang.
87. Sawaki, Y. (2007). Construct validation of analytic rating scales in a speaking assessment: Reporting a score profile and a composite. Language Testing, 24, 355-390.
88. Schoonen, R. (2005). Generalizability of writing scores: An application of structural equation modeling. Language Testing, 22, 1-30.
89. Schoonen, R., Hulstijn, J., & Bossers, B. (1998). Metacognitive and language-specific knowledge in native and foreign language reading comprehension: An empirical study among Dutch students in Grades 6, 8 and 10. Language Learning, 48, 71-106.
90. Schoonen, R., van Gelderen, A., de Glopper, K., Hulstijn, J., Simis, A., Snellings, P., & Stevenson, M. (2003). First language and second language writing: The role of linguistic knowledge, speed of processing, and metacognitive knowledge. Language Learning, 53, 165-202.
91. Schoonen, R., Vergeer, M., & Eiting, M. (1997). The assessment of writing ability: Expert readers versus lay readers. Language Testing, 14, 157-184.
92. Shin, S.-K. (2005). Did they take the same test? Examinee language proficiency and the structure of language tests. Language Testing, 22, 31-57.
93. Shiotsu, T., & Weir, C. J. (2007). The relative significance of syntactic knowledge and vocabulary breadth in the prediction of reading comprehension test performance. Language Testing, 24, 99-128.
94. Song, M.-Y. (2008). Do divisible subskills exist in second language (L2) comprehension? A structural equation modeling approach. Language Testing, 25, 435-464.
95. Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 173-180.
96. Tremblay, P. F., & Gardner, R. C. (1995). Expanding the motivation construct in language learning. Modern Language Journal, 79, 505-518.
97. Tseng, W.-T., Dörnyei, Z., & Schmitt, N. (2006). A new approach to assessing strategic learning: The case of self-regulation in vocabulary acquisition. Applied Linguistics, 27, 78-102.
98. Tseng, W.-T., & Schmitt, N. (2008). Toward a model of motivated vocabulary learning: A structural equation modeling approach. Language Learning, 58, 357-400.
99. Turner, C. E. (1989). The underlying factor structure of L2 close test performance in Francophone, university-level students: Causal modelling as an approach to construct validation. Language Testing, 6, 172-197.
100. Ullman, J. E. (2007). Structural equation modeling. In B. G. Tabachnick & L. S. Fidell (Eds.), Using multivariate statistics (5th ed., pp. 676-780). Boston, MA: Allyn and Bacon.
101. Vandergrift, L., Goh, C. C. M., Mareschal, C. J., & Tafaghodtari, M. H. (2006). The metacognitive awareness listening questionnaire: Development and validation. Language Learning, 56, 431-462.
102. Widaman, K. F. (2010). Multitrait-multimethod analysis. In G. R. Hancock & R. O. Mueller (Eds.), The reviewer's guide to quantitative methods in the social sciences (pp. 299-314). New York, NY: Routledge.
103. Williams, L., Edwards, J., & Vandenberg, R. (2003). Recent advances in causal modeling methods for organizational and management research. Journal of Management, 29, 903-936.
104. Woodrow, L. (2006a). Anxiety and speaking English as a second language. RELC Journal, 37, 308-328.
105. Woodrow, L. J. (2006b). A model of adaptive language learning. Modern Language Journal, 90, 297-319.
106. Worthington, R. L., & Whittaker, T. A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34, 806-838.
107. Xi, X. (2005). Do visual chunks and planning impact performance on the graph description task in the SPEAK exam? Language Testing, 22, 463-508.
108. Yamashiro, A. D. (2002). Using structural equation modeling for construct validation of an English as a foreign language public speaking rating scale. (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3057127)
109. Yashima, T. (2002). Willingness to communicate in a second language: The Japanese EFL context. Modern Language Journal, 86, 54-66.
110. Yashima, T., Zenuk-Nishide, L., & Shimizu, K. (2004). The influence of attitudes and affect on willingness to communicate and second language communication. Language Learning, 54, 1, 119-152.
111. Yuan, K.-H. (2005). Fit indices versus test statistics. Multivariate Behavioral Research, 40, 115-148.
112. Zumbo, B. D. (2003). Does item-level DIF manifest itself in scale-level analyses? Implications for translating language tests. Language Testing, 20, 136-147.