Can structural equation models in second language testing and learning research be successfully replicated?

International Journal of Testing

Volumn 10, Issue 3, 2010, Pages 262-273

  In'nami, Yo ^a   Koizumi, Rie ^b  

^a TOYOHASHI UNIVERSITY OF TECHNOLOGY   (Japan)

^b TOKIWA UNIVERSITY   (Japan)

Author keywords

Replication; Second language learning; Second language testing; Structural equation modeling

Indexed keywords

EID: 79960485169     PISSN: 15305058     EISSN: 15327574     Source Type: Journal    
DOI: 10.1080/15305058.2010.482219     Document Type: Article

References (66)

1. American Psychological Association. (2001). Publication manual of the American Psychological Association (5th ed.). Washington, DC: Author.
2. Arbuckle, J. L. (1999). Amos (Version 4.01) [Computer software]. Chicago: Smallwaters.
3. Bachman, L. F. (1982). The trait structure of cloze test scores. TESOL Quarterly, 16, 61-70.
4. Bachman, L. F., & Palmer, A. S. (1981). The construct validation of the FSI oral interview. Language Learning, 31, 67-86.
5. Bachman, L. F., & Palmer, A. (1982). The construct validation of some components of communicative proficiency. TESOL Quarterly, 16, 449-465.
6. Bachman, L. F., & Palmer, A. (1989). The construct validation of self-ratings of communicative language ability. Language Testing, 6, 14-29.
7. Bae, J., & Bachman, L. F. (1998). A latent variable approach to listening and reading: Testing factorial invariance across two groups of children in the Korean/English Two-Way Immersion Program. Language Testing, 15, 380-414.
8. Blais, J. G., & Laurier, M. D. (1995). The dimensionality of a placement test from several analytical perspectives. Language Testing, 12, 72-98.
9. Bollen, K. A. (1989). Structural equations with latent variables. New York: John Wiley & Sons.
10. Carr, N. T. (2006). The factor structure of test task characteristics and examinee performance. Language Testing, 23, 269-289.
11. Chen, F., Bollen, K. A., Paxton, P., Curran, P. J., & Kirby, J. B. (2001). Improper solution in structural equation models: Causes, consequences, and strategies. Sociological Methods & Research, 29, 468-508.
12. Chen, J. F., Warden, C. A., & Chang, H.-T. (2005). Motivators that do not motivate: The case of Chinese EFL learners and the influence of culture on motivation. TESOL Quarterly, 39, 609-633.
13. Choi, I.-C., Kim, K. S., & Boo, J. (2003). Comparability of a paper-based language test and a computer-based language test. Language Testing, 20, 295-320.
14. Craig, J. R., & Reese, S. C. (1973). Retention of raw data: A problem revisited. American Psychologist 28, 723.
15. Csizér, K., & Dörnyei, Z. (2005). The internal structure of language learning motivation and its relationship with language choice and learning effort. Modern Language Journal, 89, 19-36.
16. Csizér, K., & Kormos, J. (2009). Modelling the role of inter-cultural contact in the motivation of learning English as a foreign language. Applied Linguistics, 30, 166-185.
17. Cudeck, R. (1989). Analysis of correlation matrices using covariance structure models. Psychological Bulletin, 105, 317-327.
18. Eckes, T., & Grotjahn, R. (2006). A closer look at the construct validity of C-tests. Language Testing, 23, 290-325.
19. Ellis, R., & Loewen, S. (2007). Confirming the operational definitions of explicit and implicit knowledge in Ellis (2005): Responding to Isemonger. Studies in Second Language Acquisition, 29, 119-126.
20. Furlan, L. A., Cassady, J. C., & Pérez, E.R. (2009). Adapting the Cognitive Test Anxiety Scale for use with Argentinean university students. International Journal of Testing, 9, 3-19.
21. Gardner, R. C., Tremblay, P. F., & Masgoret, A. M. (1997). Towards a full model of second language learning: An empirical investigation. Modern Language Journal, 81, 344-362.
22. Gorsuch, G. J. (2000). EFL educational policies and educational cultures: Influences on teachers' approval of communicative activities. TESOL Quarterly, 34, 675-710.
23. Hedges, L. V. (1987). How hard is hard science, how soft is soft science? The empirical cumulativeness of research. American Psychologist, 42, 443-455.
24. Hsiao, T.-Y., & Oxford, R. L. (2002). Comparing theories of language learning strategies: A confirmatory factor analysis. Modern Language Journal, 86, 368-383.
25. Hulland, J., Chow, Y.-H., & Lam, S. (1996). Use of causal models in marketing research: A review. International Journal of Research in Marketing, 13, 181-197.
26. In'nami, Y. (2006). The effects of test anxiety on listening test performance. System, 34, 317-340.
27. Isemonger, I., & Watanabe, K. (2007). The construct validity of scores on a Japanese version of the perceptual component of the Style Analysis Survey. System, 35, 134-147.
28. King, G. (1995). Replication, replication. PS: Political Science and Politics, 28, 443-451.
29. Kline, R. B. (2004). Beyond significance testing: Reforming data analysis methods in behavioral research. Washington, DC: American Psychological Association.
30. Kline, R. B. (2005). Principles and practice of structural equation modeling (2nd ed.). New York: Guilford.
31. Kunnan, A.J.(1994). Modelling relationships among some test-taker characteristics and performance on EFL tests: An approach to construct validation. Language Testing, 11, 225-250.
32. Lee, S.-y. (2005). Facilitating and inhibiting factors in English as a foreign language writing performance: A model testing with structural equation modeling. Language Learning, 55, 335-374.
33. Llosa, L. (2007). Validating a standards-based classroom assessment of English proficiency: A multitrait-multimethod approach. Language Testing, 24, 489-515.
34. Matsumura, S. (2003). Modelling the relationships among interlanguage pragmatic development, L2 proficiency, and exposure to L2. Applied Linguistics, 24, 465-491.
35. Onwuegbuzie, A. J., Bailey, P., & Daley, C. E. (2000). The validation of three scales measuring anxiety at different stages of the foreign language learning process: The input anxiety scale, the processing anxiety scale, and the output anxiety scale. Language Learning, 50, 87-117.
36. Pae, T.-I., & Park, G.-P. (2006). Examining the relationship between differential item functioning and differential test functioning. Language Testing, 23, 475-496.
37. Phakiti, A. (2008a). Construct validation of Bachman and Palmer's (1996) strategic competence model over time in EFL reading tests. Language Testing, 25, 237-272.
38. Phakiti, A. (2008b). Strategic competence as a fourth-order factor model: A structural equation modeling approach. Language Assessment Quarterly, 5, 20-42.
39. Purpura, J. E. (1997). An analysis of the relationships between test takers' cognitive and metacognitive strategy use and second language test performance. Language Learning, 47, 289-325.
40. Purpura, J. E. (1998). Investigating the effects of strategy use and second language test performance with high- and low-ability test takers: A structural equation modelling approach. Language Testing, 15, 333-379.
41. Raykov, T., & Marcoulides, G. A. (2006). A first course in structural equation modelilng (2nd ed.). Mahwah, NJ: Erlbaum.
42. Ross, S. J. (2005). The impact of assessment method on foreign language proficiency growth. Applied Linguistics, 26, 317-342.
43. Sang, F., Schmitz, B., Vollmer, H. J., Baumert, J., & Roeder, P. M. (1986). Models of second language competence: A structural equation approach. Language Testing, 3, 54-79.
44. Sasaki, M. (1993). Relationships among second language proficiency, foreign language aptitude, and intelligence: A structural equation modeling approach. Language Learning, 43, 313-344.
45. Sawaki, Y. (2007). Construct validation of analytic rating scales in a speaking assessment: Reporting a score profile and a composite. Language Testing, 24, 355-390.
46. Schoonen, R. (2005). Generalizability of writing scores: An application of structural equation modeling. Language Testing, 22, 1-30.
47. Schoonen, R., Hulstijn, J., & Bossers, B. (1998). Metacognitive and language-specific knowledge in native and foreign language reading comprehension: An empirical study among Dutch students in grades 6, 8 and 10. Language Learning, 48, 71-106.
48. Schoonen, R., van Gelderen, A., de Glopper, K., Hulstijn, J., Simis, A., Snellings, P., & Stevenson, M. (2003). First language and second language writing: The role of linguistic knowledge, speed of processing, and metacognitive knowledge. Language Learning, 53, 165-202.
49. Schoonen, R., Vergeer, M., & Eiting, M. (1997). The assessment of writing ability: Expert readers versus lay readers. Language Testing, 14, 157-184.
50. nd ed.). Mahwah, NJ: Erlbaum.
51. Shin, S.-K. (2005). Did they take the same test? Examinee language proficiency and the structure of language tests. Language Testing, 22, 31-57.
52. Shiotsu, T., & Weir, C. J. (2007). The relative significance of syntactic knowledge and vocabulary breadth in the prediction of reading comprehension test performance. Language Testing, 24, 99-128.
53. Song, M.-Y. (2008). Do divisible subskills exist in second language (L2) comprehension? A structural equation modeling approach. Language Testing, 25, 435-464.
54. Tremblay, P. F., & Gardner, R. C. (1995). Expanding the motivation construct in language learning. Modern Language Journal, 79, 505-518.
55. Tseng, W.-T., Do rnyei, Z., & Schmitt, N. (2006). A new approach to assessing strategic learning: The case of self-regulation in vocabulary acquisition. Applied Linguistics, 27, 78-102.
56. Tseng, W.-T., & Schmitt, N. (2008). Toward a model of motivated vocabulary learning: A structural equation modeling approach. Language Learning, 58, 357-400.
57. Turner, C. E. (1989). The underlying factor structure of L2 cloze test performance in Francophone, university-level students: Causal modellingasanapproachtoconstruct validation. Language Testing, 6, 172-197.
58. Vandergrift, L., Goh, C. C. M., Mareschal, C. J., & Tafaghodtari, M. H. (2006). The metacognitive awareness listening questionnaire: Development and validation. Language Learning, 56, 431-462.
59. Wickerts, J. M., Borsboom, D., Kats, J., & Molenaar, D. (2006). The poor availability of psychological research data for reanalysis. American Psychologist, 61, 726-728.
60. Wolins, L. (1962). Responsibility for raw data. American Psychologist, 17, 657-658.
61. Woodrow, L. J. (2006a). A model of adaptive language learning. Modern Language Journal, 90, 297-319.
62. Woodrow, L. (2006b). Anxiety and speaking English as a second language. RELC Journal, 37, 308-328.
63. Xi, X. (2005). Do visual chunks and planning impact performance on the graph description task in the SPEAK exam? Language Testing, 22, 463-508.
64. Yashima, T. (2002). Willingness to communicate in a second language: The Japanese EFL context. Modern Language Journal, 86, 54-66.
65. Yashima, T., Zenuk-Nishide, L., & Shimizu, K. (2004). The influence of attitudes and affect on willingness tocommunicate and second language communication. Language Learning, 54, 119-152.
66. Zumbo, B. D. (2003). Does item-level DIF manifest itself in scale-level analyses? Implications for translating language tests. Language Testing, 20, 136-147.