FACTOR STRUCTURE OF THE TOEFL® INTERNET-BASED TEST (IBT): EXPLORATION IN A FIELD TRIAL SAMPLE

ETS Research Report Series

Volumn 2008, Issue 1, 2008, Pages i-67

  Sawaki, Yasuyo ^a   Stricker, Lawrence ^a   Oranje, Andreas ^a  

^a EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

confirmatory factor analysis; construct validity; integrated task; score reporting

Indexed keywords

EID: 85164622249     PISSN: None     EISSN: 23308516     Source Type: Journal    
DOI: 10.1002/j.2333-8504.2008.tb02095.x     Document Type: Article

References (54)

1. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2. Bachman, L. F. (2005). Building and supporting a case for test use. Language Assessment Quarterly, 2, 1–34.
3. Bachman, L. F., Davidson, F., Ryan, K., & Choi, I.-C. (1995). An investigation into the comparability of two tests of English as a foreign language: The Cambridge-TOEFL comparability study. Cambridge, England: Cambridge University Press.
4. Bachman, L. F., & Palmer, A. (1981). The construct validation of the FSI oral interview. Language Learning, 31, 67–86.
5. Bachman, L. F., & Palmer, A. (1982). The construct validation of some components of communicative proficiency. TESOL Quarterly, 16, 449–465.
6. Bagozzi, R. P., & Yi, Y. (1992). On the evaluation of structural equation models. Journal of the Academy of Marketing Science, 16, 74–94.
7. Bejar, I., Douglas, D., Jamieson, J., Nissan, S., & Turner, J. (2000). TOEFL 2000 listening framework: A working paper (TOEFL Monograph Series No. MS-21). Princeton, NJ: ETS.
8. Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136–162). Newbury Park, CA: Sage Publications.
9. Butler, F. A., Eignor, D., Jones, S., McNamara, T., & Suomi, B. (2000). TOEFL 2000 speaking framework: A working paper (TOEFL Monograph Series No. MS-20). Princeton, NJ: ETS.
10. Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.
11. Carr, N. T. (2003). An investigation into the structure of text characteristics and reader abilities in a test of second language reading. Unpublished doctoral dissertation, University of California, Los Angeles.
12. Carroll, J. B. (1983). Psychometric theory and language testing. In J.W. Oller Jr. (Ed.), Issues in language testing research (pp. 80–107). Rowley, MA: Newbury House.
13. Chapelle, C., Enright, M. K., & Jamieson, J. (Eds.). (2008). Building a validity argument for the Test of English as a Foreign Language. New York: Taylor & Francis.
14. Chen, F. F., West, S. G., & Sousa, K. H. (2006). Comparison of bifactor and second-order models of quality of life. Multivariate Behavioral Research, 41, 189–225.
15. nd ed.). Hillsdale, NJ: Erlbaum.
16. Cumming, A., Kantor, R., Powers, D., Santos, T., & Taylor, C. (2000). TOEFL 2000 writing framework: A working paper (TOEFL Monograph Series No. MS-18). Princeton, NJ: ETS.
17. Davidson, F. D. (1988). An exploratory modeling of the trait structures of some existing language test datasets. Unpublished doctoral dissertation, University of California, Los Angeles.
18. Enright, M. K., Grabe, W., Koda, K., Mosenthal, P., Mulcahy-Ernt, & Schedl, M. (2000). TOEFL 2000 reading framework: A working paper (TOEFL Monograph Series No. MS-17). Princeton, NJ: ETS.
19. ETS. (2003). TOEFL test and score data summary: 2002–2003 test year data. Princeton, NJ: Author.
20. Farhady, H. (1983). On the plausibility of the unitary language proficiency factor. In J. W. Oller Jr. (Ed.), Issues in language testing research (pp. 11–28). Rowley, MA: Newbury House.
21. Hale, G. A., Rock, D. A., & Jirele, T. (1989). Confirmatory factor analysis of the Test of English as a Foreign Language (TOEFL Research Rep. No. 32). Princeton, NJ: ETS.
22. Hale, G. A., Stansfield, C. W., Rock, D. A., Hicks, M. M., Butler, F. A., & Oller, J. W., Jr. (1988). Multiple-choice Cloze items and the Test of English as a Foreign Language (TOEFL Research Rep. No. 26). Princeton, NJ: ETS.
23. Henning, G. (1992). Dimensionality and construct validity of language tests. Language Testing, 9, 1–11.
24. Hoyle, R. H., & Panter, A. T. (1995). Writing about structural equation models. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues, and applications (pp. 158–176). Thousand Oaks, CA: Sage Publications.
25. Jöreskog, K. G. (1974). Analyzing psychological data by structural analysis of covariance matrices. In R. C. Atkinson, D. H. Krantz, R. D. Luce, & P. Suppes (Eds.), Contemporary developments in mathematical psychology: Measurement, psycho-physics, and neural information processing (Vol. 2, pp. 1–56). San Francisco: Freeman.
26. Jöreskog, K. G., & Sörbom, D. (1996). PRELIS 2: User's reference guide. Chicago, IL: Science Software International.
27. Jöreskog, K., & Sörbom, D. (2003a). PRELIS (Version 2.54) [Computer software]. Chicago, IL: Scientific Software International.
28. Jöreskog, K., & Sörbom, D. (2003b). LISREL (Version 8.54) [Computer software]. Chicago, IL: Scientific Software International.
29. Kline, R. B. (1998). Principles and practice of structural equation modeling. New York: Guilford Press.
30. Kunnan, A. (1995). Test taker characteristics and test performance: A structural equation modeling approach. Cambridge, England: Cambridge University Press.
31. Manning, W. H. (1987). Development of cloze-elide tests of English as a second language (TOEFL Research Rep. No. RR-87-18). Princeton, NJ: ETS.
32. Marsh, H. W. (1988). Multitrait-multimethod analysis. In J. P. Keeves (Ed.), Educational research methodology, measurement, and evaluation: An international handbook (pp. 570–580). Oxford, England: Pergamon.
33. Marsh, H. W., & Bailey, M. (1991). Confirmatory factor analyses of multitrait-multimethod data: A comparison of alternative models. Applied Psychological Measurement, 15(1), 47–70.
34. Marsh, H. W., & Grayson, D. (1995). Latent variable models of multitrait-multimethod data. In R. H. Hoyle (Ed.), Structural equation modeling: Concepts, issues and applications (pp. 177–198). Thousand Oaks, CA: Sage.
35. McLeod, L. D., Swygert, K. A., & Thissen, D. (2001). Factor analysis for items scored in two categories. In D. Thissen & H. Wainer (Eds.), Test scoring (pp. 189–216). Mahwah, NJ: Lawrence Erlbaum.
36. Nissan, S., & Sawaki, Y. (2005, March). Can TOEFL help with classroom harmony? Paper presented at the 39th TESOL convention and exhibit, San Antonio, Texas.
37. Oller, J. W. (1976). Evidence of a general language proficiency factor: An expectancy grammar. Die Neuen Sprachen, 76, 165–174.
38. Oller, J. W. (1983). A consensus for the eighties? In J. W. Oller Jr. (Ed.), Issues in language testing research (pp. 351–356). Rowley, MA: Newbury House.
39. Oller, J. W., & Hinofotis, F. A. (1980). Two mutually exclusive hypotheses about second language ability: Factor analytic studies of a variety of language subtests. In J. W. Oller Jr., & K. Perkins (Eds.), Research in language testing (pp. 13–23). Rowley, MA: Newbury House.
40. Raykov, T., & Marcoulides, G. A. (2000). A first course in structural equation modeling. Mahwah, NJ: Erlbaum.
41. Rindskopf, D., & Rose, T. (1988). Some theory and applications of confirmatory second-order factor analysis. Multivariate Behavioral Research, 23, 51–67.
42. Sasaki, M. (1996). Second language proficiency, foreign language aptitude, and intelligence: Quantitative and qualitative analyses. New York: Lang.
43. Satorra, A. (1990). Robustness issues in structural equation modeling: A review of recent developments. Quality & Quantity, 24, 367–386.
44. Satorra, A., & Bentler, P. (1999). A scaled difference chi-square test statistic for moment structure analysis (UCLA Statistics Series No. 260). Los Angeles: University of California.
45. Sawaki, Y., & Lee, Y-.W. (2006, April). A comparison of three cognitive diagnosis models in modeling examinees' reading and listening skill mastery patterns. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco, CA.
46. Schedl, M. A., Gordon, P. C., & Tang, K. (1996). An analysis of the dimensionality of TOEFL reading comprehension items (TOEFL Research Rep. No. 53). Princeton, NJ: ETS.
47. Schmid, J., & Leiman, J. M. (1957). The development of hierarchical factor solutions. Psychometrika, 22, 53–61.
48. Shin, S.-K. (2005). Did they take the same test? Examinee language proficiency and the structure of language tests. Language Testing, 22, 31–57.
49. Stricker, L. J., & Rock, D. A. (2005). Factor analysis of New Generation TOEFL. Unpublished manuscript.
50. Stricker, L. J., Rock, D. A., & Lee, Y.-W. (2005). Factor structure of the LanguEdge test across language groups (TOEFL Monograph Series No. MS-32). Princeton, NJ: ETS.
51. Swinton, S. S., & Powers, D. E. (1980). Factor analysis of the Test of English as a Foreign Language for several language groups (TOEFL Research Rep. No. 6). Princeton, NJ: ETS.
52. von Davier, M. (2005). A general diagnostic model applied to language testing data (ETS Research Rep. No. RR-05-16). Princeton, NJ: ETS.
53. Widaman, K. F. (1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9, 1–26.
54. Zhang, Y., DiBello, L., Puhan, G., Henson, R., & Templin, J. (2006, April). Estimating skills classification reliability of student profiles scores for TOEFL Internet-based test. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco, CA.