Investigating the relative effects of persons, items, sections, and languages on TOEIC score dependability

Language Testing

Volumn 23, Issue 3, 2006, Pages 351-369

  Zhang, Su ^a  

^a UNIVERSITY OF IOWA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33745743484     PISSN: 02655322     EISSN: 14770946     Source Type: Journal    
DOI: 10.1191/0265532206lt332oa     Document Type: Article

References (30)

1. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (AERA, APA and NCME) 1999: Standards for educational and psychological testing. Washington, DC: American Educational Research Association.
2. Angoff, W.H. and Sharon, A.T. 1974: The evaluation of differences in test performance of two or more groups. Educational and Psychological Measurement 34, 807-16.
3. Bachman, L.F. 1997: Generalizability theory. In Clapham, C. and Corson, D., editors, Encyclopedia of language and education. Volume 7. Language testing and assessment. Boston, MA: Kluwer Academic, 255-62.
4. Bachman, L.F., Lynch, B.K. and Mason, M. 1995: Investigating variability in tasks and rater judgements in a performance test of foreign language speaking. Language Testing 12, 239-57.
5. Bolus, R.E., Hinofotis, F.B. and Bailey, K.M. 1982: An introduction to generalizability theory in second language research. Language Learning 32, 245-58.
6. Brennan, R.L. 1983: Elements of generalizability theory. Iowa City, IA: American College Testing.
7. Brennan, R.L. 2001: Generalizability theory. New York: Springer.
8. Brown, J.D. 1999: The relative importance of persons, items, subtests and languages to TOEFL test variance. Language Testing 16, 217-38.
9. Brown, J.D. and Bailey, K.M. 1984: A categorical instrument for scoring second language writing skills. Language Learning 34, 21-42.
10. Brown, J.D. and Ross, J.A. 1996: Decision dependability of item types, sections, tests, and the overall TOEFL test battery. In Milanovic, M. and Saville, N., editors, Performance testing, congnition and assessment. Cambridge: Cambridge University Press, 231-65.
11. Crick, G.E. and Brennan, R.L. 1983: Manual for GENOVA: a generalized analysis of variance system. American College Testing, Technical Bulletin No. 43. Iowa City, IA: ACT.
12. Cronbach, L.J., Gleser, G.C., Nanda, H. and Rajaratnam, N. 1972: The dependability of behavioral measurements: theory of generalizability for scores and profiles. New York: Wiley.
13. Douglas, D. 1992: Review of the Test of English for International Communication. In Kramer, J.J. and Conoley, J.C., editors, The eleventh mental measurements yearbook. Lincoln, NE: Buros Institute of Mental Measurements of The University of Nebraska-Lincoln, 950-52.
14. Educational Testing Service No date a: TOEIC User's Guide. Princeton, NJ: Educational Testing Service. Available at: http://www.ets.org/Media/ Tests/Test_of_English_for_International_Communication/ TOEIC_User_Gd.pdf (March 2006).
15. Educational Testing Service No date b: TOEIC Technical Manual. Princeton, NJ: Educational Testing Service.
16. Eggly, S., Musial, J., and Smulowitz, J. 1999: The relationship between English language proficiency and success as a medical resident. English for Specific Purposes 18, 201-08.
17. Farhady, H. 1982: Measures of language proficiency from the learner's perspective. TESOL Quarterly 16, 43-59.
18. Kane, M.T. 1992: An argument-based approach to validation. Psychological Bulletin 112, 527-35.
19. Kunnan, A.J. 1990: DIF in native language proficiency from the learner's perspective. TESOL Quarterly 24, 741-46.
20. Kunnan, A.J. 1992: An investigation of a criterion-referenced test using G-theory, and factor and cluster analysis. Language Testing 9, 30-49.
21. Kunnan, A.J. editor 1997: Fairness and validation in language assessment. Cambridge: Cambridge University Press.
22. Lee, Y.W., Kantor, R. and Mollaun, P. 2002: Score reliability as an essential prerequisite for validating new writing and speaking tasks for TOEFL. Paper presented at the annual meeting of Teachers of English to the Speakers of Other Languages (TESOL), Salt Lake City, UT.
23. Lynch, B.K. and McNamara, T.F. 1998: Using G-theory and many-facet Rasch measurement in the development of performance assessments of the ESL speaking skills of immigrants. Language Testing 15, 158-80.
24. Messick, S. 1989: Validity. In Linn, R.L., editor, Educational measurement. 3rd ed. New York: Macmillan, 13-103.
25. Sasaki, M. 1991: A comparison of two methods for detecting differential item functioning in an ESL placement test. Language Testing 8, 95-111.
26. Shavelson, R.J. and Webb, N. 1991: Generalizability theory: a primer. Newbury Park, CA: Sage.
27. Stansfield, C.W. and Kenyon, D.M. 1992: Research of the comparability of the oral proficiency interview and the simulated oral proficiency interview. System, 20, 347-64.
28. van Weeren, J. and Theunissen, T.J.J.M. 1987: Testing pronunciation: An application of generalizability theory. Language Learning 37, 109-22.
29. Wilson, K.M. 1989: Enhancing the interpretations of a norm-referenced second-language test through criterion-referencing: a research assessment of experience in the TOEIC testing context. TOEIC Research Report No. 1. Princeton, NJ: Educational Testing Service.
30. Woodford, P.E. 1982: The Test of English for International Communication (TOEIC). In Brumfit, C., editor, English for international communication. Oxford: Pergamon, 61-71.