Dependability of scores for a new ESL speaking assessment consisting of integrated and independent tasks

Language Testing

Volumn 23, Issue 2, 2006, Pages 131-166

  Lee, Yong Won ^a  

^a EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33646232741     PISSN: 02655322     EISSN: 14770946     Source Type: Journal    
DOI: 10.1191/0265532206lt325oa     Document Type: Article

References (36)

1. Bachman, L.F. 1990: Fundamental considerations in language testing. New York: Oxford University Press.
2. Bachman, L.F., Lynch, B.K. and Mason, M. 1995: Investigating variability in tasks and rater judgment in a performance test of foreign speaking. Language Testing, 12, 238-57.
3. Bolus, R.E., Hinofotis, F.B. and Bailey, K.M. 1982: An introduction of generalizability theory in second language acquisition research. Language Learning, 32, 245-58.
4. Breland, H., Bridgeman, B. and Fowles, M.E. 1999: Writing assessment in admission to higher education: review and framework. ETS RR-99-3. Princeton, NJ: Educational Testing Service.
5. Brennan, R.L. 1992: Elements of generalizability theory. Iowa City, IA: ACT.
6. Brennan, R.L. 1999: GENOVA (Version 2.0) [Computer software]. Iowa City, IA: The University of Iowa, Iowa Testing Programs.
7. Brennan, R.L. 2000: Performance assessments from the perspective of generalizability theory. Applied Psychological Measurement 24, 339-53.
8. Brennan, R.L. 2001: Generalizability theory. New York: Springer.
9. Brennan, R.L., Gao, X. and Colton, D.A. 1995: Generalizability analyses of work keys listening and writing tests. Educational and Psychological Measurement 55, 157-76.
10. Butler, F.A., Eignor, D., Jones, S., McNamara, T. and Suomi, B.K. 2000: TOEFL 2000 speaking framework: a working paper. TOEFL Monograph MS-20. Princeton, NJ: Educational Testing Service.
11. Chalhoub-Deville, M. 1995: A contextualized approach to describing oral language proficiency. Language Learning 45, 251-81.
12. Chapelle, C., Enright, M.K. and Jamieson, J., editors, in press: Building a validity argument for the test of English as a foreign language. Mahwah, NJ: Erlbaum.
13. Crick, G.E. and Brennan, R.L. 1983: GENOVA [Computer software]. Iowa City, IA: The University of Iowa, Iowa Testing Programs.
14. Cronbach, L.J., Gleser, G.C., Nanda, H. and Rajaratnam, N. 1972: The dependability of behavioral measurements: theory of generalizability. New York: John Wiley.
15. Cumming, A., Kantor, R., Powers, D., Santos, T. and Taylor, C. 2000: TOEFL® 2000 writing framework: a working paper. TOEFL® monograph MS-18. Princeton, NJ: Educational Testing Service.
16. Dunbar, S.B., Koretz, D.M. and Hoover, H.D. 1991: Quality control in the development and use of performance assessments. Applied Measurement in Education 4, 298-303.
17. Enright, M.K., Bridgeman, B., Eignor, D., Lee, Y.-W. and Powers, D. (in press): Formative investigations of prototype measures of listening, reading, writing, and speaking. In Chapelle, C., Enright, M.K. and Jamieson, J., editors, Building a validity argument for the test of English as a foreign language. Mahwah, NJ: Erlbaum.
18. Fulcher, G. 2003: Testing second language speaking. Harlow: Pearson Professional Education.
19. Gao, X., Shavelson, R.J. and Baxter, G.P. 1994: Generalizability of large-scale performance assessments in science: promises and problems. Applied Measurement in Education 7, 323-42.
20. Henning, G. 1996: Accounting for nonsystematic error in performance testing. Language Testing 13, 53-61.
21. Kane, M.T., Crooks, T.J. and Cohen, A. 1999: Validating measures of performance. Educational Measurement: Issues and Practice 18, 5-17.
22. Lee, Y.-W. 2005: Dependability of scores for a new ESL speaking test: evaluating prototype tasks. TOEFL® Monograph MS-28. Princeton, NJ: ETS.
23. Lee, Y.-W., Golub-Smith, M., Payton, C. and Carey, J. 2001: The score reliability of the test of spoken English (TSE) from the generalizability theory perspective: validating the current procedure. Paper presented at the annual conference of American Educational Research Association (AERA) in Seattle, WA.
24. Lee, Y.-W. and Kantor, R. 2005: Dependability of new ESL writing test scores: evaluating prototype tasks and alternative rating schemes. TOEFL® Monograph MS-30. Princeton, NJ: ETS.
25. Lewkowicz, J.A. 1997: The integrated testing of a second language. In Clapham, C. and Corson, D., editors, Encyclopedia of language and education: Volume 7. Language testing and assessment. Dordrecht: Kluwer Academic, 121-30.
26. Linn, R.L. 1993: Performance-based assessments: implications of task specificity. Educational Measurement: Issues and Practice 13, 5-8, 15.
27. Linn, R.L., Burton, E., DeStefano, L. and Hanson, M. 1996: Generalizability of new standards project 1993 pilot study tasks in mathematics. Applied Measurement in Education 9, 201-14.
28. Lynch, B.K. and McNamara, T.F. 1998: Using G-theory and many-facet Rasch measurement in the development of performance assessments of the ESL speaking skills of immigrants. Language Testing 15, 158-80.
29. Miller, M.D. and Linn, R.L. 2000: Validation of performance-based assessments. Applied Psychological Measurement 24, 367-78.
30. Read, J. 1990: Providing relevant content in an EAP writing test. English for Specific Purposes 9, 109-21.
31. Shavelson, R.J., Baxter, G.P. and Gao, X. 1993: Sampling variability of performance-based assessments. Journal of Educational Measurement 30, 215-32.
32. Shavelson, R.J. and Webb, N.R. 1991: Generalizability theory: a primer. Newbury Park, CA: Sage.
33. Suen, H.K. 1990: Principles of test theories. Hillsdale, NJ: Lawrence Erlbaum.
34. van Weeren, J. and Theunissen, T.J.J. 1987: Testing pronunciation: an application of generalizability theory. Language Learning 37, 109-22.
35. Weir, C.J. 1993: Understanding and developing language tests. Hemel Hemstead: Prentice Hall.
36. Wesche, B. 1987: Second language performance testing: the Ontario test of ESL as an example. Language Testing 4, 28-47.