A teacher-verification study of speaking and writing prototype tasks for a new TOEFL

Language Testing

Volumn 21, Issue 2, 2004, Pages 107-145

  Cumming, Alister ^a   Grant, Leslie ^b   Mulcahy Ernt, Patricia ^c   Powers, Donald E ^d  

^a UNIVERSITY OF TORONTO   (Canada)

^b CENTRAL MICHIGAN UNIVERSITY   (United States)

^c UNIVERSITY OF BRIDGEPORT   (United States)

^d EDUCATIONAL TESTING SERVICE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 18844376740     PISSN: 02655322     EISSN: None     Source Type: Journal    
DOI: 10.1191/0265532204lt278oa     Document Type: Article

References (57)

1. Aaron, B., Kromrey, J. D. and Ferron, J. M. 1998: Equating r-based and d-based effect size indices: Problems with the commonly recommended formula. Paper presented at the annual meeting of the Florida Educational Research Association, Orlando, FL, November. ERIC Document Reproduction Service No. ED 433-353.
2. Alderson, J. C. and Hamp-Lyons, L. 1996: TOEFL preparation courses: a study of washback. TESOL Quarterly 13, 280-297.
3. Bachman, L. 1990: Fundamental considerations in language testing. Oxford: Oxford University Press.
4. Bachman, L. 2000: Modern language testing at the turn of the century: assuring that what we count counts. Language Testing 17, 1-42.
5. Bachman, L. and Palmer, A. 1996: Language testing in practice. Oxford: Oxford University Press.
6. Bailey, K. 1999: Washback in language testing. TOEFL Monograph Series, Report No. 15. Princeton, NJ: Educational Testing Service.
7. Bejar, I., Douglas, D., Jamieson, J., Nissan, S. and Turner, J. 2000: TOEFL 2000 listening framework: a working paper. TOEFL Monograph Series, Report No. 19. Princeton, NJ: Educational Testing Service.
8. Bridgeman, B., Cline, F. and Powers, D. 2002: Evaluating new tasks for TOEFL: relationships to external criteria. Paper presented at the annual meeting of Teachers of English to Speakers of Other Languages (TESOL), Salt Lake City, UT April.
9. Brindley, G. 1998: Outcomes-based assessment and reporting in language learning programs: a review of the issues. Language Testing 15, 45-85.
10. Brindley, G. editor, 2000: Studies in immigrant English assessment, Vol. 1. Sydney, Australia: National Centre for English Language Teaching and Research, Macquarie University.
11. Butler, F., Eignor, D., Jones, S., McNamara, T. and Suomi, B. 2000: TOEFL 2000 speaking framework: a working paper. TOEFL Monograph Series, Report No. 20. Princeton, NJ: Educational Testing Service.
12. Chalhoub-Deville, M. 1995: Deriving oral assessment scales across different tests and rater groups. Language Testing 12, 16-33.
13. Chapelle, C. 1998: Construct definition and validity inquiry in SLA research. In L. Bachman and A. Cohen, editors, Interfaces between second language acquisition and language testing research. New York: Cambridge University Press, 32-70.
14. Chapelle, C. 1999: Validity in language assessment. Annual Review of Applied Linguistics 19, 254-272.
15. Chapelle, C., Grabe, W. and Berns, M. 1997: Communicative language prociency: definition and implications for TOEFL 2000. TOEFL Monograph Series Report No. 10. Princeton, NJ: Educational Testing Service.
16. Clapham, C. 1996: The development of the IELTS: a study of the effect of background knowledge on reading comprehension. Cambridge: Cambridge University Press.
17. Cumming, A. 1996: The concept of validation in language testing. In Cumming, A and Berwick R., editors, Validation in language testing. Clevedon Multilingual Matters, 1-14.
18. Cumming, A., Kantor, R., Powers, D., Santos, T. and Taylor, C. 2000: TOEFL 2000 writing framework: a working paper. TOEFL Monograph Series, Report No. 18. Princeton, NJ: Educational Testing Service.
19. Cummins, J. 1984: Bilingualism and special education: issues in assessment and pedagogy. Clevedon, UK: Multilingual Matters.
20. Darling-Hammond, L. 1994: Performance-based assessment and educational equity. Harvard Educational Review 64, 5-30.
21. Educational Testing Service (ETS) 2001: New TOEFL: phase 3 prototyping interim report. Unpublished report. Princeton, NJ: Authors.
22. Educational Testing Service (ETS) 2002: LanguEdge courseware: handbook for scoring speaking and writing. Princeton, NJ: Educational Testing Service.
23. Elder, C. 1993: How do subject specialists construe classroom language proficiency? Language Testing 10, 233-254.
24. Enright, M. and Cline, F. 2002: Evaluating new task types for TOEFL: relationship between skills. Paper presented at the annual meeting of Teachers of English to Speakers of Other Languages (TESOL), Salt Lake City, UT, April.
25. Enright, M., Grabe, B., Koda, K., Mosenthal, P., Mulcahy-Ernt, P. and Schedl, M. 2000: TOEFL 2000 reading framework: a working paper. TOEFL Monograph Series, Report No. 17. Princeton, NJ: Educational Testing Service.
26. Epp, L. and Stawychny, M. 2001: Using the Canadian Language Benchmarks (CLB) to benchmark college programs=courses and language proficiency tests. TESL Canada Journal 18, 32-47.
27. Freedman, S. 1991: Evaluating writing: linking large-scale testing and classroom assessment. Occasional Paper 27. University of California, Berkeley: Center for the Study of Writing.
28. Fulcher, G. 1999: Assessment in English for academic purposes: putting content validity in its place. Applied Linguistics 20, 221-236.
29. Ginther, A. 2001: effects of the presence and absence of visuals on performance on TOEFL CBT listening-comprehensive stimuli. TOEFL Research Report No. 66. Princeton, NJ: Educational Testing Service.
30. Gipps, C. 1994: Beyond testing: towards a theory of educational assessment. London: Falmer Press.
31. Grant, L. 1997: Testing the language proficiency of bilingual teachers: Arizona's Spanish proficiency test. Language Testing 14, 23-46.
32. Griffin, P. 1995: The American literacy profile scales: a framework for authentic assessment. Portsmouth, NH: Heinemann.
33. Halleck, G. B. and Moder, C. L. 1995: Testing language and teaching skills of international teaching assistants: The limits of compensatory strategies. TESOL Quarterly 29, 733-758.
34. Hamp-Lyons, L. and Kroll, B. 1997: TOEFL 2000- writing: composition, community, and assessment. TOEFL Monograph Report No. 5. Princeton, NJ: Educational Testing Service.
35. Hamp-Lyons, L. and Mathias, S. P. 1994: Examining expert judgements of task difficulty on essay tests. Journal of Second Language Writing 3, 49-68.
36. Hoge, R. and Coladarci, T. 1989: Teacher-based judgments of academic achievement: a review of literature. Review of Educational Research 59, 297-313.
37. Jamieson, J., Jones, S., Kirsch, I., Mosenthal, P. and Taylor, C. 1999: TOEFL 2000 framework: a working paper. TOEFL Monograph Series Report No. 16. Princeton, NJ: Educational Testing Service.
38. Kunnan, A. 1998: Approaches to validation in language assessment. In A. Kunnan, editor, Validation in language assessment. Mahwah, NJ: Lawrence Erlbaum, 1-18.
39. Lewkowicz, J. 2000: Authenticity in language testing. Language Testing 17, 43-64.
40. Linn, R., Baker, E. and Dunbar, S. 1991: Complex, performance-based assessment: expectations and validation criteria. Educational Researcher 20, 15-21.
41. Messick, S. 1989: Meaning and values in test validation: the science and ethics of assessment. Educational Researcher 18, 5-11.
42. Miesels, S., Bickel, D., Nicholson, J., Xue, Y. and Atkins-Burnett, S. 2001: Trusting teachers” judgements: a validity study of a curriculum-embedded performance assessment in kindergarten to grade 3. American Educational Research Journal 38, 73-95.
43. Miles, M. and Huberman, A. 1994: Qualitative data analysis: an expanded sourcebook, second edition. Thousand Oaks, CA:, Sage Publications.
44. Moss, P. 1992: Shifting conceptions of validity in educational measurement: implications for performance assessment. Review of Educational Research 62, 229-258.
45. North, B. 1995: The development of a common framework scale of descriptors of language proficiency based on a theory of measurement. System 23, 445-465.
46. North, B. 2000: The development of a Common Framework Scale of language proficiency. Oxford: Peter Lang.
47. North, B. and Schneider, B. 1998: Scaling descriptors for language prociency scales. Language Testing 15, 217-262.
48. O”Sullivan, B., Weir, C. and Saville, N. 2002: Using observation checklists to validate speaking-test tasks. Language Testing 19, 33-56.
49. Raimes, A. 1990: The TOEFL Test of Written English: causes for concern. TESOL Quarterly 24, 427-442.
50. Roberts, M. 2002: TOEFL preparation: what are our Korean students doing and why? Korea TESOL Journal 5, 81-105.
51. Rosenfeld, M., Leung, S. and Oltman, P. 2001: The reading, writing, speaking, and listening tasks important for academic success at the undergraduate and graduate levels. TOEFL Monograph 21. Princeton, NJ: Educational Testing Service.
52. Sharpley, C. and Edgar, E. 1986: Teachers” ratings vs. standardized tests: an empirical investigation of agreement between two indices of achievement. Psychology in the Schools 23, 106-111.
53. Spence-Brown, R. 2001: The eye of the beholder: authenticity in an embedded assessment task. Language Testing 18, 463-481.
54. Spolsky, B. 1985: The limits of authenticity in language testing. Language Testing 2, 31-40.
55. Stansfield, C. and Kenyon, D. 1996: Comparing the scaling of speaking tasks by language teachers and by the ACTFL guidelines. In Cumming, A. and Berwick, R., editors, Validation in language testing. Clevedon, UK: Multilingual Matters, 124-153.
56. Teachers of English to Speakers of Other Languages (TESOL) 1998: Managing the assessment process: a framework for measuring student attainment of the ESL standards. Alexandria, VA: Authors.
57. Thompson, B. 2000: A suggested revision to the forthcoming 5th edition of the APA Publication Manual. Available online at: http://www.coe.tamu.edu/%7Ebthompson/apaeffec.htm (December 2003).