Assessing paired orals: Raters' orientation to interaction

Language Testing

Volumn 26, Issue 3, 2009, Pages 423-443

  Ducasse, Ana Maria ^a   Brown, Annie ^b  

^a LA TROBE UNIVERSITY   (Australia)

^b Ministry of Higher Education and Scientific Research   (United Arab Emirates)

Author keywords

Paired interaction construct; Rating oral proficiency

Indexed keywords

EID: 68049147395     PISSN: 02655322     EISSN: 14770946     Source Type: Journal    
DOI: 10.1177/0265532209104669     Document Type: Article

References (52)

1. Berry, V. (1997). Ethical considerations when assessing oral proficiency in pairs. In A. K. Huhta, V. Kurki-Suonio, & S. Luoma (Eds.), Current developments in language testing. Jyvaskyla: Jyvaskyla University Press.
2. Brown, A. (2000). An investigation of the rating process in the IELTS speaking module. In Tulloch, R. (Ed.), IELTS research reports, Vol. 3, ELICOS, Sydney.
3. Brown, A. (2003). Interviewer variation and the co-construction of speaking proficiency. Language Testing, 20(1), 1-25.
4. Brown, A. (2005). Interviewer variability in oral proficiency interviews. Frankfurt am Main: Peter Lang.
5. Brown, A., Iwashita, N., & McNamara, T. (2005). An examination of rater orientations and test-taker performance on English-for-academic-purposes speaking tasks (TOEFL Monograph No. TOEFL-MS-29), Educational Testing Service, Princeton, NJ.
6. Csepes, I. (2002). Measuring oral proficiency through paired performance. Unpublished PhD dissertation, Eötvös Loránd University, Budapest.
7. Cumming, A., Kantor, R. & Powers, D.E. (2002). Decision making while rating ESL/EFL writing tasks: A descriptive framework. Modern Language Journal, 86, 67-96.
8. Ducasse, A. (2007). How do candidates view interaction in a paired oral? In C. Gitsaki (Ed.), Language and Languages: Global and Local Tensions (pp. 184-200). Newcastle, UK: Cambridge Scholars Publishing.
9. Ducasse, A. (2009). An empirically based rating scale for interaction in a Paired Test. In Brown, A., & Hill, K. (Eds.), Tasks and criteria in performance assessment: Proceedings of the 28th Annual Language Testing Research Colloquium (pp. 1-22). Frankfurt: Peter Lang.
10. Egyud, G., & Glover, P. (2001). Oral testing in pairs: A secondary school perspective. ELT Journal, 55(1), 70-76.
11. Ericsson, K., & Simon, H. (1993). Protocol analysis: Verbal reports as data (revised edition). Cambridge, MA: MIT Press.
12. ffrench, A. (1999). Study of qualitative differences between CPE individuals and paired test formats. Internal UCLES EFL Report.
13. Folland, D., & Robertson, D. (1976). Towards objectivity in group oral testing. ELT Journal, 30, 156-167.
14. Fulcher, G. (1996). Does thick description lead to smarter tests? Language Testing, 13(2), 23-51.
15. Fulcher, G. (2003). Testing second language speaking. London: Pearson Education.
16. Galaczi, E. (2004). Peer-peer interaction in a paired speaking test: The case of the First Certificate in English. Unpublished PhD dissertation, Teachers College, Columbia University, New York.
17. Gardner, R. (2001). When listeners talk: Response tokens and listener stance. Amsterdam: John Benjamins.
18. Gass, S., & Varonis, E. (1985). Task variation and non-native/non-native negotiation of meaning. In S. Gass & C. Madden (Eds.), Input in second language acquisition (pp. 149-161). Rowley, MA: Newbury House.
19. Green, A. (1998). Verbal Protocol analysis in language testing research: A handbook (Vol. 5). Cambridge: Cambridge University Press.
20. Hamp-Lyons, L., & Lynch, B. (1998). Perspectives on validity: An historical analysis of language testing abstracts. In A. J. Kunnan (Ed.), Validation in language assessment. Mahwah, NJ: Lawrence Erlbaum.
21. Hatch, E.M. & Lazaraton, A. (1991). The research manual: Design and statistics for applied linguistics. New York: Newbury House.
22. He, A.W. & Young, R. (1998). Language proficiency interviews: A discourse approach. In R. Young & A. W. He (Eds.), Talking and testing: Discourse approaches to the assessment of oral proficiency (pp. 1-24). Amsterdam: John Benjamins.
23. Hildson, J. (1991). The group oral exam: Advantages and limitations. In J. C. Alderson & B. North (Eds.), Language testing in the 90's: The communicative legacy. London: Modern English Publications and the British Council.
24. Iwashita, N. (1996). The validity of the paired interview in oral performance assessment. Melbourne Papers in Language Testing, 5(2), 51-65.
25. Jacoby, S., & Ochs, E. (1995). Co-construction: An introduction. Research on Language and Social Interaction, 28, 171-183.
26. Johnson, M. (2001). The art of non-conversation: A re-examination of the validity of the oral proficiency interview. New Haven, CT: Yale University Press.
27. Kormos, J. (1999). Simulating conversations in oral proficiency assessment: A conversation analysis of role plays and non-scripted interviews in language exams. Language Testing, 16(2), 163-188.
28. Kramsch, C. (1986). From language proficiency to interactional competence. The Modern Language Journal, 70, 366-372.
29. Lazaraton, A. (1992). The structural organisation of a language interview: A conversation analytic perspective. System, 20, 373-386.
30. Lazaraton, A. (1997). Preference organization in oral proficiency interviews. Research on Language and Social Interaction, 30, 53-72.
31. Lazaraton, A. (2002). A qualitative approach to the validation of oral language tests. Cambridge: UCLES/Cambridge University Press.
32. Lazaraton, A. (2004). Gesture and speech in the vocabulary explanations of one ESL teacher: A microanalytic inquiry. Language Learning, 54(1), 79-117.
33. Long, M. (1983). Native speaker/non-native speaker conversation and the negotation of comprehensive input. Applied Linguistics, 5, 177-193.
34. McNamara, T. (2001). Language assessment as social practice: Challenges for research, Language Testing, 18(4), 333-349.
35. May, L. (2006a). An examination of of rater orientations on a paired candidate discussion task through stimulated recall. Melbourne Papers in Language Testing, 11(1), 29-51.
36. May, L. (2006b). 'Effective interaction' in a paired candidate EAP speaking test. Paper presented at the 28th Annual Language Testing Research Colloquium in Melbourne, Australia, July 2006.
37. Meiron, B.E. (1998). Rating oral proficiency tests: A triangulated study of rater thought processes. Unpublished master's thesis. University of California Los Angeles.
38. Messick, S. (1996). Validity and washback in language testing. Language Testing 13, 241-256.
39. Morrison, D.M., & Lee, N. (1985). Simulating an academic tutorial: A test validation study. In Y. P. Lee (Ed.), New directions in language testing (pp. 85-92). Oxford: Pergamon Institute of English.
40. Norris, J. (2001). Identifying rating criteria for task-based EAP assessment. In T. Hudson & J. D. Brown (Eds), A focus on language test development: Expanding the language proficiency construct across a variety of tests. Technical report 2, 163-204. Honolulu: University of Hawai'i, Second Language Teaching and Curriculum Center.
41. North, B., & Schneider, G. (1998). Scaling descriptors for language proficiency scales. Language Testing, 15(2), 217-263.
42. Norton, J. (2005). The paired format in the Cambridge Speaking Tests. ELT Journal, 59(4), 287-297.
43. O'Sullivan, B. (2002). Using observation checklists to validate speaking test pair tasks. Language Testing, 19(1), 33-56.
44. Perret, G. (1990). The language testing interview: A reappraisal. In J. de Jong & D. K. Stevenson (Eds.), Individualising the assessment of language abilities (pp. 225-228). Philadelphia, PA: Multilingual Matters.
45. Politt, A., & Murray, N.L. (1996). What do raters really pay attention to? In M. Milanovic & N. Saville (Eds.), Performance testing, cognition and assessment: Selected papers from the 15th Language Testing Research Colloquium (pp. 74-91). Cambridge: Cambridge University Press.
46. Saville, N., & Hargreaves, P. (1999). Assessing speaking in the revised FCE. ELT Journal, 53, 42-51.
47. Storch, N. (2001). Role relationships in dyadic interactions and their effect on language uptake. Unpublished Ph D Thesis. The University of Melbourne, Melbourne.
48. Swain, M. (2001). Examining dialogue: Another approach to content specification and to validating inferences drawn from test scores. Language Testing, 18, 275-300.
49. Taylor, L. (2001). The paired speaking test format: Recent studies. UCLES Research Notes, Vol. 6, retrieved from http://www.cambridgeesol.org/rs_notes/rs_nts6.pdf
50. Turner, C., & Upshur, J. (1996). Developing rating scales for the assessment of second language performance. In G. Wigglesworth & C. Elder (Eds.), The language testing cycle: From inception to washback. Australian Review of Applied Linguistics, Series S (13), 55-79.
51. van Lier, L. (1989). Reeling, writhing, drawling, stretching, and fainting in coils: Oral proficiency interviews as conversation. TESOL Quarterly, 23, 489-508.
52. Young, R., & Milanovic, M. (1992). Discourse variation in oral proficiency interviews. Studies in Second Language Acquisition, 14, 403-424.