Judgments of oral proficiency by non-native and native english speaking teacher raters: Competing or complementary constructs?

Language Testing

Volumn 28, Issue 1, 2011, Pages 31-50

  Zhang, Ying ^a   Elder, Catherine ^b  

^a MONASH UNIVERSITY   (Australia)

^b UNIVERSITY OF MELBOURNE   (Australia)

Author keywords

CET SET; holistic rating; language testing; many facet Rasch measurement (MFRM); NNES and NES; oral English proficiency construct

Indexed keywords

EID: 79751476224     PISSN: 02655322     EISSN: 14770946     Source Type: Journal    
DOI: 10.1177/0265532209360671     Document Type: Article

References (85)

1. Alderson JC (1991). Bands and scores. In J. C. Alderson and B. North (Eds.), Language testing in the 1990s: The communicative legacy (pp. 71-86). London: Macmillan.
2. Bachman LF, Lynch B., and Mason M. (1995). Investigating variability in tasks and rater judgments in a performance test of foreign language speaking. Language Testing, 12(2), 238-256.
3. Barnwell D. (1989). 'Naive' native speakers and judgements of oral proficiency in Spanish. Language Testing, 6(2), 152-163.
4. Beneke J. (1991). English als lingua franca oder als Medium interkultureller Kommunikation. In R. Grebing (Ed.), Grenzenloses Sprachenlernen (pp. 54-66). Berlin: Cornelsen.
5. Berns M. (2005). Expanding on the Expanding Circle: Where do WE go from here? World Englishes, 24(1), 85-93.
6. Bolton K. (2002). Chinese Englishes: From Canton jargon to global English. World Englishes, 21(2), 181-199.
7. Bonk WJ, Ockey GJ (2003). A many-facet Rasch analysis of the second language group oral discussion task. Language Testing, 20(1), 89-110.
8. Brown A. (1995). The effect of rater variables in the development of an occupation-specific language performance test. Language Testing, 12(1), 1-15.
9. Brown A. (2000). An investigation of the rating process in the IELTS oral interview. In R. Tullon (Ed.), IELTS Research Reports 2000 (Vol. 3, pp. 49-84). Canberra: IELTS Australia, 2000.
10. Brown A. (2004). Interviewer variability in oral proficiency. Unpublished PhD thesis. University of Melbourne.
11. Brown A., Iwashita N., and McNamara T. (2005). An examination of rater orientations and test-taker performance on English-for-Academic-Purposes speaking tasks (No. RR-05-05).
12. Bruthiaux P. (2003). Squaring the circles: Issues in modeling English worldwide. International Journal of Applied Linguistics, 13(2), 159-178.
13. Brutt-Griffler J., Samimy KK (2001). Transcending the nativeness paradigm. World Englishes, 20(1), 99-106.
14. Chalhoub-Deville M. (1995a). A contextualized approach to describing oral language proficiency. Language Learning, 45(2), 251-281.
15. Chalhoub-Deville M. (1995b). Deriving oral assessment scales across different tests and rater groups. Language Testing, 12, 16-33.
16. Cook V. (1999). Going beyond the native speaker in language teaching. TESOL Quarterly, 33(2), 185-209.
17. Crewe W. (1977). Singapore English as a non-native dialect. In W. Crewe (Ed.), The English language in Singapore (pp. 96-119). Singapore: Eastern Universities Press.
18. Crystal D. (1997). English as a global language. Cambridge: Cambridge University Press.
19. Davies A. (1991). The native speaker in applied linguistics. Edinburgh: Edinburgh University Press.
20. Davies A. (1996). Proficiency or the native speaker: What are we trying to achieve in ELT? In Guy Cook and B. Seidlhofer (Eds.), Principle and practice in applied linguistics (pp. 145-159). Oxford: Oxford University Press.
21. Davies A. (1999). Standard English: Discordant voices. World Englishes, 18(2), 171-186.
22. Davies A. (2003). The native speaker: Myth and reality (2nd ed.). Buffalo, NY: Multilingual Matters.
23. Davies A. (2004). The native speaker in applied linguistics. In Alan Davies and C. Elder (Eds.), The handbook of applied linguistics (pp. 431-450). Malden, MA: Blackwell.
24. Davies A., Hamp-Lyons L., and Kemp C. (2003). Whose norms? International proficiency tests in English. World Englishes, 22(4), 571-584.
25. Elder C., Davies A. (2006). Assessing English as a lingua franca. Annual Review of Applied Linguistics, 26, 282-304.
26. Elder C., Harding L. (2008). Language testing and English as an international language. Australian Review of Applied Linguistics, 31(3), 34.31-34.11.
27. Fayer JM, Krasinski E. (1987). Native and non-native judgments of intelligibility and irritation. Language Learning, 37(3), 313-326.
28. Fulcher G. (2003). Testing second language speaking. Pearson Longman.
29. Galloway VB (1980). Perceptions of the communicative efforts of American students of Spanish. Modern Language Journal, 64, 428-433.
30. Gnutzmann C. (2000). Lingua franca. In M. Byram (Ed.), The Routledge encyclopedia of language teaching and learning (pp. 356-359). London: Routledge.
31. Graddol D. (1997). The future of English?: A guide to forecasting the popularity of the English language in the 21st century. London: British Council.
32. Graddol D. (2001). The decline of the native speaker. In AILA Review 13: English in a changing world (pp. 57-68).
33. Green A. (1998). Verbal protocol analysis in language testing research: A handbook. Cambridge, UK: University of Cambridge, Local Examinations Syndicate.
34. Hadden BL (1991). Teacher and nonteacher perceptions of second-language communication. Language Learning, 41, 1-24.
35. Hamilton J., Lopes M., McNamara T., and Sheridan E. (1993). Rating scales and native speaker performance on a communicatively oriented EAP test. Language Testing, 10, 337-353.
36. Hamp-Lyons L., Zhang BW (2001). World Englishes: Issues in and from academic writing assessment. In J. Flowerdew and M. Peacock (Eds.), Research perspectives on English for academic purposes. Cambridge: Cambridge University Press.
37. Hill K. (1996). Who should be the judge? The use of non-native speakers as raters on a test of English as an international language. Melbourne Papers in Language Testing, 5(2), 29-49.
38. Hinofotis FB, Bailey KM, and Stern SL (1981). Assessing the oral proficiency of prospective foreign teaching assistants: Instrument development. In A. Palmer, P. J. Groot and G. H. Trosper (Eds.), The construct validation of tests of communicative competence (pp. 106-126). Washington, DC: TESOL.
39. Iwashita N., McNamara T., and Elder C. (2001). Can we predict task difficulty in an oral proficiency test? Exploring the potential of an information-processing approach to task design. Language Learning, 51(3), 401-436.
40. Jenkins J. (1998). Which pronunciation norms and models for English as an International Language? ELT J, 52, 119-126.
41. Jenkins J. (2000). The phonology of English as an International Language: New models, new norms, new goals. Oxford: Oxford University Press.
42. Jenkins J. (2002). A sociolinguistically based, empirically researched pronunciation syllabus for English as an International Language. Applied Linguistics, 23, 89-103.
43. Jin Y. (2000). Washback study of CET Spoken English Test on teaching. Foreign Language World, 80(4), 56-61.
44. Joseph JE (2004). Language and identity: National, ethnic, religious. Basingstoke, UK: Pal-grave Macmillan.
45. Kachru BB (1985). Standards, codification and sociolinguistic realism: The English language in the outer circle. In R. Quirk and H. G. Widdowson (Eds.), English in the world: Teaching and learning the language and literatures (pp. 11-30). Cambridge: Cambridge University Press.
46. Kim Y. (2009). An investigation into native and non-native teachers' judgments of oral English performance: A mixed methods approach. Language Testing, 26(2), 187-217.
47. Kirkpatrick A., Xu Z. (2002). Chinese pragmatic norms and 'China English'. World Englishes, 21(2), 269-279.
48. Kramsch C. (2007). The uses of communicative competence in a global world. In J. Liu (Ed.), English language teaching in China: New approaches, perspectives and standards (pp. 55-74). London, New York: Continuum International Publishing.
49. Lazaraton A. (2005). Non-native speakers as language assessors: Recent research and implications for assessment practice. Paper presented at the BAAL, Bristol.
50. Linacre JM (2005). Facets Rasch measurement computer program. Chicago: Winsteps.com.
51. Lowenberg PH (1993). Issues of validity in tests of English as a world language: Whose standards? World Englishes, 12(1), 95-106.
52. Lowenberg PH (2000). Non-native varieties and issues of fairness in testing English as a world English. In A. J. Kunnan (Ed.), Fairness and validation in language assessment: Selected papers from the 19th Language Testing Research Colloquium. Orlando, Florida.
53. Lowenberg PH (2002). Assessing English proficiency in the Expanding Circle. World Englishes, 21(3), 431-435.
54. Lukmani Y. (2002). English in India: Assessment issues. Paper presented at the Hong Kong Seminar, Hong Kong.
55. Lynch BK, McNamara TF (1998). Using G-theory and Many-facet Rasch Measurement in the development of performance assessments of the ESL speaking skills of immigrants. Language Testing, 15(2), 158-180.
56. Malabonga V. (2004). Davies A 2003: The native speaker: myth and reality. Language Testing, 21(2), 245-248.
57. McArthur T. (1998). The English language. Cambridge: Cambridge University.
58. McArthur T. (2004). Is it world or international or global English, and does it matter? English Today, 20(3), 3-15.
59. McKay S. (2002). Teaching English as an International Language. Oxford: Oxford University Press.
60. McNamara TF (1996). Measuring second language performance. London and New York: Addison Wesley Longman.
61. McNamara TF, Lumley T. (1997). The effect of interlocutor and assessment mode variables in overseas assessments of speaking skills in occupational settings. Language Testing, 14(2), 140-156.
62. Medgyes P. (1999). Language training: A neglected area in teacher education. In G. Braine (Ed.), Non-native educators in English language teaching. Mahwah, NJ: Lawrence Erlbaum.
63. Melchers G., Shaw P. (2003). World Englishes. London: Arnold.
64. Mullis IVS (1980). Using the primary trait system for evaluating writing (Manuscript No. 10-W51). Princeton, NJ: Educational Testing Service.
65. Myford CM, Wolfe EW (2003). Detecting and measuring rater effects using many-facet Rasch measurement: Part I. Journal of Applied Measurement, 4(4), 386-421.
66. Myford CM, Wolfe EW (2004). Detecting and measuring rater effects using many-facet Rasch measurement: Part II. Journal of Applied Measurement, 5(2), 189-227.
67. National College English Testing Committee (1999). CET-SET syllabus. Shanghai: Shanghai Foreign Language Education Press.
68. North B., Schneider G. (1998). Scaling descriptors for language proficiency scales. Language Testing, 15(2), 217-262.
69. Pennycook A. (2003). Global Englishes, rip slyme, and performativity. Journal of Sociolinguistics, 7(4), 513-533.
70. Phillipson R. (1992). Linguistic imperialism. Oxford [England]; New York: Oxford University Press.
71. Pollitt A., Murray NL (1996). What raters really pay attention to. In M. Milanovic and N. Saville (Eds.), Performance testing, cognition and assessment (pp. 74-91). Cambridge: University of Cambridge Local Examinations Syndicated and Cambridge University Press.
72. Quirk R. (1990). What is Standard English? In R. Quirk and G. Stein (Eds.), English in use (pp. 112-125). London: Longman.
73. Rampton MBH (1990). Displacing the 'native speaker': Expertise, affiliation, and inheritance. ELT Journal, 44, 97-101.
74. Ross JR (1979). Where's English? In C. J. Fillmore, D. Kempler and W. S.-Y. Wang (Eds.), Individual differences in language ability and language behavior (pp. 127-163). New York: Academic Press.
75. Santos TA (1988). Professors' reactions to the academic writing of non-native speaking students. TESOL Quarterly, 22(1), 69-90.
76. Schnitzer E. (1995). English as an international language: Implications for interculturalists and language educators. International Journal of Intercultural Relations, 19(2), 227-236.
77. Seidlhofer B. (2001). Closing a conceptual gap: The case for a description of English as a lingua franca. International Journal of Applied Linguistics, 11(2), 133-158.
78. Sheorey R. (1985). Goof gravity in ESL: Native vs. nonnative perceptions. Paper presented at the 19th annual TESOL convention, New York.
79. Shi L. (2001). Native- and nonnative-speaking EFL teachers' evaluation of Chinese students' English writing. Language Testing, 18, 303-325.
80. Stanlaw J. (2004). Japanese English: Language and culture contact. Hong Kong Hong Kong University Press.
81. Timmis I. (2002). Native-speaker norms and International English: A classroom view. ELT J, 56(3), 240-249.
82. Weigle SC (1998). Using FACETS to model rater training effects. Language Testing, 15(2), 263-267.
83. Wigglesworth G. (1997). An investigation of planning time and proficiency level on oral test discourse. Language Testing, 14(1), 85-106.
84. Wright BD, Linacre JM (1994). Reasonable mean-square fit values. Rasch Measurement: Transactions of the Rasch Measurement SIG, 8, 370.
85. Yano Y. (2001). World Englishes in 2000 and beyond. World Englishes, 20(2), 119-132.