Investigating Score Dependability in English/Chinese Interpreter Certification Performance Testing: A Generalizability Theory Approach

Language Assessment Quarterly

Volumn 13, Issue 3, 2016, Pages 186-201

  Han, Chao ^a  

^a SOUTHWEST UNIVERSITY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84984620345     PISSN: 15434303     EISSN: 15434311     Source Type: Journal    
DOI: 10.1080/15434303.2016.1211132     Document Type: Article

References (54)

1. American Educational Research Association, American Psychological Association, and National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Washington, DC:American Educational Research Association.
2. C.Angelelli, (2007). Assessing medical interpreters. The Translator, 13(1), 63–82. doi:10.1080/13556509.2007.10799229
3. C.Angelelli, & H.E.Jacobson, (Eds.). (2009). Testing and assessment in translation and interpreting studies. Amsterdam, Netherlands:John Benjamins.
4. I.S.Arocha, & L.Joyce, (2013). Patient safety, professionalization, and reimbursement as primary drivers for National Medical Interpreter Certification in the United States. Translation & Interpreting, 5(1), 127–142. doi:ti.105201.2013.a07
5. L.F.Bachman, B.K.Lynch, & M.Mason, (1995). Investigating variability in tasks and rater judgements in a performance test of foreign language speaking. Language Testing, 12(2), 238–257. doi:10.1177/026553229501200206
6. R.L.Brennan, (2000). Performance assessments from the perspective of generalizability theory. Applied Psychological Measurement, 24(4), 339–353. doi:10.1177/01466210022031796
7. R.L.Brennan, (2001a). Generalizability theory. New York, NY:Springer-Verlag.
8. R.L.Brennan, (2001b). Manual for mGENOVA Version 2.1. Retrieved from http://www.education.uiowa.edu/centers/casma/computer-programs#8f748e48-f88c-6551-b2b8-ff00000648cd
9. J.D.Brown, (1984). A norm-referenced engineering reading test. In A.K.Pugh & J.M.Ulijn (Eds.), Reading for professional purposes:Studies and practices in native and foreign languages (pp. 213–222). London, UK:Heinemann Educational Books.
10. J.D.Brown, (1999). The relative importance of persons, items, subtests and languages to TOEFL test variance. Language Testing, 16(2), 217–238.
11. H.Bühler, (1986). Linguistic (semantic) and extra-linguistic (pragmatic) criteria for the evaluation of conference interpretation and interpreters. Multilingua, 5(4), 231–235.
12. S.Campbell, & S.Hale, (2003). Translation and interpreting assessment in the context of educational measurement. In G.Anderman & M.Rogers (Eds.), Translation today:Trends and perspectives (pp. 205–224). Clevedon, UK:Multilingual Matters.
13. J.Cardinet, S.Johnson, & G.Pini, (2010). Applying generalizability theory using EduG. New York, NY:Routledge.
14. J.Chen, (2009). Authenticity in accreditation tests for interpreters in China. The Interpreter and Translator Trainer, 3(2), 257–273. doi:10.1080/1750399X.2009.10798791
15. J.Chen, (2011). Language assessment:Its development and future—An interview with Lyle F. Bachman. Language Assessment Quarterly, 8(3), 277–290. doi:10.1080/15434303.2011.591464
16. A.Clifford, (2005). Putting the exam to the test:Psychometric validation and interpreter certification. Interpreting, 7(1), 97–131. doi:10.1075/intp.7.1
17. L.Feuerle, (2013). Testing interpreters:Developing, administering, and scoring court interpreter certification exams. Translation & Interpreting, 5(1), 80–93. doi:10.12807/ti.105201.2013.a04
18. A.Gebril, (2009). Score generalizability of academic writing tasks:Does one test method fit all? Language Testing, 26(4), 507–531. doi:10.1177/0265532209340188
19. A.Gebril, (2010). Bringing reading-to-writing and writing-only assessment tasks together:A generalizability analysis. Assessing Writing, 15(2), 100–117. doi:10.1016/j.asw.2010.05.002
20. D.Gile, (1999). Variability in the perception of fidelity in simultaneous interpretation. Hermes, 22, 51–79.
21. S.Hale, I.Garcia, J.Hlavac, M.Kim, M.Lai, B.Turner, & H.Slatyer, (2012). Development of a conceptual overview for a new model for NAATI standards, testing and assessment. Retrieved from http://www.naati.com.au/PDF/INT/INTFinalReport.pdf
22. C.Han, (2015a). Assessing interpretation:Practices, issues and prospects. Manuscript submitted for publication.
23. C.Han, (2015b). Investigating rater severity/leniency in interpreter performance testing:A multifaceted Rasch measurement approach. Interpreting, 17(2), 255–283. doi:10.1075/intp
24. C.Han, & M.Riazi, (2015). Investigating the effects of speech rate and accent on simultaneous interpretation:A mixed-methods approach. Manuscript submitted for publication.
25. J.Hlavac, (2013). A cross-national overview of translator and interpreter certification procedures. Translation & Interpreting. 5(1), 32–65. Retrieved from http://www.trans-int.org/index.php/transint/article/view/184/108
26. I.Kurz, (1989). Conference interpreting - user expectations. In D.L.Hammond (Ed.), Coming of age:Proceedings of the 30th annual conference of the American translators association (pp. 143–148). Medford, NJ:Learned Information.
27. J.Lee, (2008). Rating scales for interpreting performance assessment. The Interpreter and Translator Trainer, 2(2), 165–184. doi:10.1080/1750399X.2008.10798772
28. S.-B.Lee, (2015). Developing an analytic scale for assessing undergraduate students’ consecutive interpreting performances. Interpreting, 17(2), 226–254. doi:10.1075/intp
29. Y.-W.Lee, (2006). Dependability of scores for a new ESL speaking assessment consisting of integrated and independent tasks. Language Testing, 23(2), 131–166. doi:10.1191/0265532206lt325oa
30. Y.-W.Lee, & R.Kantor, (2007). Evaluating prototype tasks and alternative rating schemes for a new ESL writing test through G-theory. International Journal of Testing, 7(4), 353–385. doi:10.1080/15305050701632247
31. M.-H.Liu, (2013). Design and analysis of Taiwan’s interpretation certification examination. In D.Tsagari & R.van Deemter (Eds.), Assessment issues in language translation and interpreting (pp. 163–178). Frankfurt, Germany:Peter Lang.
32. B.K.Lynch, & T.F.McNamara, (1998). Using G-theory and many-facet Rasch measurement in the development of performance assessments of the ESL speaking skills of immigrants. Language Testing, 15(2), 158–180.
33. W.A.Mehrens, (1992). Using performance assessment for accountability purposes. Educational Measurement:Issues and Practice, 11(1), 3–9. doi:10.1111/j.1745-3992.1992.tb00220.x
34. S.Messick, (1989). Validity. In R.L.Linn (Ed.), Educational measurement (3rd ed.)(pp. 13–103). New York, NY:American Council on Education and Macmillan.
35. Office of CATTI. (2005). 二级口译英语同声传译类考试大纲. 外文出版社. [Syllabus of CATTI Level IV Simultaneous Interpreting Test]. Beijing, China:Foreign Languages Press.
36. C.E.Roat, (2006). Certification of Health Care Interpreters in the United States:A Primer, a Status Report and Considerations for National Certification. Retrieved from http://www.calendow.org/uploadedFiles/certification_of_health_care_interpretors.pdf
37. D.Russell, & K.Malcolm, (2009). Assessing ASL–English interpreters:The Canadian model of national certification. In C.V.Angelelli & H.E.Jacobson (Eds.), Testing and assessment in translation and interpreting studies (pp. 331–376). Amsterdam, Netherlands:John Benjamins.
38. Y.Sawaki, (2007). Construct validation of analytic rating scales in a speaking assessment:Reporting a score profile and a composite. Language Testing, 24(3), 355–390. doi:10.1177/0265532207077205
39. D.B.Sawyer, (2000). Towards meaningful, appropriate, and useful assessment:How the false dichotomy between theory and practice undermines interpreter education. ATA Chronicle, 29(2), 32–40.
40. D.B.Sawyer, (2004). Fundamental aspects of interpreter education:Curriculum and assessment. Amsterdam, Netherlands & Philadelphia, PA:John Benjamins.
41. H.Skaaden, (2013). Assessing interpreter aptitude in a variety of languages. In D.Tsagari & R.van Deemter (Eds.), Assessment issues in language translation and interpreting (pp. 35–50). Frankfurt, Germany:Peter Lang.
42. M.Strong, & S.F.Rudser, (1985). An assessment instrument for sign language interpreters. Sign Language Studies, 1049, 343–362. doi:10.1353/sls.1985.0008
43. The Institute of Linguists Educational Trust. (2010). Diploma in Public Service Interpreting:Handbook for candidates. Retrieved from http://www.iol.org.uk/qualifications/DPSI/Handbook/DPSIHB11.pdf
44. E.Tiselius, (2009). Revisiting Carroll’s Scales. In C.Angelelli & H.E.Jacobson (Eds.), Testing and assessment in translation and interpreting studies (pp. 95–121). Amsterdam, Netherlands:John Benjamins.
45. D.Tsagari, & R.van Deemter, (Eds.). (2013). Assessment issues in language translation and interpreting. Frankfurt, Germany:Peter Lang.
46. B.Turner, M.Lai, & N.Huang, (2010). Error deduction and descriptors – A comparison of two methods of translation test assessment. Translation & Interpreting, 2(1), 11–23.
47. H.Vermeiren, J.V.Gucht, & L.De Bontridder, (2009). Standards as critical success factors in assessments:Certifying social interpreters in Flanders, Belgium. In C.V.Angelelli & H.E.Jacobson (Eds.), Testing and assessment in translation and interpreting studies (pp. 291–330). Amsterdam, Netherlands:John Benjamins.
48. J.-H.Wang, J.Napier, D.Goswell, & A.Carmichael, (2015). The design and application of rubrics to assess signed language interpreting performance. The Interpreter and Translator Trainer, 9(1), 83–103. doi:10.1080/1750399X.2015.1009261
49. M.W.Wang, & J.C.Stanley, (1970). Differential weighting:A review of methods and empirical studies. Review of Educational Research, 40(5), 663–705. doi:10.3102/00346543040005663
50. J.Wu, M.Liu, & C.Liao, (2013). Analytic scoring in interpretation test:Construct validity and the halo effect. In -H.-H.Liao, T.-E.Kao, & Y.Lin (Eds.), The making of a translator:Multiple perspectives (pp. 277–292). Taipei, Taiwan:Bookman.
51. S.C.Wu, (2013). How do we assess students in the interpreting examinations. In D.Tsagari & R.van Deemter (Eds.), Assessment issues in language translation and interpreting (pp. 15–33). Frankfurt, Germany:Peter Lang.
52. ® Academic Speaking Test (TAST) for operational use. Language Testing, 24(2), 251–286. doi:10.1177/0265532207076365
53. C.S.Yang, (2005). 口译教学研究 – 理论与实践 [Studies on Interpreter Training:Theories and Practices]. Beijing, China:China Translation and Publishing Corporation.
54. S.Zhang, (2006). Investigating the relative effects of persons, items, sections, and languages on TOEIC score dependability. Language Testing, 23(3), 351–369. doi:10.1191/0265532206lt332oa