A closer look at the construct validity of C-tests

Language Testing

Volumn 23, Issue 3, 2006, Pages 290-325

  Eckes, Thomas ^a   Grotjahn, Rüdiger ^b  

^a UNIVERSITY OF HAGEN   (Germany)

^b RUHR UNIVERSITY BOCHUM   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33745752991     PISSN: 02655322     EISSN: 14770946     Source Type: Journal    
DOI: 10.1191/0265532206lt330oa     Document Type: Article

References (111)

1. Alderson, J.C. 2002: Testing proficiency and achievement: principles and practice. In Coleman, J.A., Grotjahn, R. and Raatz, U., editors, University language testing and the C-Test. Bochum: AKS-Verlag, 15-30.
2. Alderson, J.C., Clapham, C. and Steel, D. 1997: Metalinguistic knowledge, language aptitude and language proficiency. Language Teaching Research 1, 93-121.
3. Andrich, D. 1978: A rating formulation for ordered response categories. Psychometrika 43, 561-73.
4. Arras, U., Eckes, T. and Grotjahn, R. 2002: C-Tests im Rahmen des 'Test Deutsch als Fremdsprache' (TestDaF): Erste Forschungsergebnisse [C-tests within the 'Test of German as a foreign language' (TestDaF): preliminary research findings]. In Grotjahn, R., editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen [The C-test: theoretical foundations and practical applications]. Volume 4. Bochum: AKS-Verlag, 175-209.
5. Babaii, E. and Ansary, H. 2001: The C-test: a valid operationalization of reduced redundancy principle? System 29, 209-19.
6. Bachman, L.F. 1990: Fundamental considerations in language testing. Oxford: Oxford University Press.
7. Bachman, L.F. 2000: Modern language testing at the turn of the century: assuring that what we count counts. Language Testing 17, 1-42.
8. Bachman, L.F. 2004: Statistical analyses for language assessment. Cambridge: Cambridge University Press.
9. Bachman, L.F., Davidson, F., Ryan, K. and Choi, I.-C. 1995: An investigation into the comparability of two tests of English as a foreign language: the Cambridge-TOEFL comparability study. Cambridge: Cambridge University Press.
10. Bachman, L.F. and Palmer, A.S. 1996: Language testing in practice: designing and developing useful language tests. Oxford: Oxford University Press.
11. Bae, J. and Bachman, L.F. 1998: A latent variable approach to listening and reading: testing factorial invariance across two groups of children in the Korean/English two-way immersion program. Language Testing 15, 380-414.
12. Baker, R. 1997: Classical test theory and item response theory in test analysis. Lancaster: Centre for Research in Language Education, Lancaster University.
13. Bejar, I.I. 1983: Achievement testing: recent advances. Beverly Hills, CA: Sage.
14. Bond, T.G. and Fox, C.M. 2001: Applying the Rasch model: fundamental measurement in the human sciences. Mahwah, NJ: Lawrence Erlbaum.
15. Boonsathorn, S. 1987: C-Tests, proficiency, and reading strategies in ESL. Unpublished PhD dissertation, University of Alberta, Canada.
16. Byrne, B.M. 1998: Structural equation modeling with LISREL, PRELIS, and SIMPLIS: basic concepts, applications, and programming. Mahwah, NJ: Lawrence Erlbaum.
17. Carroll, J.B. 1983: Psychometric theory and language testing. In Oller, J.W. Jr., editor, Issues in language testing research. Rowley, MA: Newbury House, 80-107.
18. Carroll, J.B. 1987: Review of Klein-Braley, C. and Raatz, U. 1985: C-Tests in der Praxis [C-testing in practice]. Bochum: AKS. Language Testing 4, 89-106.
19. Carroll, J.B. 1993: Human cognitive abilities: a survey of factor-analytic studies. Cambridge: Cambridge University Press.
20. Chapelle, C.A. 1994: Are C-tests valid measures for L2 vocabulary research? Second Language Research 10, 157-87.
21. Chapelle, C.A. and Abraham, R.G. 1990: Cloze method: what difference does it make? Language Testing 7, 121-46.
22. Chihara, T., Cline, W.D. and Sakurai, T. 1996: If the cloze test is a question, is the C-test the answer? In Grotjahn, R., editor, Der C-Test theoretische Grundlagen und praktische Anwendungen [The C-test: theoretical foundations and practical applications]. Volume 3. Bochum: Brockmeyer, 183-95.
23. Clapham, C. 1996: The development of IELTS: a study of the effect of background knowledge on reading comprehension. Cambridge: Cambridge University Press.
24. Cohen, A.D., Segal, M. and Weiss Bar-Siman-Tov, R. 1985: The C-Test in Hebrew. In Klein-Braley, C. and Raatz, U., editors, C-Tests in der Praxis [C-testing in practice]. Bochum: AKS-Verlag, 121-27.
25. Daller, H. and Grotjahn, R. 1999: The language proficiency of Turkish returnees from Germany: an empirical investigation of academic and everyday language proficiency. Language, Culture and Curriculum 12, 156-72.
26. Daller, H. and Phelan, D. 2006: The C-test and TOEIC® as measures of students' progress in intensive short courses in EFL. In Grotjahn, R., editor, Der C-Test: Theorie, Empirie, Anwendungen/The C-test: theory, empirical research, applications. Frankfurt am Main: Peter Lang, 101-19.
27. Daller, H., van Hout, R. and Treffers-Daller, J. 2003: Lexical richness in the spontaneous speech of bilinguals. Applied Linguistics 24, 197-222.
28. Dörnyei, Z. and Katona, L. 1992: Validation of the C-test amongst Hungarian EFL learners. Language Testing 9, 187-206.
29. Douglas, D. 2000: Assessing languages for specific purposes. Cambridge: Cambridge University Press.
30. Eckes, T. 2003: Qualitätssicherung beim TestDaF: Konzepte, Methoden, Ergebnisse [Quality assuran for TestDaF: concepts, methods, results]. Fremdsprachen und Hochschule 69, 43-68.
31. Eckes, T. 2004: Beurteilerübereinstimmung und Beurteilerstrenge: Eine Multifacetten-Rasch-Analyse von Leistungsbeurteilungen im 'Test Deutsch als Fremdsprache' (TestDaF) [Rater agreement and rater severity: a many-facet Rasch analysis of performance assessments in the 'Test of German as a foreign language' (TestDaF)]. Diagnostica 50, 65-77.
32. Eckes, T. 2005a: Evaluation von Beurteilungen: Psychometrische Qualitätssicherung mit dem Multifacetten-Rasch-Modell [Evaluation of ratings: psychometric quality assurance via many-facet Rasch measurement]. Zeitschrift für Psychologie 213, 77-96.
33. Eckes, T. 2005b: Examining rater effects in TestDaF writing and speaking performance assessments: a many-facet Rasch analysis. Language Assessment Quarterly 2, 197-221.
34. Eckes, T. 2006: Rasch-Modelle zur C-Test-Skalierung [Rasch models for C-tests]. In Grotjahn, R., editor, Der C-Test: Theorie, Empirie, Anwendungenl/The C-test: theory, empirical research, applications. Frankfurt am Main: Peter Lang, 1-44.
35. Eckes, T. and Grotjahn, R. 2006: C-Tests als Anker für TestDaF: Rasch-Analysen mit dem kontinuierlichen Ratingskalen-Modell [C-tests as anchor for TestDaF: Rasch analyses via the continuous rating scale model]. In Grotjahn, R., editor, Der C-Test: Theorie, Empirie, Anwendungen [The C-test: theory, empirical research, applications]. Frankfurt am Main: Peter Lang, 167-93.
36. Eckes, T., Ellis, M., Kalnberzina, V., Pižorn, K., Springer, C., Szollás, K. and Tsagari, C. 2005: Progress and problems in reforming public language examiniations in Europe: cameos from the Baltic States, Greece, Hungary, Poland, Slovenia, France, and Germany. Language Testing 22, 355-77.
37. Enright, M.K., Bridgeman, B. and Cline, F. 2002: Prototyping a test design for a new TOEFL. Princeton, NJ: Educational Testing Service. Paper presented as part of the symposium 'Research in support of the development of new TOEFL' at the meeting of the National Council on Measurement in Education, New Orleans, LA, April 2002.
38. Farhady, H. and Jamali, F. 1999: Varieties of C-test as measures of general proficiency. Journal of the Faculty of Foreign Languages 3, 23-42 (Tehran University Press).
39. Fouly, K.A., Bachman, L.F. and Cziko, G.A. 1990: The divisibility of language competence: a confirmatory approach. Language Learning 40, 1-21.
40. Ginther, A. and Stevens, J. 1998: Language background, ethnicity, and the internal construct validity of the Advanced Placement Spanish Language Examination. In Kunnan, A.J., editor, Validation in language assessment: selected papers from the 17th Language Testing Research Colloquium, Long Beach. Mahwah, NJ: Lawrence Erlbaum, 169-94.
41. Grotjahn, R. 1987: How to construct and evaluate a C-Test: a discussion of some problems and some statistical analyses. In Grotjahn, R., Klein-Braley, C. and Stevenson, D.K., editors, Taking their measure: the validity and validation of language tests. Bochum: Brockmeyer, 219-53.
42. Grotjahn, R. 1992: Der C-Test im Französischen: quantitative analysen [Quantitative analyses of a French C-test]. In Grotjahn, R., editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen [The C-test: theoretical foundations and practical applications]. Volume 1. Bochum: Brockmeyer, 205-55.
43. Grotjahn, R. 1995: Der C-Test: state of the art. Zeitschrift für Fremdsprachenforschung 6, 37-60.
44. Grotjahn, R. 2002: 'Scrambled' C-Tests: eine Folgeuntersuchung ['Scrambled' C-tests: a follow-up study]. In Grotjahn, R., editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen [The C-test: theoretical foundations and practical applications]. Volume 4. Bochum: AKS-Verlag, 83-121.
45. Grotjahn, R. 2004: TestDaF: theoretical basis and empirical research. In Milanovic, M. and Weir, C.J., editors, European language testing in a global context. Proceedings of the ALTE Barcelona Conference July 2001. Cambridge: Cambridge University Press, 189-203.
46. Grotjahn, R. 2006: 25 Jahre C-Test: Einleitung und Übersicht über den Band [25 years of research into the C-test: introduction]. In Grotjahn, R., editor, Der C-Test: Theorie, Empirrie, Anwendungen [The C-test: theory, empirical research, applications]. Frankfurt am Main: Peter Lang, ix-xxv.
47. Grotjahn, R., Klein-Braley, C. and Raatz, U. 2002: C-Tests: an overview. In Coleman, J.A., Grotjahn, R. and Raatz, U., editors, University language testing and the C-Test. Bochum: AKS-Verlag, 93-114.
48. Grotjahn, R. and Stemmer, B. 2002: C-Tests and language processing. In Coleman, J.A., Grotjahn, R. and Raatz, U., editors, University language testing and the C-Test. Bochum: AKS-Verlag, 115-30.
49. Grotjahn, R. and Tönshoff, W. 1992: Textverständnis bei der C-Test-Bearbeitung: Pilotstudien mit Französisch- und Italienischlernern [Text comprehension and C-test-taking: pilot studies with learners of French and Italian]. In Grotjahn, R., editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen [The C-test: theoretical foundations and practical applications]. Volume 1. Bochum: Brockmeyer, 19-95.
50. Hair, J.F., Anderson, R.E., Tatham, R.L. and Black, W.C. 1998: Multivariate data analysis. 5th edition. Upper Saddle River, NJ: Prentice-Hall.
51. Hastings, A.J. 2002a: Error analysis of an English C-Test: evidence for integrated processing. In Grotjahn, R., editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen [The C-test: theoretical foundations and practical applications]. Volume 4. Bochum: AKS-Verlag, 53-66.
52. Grotjahn, R. 2002b: In defense of C-testing. In Grotjahn, R., editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen [The C-test: theoretical foundations and practical applications]. Volume 4. Bochum: AKS-Verlag, 11-25.
53. Henning, G. 1992: Dimensionality and construct validity of language tests. Language Testing 9, 1-11.
54. Huhta, A. 1996: Validating an EFL C-test for students of English philology. In Grotjahn, R., editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen [The C-test: theoretical foundations and practical applications]. Volume 3. Bochum: Brockmeyer, 197-234.
55. Jafarpur, A. 2002: A comparative study of a C-Test and a cloze test. In Grotjahn, R., editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen [The C-test: theoretical foundations and practical applications]. Volume 4. Bochum: AKS-Verlag, 31-51.
56. Jakschik, G. 1996: Validierung des C-Tests für erwachsene Zweitsprachler. Eine Längsschnittuntersuchung bei Trägern von beruflichen Bildungsmaßnahmen [C-test validation for adult second language learners: a longitudinal study with bodies responsible for vocational training programmes]. In Grotjahn, R., editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen [The C-test: theoretical foundations and practical applications]. Volume 3. Bochum: Brockmeyer, 235-77.
57. Jöreskog, K.G. and Sörbom, D. 1993: LISREL 8: structural equation modeling with the SIMPLIS command language. Chicago: Scientific Software International.
58. Jöreskog, K.G. and Sörbom, D. 1996a: LISREL 8: user's reference guide. Chicago: Scientific Software International.
59. Jöreskog, K.G. and Sörbom, D. 1996b: PRELIS 2: user's reference guide. Chicago: Scientific Software International.
60. Kamimoto, T. 1992: An inquiry into what a C-Test measures. Fukuoka Women's Junior College Studies 44, 67-79.
61. Kenyon, D.M. 2000: Tape-mediated oral proficiency testing: considerations in developing Simulated Oral Proficiency Interviews (SOPIs). In Bolton, S., editor, TESTDAF: Grundlagen für die Entwicklung eines neuen Sprachtests. Beiträge aus einem Expertenseminar [TESTDAF: foundations of the development of a new language test]. Köln: VUB Gilde, 87-106.
62. Klein-Braley, C. 1985: A cloze-up on the C-test: a study in the construct validation of authentic tests. Language Testing 2, 76-104.
63. Klein-Braley, C. 1994: Language testing with the C-Test: a linguistic and statistical investigation into the strategies used by C-Test takers, and the prediction of C-Test difficulty. Unpublished Higher Thesis, Department of Linguistics and Literature, University of Duisburg, Germany.
64. Klein-Braley, C. 1996a: Hunting unicorns: C-Tests, cloze tests and the RIP (Removal of Information Procedure). In Grotjahn, R., editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen [The C-test: theoretical foundations and practical applications]. Volume 3. Bochum: Brockmeyer, 127-37.
65. Klein-Braley, C. 1996b: Towards a theory of C-Test processing. In Grotjahn, R., editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen [The C-test: theoretical foundations and practical applications]. Volume 3. Bochum: Brockmeyer, 23-94.
66. Klein-Braley, C. 1997: C-Tests in the context of reduced redundancy testing: an appraisal. Language Testing 14, 47-84.
67. Klein-Braley, C. and Raatz, U. 1984: A survey of research on the C-Test. Language Testing 1, 134-46.
68. Kline, R.B. 1998: Principles and practice of structural equation modeling. New York: Guilford.
69. Kolen, M.J. and Brennan, R.L. 2004: Test equating, scaling, and linking: methods and practices. 2nd edition. New York: Springer.
70. Kontra, E.H. and Kormos, J. 2006: Strategy use and the construct of C-tests. In Grotjahn, R., editor, Der C-Test: Theorie, Empirie, Anwendungen/The C-test: theory, empirical research, applications. Frankfurt am Main: Peter Lang, 121-38.
71. Krekeler, C. 2003: Der kleine Unterschied - und keine Folgen? Grammatik in Sprachtests für den Hochschulzugang [The role of grammar in language tests for admission to higher education]. Zeitschrift für Fremdsprachenforschung 14, 107-47.
72. Kunnan, A.J. 1998: An introduction to structural equation modelling for language assessment research. Language Testing 15, 295-332.
73. 3 statistics in an EFL reading comprehension test. Language Testing 21, 74-100.
74. Linacre, J.M. 1989: Many-facet Rasch measurement. Chicago: MESA Press.
75. Linacre, J.M. 2002: What do infit and outfit, mean-square and standardized mean? Rasch Measurement Transactions 16, 878.
76. Linacre, J.M. 2004a: A user's guide to FACETS: Rasch-model computer programs. Chicago, IL: Winsteps.com
77. Linacre, J.M. 2004b: A user's guide to WINSTEPS-MINISTEP: Rasch-model computer programs. Chicago, IL: Winsteps.com
78. Linacre, J.M. and Wright, B.D. 2002: Construction of measures from many-facet data. Journal of Applied Measurement 3, 484-509.
79. Little, D. and Singleton, D. 1992: The C-test as an elicitation instrument in second language research. In Grotjahn, R., editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen [The C-test: theoretical foundations and practical applications]. Volume 1. Bochum: Brockmeyer, 173-92.
80. McNamara, T.F. 1996: Measuring second language performance. London: Longman.
81. Messick, S. 1989: Validity. In Linn, R.L., editor, Educational measurement. 3rd edition. New York: Macmillan, 13-103.
82. Messick, S. 1995: Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning. American Psychologist 50, 741-49.
83. Myers, R. 1990: Classical and modern regression with applications. 2nd edition. Boston, MA: Duxbury.
84. Nassaji, H. 2003: Higher-level and lower-level text processing skills in advanced ESL reading comprehension. The Modem Language Journal 87, 261-76.
85. Negishi, M. 1987: The C-test: an integrative measure? IRLT Bulletin 1, 3-26.
86. Oller, J.W., Jr. 1983: A consensus for the eighties? In Oller, J.W., Jr., editor, Issues in language testing research. Rowley, MA: Newbury House, 351-56.
87. Purpura, J.E. 1998: Investigating the effects of strategy use and second language test performance with high- and low-ability test takers: a structural equation modelling approach. Language Testing 15, 333-79.
88. Raatz, U., Grotjahn, R. and Wockenfuß, V. 2006: Das TESTATT-Projekt: Entwicklung von C-Tests zur Evaluation des Fremdsprachenstudiums [The TESTATT project: development of C-tests for the evaluation of foreign language courses]. In Grotjahn, R., editor, Der C-Test: Theorie, Empirie, Anwendungen [The C-test: theory, empirical research, applications]. Frankfurt am Main: Peter Lang, 85-99.
89. Rasch, G. 1960: Probabilistic models for some intelligence and attainment tests. Copenhagen: Nielsen & Lydiche. Expanded edition, 1980, Chicago, IL: University of Chicago Press.
90. Read, J. 2000: Assessing vocabulary. Cambridge: Cambridge University Press.
91. Rea-Dickins, P. 1997: The testing of grammar in a second language. In Clapham, C. and Corson, D., editors, Encyclopedia of language and education. Volume 7. Language testing and assessment. Dordrecht: Kluwer, 87-97.
92. Reckase, M.D., Ackerman, T.A. and Carlson, J.E. 1988: Building a unidimensional test using multidimensional items. Journal of Educational Measurement 25, 193-203.
93. Schermelleh-Engel, K., Moosbrugger, H. and Müller, H. 2003: Evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online 8, 23-74.
94. Schumacker, R.E. and Lomax, R.G. 2004: A beginner's guide to structural equation modeling. 2nd edition. Mahwah, NJ: Lawrence Erlbaum.
95. Schwibbe, G. and Schwibbe, M. 1987: Validierungsuntersuchungen zum Duisburg English Language Test for Advanced Students (DELTA) [Validating the Duisburg English Language Test for Advanced Students (DELTA)]. In Grotjahn, R., Klein-Braley, C. and Stevenson, D.K., editors, Taking their measure: the validity and validation of language tests. Bochum: Brockmeyer, 255-74.
96. Shohamy, E. 1982: Predicting speaking proficiency from Cloze tests: theoretical and practical considerations for test substitution. Applied Linguistics 3, 161-71.
97. Sigott, G. 2002: High-level processes in C-Test taking? In Grotjahn, R., editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen [The C-Test: theoretical foundations and practical applications]. Volume 4. Bochum: AKS-Verlag, 67-82.
98. Sigott, G. 2004: Towards identifying the C-Test construct. Frankfurt am Main: Peter Lang.
99. Sigott, G. 2006: How fluid is the C-Test construct? In Grotjahn, R., editor, Der C-Test: Theorie, Empirie, Anwendungen [The C-Test: theory, empirical research, applications]. Frankfurt am Main: Peter Lang, 139-46.
100. Singleton, D. 1999: Exploring the second language mental lexicon. Cambridge: Cambridge University Press.
101. Singleton, D. and Singleton, E. 2002: The C-Test and L2 acquisition/ processing research. In Coleman, J.A., Grotjahn, R. and Raatz, U., editors, University language testing and the C-Test. Bochum: AKS-Verlag, 143-68.
102. Spolsky, B. 2001: Closing the cloze. In Pürschel, H. and Raatz, U., editors, Tests and translation: papers in memory of Christine Klein-Braley. Bochum: AKS-Verlag, 1-20.
103. Stemmer, B. 1991: What's on a C-Test taker's mind: mental processes in C-Test taking. Bochum: Brockmeyer.
104. Stevens, J.P. 2002: Applied multivariate statistics for the social sciences. 4th edition. Mahwah, NJ: Lawrence Erlbaum.
105. Upshur, J.A. and Homburg, T.J. 1983: Some relations among language tests at successive ability levels. In Oller, J.W., Jr., editor, Issues in language testing research. Rowley, MA: Newbury House, 188-202.
106. Vollmer, H.J. 1981: Why are we interested in 'General Language Proficiency'? In Alderson, J.C. and Hughes, A., editors, Issues in language testing. London: The British Council, 152-71.
107. Vollmer, H.J. and Sang, F. 1983: Competing hypotheses about second language ability: a plea for caution. In Oller, J.W., Jr., editor, Issues in language testing research. Rowley, MA: Newbury House, 29-79.
108. Weir, C.J. 2005: Language testing and validation: an evidence-based approach. Basingstoke: Palgrave Macmillan.
109. Wright, B.D. and Masters, G.N. 1982: Rating scale analysis. Chicago: MESA Press.
110. Wright, B.D. and Masters, G.N. 2002: Number of person or item strata. Rasch Measurement Transactions 16, 888.
111. Wright, F. 1990: Testing foreign language ability: two approaches in theory and practice. Unpublished M. Litt. Thesis, Trinity College Dublin, Ireland.