Comparability of a paper-based language test and a computer-based language test

Language Testing

Volumn 20, Issue 3, 2003, Pages 295-320

  Choi, Inn Chull ^a   Kim, Kyoung Sung ^b   Boo, Jaeyool ^c  

^a Sungshin Women's University   (South Korea)

^b Seoul National University of Education   (South Korea)

^c Jeonju National University of Education   (South Korea)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 21644479882     PISSN: 02655322     EISSN: None     Source Type: Journal    
DOI: 10.1191/0265532203lt258oa     Document Type: Article

References (62)

1. Alderson, C. 1999: Reading constructs and reading assessment. In Chalhoub-Deville, M., editor, Studies in language testing, Vol. 10. Issues in computer-adaptive testing of reading. Cambridge: University of Cambridge Press, 49-70.
2. Allen, M. J. and Yen, W. M. 1979: Introduction to measurement theory. Monterey, CA: Brooks/Cole.
3. APA (American Psychological Association) 1986: Guidelines for computer-based texts and interpretations. Washington, DC: Author.
4. Bachman, L. F. 1990: Fundamental considerations in language testing. Oxford: Oxford University Press.
5. Bachman, L. F., Davidson, F., Ryan, K. and Choi, I.-C. 1995: Studies in language testing 1: an investigation into the comparability of two tests of English as a Foreign Language. Cambridge: Cambridge University Press.
6. Bachman, L. F. and Palmer, A. 1981: The construct validation of the FSI oral interview. Language Learning 31, 67-86.
7. Bachman, L. F. and Palmer, A. 1996: Language testing in practice. Oxford: Oxford University Press.
8. Bentler, P. M. and Wu, E. J. C. 1995: EQS for Windows user's guide. Encino, CA: Multivariate Software, Inc.
9. Bollen, A. K. 1989: Structural equations with latent variables. New York: A Wiley-International Publications.
10. Bollen, A. K. and Long, S. J. 1987: Testing structural equation models. Newbury Park, CA: Sage.
11. Boo, J. 1997: Computerized versus paper-and-pencil assessment of educational development: score comparability and examinee preferences. Unpublished dissertation, University of Iowa.
12. Brown, J. D. 1997: Computers in language testing: present research and some future directions. Language Learning & Technology 1, 44-59 ; available at http://llt.msu.edu/vol1num1/brown/default.html (last accessed March 2003).
13. Bugbee, A. C. 1996: The equivalence of paper-and-pencil and computer-based testing. Journal of Research on Computing in Education 28, 282-299.
14. Canale, Michael 1983: On some dimensions of language proficiency. In Oller, J. W. Jr., editor, Issues in language testing research. Rowley, MA: Newbury House, 333-342.
15. Chalhoub-Deville, M., editor, 1999: Issues in computer-adaptive testing of reading proficiency. In Milanovic, M. ; series editor, Studies in Language Testing, Vol. 10. Cambridge: University of Cambridge Press, 6-16.
16. Chapelle, C. 1998: Construct definition and validity inquiry in SLA research. In Bachman, L. F. and Cohen, A. D., editors, Interfaces between second language acquisition and language testing research. New York: Cambridge University Press, 32-70.
17. Chapelle, C. 1999: Validity in language assessment. Annual Review of Applied Linguistics 19, 254-272.
18. Chapelle, C. 2001: Computer applications in second language acquisition: foundations for teaching, testing, and research. Cambridge: Cambridge University Press.
19. Choi, I.-C. 1991: Theoretical studies in second language acquisition: application of item response theory to language testing. New York: Peter Lang Publishing.
20. Choi, I.-C. 1994: Content and construct validation of a criterion-referenced English proficiency test. English Teaching 48, 311-348.
21. Choi, I.-C. 1997: Essential test method facets of a general English proficiency test and their validity as perceived by test-takers. Language Research 33, 773-799.
22. Choi, I.-C. 1999: Test fairness and validity of the TEPS. Language Research 35, 571-603.
23. Choi, I.-C. 2000: Language testing in the 21st century: prospects of computer adaptive testing and performance testing. Language Research 36, 205-241.
24. Choi, I.-C. and Bachman, F. 1992: An investigation into the adequacy of three IRT models for data from two EFL reading tests. Language Testing 9, 51-78.
25. Cummins, J. P. 1983: Language proficiency and academic achievement. In Oller, J. W. Jr., editor, Issues in language testing research. Rowley, MA: Newbury House, 108-126.
26. Dunkel, P. 1991: The effectiveness research on computer-assisted instruction and computer assisted language learning. In Dunkel, P., editor, Computer-assisted language learning and testing: research issues and practice. New York: Newbury House, 5-36.
27. Dunkel, P. 1999: Considerations in developing or using second/foreign language proficiency computer-adaptive tests. Language Learning & Technology 2, 77-93. http://llt.msu.edu/vol2num2/article4/index.html (last accessed March 2003).
28. Fulcher, G. 1999: Computerizing an English language placement test. ELT Journal 53, 289-299.
29. Green, B. F. 1990: System design and operations. In Wainer, H., editor, Computerized adaptive testing: a primer. Hillsdale, NJ: Lawrence Erlbaum, 23-40.
30. Green, B. F., Bock, R. D., Humphreys, L. G., Linn, R. L. and Reckase M. D. 1984: Technical guidelines for assessing computerized adaptive tests. Journal of Educational Measurement 21, 347-359.
31. Hair, J. F., Anderson, R., Tatham, R. and Black, W. 1998: Multivariate data analysis. Englewood Cliffs, NJ: Prentice Hall.
32. Hambleton, R. K., Swaminathan, H. and Rogers, H. J. 1991: Fundamentals of item response theory. Newbury Park, CA:, Sage Publications.
33. Henning, G. and Cascallar, E. 1992: A preliminary study of the nature of communicative competence. TOEFL Research Reports 36. Feb. Princeton, NJ: Educational Testing Service.
34. Joreskog, K. G. and Sorbom, D. 1997: Lisrel 7 user's reference guide. Chicago, IL: Scientific Software International Inc.
35. Jurafsky, D. and Martin, J. H. 2000: Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition. Upper Saddle River, NJ: Prentice Hall.
36. Kenyon, D. and Malabonga, V. 2001: Comparing examinee attitudes toward computer-assisted and other oral proficiency assessments. Language Learning and Technology 5, 60-83. http://llt.msu.edu/vol5num2/kenyon/default.html (last accessed March 2003).
37. Kirsch, I., Jamieson, J., Taylor, C. and Eignor. D. 1998: Computer familiarity among TOEFL examinees. TOEFL Research Reports 59. March: Princeton, NJ: Educational Testing Service.
38. Lancashire, I., Bradley, J., McCarty W., Stairs, M. and Wooldridge T. R. 1996: Using TACT with electronic texts. New York: The Modern Language Association of America.
39. Larson, J. 1999: Considerations for testing reading proficiency via computer-adaptive testing. In Chalhoub-Deville, M., editor, Studies in language testing, Vol. 10. Issues in computer-adaptive testing of reading proficiency. Cambridge: University of Cambridge Press, 71-90.
40. Marsh, H. W. and Redmayne, R. S. 1994: A multidimensional physical self-concept and its relations to multiple components of physical fitness. Journal of Sport and Exercise Psychology 16, 43-55.
41. Mazzeo, J. and Harvey, A. L. 1988: The equivalence of scores from automated and conventional educational and psychological tests: a review of the literature. Report No. CBR 87-8, ETS RR 88-21. Princeton, NJ: Educational Testing Service.
42. Mead, A. D. and Drasgow, F. 1993: Equivalence of computerized and paper-and-pencil cognitive ability tests: a meta-analysis. Psychological Bulletin 114, 449-458.
43. Miller, G. 1999: Cracking the TOEFL CBT. New York: The Princeton Review.
44. Neuman, G. and Baydoun, R. 1998: Computerization of paper-and-pencil tests: when are they equivalent? Applied Psychological Measurement 22, 71-83.
45. Oller, J. W. Jr. 1995: Review of content and construct validation of a criterion-referenced English proficiency test by Choi (1994). English Teaching 50, 161-168.
46. Russel, M. and Haney, W. 1997: Testing writing on computers: an experiment comparing student performance on tests conducted via computer and via paper-and-pencil. Educational Policy Analysis Archives 5(3).
47. Sands, W. A., Waters, B. K. and McBride, J. R., editors, 1997: Computerized adaptive testing: from inquiry to operation. Washington, DC: American Psychological Association.
48. Sawaki, Y. 2001: Comparability of Conventional and Computerized Tests of Reading in a Second Language. Language Learning & Technology 5, 38-59. http://llt.msu.edu/vol5num2/sawaki/default.html (last accessed March 2003).
49. Schaeffer, G. A., Reese, C. M., Steffen, M., McKinley, R. L. and Mills, C. N. 1993: Field test of computer-based GRE general test. Research Report 93-07. Princeton, NJ: Educational Testing Service.
50. Scott, M. and Oxford University Press 1995: Wordsmith Tools. Ver. 2.0. Oxford: Oxford University Press.
51. SPSS (Statistical Package for Social Science) 2000: SPSSPC User's Guide Ver. 9.0. Chicago: SPSS Inc.
52. Steinberg, L., Thissen, D. and Wainer, H. 1990: Validity. In Wainer, H., editor, Computerized adaptive testing: a primer. Hillsdale, NJ: Lawrence Erlbaum, 187-232.
53. Stout, W. R., Junker, N. B. and Chang, H. H. 1991: Dimtest and Testsim. Urbana, IL: University of Illinois at Urbana-Champaign.
54. Taylor C., Jamieson, J., Eignor, D. and Kirsch, I. 1998: The relationship between computer familiarity and performance on computer-based TOEFL test takers. TOEFL Research Reports 61. March. Princeton, NJ: Educational Testing Service.
55. UCLES (University of Cambridge Local Examination Syndicate) 1998: CommuniCAT. (Software.) Cambridge: University of Cambridge Local Examination Syndicate.
56. Van de Vijver, F. J. R. and Harsveld, M. 1994: The incomplete equivalence of the paper-and-pencil and computerized versions of the General Aptitude Test Battery. Journal of Applied Psychology 79, 852-859.
57. Van der Linden, W. J. and Hambleton, R. K. 1997: Handbook of modern item response theory. New York, Springer.
58. Vispoel, W. P., Bleiler, T., Boo, J., Steger-May, K., Lin, C. and Turhan, A. 1997: Computer versus paper-and-pencil assessment of self-concept: psychometric equivalence and respondent preferences. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL, March.
59. Vispoel, W. P., Boo, J. and Bleiler, T. 2001: Computerized and paper-and-pencil versions of the Rosenberg self-esteem scale: a comparison of psychometric features and respondent preferences. Educational Psychological Measurement 61, 461-474.
60. Wainer, H., Dorans, N. J., Green, B. F., Mislevy, R. J., Steinberg, L. and Thissen, D. 1990: Future challenges. In Wainer, H., Dorans, N. J., Flaugher, R., Green, B. F., Mislevy, R. J., Steinberg, L. and Thissen, D., editors, Computerized adaptive testing: a primer. Hillsdale, NJ: Lawrence Erlbaum, 233-272.
61. Wainer, H., Dorans, N. J., Eignor, D., Flaugher, R., Green, B. F., Mislevy, R., Steinberg, L. and Thissen, D., editors, 2000: Computerized adaptive testing: a primer. 2nd Edition. Hillsdale, NJ: Lawrence Erlbaum.
62. Wang, T. and Kolen, M. J. 2001: Evaluating comparability in computerized adaptive testing: issues, criteria and an example. Journal of Educational Measurement 38, 19-49.