Automatically Assessing Lexical Sophistication: Indices, Tools, Findings, and Application

TESOL Quarterly

Volumn 49, Issue 4, 2015, Pages 757-786

  Kyle, Kristopher ^a   Crossley, Scott A ^a  

^a GEORGIA STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84955488987     PISSN: 00398322     EISSN: 15457249     Source Type: Journal    
DOI: 10.1002/tesq.194     Document Type: Article

References (59)

1. Attali, Y., & Burstein, J. (2006). Automated essay scoring with e-rater V.2. Journal of Technology, Learning, and Assessment, 4(3)http://www.jtla.org
2. Biber, D., Conrad, S., & Cortes, V. (2004). If you look at ...: Lexical bundles in university teaching and textbooks. Applied Linguistics, 25, 371-405. doi:10.1093/applin/25.3.371
3. BNC Consortium. (2007), British National Corpus, version 3 (BNC XML ed.)http://www.natcorp.ox.ac.uk
4. Brown, G. D. A. (1984). A frequency count of 190, 000 words in the London-Lund Corpus of English Conversation. Behavior Research Methods, Instrumentation & Computers, 16, 502-532. doi:10.3758/BF03200836
5. Brysbaert, M., & New, B. (2009). Moving beyond Kucera and Francis: A critical evaluation of current word frequency norms and the introduction of a new and improved word frequency measure for American English. Behavior Research Methods, 41, 977-990. doi:10.3758/BRM.41.4.977
6. Brysbaert, M., Warriner, A. B., & Kuperman, V. (2014). Concreteness ratings for 40 thousand generally known English word lemmas. Behavior Research Methods, 46, 904-911. doi:10.3758/s13428-013-0403-5
7. Cobb, T. (2007). Computing the vocabulary demands of L2 reading. Language Learning & Technology, 11(3), 38-63http://llt.msu.edu
8. Cobb, T. (2013). Web Vocabprofilehttp://www.lextutor.ca/vp
9. Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum.
10. Coltheart, M. (1981). The MRC psycholinguistic database. Quarterly Journal of Experimental Psychology Section A, 33, 497-505. doi:10.1080/14640748108400805
11. Coxhead, A. (2000). A new academic word list. TESOL Quarterly, 34, 213-238. doi:10.2307/3587951
12. Coxhead, A. (2011). The academic word list 10 years on: Research and teaching implications. TESOL Quarterly, 45, 355-362. doi:10.5054/tq.2011.254528
13. Crossley, S. A., Cai, Z., & McNamara, D. S. (2012). Syntagmatic, paradigmatic, and automatic n-gram approaches to assessing essay quality. In P. M. McCarthy & G. M. Youngblood (Eds.), Proceedings of the 25th International Florida Artificial Intelligence Research Society (FLAIRS) Conference (pp. 214-219) Menlo Park, CA: AAAI Press.
14. Crossley, S. A., Clevinger, A., & Kim, Y. (2014). The role of lexical properties and cohesive devices in text integration and their effect on human ratings of speaking proficiency. Language Assessment Quarterly, 11(3), 250-270. doi:10.1080/15434303.2014.926905
15. Crossley, S. A., Cobb, T., & McNamara, D. S. (2013). Comparing count-based and band-based indices of word frequency: Implications for active vocabulary research and pedagogical applications. System, 41, 965-981. doi:10.1016/j.system.2013.08.002
16. Crossley, S. A., Greenfield, J., & McNamara, D. S. (2008). Assessing text readability using cognitively based indices. TESOL Quarterly, 42, 475-493. doi:10.1002/j.1545-7249.2008.tb00142.x
17. Crossley, S. A., & McNamara, D. S. (2012). Predicting second language writing proficiency: The role of cohesion, readability, and lexical difficulty. Journal of Research in Reading, 35, 115-135. doi:10.1111/j.1467-9817.2010.01449.x
18. Crossley, S. A., & McNamara, D. S. (2013). Applications of text analysis tools for spoken response grading. Language Learning & Technology, 17, 171-192.
19. Crossley, S., Salsbury, T., & McNamara, D. (2010). The development of polysemy and frequency use in English second language speakers. Language Learning, 60, 573-605. doi:10.1111/j.1467-9922.2010.00568.x
20. Crossley, S. A., Salsbury, T., McNamara, D. S., & Jarvis, S. (2011a). Predicting lexical proficiency in language learners using computational indices. Language Testing, 28, 561-580. doi:10.1177/0265532210378031
21. Crossley, S. A., Salsbury, T., McNamara, D. S., & Jarvis, S. (2011b). What is lexical proficiency? Some answers from computational models of speech data. TESOL Quarterly, 45, 182-193. doi:10.5054/tq.2010.244019
22. Crossley, S. A., Subtirelu, N., & Salsbury, T. (2013). Frequency effects or context effects in second language word learning: What predicts early lexical production? Studies in Second Language Acquisition, 35, 727-755. doi:10.1017/S0272263113000375
23. Crossley, S. A., Weston, J., McLain Sullivan, S. T., & McNamara, D. S. (2011). The development of writing proficiency as a function of grade level: A linguistic analysis. Written Communication, 28, 282-311. doi:10.1177/0741088311410188
24. Deane, P., & Quinlan, T. (2010). What automated analyses of corpora can tell us about students' writing skills. Journal of Writing Research, 2, 151-177.
25. Ellis, N. C. (2002). Frequency effects in language processing. Studies in Second Language Acquisition, 24(2), 143-188. doi:10.1017/S0272263102002024
26. Ferris, D. (1994). Lexical and syntactic features of ESL writing by students at different levels of L2 proficiency. TESOL Quarterly, 28, 414-420. doi:10.2307/3587446
27. Graesser, A. C., McNamara, D. S., Louwerse, M. M., & Cai, Z. (2004). Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments, & Computers, 36, 193-202. doi:10.3758/BF03195564
28. Gries, S. T. (2008). Dispersions and adjusted frequencies in corpora. International Journal of Corpus Linguistics, 13, 403-437. doi:10.1075/ijcl.13.4.02gri
29. Heatley, A., Nation, I. S. P., & Coxhead, A. (2002). RANGE and FREQUENCY programshttp://www.victoria.ac.nz/lals/staff/paul-nation.aspx
30. Hyland, K. (2004). Disciplinary discourses: Social interactions in academic writing. Ann Arbor: University of Michigan Press.
31. Jarvis, S., & Crossley, S. A. (2012). Approaching language transfer through text classification: Explorations in the detection-based approach. Bristol, England: Multilingual Matters.
32. Koda, K. (2005). Insights into second language reading. Cambridge, England: Cambridge University Press. doi:10.1017/CBO9781139524841
33. Kucera, H., & Francis, W. N. (1967). Computational analysis of present-day American English. Providence, RI: Brown University Press.
34. Kuperman, V., Stadthagen-Gonzales, H., & Brysbaert, M. (2012). Age-of-acquisition ratings for 30 thousand English words. Behavior Research Methods, 44, 978-990. doi:10.3758/s13428-012-0210-4
35. Kyle, K., Crossley, S. A., Dai, J., & McNamara, D. S. (2013, June). Native language identification: A key ngrams approach. Paper presented at the 8th Workshop on Innovative Use of NLP for Building Educational Applications, Atlanta, GA.
36. Laufer, B. (1994). The lexical profile of second language writing: Does it change over time? RELC Journal, 25(2), 21-33. doi:10.1177/003368829402500202
37. Laufer, B., & Nation, P. (1995). Vocabulary size and use: Lexical richness in L2 written production. Applied Linguistics, 16, 307-322. doi:10.1093/applin/16.3.307
38. McCarthy, P. M., & Jarvis, S. (2010). MTLD, vocd-D, and HD-D: A validation study of sophisticated approaches to lexical diversity assessment. Behavior Research Methods, 42, 381-392. doi:10.3758/BRM.42.2.381
39. McNamara, D. S., Crossley, S. A., & McCarthy, P. M. (2010). Linguistic features of writing quality. Written Communication, 27, 57-86. doi:10.1177/0741088309351547
40. McNamara, D. S., & Graesser, A. C. (2012). Coh-Metrix: An automated tool for theoretical and applied natural language processing. In P. M. McCarthy & C. Boonthum-Denecke (Eds.), Applied natural language processing and content analysis: Identification, investigation, and resolution (pp. 188-205). Hershey, PA: IGI Global. doi:10.4018/978-1-60960-741-8
41. McNamara, D. S., Graesser, A. C., McCarthy, P., & Cai, Z. (2014). Automated evaluation of text and discourse with Coh-Metrix. Cambridge, England: Cambridge University Press. doi:http://dx.doi.org/10.1017/CBO9780511894664
42. Meara, P. (1992). EFL vocabulary tests. Swansea, Wales: Centre for Applied Language Studies.
43. Meara, P. (1996). The dimensions of lexical competence. In G. Brown, K. Malmkjaer & J. Williams (Eds.), Performance and competence in second language acquisition (pp. 35-53). Cambridge, England: Cambridge University Press.
44. Meara, P. (2005a). Designing vocabulary tests for English, Spanish and other languages. In C. Butler, S. Christopher, M. Á. Gómez González, & S. M. Doval-Suárez (Eds.), The dynamics of language use (pp. 271-285). Amsterdam, the Netherlands: John Benjamins.
45. Meara, P. (2005b). Lexical frequency profiles: A Monte Carlo analysis. Applied Linguistics, 26, 32-47. doi:10.1093/applin/amh037
46. Morris, L., & Cobb, T. (2004). Vocabulary profiles as predictors of TESL student performance. System, 32, 75-87. doi:10.1016/j.system.2003.05.001
47. Nation, I. S. P. (1990). Teaching and learning vocabulary. Boston, MA: Heinle and Heinle. doi:10.1016/0346-251X(94)90065-5
48. Nation, I. S. P. (2001). Learning vocabulary in another language. Cambridge, England: Cambridge University Press. doi:10.1017/CBO9781139524759
49. Pennebaker, J. W., Booth, R. J., & Francis, M. E. (2007). LIWC2007: Linguistic inquiry and word counthttp://homepage.psy.utexas.edu/HomePage/Faculty/Pennebaker/Reprints/LIWC2007_OperatorManual.pdf
50. Read, J. (1998). Validating a test to measure depth of vocabulary knowledge. In A. Kunnan (Ed.), Validation in language assessment (pp. 41-60). Mahwah, NJ: Lawrence Erlbaum.
51. Salsbury, T., Crossley, S. A., & McNamara, D. S. (2011). Psycholinguistic word information in second language oral discourse. Second Language Research, 27, 343-360. doi:10.1177/0267658310395851
52. Schmitt, N., Schmitt, D., & Clapham, C. (2001). Developing and exploring the behaviour of two new versions of the vocabulary levels test. Language Testing, 18, 55-88. doi:10.1191/026553201668475857
53. Simpson, R., Briggs, S., Ovens, J., & Swales, J. M. (2002). The Michigan corpus of academic spoken English. Ann Arbor: Regents of the University of Michigan.
54. Simpson-Vlach, R., & Ellis, N. C. (2010). An academic formulas list: New methods in phraseology research. Applied Linguistics, 31, 487-512. doi:10.1093/applin/amp058
55. Svartvik, J., & Quirk, R. (1980). A corpus of English conversation. Lund, Sweden: Gleerup.
56. Thorndike, E. L., & Lorge, I. (1944). The teacher's word book of 30, 000 words. New York, NY: Teachers College, Columbia University.
57. Toglia, M. P., & Battig, W. R. (1978). Handbook of semantic word norms. New York, NY: Lawrence Erlbaum.
58. Witten, I. A., Frank, E., & Hall, M. A. (2011). Data mining. San Francisco, CA: Elsevier.
59. Xue, G., & Nation, I. S. P. (1984). A university word list. Language Learning and Communication, 3, 215-222.