Predicting EFL writing ability from levels of mental representation measured by Coh-Metrix: A structural equation modeling study

Assessing Writing

Volumn 24, Issue , 2015, Pages 35-58

  Aryadoust, Vahid ^a   Liu, Sha ^b  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b CHINA WEST NORMAL UNIVERSITY   (China)

Author keywords

Coh Metrix; Construction integration model; Situation model; Structural equation modeling; Surface structure; Textbase

Indexed keywords

EID: 84961290657     PISSN: 10752935     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asw.2015.03.001     Document Type: Article

References (52)

1. Alderson C.J. Assessing reading. The Cambridge language assessment series 2000, Cambridge University Press, Cambridge.
2. Aryadoust V. Evaluating the psychometric quality of an ESL placement test of writing: A Many-Facets Rasch study. Linguistics Journal 2012, 6:8-33.
3. Aryadoust V. Predictive modelling in Coh-Metrix-based writing research: A proposal for genetic-programing-based symbolic regression. Current trends in language testing in the Pacific Rim and the Middle East: Policies, analysis, and diagnoses 2015, Cambridge Scholars Publishing, Newcastle. V. Aryadoust, J. Fox (Eds.).
4. Aryadoust V. Understanding the growth of ESL paragraph writing skills and its relationships with linguistic features. Educational Psychology: An International Journal of Experimental Educational Psychology 2015, 10.1080/01443410.2014.950946.
5. Aryadoust V., Goh C.C.M. Predicting listening item difficulty with language complexity measures: A comparative data mining study. CaMLA Working Papers, 2014-01 2014, CaMLA, Ann Arbor, MI, Available from http://www.cambridgemichigan.org/resources/working-papers/2014-02.
6. Attali Y. Validity and reliability of automated essay scoring. Handbook of automated essay evaluation: Current applications and new directions 2013, 181-199. Routledge, New York, NY. M.D. Shermis, J. Burstein (Eds.).
7. Auerbach C., La Porte H.H., Caputo R.K. Statistical methods for estimates of interrater reliability. Evidence based practice manual: Research and outcome measures in health and human services 2004, 444-448. Oxford University Press, New York, NY. A.R. Roberts, K.R. Yeager (Eds.).
8. Bagozzi R., Yi Y. Specification, evaluation and interpretation of structural equation models. Journal of the Academy of Marketing Science 2012, 40:8-34.
9. Charniak E. A maximum-entropy-inspired parser. Proceedings of the NAACL 2000 2000, 132-139.
10. Coltheart M. The MRC psycholinguistic database. Quarterly Journal of Experimental Psychology 1981, 33A:497-505.
11. Connor U. Linguistic/rhetorical measures for international persuasive student writing. Research in the Teaching of English 1990, 24:67-87.
12. Crossley S.A., McNamara D.S. Cohesion coherence, and expert evaluations of writing proficiency. Proceedings of the 32nd annual conference of the cognitive science society 2010, 984-989. Cognitive Science Society, Austin, TX. R. Catrambone, S. Ohlsson (Eds.).
13. Crossley S.A., Greenfield J., McNamara D.S. Assessing text readability using cognitively based indices. TESOL Quarterly 2008, 42:475-493.
14. Crossley S.A., McNamara D.S., Weston J., McLain Sullivan S.T. The development of writing proficiency as a function of grade level: A linguistic analysis technical report 2010, The University of Memphis, Memphis, TN, 〈http://w-pal.memphis.edu/main/publications.html.〉.
15. Crossley S.A., Salsbury T., McNamara D.S. The development of polysemy and frequency use in English second language speakers. Language Learning 2010, 60:573-605.
16. Crossley S.A., Salsbury T., McNamara D.S., Jarvis S. What is lexical proficiency? Some answers from computational models of speech data. TESOL Quarterly 2011, 45:182-193.
17. Dancey C., Reidy J. Statistics without maths for psychology: Using SPSS for Windows 2004, Prentice Hall, London.
18. Dumais S.T. Latent semantic analysis. Annual Review of Information Science and Technology 2005, 38:188.
19. Eckes T. On common ground? How raters perceive scoring criteria in oral proficiency testing. Tasks and criteria in performance assessment 2009, 43-73. Peter Lang, Frankfurt, Germany, Proceedings of the 28th language testing research colloquium. A. Brown, K. Hill (Eds.).
20. Eckes T. Introduction to many-facet Rasch measurement: Analyzing and evaluating rater-mediated assessments 2011, Peter Lang, Frankfurt.
21. WordNet: An electronic lexical database 1998, MIT Press, Cambridge, MA. C. Fellbaum (Ed.).
22. Fulcher G. Practical language testing 2010, Hodder Education/Routledge, London.
23. Garson G.D. Testing statistical assumption (blue book series) 2012, 17-18. Statistical Associates Publishing, 〈www.statisticalassociates.com/assumptions.pdf.〉.
24. Graesser A.C., McNamara D.S., Louwerse M.M., Cai Z. Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments and Computers 2004, 36:193-202.
25. Hair J.F., Black W.C., Babin B.J., Anderson R.E. Multivariate data analysis 2010, Prentice Hall, Upper Saddle River, NJ. (7th ed.).
26. Jarvis S., Grant L., Bikowski D., Ferris D. Exploring multiple profiles of highly rated learner compositions. Journal of Second Language Writing 2003, 12:377-403.
27. Kenny D.A., McCoach D.B. Effect of the number of variables on measures of fit in structural equation modeling. Structural Equation Modeling 2003, 10:333-3511.
28. Kintsch W. Comprehension: A paradigm for cognition 1998, Cambridge University Press, New York.
29. Kintsch W. The construction-integration model of text comprehension and its implications for instruction. Theoretical models and processes of reading 2004, 1270-1328. International Reading Association, Newark, DE. (5th ed.). R. Ruddell, N. Unrau (Eds.).
30. Kintsch W., Mangalath P. The construction of meaning. Topics in Cognitive Science 2011, 3:346-370.
31. Kline R.B. Principles and practice of structural equation modeling 2011, The Guilford Press, New York. (3rd ed.).
32. Linacre J.M. Sample size and item calibrations stability. Rasch Measurement Transactions 1994, 7:328.
33. Linacre J.M. Facets Rasch measurement 2014, Winsteps.com, Chicago, IL, (Computer program).
34. Linacre J.M. A user's guide to Facets Rasch-model computer programs 2014, Winsteps.com, Chicago, IL.
35. Mardia K.V. Measures of multivariate skewness and kurtosis with applications. Biometrika 1970, 57:519-530.
36. McCarthy P.M., Guess R.H., McNamara D.S. The components of paraphrase evaluations. Behavioral Research Methods 2009, 41:682-690.
37. McNamara D.S., Kintsch W. Learning from text: Effects of prior knowledge and text coherence. Discourse Processes 1996, 22:247-288.
38. McNamara D.S., Crossley S.A., McCarthy P.M. Linguistic features of writing quality. Written Communication 2010, 27:57-86.
39. McNamara D.S., Graesser A.C., McCarthy P.M., Cai Z. Automated evaluation of text and discourse with Coh-Metrix 2014, Cambridge University Press, Cambridge.
40. Messick S. Validity. Educational measurement 1989, 13-103. Macmillan, New York. R.L. Linn (Ed.).
41. Millis K.K., Graesser A.C., Haberlandt K.F. The impact of connectives on the memory for expository texts. Applied Cognitive Psychology 1993, 7:317-339.
42. Nation I.S.P. Learning vocabulary in another language 2001, Cambridge University Press, Cambridge.
43. Page E.B., Petersen N.S. The computer moves into essay grading: Updating the ancient test. Phi Delta Kappan 1995, 76:561-565.
44. Pennebaker J.W. The secret life of pronouns: What our words say about us 2011, Bloomsbury Press, London, UK.
45. Powers D.E. Scoring the TOEFL independent essay automatically: Reactions of test takers and test score users. ETS Research Manuscript 2011, (RM-11-34).
46. Powers D.E., Burstein J., Chodorow M. Stumping e-rater: Challenging the validity of automated essay scoring. Computers in Human Behavior 2002, 18(2):103-134.
47. Reid J. A computer text analysis of four cohesion devices in English discourse by native and non-native writers. Journal of Second Language Writing 1992, 1:79-107.
48. Schumacker R.E., Lomax R.G. A beginner's guide to structural equation modeling 2010, Routledge, New York.
49. Sekine S., Grishman R. A corpus-based probabilistic grammar with only two nonterminals. Proceedings of the international workshop on parsing technologies 1995, 216-223.
50. Wolfe E.W. Uncovering rater's cognitive processing and focus using think-aloud protocols. Journal of Writing Assessment 2005, 2(1):37-56.
51. Wolf E.J., Harrington K.M., Clark S.L., Miller M.W. Sample size requirements for structural equation models an evaluation of power, bias, and solution propriety. Educational and Psychological Measurement 2013, 73(6):913-934.
52. Zwaan R.A., Radvansky G.A. Situation models in language comprehension and memory. Psychological Bulletin 1998, 123:162-1855.