All mixed up?: Finding the optimal feature set for general readability prediction and its application to English and Dutch

Computational Linguistics

Volumn 42, Issue 3, 2016, Pages 457-490

  De Clercq, Orphée ^a   Hoste, Véronique ^a  

^a Faculty of Arts and Philosophy   (Belgium)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION (OF INFORMATION); GENETIC ALGORITHMS; PIPELINES; SEMANTICS; SUPERVISED LEARNING;

FEATURE COMBINATION; GENETIC-ALGORITHM OPTIMIZATIONS; LINGUISTIC PROCESSING; OPTIMAL FEATURE SETS; PREDICTION SYSTEMS; PREDICTION TASKS; SEMANTIC INFORMATION; SUPERVISED MACHINE LEARNING;

FORECASTING;

EID: 84988654497     PISSN: 08912017     EISSN: 15309312     Source Type: Journal    
DOI: 10.1162/COLI_a_00255     Document Type: Article

References (75)

1. Aha, David W. and Richard L. Bankert. 1996. A comparative evaluation of sequential feature selection algorithms. In D. Fischer and J.-H. Lenz, editors, Artificial Intelligence and Statistics V. New York: Springer Verlag, pages 199-206.
2. Alderson, J. Charles. 1984. Reading in a Foreign Language: A Reading Problem or a Language Problem. Longman.
3. Aston, Guy and Lou Burnard. 1998. The BNC Handbook: Exploring the British National Corpus with SARA. Edinburgh University Press.
4. Bailin, Alan and Ann Grafstein. 2001. The linguistic assumptions underlying readability formulae: A critique. Language & Communication, 21(3):285-301.
5. Barzilay, Regina and Mirella Lapata. 2005. Modeling local coherence: An entity-based approach. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL-2005), pages 141-148, Ann Arbor, MI.
6. Barzilay, Regina and Mirella Lapata. 2008. Modeling local coherence: An entity-based approach. Computational Linguistics, 34(1):1-34.
7. Benjamin, Rebekah George. 2012. Reconstructing readability: Recent developments and recommendations in the analysis of text difficulty. Educational Psychology Review, 24(1):63-88.
8. Björkelund, Anders, Love Hafdell, and Pierre Nugues. 2009. Multilingual semantic role labeling. In Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL 2009): Shared Task, pages 43-48, Boulder, CO.
9. Caruana, Rich and Dayne Freitag. 1994. Greedy Attribute Selection. In Proceedings of the International Conference on Machine Learning (ICML-1994), pages 28-36, New Brunswick, NJ.
10. Chall, Jeanne and Edgar Dale. 1995. Readability Revisited: The New Dale-Chall Readability Formula. Brookline Books.
11. Collins-Thompson, Kevin. 2014. Computational assessment of text readability: A survey of current and future research. Special Issue of the International Journal of Applied Linguistics, 165(2):97-135.
12. Collins-Thompson, Kevin and Jamie Callan. 2004. A language modeling approach to predicting reading difficulty. In Proceedings of the Human Language Technology Conference and the North American Chapter of the Association for Computational Linguistics Annual Meeting (HLT - NAACL-2004), pages 193-200, Boston, MA.
13. Collins-Thompson, Kevyn and Jamie Callan. 2005. Predicting reading difficulty with statistical language models. Journal of the American Society for Information Science and Technology, 56:1448-1462.
14. Cristianini, Nello and John Shawe-Taylor. 2000. An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press.
15. Crossley, Scott A., Jerry Greenfield, and Danielle S. McNamara. 2008. Assessing text readability using cognitively based indices. TESOL Quarterly, 43(3):475-493.
16. Davison, Alice and Robert N. Kantor. 1982. On the failure of readability formulas to define readable texts: A case study from adaptations. Reading Research Quarterly, 17(2):187-209.
17. Orphée De Clercq, Bart Desmet, Sarah Schulz, Els Lefever, and Veronique Hoste. 2013. Normalization of Dutch user-generated content. In Proceedings of Recent Advances in Natural Language Processing, pages 179-188.
18. Orphée De Clercq, Iris Hendrickx, and Véronique Hoste. 2011. Cross-domain Dutch coreference resolution. In Proceedings of the 8th International Conference on Recent Advances in Natural Language Processing (RANLP-2011), pages 186-193, Hissar.
19. Orphée De Clercq, Paola Monachesi, and Véronique Hoste. 2012. Evaluating automatic cross-domain semantic role annotation. In Proceedings of the 8th International Conference on Language Resources and Evaluation (LREC-2012), pages 88-93, Istanbul.
20. Orphée De Clercq, Véronique Hoste, Bart Desmet, Philip van Oosten, Martine De Cock, and Lieve Macken. 2014. Using the crowd for readability prediction. Natural Language Engineering, Cambridge Journals Online, 20(3):293-325.
21. Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC-2006), pages 449-454, Genoa.
22. Desmet, Bart. 2014. Finding the Online Cry for Help: Automatic Text Classification for Suicide Prevention. Ph.D. thesis, Ghent University.
23. Desmet, Bart and Véronique Hoste. 2013. Fine-grained Dutch named entity recognition. Language Resources and Evaluation, 48(2):307-343.
24. DuBay, William H. 2004. The Principles of Readability. Impact Information.
25. DuBay, William H., editor. 2007. Unlocking Language: the Classic Readability Studies. BookSurge.
26. Falkenjack, Johan and Arne Jonsson. 2014. Classifying easy-to-read texts without parsing. In Proceedings of the 3rd Workshop on Predicting and Improving Text Readability for Target Reader Populations (PITR), pages 114-122, Gothenburg.
27. Lijun Feng, Noémie Elhadad, and Matt Huenerfauth. 2009. Cognitively motivated features for readability assessment. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics (EACL-2009), pages 229-237, Athens.
28. Lijun Feng, Martin Jansche, Matt Huenerfauth, and Noémie Elhadad. 2010. A comparison of features for automatic readability assessment. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING-2010), pages 276-284, Beijing.
29. Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. Incorporating non-local information into information extraction systems by Gibbs sampling. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL-2005), pages 363-370, Ann Arbor, MI.
30. Flesch, Rudolph. 1948. A new readability yardstick. Journal of Applied Psychology, 32(3):221-233.
31. François, Thomas. 2009. Combining a statistical language model with logistic regression to predict the lexical and syntactic difficulty of texts for FFL. In Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics: Student Research Workshop (EACL-2009), pages 19-27, Athens.
32. François, Thomas. 2011. Les apports du traitement automatique du language à la lisibilité du français langue étrangère. Ph.D. thesis, Université catholique de Louvain.
33. François, Thomas and Eleni Miltsakaki. 2012. Do NLP and machine learning improve traditional readability formulas? In Proceedings of the 1st Workshop on Predicting and Improving Text Readability for Target Reader Populations (PITR2012), pages 49-57, Montreal.
34. Galley, Michel and Kathleen Mckeown. 2003. Improving word sense disambiguation in lexical chaining. In Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI-2003), pages 1486-1488, Acapulco.
35. Goldberg, David. 1989. Genetic Algorithms in Search, Optimization and Machine Learning. Addison Wesley.
36. Goldberg, David and Kalyanmoy Deb. 1991. A comparative analysis of selection schemes used in genetic algorithms. In W. N. Martin and W. M. Spears, editors, Foundations of Genetic Algorithms. Morgan Kaufmann Publishers, pages 69-93.
37. Arthur C. Graesser, Danielle S. McNamara, Max M. Louwerse, and Zhiqiang Cai. 2004. Coh-Metrix: Analysis of text on cohesion and language. Behavior Research Methods, Instruments and Computers, 36:193-202.
38. Gunning, Robert. 1952. The Technique of Clear Writing. McGraw-Hill, New York.
39. Halliday, Michael and Ruqaiya Hasan. 1976. Cohesion in English. Longman Group Ltd.
40. Heilman, Michael, Kevyn Collins-Thompson, and Maxine Eskenazi. 2008. An analysis of statistical models and features for reading difficulty prediction. In Proceedings of the 3rd ACL Workshop on Innovative Use of NLP for Building Educational Applications (EANL-2008), pages 71-79, Columbus, OH.
41. Michael J. Heilman, Kevyi Collins-Thompson, Jamie Callan, and Maxine Eskenazi. 2007. Combining lexical and grammatical features to improve readability measures for first and second language texts. In Proceedings of the Human Language Technology Conference and the North American Chapter of the Association for Computational Linguistics Annual Meeting (HLT-NAACL 2007), pages 460-467, Rochester, NY.
42. Hoste, Véronique. 2005. Optimization Issues in Machine Learning of Coreference Resolution. Ph.D. thesis, Antwerp University.
43. George H. John, Ron Kohavi, and Karl Pfleger. 1994. Irrelevant Features and the Subset Selection Problem. In International conference on Machine Learning, pages 121-129, New Brunswick, NJ.
44. Rohit J. Kate, Xiaoqiang Luo, Siddharth Patwardhan, Martin Franz, Radu Florian, Raymond J. Mooney, Salim Roukos, and Chris Welty. 2010. Learning to predict readability using diverse linguistic features. In Proceedings of the 23rd International Conference on Computational Linguistics (COLING-2010), pages 546-554, Beijing.
45. Kincaid, J. Peter, Robert P. Fishburne, Richard L. Rogers, and Brad S. Chissom. 1975. Derivation of new readability formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy enlisted personnel. Research Branch Report RBR-8-75, Naval Technical Training Command Millington Tenn Research Branch, Springfield, Virginia.
46. George R. Klare 1976. A second look at the validity of readability formulas. Journal of Literacy Research, 8:129-152.
47. Kraf, Rogier and Henk Pander Maat. 2009. Leesbaarheidsonderzoek: Oude problemen, nieuwe kansen. Tijdschrift voor Taalbeheersing, 31(2):97-123.
48. Lee, Heeyoung, Angel Chang, Yves Peirsman, Nathanael Chambers, Mihai Surdeanu, and Dan Jurafsky. 2013. Deterministic coreference resolution based on entity-centric, precision-ranked rules. Computational Linguistics, 39(4):885-916.
49. Leroy, Gondy and James E. Endicott. 2011. Term familiarity to indicate perceived and actual difficulty of text in medical digital libraries. In International Conference on Asia-Pacific Digital Libraries (ICADL-2011), pages 307-310, Beijing.
50. Leroy, Gondy, Steven Helmreich, James R. Cowie, Trudi Miller, and Wei Zheng. 2008. Evaluating online health information: Beyond readability formulas. In Proceedings of the American Medical Informatics Association Annual Symposium (AMIA-2008), pages 394-398.
51. Lieve Macken, Orphée De Clercq, and Hans Paulussen. 2011. Dutch Parallel Corpus: A balanced copyright-cleared parallel corpus. Meta, 56(2).
52. Danielle S. McNamara, Max M. Louwerse, Philip M. McCarthy, and Arthur C. Graesser. 2010. Coh-Metrix: Capturing linguistic features of cohesion. Discourse Processes, 47(4):292-330.
53. Mitchell, Melanie. 1996. An Introduction to Genetic Algorithms. MIT Press.
54. Ani Nenkova, Jieun Chae, Annie Louis, and Emily Pitler. 2010. Structural features for predicting the linguistic quality of text: Applications to machine translation, automatic summarization and human-authored text. Empirical Methods in NLG, Lecture Notes in Artificial Intelligence, 5790:222-241.
55. OECD. 2013. OECD Skills Outlook 2013. Technical report, OECD Publishing.
56. Nelleke Oostdijk, Martin Reynaert, Véronique Hoste, and Ineke Schuurman. 2013. The construction of a 500-million-word reference corpus of contemporary written Dutch. In P. Spyns and J. Odijk, editors, Essential Speech and Language Technology for Dutch, Theory and Applications of Natural Language Processing. Springer, pages 219-247.
57. Martha Palmer, Daniel Gildea, and Paul Kingsbury. 2005. The proposition bank: A corpus annotated with semantic roles. Computational Linguistics, 31(1):71-106.
58. Sarah E. Petersen and Mari Ostendorf. 2009. A machine learning approach to reading level assessment. Computer Speech & Language, 23(1):89-106.
59. Pitler, Emily and Ani Nenkova. 2008. Revisiting readability: A unified framework for predicting text quality. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing (EMNLP-2008), pages 186-195, Honolulu, HI.
60. Rashmi Prasad, Nikhil Dinesh, Alan Lee, Eleni Miltsakaki, Livio Robaldo, Aravind Joshi, and Bonnie Webber. 2008. The Penn discourse treebank 2.0. In Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC-2008), pages 2961-2968, Marrakech.
61. Rayson, Paul and Roger Garside. 2000. Comparing corpora using frequency profiling. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics Workshop on Comparing Corpora, (ACL-2000), pages 1-6, Hong Kong.
62. Marta Sabou, Kalina Bontcheva, and Arno Scharl. 2012. Crowdsourcing research opportunities: Lessons from natural language processing. In Proceedings of the 12th International Conference on Knowledge Management and Knowledge Technologies (i-Know-2012), pages 17:1-17:8, Graz.
63. Salton, Gerard. 1989. Automatic Text Processing: The Transformation, Analysis and Retrieval of Information by Computer. Addison-Wesley Longman Publishing Co.
64. Schwarm, Sarah E. and Mari Ostendorf. 2005. Reading level assessment using support vector machines and statistical language models. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL-2005), pages 523-530, Ann Arbor, MI.
65. Si, Luo and Jamie Callan. 2001. A statistical model for scientific readability. In Proceedings of the 10th International Conference on Information Knowledge Management (ICKM-2001), pages 574-576, Atlanta, GA.
66. Staphorsius, Gerrit. 1994. Leesbaarheid en leesvaardigheid. De ontwikkeling van een domeingericht meetinstrument. Cito.
67. Stolcke, Andreas. 2002. SRILM—an extensible language modeling toolkit. In Proceedings of the 7th International Conference on Spoken Language Processing (ICSLP-2002), pages 901-904, Denver, CO.
68. Kumiko Tanaka-Ishii, Satoshi Tezuka, and Hiroshi Terada. 2010. Sorting texts by readability. Computational Linguistics, 36(2):203-227.
69. Amalia Todirascu, Thomas François, Nuria Gala, Cédric Fairon, Anne-Laure Ligozat, and Delphine Bernhard. 2013. Coherence and cohesion for the assessment of text readability. In Proceedings of 10th International Workshop on Natural Language Processing and Cognitive Science (NLPCS-2013), pages 11-19, Marseille.
70. Willem H. van Boom 2014. Begrijpelijke hypotheekvoorwaarden en consumentengedrag. In T.M. Berkhout en A.A. van Velten, editors, Perspectieven voor vastgoedfinanciering (Congresbundel Stichting Fundatie Bachiene). Stichting Fundatie Bachiene, pages 45-80.
71. Marjan Van de Kauter, Geert Coorman, Els Lefever, Bart Desmet, Lieve Macken, and Véronique Hoste. 2013. LeTs Preprocess: The multilingual LT3 linguistic preprocessing toolkit. Computational Linguistics in the Netherlands Journal, 3:103-120.
72. Antal van den Bosch, Bertjan Busser, Walter Daelemans, and Sander Canisius. 2007. An efficient memory-based morphosyntactic tagger and parser for Dutch. In Proceedings of the Seventeenth Computational Linguistics in the Netherlands (CLIN), pages 191-206, Nijmegen.
73. Gertjan J. M. van Noord, Gosse Bouma, Frank van Eynde, Daniel de Kok, Jelmer van der Linde, Ineke Schuurman, Erik Tjong Kim Sang, and Vincent Vandeghinste. 2013. Large scale syntactic annotation of written Dutch: LASSY. In P. Spyns and J. Odijk, editors, Essential Speech and Language Technology for Dutch, Theory and Applications of Natural Language Processing. Springer, pages 231-254.
74. Philip van Oosten, Dries Tanghe, and Véronique Hoste. 2010. Towards an improved methodology for automated readability prediction. In Proceedings of the 7th International Conference on Language Resources and Evaluation (LREC-2010), pages 775-782, Valletta.
75. Wolf, Allison. 2005. Basic skills in the workplace: Opening doors to learning. Technical report 3053, Chartered Institute of Personnel and Development.