Word normalization in Twitter using finite-state transducers

CEUR Workshop Proceedings

Volumn 1086, Issue , 2013, Pages 49-53

  Porta, Jordi ^a   Sancho, José Luis ^a  

^a Centro de Estudios de la Real Academia Espanõla   (Spain)

Author keywords

Finite state transducers; Lexical normalisation; Statistical language models; Tweet messages

Indexed keywords

LINGUISTICS; NATURAL LANGUAGE PROCESSING SYSTEMS; OPTICAL CHARACTER RECOGNITION; SOCIAL NETWORKING (ONLINE); TRANSDUCERS;

FINITE STATE TRANSDUCERS; LINGUISTIC APPROACH; LINGUISTIC MODELS; NORMALISATION; STATISTICAL LANGUAGE MODELING; STATISTICAL LANGUAGE MODELS; TWEET MESSAGES; WEIGHTED FINITE-STATE TRANSDUCERS;

COMPUTATIONAL LINGUISTICS;

EID: 84921802555     PISSN: 16130073     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (16)

1. Allauzen, Cyril and Mehryar Mohri. 2008. 3-way composition of weighted finite-state transducers. In Proc. of the 13th Int. Conf. on Implementation and Application of Automata (CIAA-2008), pages 262-273, San Francisco, California, USA.
2. Atserias, Jordi, Bernardino Casas, Elisabet Comelles, Meritxell González, Lluís Padró, and Muntsa Padró. 2006. Free-Ling 1.3: Syntactic and semantic services in an open-source NLP library. In Proc. of the 5th Int. Conf. on Language Resources and Evaluation (LREC-2006), pages 48-55, Genoa, Italy, May.
3. Baroni, Marco, Silvia Bernardini, Adriano Ferraresi, and Eros Zanchetta. 2009. The wacky wide web: A collection of very large linguistically processed web-crawled corpora. Language Resources and Evaluation, 43(3): 209-226.
4. BNC. 2001. The British National Corpus, version 2 (BNC World). Distributed by Oxford University Computing Services on behalf of the BNC Consortium. http://www.natcorp.ox.ac.uk.
5. Chomsky, Noam and Morris Halle. 1968. The sound pattern of English. Harper & Row, New York.
6. Crystal, David. 2008. Txtng: The Gr8 Db8. Oxford University Press.
7. Gomez Hidalgo, José María, Andrés Alfonso Caurcel Díaz, and Yovan Iñiguez del Rio. 2013. Un método de análisis de lenguaje tipo SMS para el castellano. Linguamática, 5(1): 31-39, July.
8. Levenshtein, Vladimir I. 1966. Binary codes capable of correcting deletions, insertions, and reversals. Soviet Physics Doklady, 10(8): 707-710.
9. Lita, Lucian Vlad, Abe Ittycheriah, Salim Roukos, and Nanda Kambhatla. 2003. tRuEcasIng. In Proc. of the 41st Annual Meeting on ACL - Volume 1, ACL'03, pages 152-159, Stroudsburg, PA, USA.
10. Mosquera, Alejandro and Paloma Moreda. 2012. TENOR: A lexical normalisation tool for Spanish Web 2.0 texts. In Petr Sojka, Aleš Horák, Ivan Kopeček, and Karel Pala, editors, Text, Speech and Dialogue, volume 7499 of LNCS. Springer, pages 535-542.
11. Oliva, J., J. I. Serrano, M. D. Del Castillo, and Á. Iglesias. 2013. A SMS normalization system integrating multiple grammatical resources. Natural Language Engineering, 19: 121-141, 1.
12. Pinto, David, Darnes Vilariño Ayala, Yuridiana Alemán, Helena Gómez, Nahun Loya, and Héctor Jiménez-Salazar. 2012. The Soundex phonetic algorithm revisited for SMS text representation. In Petr Sojka, Aleš Horák, Ivan Kopeček, and Karel Pala, editors, Text, Speech and Dialogue, volume 7499 of LNCS, pages 47-55. Springer.
13. Porta, Jordi, José-Luis Sancho, and Javier Gómez. 2013. Edit transducers for spelling variation in Old Spanish. In Proc. of the workshop on computational historical linguistics at NODALIDA 2013. NEALT Proc. Series 18, pages 70-79, Oslo, Norway, May 22-24.
14. RAE. 2001. Diccionario de la lengua española. Espasa, Madrid, 22th edition.
15. Roark, Brian, Richard Sproat, Cyril Allauzen, Michael Riley, Jeffrey Sorensen, and Terry Tai. 2012. The OpenGrm opensource finite-state grammar software libraries. In Proc. of the ACL 2012 System Demonstrations, pages 61-66, Jeju Island, Korea, July.
16. Tai, Terry, Wojciech Skut, and Richard Sproat. 2011. Thrax: An Open Source Grammar Compiler Built on OpenFst. In ASRU 2011, Waikoloa Resort, Hawaii, December.