Microblog language identification: Overcoming the limitations of short, unedited and idiomatic text

Language Resources and Evaluation

Volumn 47, Issue 1, 2013, Pages 195-215

  Carter, Simon ^a   Weerkamp, Wouter ^a   Tsagkias, Manos ^a  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

Author keywords

Content analysis; Language identification; Microblogs; Text classification

Indexed keywords

EID: 84874727608     PISSN: 1574020X     EISSN: 15728412     Source Type: Journal    
DOI: 10.1007/s10579-012-9195-y     Document Type: Article

References (22)

1. Altheide, D. L. (1996). Qualitative media analysis (Qualitative research methods). London: Sage Pubn Inc.
2. Baldwin, T., & Lui, M. (2010). Language identification: The long and the short of the matter. In Human language technologies: The 2010 annual conference of the North American chapter of the association for computational linguistics (NAACL HLT 2010) (pp. 229-237). Association for Computational Linguistics.
3. Bhargava, A., & Kondrak, G. (2010). Language identification of names with SVMs. In Human language technologies: The 2010 annual conference of the North American chapter of the association for computational linguistics (HLT-NAACL 2010) (pp. 693-696). Association for Computational Linguistics.
4. Carter, S., Tsagkias, M., & Weerkamp, W. (2011). Semi-supervised priors for microblog language identification. In Dutch-Belgian information retrieval workshop (DIR 2011).
5. Cavnar, W., & Trenkle, J. (1994). N-gram-based text categorization. In Proceedings of third annual symposium on document analysis and information retrieval (pp. 161-175).
6. Dietterich, T. G. (2000). Ensemble methods in machine learning. In Proceedings of the first international workshop on multiple classifier systems (pp. 1-15).
7. Golovchinsky, G., & Efron, M. (2010). Making sense of twitter search. In Proceedings of CHI 2010 workshop on microblogging: What and how can we learn from it?.
8. Gottron, T., & Lipka, N. (2010). A comparison of language identification approaches on short, query-style texts. In Advances in Information retrieval, 32nd European conference on IR research (ECIR 2010) (pp. 611-614).
9. Hong, L., Convertino, G., & Chi, E. (2011). Language matters in twitter: A large scale study. In Proceedings of AAAI conference on weblogs and social media (ICWSM 2011) (pp. 518-521).
10. Jansen, B. J., Zhang, M., Sobel, K., & Chowdury, A. (2009). Twitter power: Tweets as electronic word of mouth. Journal of the American Society for Information Science and Technology, 60(11), 2169-2188.
11. Koehn, P. (2005). Europarl: A parallel corpus for statistical machine translation. In: MT Summit.
12. Martins, B., & Silva, M. (2005). Language identification in web pages. In Proceedings of the 2005 ACM symposium on applied computing (pp. 764-768).
13. Massoudi, K., Tsagkias, M., de Rijke, M., & Weerkamp, W. (2011). Incorporating query expansion and quality indicators in searching microblog posts. In 33rd European conference on information retrieval (ECIR 2011) (pp. 362-367). New York: Springer.
14. Oghina, A., Breuss, M., Tsagkias, M., & de Rijke, M. (2012). Predicting IMDb movie ratings using social media. In 34th European conference on information retrieval (ECIR 2012). New York: Springer.
15. Poblete, B., Garcia, R., Mendoza, M., & Jaimes, A. (2011). Do all birds tweet the same? Characterizing twitter around the world. In Proceedings of the 20th ACM international conference on information and knowledge management (CIKM 2011) (pp. 1025-1030).
16. Sakaki, T., Okazaki, M., & Matsuo, Y. (2010). Earthquake shakes twitter users: Real-time event detection by social sensors. In Proceedings of the 19th international conference on world wide web (WWW 2010) (pp. 851-860).
17. Semiocast. (2010). Half of messages on twitter are not in English, Japanese is the second most used language. http://semiocast. com/downloads/Semiocast_Half_of_messages_on_Twitter_are_not_in_English_20100224. pdf.
18. Tumasjan, A., Sprenger, T., Sandner, P., & Welpe, I. (2010). Predicting elections with twitter: What 140 characters reveal about political sentiment. In International AAAI Conference on Weblogs and Social Media (ICWSM 2010), (pp. 178-185).
19. Vieweg, S., Hughes, A. L., Starbird, K., & Palen, L. (2010). Microblogging during two natural hazards events: What twitter may contribute to situational awareness. In Proceedings of the 28th international conference on human factors in computing systems (CHI 2010) (pp. 1079-1088).
20. Weerkamp, W., Tsagkias, M., & Carter, S. (2011). How people use twitter in different languages. In Proceedings of the 3rd international conference on web science (WebSci 2011).
21. Yang, Y., & Liu, X. (1999). A re-examination of text categorization methods. In Proceedings of the 22nd annual international ACM SIGIR conference on research and development in information retrieval (pp. 42-49).
22. Yu, D., Wang, S., Karam, Z., & Deng, L. (2010). Language recognition using deep-structured conditional random fields. In 2010 IEEE International conference on acoustics speech and signal processing (ICASSP 2010) (pp. 5030-5033).