Using LDA to detect semantically incoherent documents

CoNLL 2008 - Proceedings of the Twelfth Conference on Computational Natural Language Learning

Volumn , Issue , 2008, Pages 41-48

  Misra, Hemant ^a   Cappé, Olivier ^a   Yvon, François ^b  

^a CNRS   (France)

^b UNIVERSITÉ PARIS SACLAY   (France)

Author keywords

[No Author keywords available]

Indexed keywords

CONTENT DETECTION; DIRICHLET; DOCUMENT SETS; TEXT SEGMENTATION; TOPIC DETECTION; TOPIC DISTRIBUTIONS;

SEMANTICS;

EID: 84865096908     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.3115/1596324.1596332     Document Type: Conference Paper

References (21)

1. Blei, David and John Lafferty. 2005. Correlated topic models. In Advances in Neural Information Processing Systems (NIPS'18), Vancouver, Canada.
2. Blei, David M., Andrew Y. Ng, and Michael I. Jordan. 2002. Latent Dirichlet allocation. In Dietterich, Thomas G., Suzanna Becker, and Zoubin Ghahramani, editors, Advances in Neural Information Processing Systems (NIPS), volume 14, pages 601-608, Cambridge, MA. MIT Press.
3. Blei, David M., Thomas L. Griffiths, Michael I. Jordan, and Joshua B. Tenenbaum. 2004. Hierarchical topic models and the nested Chinese restaurant process. In Advances in Neural Information Processing Systems (NIPS), volume 16, Vancouver, Canada.
4. Buntine, Wray and Aleks Jakulin. 2004. Applying discrete PCA in data analysis. In Chickering, M. and J. Halpern, editors, Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence (UAI'04), pages 59-66. AUAI Press 2004.
5. Buntine, Wray, Jaakko Löfström, Jukka Perkiö, Sami Perttu, Vladimir Poroshin, Tomi Silander, Henry Tirri, Antti Tuominen, and Ville Tuulos. 2004. A scalable topic-based open source search engine. In Proceedings of the IEEE/WIC/ACM International Conference on Web Intelligence, pages 228-234, Beijing, China.
6. Foltz, P.W., W. Kintsch, and T.K. Landauer. 1998. The measurement of textual coherence with Latent Semantic Analysis. Discourse Processes, 25(2-3):285-307.
7. Globerson, Amir, Terry Y. Koo, Xavier Carreras, and Michael Collins. 2007. Exponentiated gradient algorithms for log-linear structured prediction. In International Conference on Machine Learning, Corvallis, Oregon.
8. Griffiths, Thomas L. and Mark Steyvers. 2004. Finding scientific topics. Proceedings of the National Academy of Sciences, 101 (supl 1):5228-5235.
9. Griffiths, Thomas L., Mark Steyvers, David M. Blei, and Joshua Tenenbaum. 2005. Integrating topics and syntax. In Proceedings of NIPS, 17, Vancouver, CA.
10. Gruber, Amit, Michal Rosen-Zvi, and Yair Weiss. 2007. Hidden topic Markov models. In Proceedings of International Conference on Artificial Intelligence and Statistics, San Juan, Puerto Rico, March.
11. Hearst, Marti. 1997. TextTiling: Segmenting texts into multi-paragraph subtopic passages. Computational Linguistics, 23(1):33-64.
12. Heidel, Aaron, Hung an Chang, and Lin shan Lee. 2007. Language model adaptation using latent Dirichlet allocation and an efficient topic inference algorithm. In Proceedings of European Conference on Speech Communication and Technology, Antwerp, Belgium.
13. Hofmann, Thomas. 2001. Unsupervised learning by probabilistic latent semantic analysis. Machine Learning Journal, 42(1):177-196.
14. Hsu, Bo-June (Paul) and Jim Glass. 2006. Style & topic language model adaptation using HMMLDA. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Sydney, Australia.
15. Kivinen, Jyrki and Manfrud K. Warmuth. 1997. Exponentiated gradient versus gradient descent for linear predictors. Information and Computation, 132:1-63.
16. Lewis, David D., Yiming Yang, Tony Rose, and Fan Li. 2004. RCV1: A new benchmark collection for text categorization research. Machine Learning Research, 5:361-397.
17. Minka, Thomas and John Lafferty. 2002. Expectationpropagation for the generative aspect model. In Proceedings of the conference on Uncertainty in Artificial Intelligence (UAI).
18. Nigam, K., A. K. McCallum, S. Thrun, and T. M. Mitchell. 2000. Text classification from labeled and unlabeled documents using EM. Machine Learning, 39(2/3):103-134.
19. Ohta, Tomoko, Yuka Tateisi, Hideki Mima, Jun ichi Tsujii, and Jin-Dong Kim. 2002. The GENIA corpus: an annotated research abstract corpus in molecular biology domain. In Proceedings of Human Language Technology Conference, pages 73-77.
20. Rigouste, Loïs, Olivier Cappé, and François Yvon. 2007. Inference and evaluation of the multinomial mixture model for text clustering. Information Processing and Management, 43(5):1260-1280, September.
21. Tam, Yik-Cheung and Tanja Schultz. 2007. Correlated latent semantic model for unsupervised LM adaptation. In Proceedings of the International Conference on Acoustics, Speech and Signal Processing, Honolulu, Hawaii, U.S.A.