Semantic passage segmentation based on sentence topics for question answering

Information Sciences

Volumn 177, Issue 18, 2007, Pages 3696-3717

  Oh, Hyo Jung ^a,b   Myaeng, Sung Hyon ^b   Jang, Myung Gil ^a  

^a Electronics and Telecommunications Research Institute (ETRI)   (South Korea)

^b Korea Advanced Institute of Science and Technology (KAIST)   (South Korea)

Author keywords

Passage retrieval; Passage segmentation; Question answering; Semantic passages; Sentence classification; Topic assignment

Indexed keywords

INFORMATION RETRIEVAL; QUERY PROCESSING; STATISTICAL METHODS;

PASSAGE SEGMENTATION; QUESTION ANSWERING; SEMANTIC PASSAGES; SENTENCE CLASSIFICATION; TOPIC ASSIGNMENT;

SEMANTICS;

EID: 34250709079     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2007.02.038     Document Type: Article

References (41)

1. Berger A.L., Pietar S.A., and Pietra V.J. Maximum entropy approach to natural language processing. Computational Linguistics 22 1 (1996) 39-71
2. T. Brants, F. Chen, I. Tsochantaridis, Topic-based document segmentation with probabilistic latent semantic analysis, in: Proceedings of the 11th International Conference on Information and Knowledge management (CIKM-02), 2002, pp. 211-218.
3. M. Clillet, J. Pessiot, M. Amini, P. Gallinari, Unsupervised learning with term clustering for thematic segmentation of texts, in: Proceedings of the 7th Recherche d'Information Assistée par Ordinateur (RIAO-04), 2004, pp. 1-11.
4. J.P. Callan, Passage-retrieval evidence in document retrieval, in: Proceedings of 17th annual international ACM-SIGIR, 1994, pp. 302-310.
5. Chakik F.E., Shahine A., Jaam J., and Hasnah A. An approach for constructing complex discriminating surfaces based on Bayesian interference of the maximum entropy. Information Sciences 163 4 (2004) 275-291
6. G. Chao, M.G. Dyer, Maximum entropy models for word sense disambiguation, in: Proceedings of 19th International Conference on Computational Linguistics (COLING-02), 2002, pp. 1-7.
7. H. Christensen, B. Kolluru, Y. Gotoh, S. Renals, Maximum entropy segmentation on broadcast news, in: Proceedings of the IEEE International Conference on Acoustic, Speech, and Signal Processing (ICASSP-05), 2005, pp. 1029-1032.
8. M.R. Choi, J. Hur, M.G. Jang, Constructing Korean lexical concept network for encyclopedia question-answering system, in: Proceedings of IECON 2004 - 30th Annual Conference of IEEE Industrial Electronics Society, 2004, pp. 3115-3119.
9. C.L.A. Clarke, G.V. Cormack, T.R. Lynam, C.M. Li, G.L. McLearn, Web reinforced question answering (MultiText experiments for TREC 2001), in: Proceedings of Text Retrieval Conference (TREC), 2001, pp. 673-679.
10. Darroch J., and Ratcliff D. Generalized iterative scaling for log-linear models. The Annals of Mathematical Statistics 43 5 (1972) 1470-1480
11. Della Pietra S., Della Pietra V., and Lafferty J. Inducing features of random fields. IEEE Transaction on Pattern Analysis and Machine Intelligence 19 4 (1997) 380-393
12. Fellbaum C. WordNet, an electronic lexical database (1998), The MIT Press
13. S.M. Harabagiu, S.J. Maiorano, Finding answers in large collections of texts: paragraph indexing + abductive inference, in: Proceeding of AAAI, 1999, pp. 63-71.
14. M. Hearst, Multi-paragraph segmentation of expository text, in: Proceedings of the 32nd Annual meeting of the Association of Computational Linguistics (ACL-94), 1994, pp. 9-16.
15. P. Hsueh, J. Moore, S. Penals, Automatic segmentation of multiparty dialogue, in: Proceeding of the 11th Conference of the European Chapter of the Association for Computational Linguistics (EACL-06), 2006, pp. 273-280.
16. X. Ji, H. Zha, Domain-independent text segmentation using anisotropic diffusion and dynamic programming, in: Proceedings of 26th annual international ACM SIGIR, 2003, pp. 322-329.
17. Joachims T. Learning to classify text using support vector machines (2002), Kluwer
18. B.Y. Kang, S.H. Myaeng, Theme assignment for sentences based on head-driven patterns, in: Proceedings of 8th Conference on Text, Speech and Dialogue (TSD), 2005, pp. 187-194.
19. Kaszkiel M., and Zobel J. Effective ranking with arbitrary passage. Journal of the American Society of Information Science 52 4 (2001) 344-364
20. H.-J. Kim, H.-J. Oh, C.-H. Lee, M.-G. Jang, The 3-step Answer Processing Method for Encyclopedia Question-Answering System: AnyQuestion 1.0, in: Proceedings of Asia Information Retrieval Symposium (AIRS-04), LNCS, vol. 3689, 2004, pp. 309-312.
21. Z. Le, Y. Tain-shun, Filtering junk mail with a maximum entropy model, in: Proceedings of 20th International Conference on Computer Processing of Oriental Languages (ICCPOL03), 2003, pp. 446-453.
22. C.K. Lee, J.H. Wang, H.J. Kim, M.G. Jang, Extracting template for knowledge-based question-answering using conditional random fields, in: Proceedings of 28th annual international ACM SIGIR Workshop on MF/IR, 2005.
23. Lee C.K., Lee G.G., and Jang M.G. Dependency structure applied to language modeling for information retrieval. ETRI Journal 28 3 (2006) 337-346
24. C.K. Lee, H.J. Oh, Y.G. Hwang, S. J. Lim, M.G. Jang, et al., Fine-grained named entity recognition using conditional random fields for question answering, in: Proceedings of Asia Information Retrieval Symposium (AIRS-06), LNCS, vol. 4182, 2006, pp. 581-587.
25. Y.S. Lee, Y.S. Hwang, H.C. Rim, Variable length passage retrieval for Q&A system, in: Proceedings of the 14th Hangul and Korean Information Processing, 2002, pp. 259-266.
26. David D. Lewis, Representation and Learning in Information Retrieval, Ph. D. thesis, Department of Computer Science, University of Massachusetts, 1992.
27. L. Li, K. Yamanishi, Topic analysis using a finite mixture model, in: Proceedings of Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, 2000, pp. 35-44.
28. K. Nigam, J. Lafferty, A. McCallum, Using maximum entropy for text classification, in: Proceedings of the IJCAI-99 Workshop on Machine Learning for Information Filtering, 1999, pp. 61-67.
29. R. Malouf, A comparison of algorithms for maximum entropy parameter estimation, in: Proceedings of the 6th Conference on Natural Language Learning, 2002, pp. 49-55.
30. E. Matusov, J. Paters, C. Meyer, H. Ney, Topic segmentation using Markov models on section level, in: Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU-03), 2003, pp. 471-476.
31. E. Mittendorf, P. Schäuble, Document and passage retrieval based on hidden Markov models, in: Proceedings of 17th annual international ACM-SIGIR, 1994, pp. 318-327.
32. Moldovan D., Pasca M., Harabagiu S., and Surdeanu M. Performance issue and error analysis in an open-domain question answering system. ACM Transactions on Information Systems (TOIS) 21 2 (2003) 113-154
33. J.M. Ponte, B. Croft, Text segmentation by topic, in: Proceedings of the 1st European Conference on Research and Advanced Technology for Digital Libraries, 1997, pp. 113-125.
34. J. Reynar, Statistical models for topic segmentation, in: Proceeding of the 37th Annual Meeting of the Association for Computational Linguistics (ACL-99), 1999, pp. 1046-1053.
35. S.E. Roberson, S. Walker, Some simple effective approximations to the 2-Poisson model for probabilistic weighted retrieval, in: Proceedings of 17th Annual International ACM-SIGIR, 1994, pp. 345-354.
36. G. Salton, J. Allan, C. Buckely, Approaches to passage retrieval in full text information systems, in: Proceedings of 16th Annual International ACM-SIGIR, 1993, pp. 49-58.
37. Salton G., Singhal A.K., Mitra M., and Buckely C. Automatic text structuring and summarization. Information Processing and Management 33 2 (1997) 193-207
38. Singhal A.K., Salton G., Mitra M., and Buckely C. Document length normalization. Information Processing and Management 32 5 (1996) 619-633
39. Stokes N., Carthy J., and Smeaton A.F. SeLeCT: A lexical cohesion based news story segmentation system. Journal of AI Communications 17 1 (2004) 3-12
40. E.M. Voorhees, Overview of TREC 2003 Question Answering Track, in: Proceedings of Text REtreival Conference (TREC-12), 2003, pp. 1-8.
41. Y. Yang, X. Liu, A re-examination of text categorization methods, in: Proceedings of 22nd annual international ACM-SIGIR, 1999, pp. 42-49.