An information-pattern-based approach to novelty detection

Information Processing and Management

Volumn 44, Issue 3, 2008, Pages 1159-1188

  Li, Xiaoyan ^a   Croft, W Bruce ^b  

^a MOUNT HOLYOKE COLLEGE   (United States)

^b Biologically Inspired Neural and Dynamical Systems Laboratory   (United States)

Author keywords

Information patterns; Information retrieval; Named entities; Novelty detection; Question answering

Indexed keywords

DATA PROCESSING; INFORMATION MANAGEMENT; INFORMATION RETRIEVAL; USER INTERFACES;

INFORMATION PATTERNS; NAMED ENTITIES; NOVELTY DETECTION; QUESTION ANSWERING;

PATTERN RECOGNITION;

EID: 40649103141     PISSN: 03064573     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ipm.2007.09.013     Document Type: Article

References (39)

1. Allan, J., Gupta, R., & Khandelwal, V. (2001). Temporal summaries of news topics. In Proceedings of the 24th annual international ACM SIGIR conference on research and development in information retrieval (pp. 10-18).
2. Allan, J., Lavrenko, V., & Jin, H. (2000). First story detection in TDT is hard. In Proceedings of the eleventh international conference on information and knowledge management (pp. 374-381).
3. Allan, J., Paka, R., & Lavrenko, V. (1998). On-line new event detection and tracking. In Proceedings of the 21st annual international ACM SIGIR conference on research and development in information retrieval (pp. 37-45).
4. Allan, J., Wade, C., & Bolivar, A. (2003). Retrieval and novelty detection at the sentence level. In Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval (pp. 314-321).
5. Bikel D.M., Schwartz R.L., and Weischedel R.M. An algorithm that learns what's in a name. Machine Leaning 3 (1999) 221-231
6. Brants, T., Chen, F., & Farahat, A. (2003). A system for new event detection. In Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval (pp. 330-337).
7. Carbonell, J., & Goldstein, J. (1998). The use of MMR, diversity-based re-ranking for reordering documents and producing summaries. In Proceedings of the 21st annual international ACM SIGIR conference on research and development in information retrieval (pp. 335-336).
8. Dai, W., & Srihari, R. (2005). Minimal document set retrieval. In Proceedings of the 14th international conference on information and knowledge management (pp. 752-759).
9. Eichmann, D., & Srinivasan, P. (2002). Novel results and some answers, the University of Iowa TREC-11 results. In NIST Special Publication: SP 500-251, the eleventh text retrieval conference. http://trec.nist.gov/pubs/trec11/papers/uiowa.eichmann.pdf.
10. Franz, M., Ittycheriah, A., McCarley, J. S., & Ward, T. (2001). First story detection, combining similarity and novelty based approach. In 2001 Topic Detection and Tracking (TDT) Workshop Report.
11. Harman, D. (2002). Overview of the TREC 2002 novelty track. In NIST Special Publication: SP 500-251, the eleventh text retrieval conference. http://trec.nist.gov/pubs/trec11/papers/NOVELTY.OVER.pdf.
12. Jin, Q., Zhao, J., & Xu, B. (2003). NLPR at TREC 2003: novelty and robust. In NIST Special Publication: SP 500-255, the twelfth text retrieval conference http://trec.nist.gov/pubs/trec12/papers/chinese-acad-sci.novelty.robust.pdf.
13. Kazawa, H., Hirao, T., Isozaki, H., & Maeda, E. (2003). A machine learning approach for QA and novelty tracks: NTT system description. In NIST Special Publication: SP 500-255, the twelfth text retrieval conference. (pp. 460-467).
14. Kim, S.-N., Ravichandran, D., & Hovy, E. (2004). ISI novelty track system for TREC 2004. In NIST Special Publication: SP 500-261, the thirteenth text retrieval conference. http://trec.nist.gov/pubs/trec13/papers/usc-isi.novelty.pdf.
15. Krovetz, R. (1993). Viewing morphology as an inference process, In Proceedings the 16th annual international ACM SIGIR conference on research and development in information retrieval (pp. 191-202).
16. Kumaran, G., & Allan, J. (2004). Text classification and named entities for new event detection. In Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval (pp. 297-304).
17. Kumaran, G., & Allan, J. (2006). Simple questions to improve pseudo-relevance feedback results. In The 29th annual international ACM SIGIR conference on research and development in information retrieval (pp. 661-662).
18. Kwok, K. L., Deng, P., Dinstl, N., Chan, M. (2002). TREC2002, novelty and filtering track experiments using PRICS. In NIST Special Publication: SP 500-251, the eleventh text retrieval conference. http://trec.nist.gov/pubs/trec11/papers/queens.kwok.pdf.
19. Lemur (2006). Lemur toolkit for language modeling and information retrieval. The LEMUR PROJECT by CMU and UMASS. http://www.lemurproject.org.
20. Li, X. (2003). Syntactic features in question answering. In Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval (pp. 383-384).
21. Li, X. (2006). Sentence level information patterns for novelty detection, PhD thesis, Department of Computer Science, University of Massachusetts at Amherst. http://www.mtholyoke.edu/xli/Xiaoyan-PhD-Thesis-2006-Final.pdf.
22. Li, X., & Croft, W. B. (2003). Time-based language models. In Proceedings of the 12th international conference on information and knowledge management (pp. 469-475).
23. Li, X., & Croft, W. B. (2005). Novelty detection based on sentence level information patterns. In Proceedings of the 14th international conference on information and knowledge management (pp. 744-751).
24. Li, X., & Croft, W. B. (2006). Improving novelty detection for general topics using sentence level information patterns. In Proceedings of the 15th international conference on information and knowledge management (pp. 238-247).
25. Litkowski, K. C. (2003). Use of metadata for question answering and novelty tasks. In NIST Special Publication: SP 500-255, the twelfth text retrieval conference. http://trec.nist.gov/pubs/trec12/papers/clresearch.qa.novelty.pdf.
26. Qi, H., Otterbacher, J., Winkel, A., & Radev, D. T. (2002). The University of Michigan at TREC2002: question answering and novelty tracks. In NIST Special Publication: SP 500-251, the eleventh text retrieval conference. http://trec.nist.gov/pubs/trec11/papers/umichigan.radev.pdf.
27. Salton G., and McGill M.J. Introduction to modern information retrieval (1983), McGraw-Hill ISBN: 007054484
28. Soboroff, I. (2004). Overview of the TREC 2004 novelty track. In NIST Special Publication: SP 500-261, the thirteenth text retrieval conference. http://trec.nist.gov/pubs/trec13/papers/NOVELTY.OVERVIEW.pdf.
29. Soboroff, I., & Harman, D. (2003). Overview of the TREC 2003 novelty track. In NIST Special Publication: SP 500-255, the twelfth text retrieval conference. http://trec.nist.gov/pubs/trec12/papers/NOVELTY.OVERVIEW.pdf.
30. Stokes, N., & Carthy, J. (2001). First story detection using a composite document representation. In Proceedings of human language technology conference (pp. 134-141).
31. Sun, J., Yang, J., Pan, W., Zhang, H., Wang, B. & Cheng, X. (2003). TREC-2003 novelty and web track at ICT. In NIST Special Publication: SP 500-255, the twelfth text retrieval conference. http://trec.nist.gov/pubs/trec12/papers/chinese-acad-sci.novelty.web.pdf.
32. Tsai, M., Hsu, M., & Chen, H. (2003). Approach of information retrieval with reference corpus to novelty detection. In NIST Special Publication: SP 500-255, the twelfth text retrieval conference. http://trec.nist.gov/pubs/trec12/papers/ntu.novelty.pdf.
33. Yang, Y., Pierce, T., & Carbonell, J. (1998). A study on retrospective and on-line event detection. In The 21st annual international ACM SIGIR conference on research and development in information retrieval (pp. 28-36).
34. Yang, Y., Zhang, J., Carbonell, J., & Jin, C. (2002). Topic-conditioned novelty detection. In Proceedings of the eighth ACM SIGKDD international conference on knowledge discovery and data mining (pp. 688-693).
35. Zhai, C. (2001). Notes on the Lemur TFIDF model, School of Computer Science, CMU www.cs.cmu.edu/lemur/1.9/tfidf.ps.
36. Zhai, C., Cohen, W. W., & Lafferty, J. (2003). Beyond independent relevance: method and evaluation metrics for subtopic retrieval. In Proceedings of the 26th annual international ACM SIGIR conference on research and development in information retrieval (pp. 10-17).
37. Zhang, Y., Callan, J., & Minka, T. (2002). Novelty and redundancy detection in adaptive filtering. In The 25th annual international ACM SIGIR conference on research and development in information retrieval (pp.81-88).
38. Zhang, M., Lin, C., Liu, Y., Zhao, L., & Ma, S. (2003). THUIR at REEC 2003: novelty, robust and web. In NIST Special Publication: SP 500-255, the twelfth text retrieval conference. http://trec.nist.gov/pubs/trec12/papers/tsinghuau.hard.pdf.
39. Zhang, M., Song, R., Lin, C., Ma, S., Jiang, Z., Jin, Y. et al. (2002). Expansion-based technologies in finding relevant and new information: THU TREC2002 novelty track experiments. In NIST Special Publication: SP 500-251, the eleventh text retrieval conference. http://trec.nist.gov/pubs/trec11/papers/tsinghuau.novelty2.pdf.