Harnessing the expertise of 70,000 human editors: Knowledge-based feature generation for text categorization

Journal of Machine Learning Research

Volumn 8, Issue , 2007, Pages 2297-2345

  Gabrilovich, Evgeniy ^a,b   Markovitch, Shaul ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

^b YAHOO RESEARCH   (United States)

Author keywords

Background knowledge; Feature generation; Text classification

Indexed keywords

ALGORITHMS; DATA ACQUISITION; FILE EDITORS; KNOWLEDGE BASED SYSTEMS; LEARNING SYSTEMS; ONTOLOGY;

BACKGROUND KNOWLEDGE; COMMON SENSE KNOWLEDGE; FEATURE GENERATION; LANGUAGE PROCESSING; OPEN DIRECTORY PROJECT; POLYSEMY; TEXT CLASSIFICATION; WEB CRAWLING;

TEXT PROCESSING;

EID: 35748946646     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (114)

1. Rie Kubota Ando and Tong Zhang. Framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research, pages 1817-1853, 2005a.
2. Rie Kubota Ando and Tong Zhang. A high-performance semi-supervised learning method for text chunking. In Proceedings of the 43rd Annual Meeting of the ACL, pages 1-9, Ann Arbor, MI, June 2005b.
3. Douglas Baker and Andrew K. McCallum. Distributional clustering of words for text classification. In Bruce Croft, Alistair Moffat, Cornelis J. Van Rijsbergen, Ross Wilkinson, and Justin Zobel, editors, Proceedings of the 21st ACM International Conference on Research and Development in Information Retrieval, pages 96-103, Melbourne, AU, 1998. ACM Press, New York, US. URL http://www.cs.cmu.edu/mccallum/papers/clustering-sigir98.ps.gz.
4. Lisa Ballesteros and Bruce Croft. Phrasal translation and query expansion techniques for cross-language information retrieval. In Proceedings of the 20th ACM International Conference on Research and Development in Information Retrieval, pages 84-91, 1997.
5. Roberto Basili, Alessandro Moschitti, and Maria T. Pazienza. Language-sensitive text classification. In Proceedings of RIAO-00, 6th International Conference "Recherche d'Information Assistee par Ordinateur", pages 331-343, Paris, France, 2000.
6. Ron Bekkerman. Distributional clustering of words for text categorization. Master's thesis, Technion, 2003.
7. Paul N. Bennett, Susan T. Dumais, and Eric Horvitz. Inductive transfer for text classification using generalized reliability indicators. In Proceedings of the ICML-2003 Workshop on The Continuum from Labeled to Unlabeled Data, 2003.
8. Paul N. Bennett, Susan T. Dumais, and Eric Horvitz. The combination of text classifiers using reliability indicators. Information Retrieval, 8(1):67-100, 2005.
9. Cathy Blake and Christopher Merz. UCI Repository of machine learning databases, 1998. http://www.ics.uci.edu/~mlearn/MLRepository.html.
10. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. Latent direchlet allocation. Journal of Machine Learning Research, 3:993-1022, 2003.
11. Stephan Bloehdorn and Andreas Hotho. Boosting for text classification with semantic features. In Proceedings of the MSW 2004 Workshop at the 10th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 70-87, 2004.
12. Janez Brank, Marko Grobelnik, Natasa Milic-Frayling, and Dunia Mladenic. Interaction of feature selection methods and linear classification models. In Workshop on Text Learning held at ICML-2002, 2002.
13. Eric Brill. Transformation-based error-driven learning and natural language processing: A case study in part of speech tagging. Computational Linguistics, 21(4):543-565, 1995.
14. Michael Cafarella, Doug Downey, Stephen Soderland, and Oren Etzioni. Knowitnow: Fast, scalable information extraction from the web. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Vancouver, Canada, October 2005.
15. Lijuan Cai and Thomas Hofmann. Text categorization by boosting automatically extracted concepts. In Proceedings of the 26th International Conference on Research and Development in Information Retrieval, pages 182-189, 2003.
16. Maria Fernanda Caropreso, Stan Matwin, and Fabrizio Sebastiani. A learner-independent evaluation of the usefulness of statistical phrases for automated text categorization. In Amita G. Chin, editor, Text Databases and Document Management: Theory and Practice, pages 78-102. Idea Group Publishing, Hershey, US, 2001. URL http://faure.iei.pi.cnr.it/fabrizio/ Publications/TD01a/TD01a.pdf.
17. Rich Caruana. Multitask learning. Machine Learning, 28(1):41-75, 1997.
18. Soumen Chakrabarti, Byron Dom, Rakesh Agrawal, and Prabhakar Raghavan. Using taxonomy, discriminants, and signatures for navigating in text databases. In Proceedings of the 23rd VLDB Conference, pages 446-455, 1997.
19. Lois Mai Chan. A Guide to the Library of Congress Classification. Libraries Unlimited, 5th edition, 1999.
20. Ming-Wei Chang, Quang Do, and Dan Roth. Multilingual dependency parsing: A pipeline approach. In Recent Advances in Natural Language Processing, pages 195-204, 2006.
21. Stanley Chen and Joshua Goodman. A high-performance semi-supervised learning method for text chunking. In Proceedings of the 34th Annual Meeting of the ACL, 1996.
22. William W. Cohen. Automatically extracting features for concept learning from the web. In Proceedings of the 17th International Conference on Machine Learning, 2000.
23. Nello Cristianini, John Shawe-Taylor, and Huma Lodhi. Latent semantic kernels. Journal of Intelligent Information Systems, 18(2/3): 127-152, 2002.
24. Dmitry Davidov, Evgeniy Gabrilovich, and Shaul Markovitch. Parameterized generation of labeled datasets for text categorization based on a hierarchical directory. In Proceedings of the 27th ACM International Conference on Research and Development in Information Retrieval, pages 250-257, 2004.
25. Franca Debole and Fabrizio Sebastiani. Supervised term weighting for automated text categorization. In Proceedings of SAC-03, 18th ACM Symposium on Applied Computing, pages 784-788, 2003.
26. Scott Deerwester, Susan Dumais, George Furnas, Thomas Landauer, and Richard Harshman. Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41 (6):391-407, 1990.
27. Gerald Dejong and Raymond Mooney. Explanation-based learning: An alternative view. Machine Learning, 1(2): 145-176, 1986.
28. Melvil Dewey, Joan S. Mitchell, Julianne Beall, Giles Martin, Winton E. Matthews, and Gregory R. New, editors. Dewey Decimal Classification and Relative Index. Online Computer Library Center (OCLC), 22nd edition, 2003.
29. Inderjit Dhillon, Subramanyam Mallela, and Rahul Kumar. A divisive information-theoretic feature clustering algorithm for text classification. Journal of Machine Learning Research, 3:1265-1287, March 2003. URL http://www.jmlr.org/papers/volume3/dhillon03a/dhillon03a.pdf.
30. Chuong Do and Andrew Ng. Transfer learning for text classification. In Proceedings of Neural Information Processing Systems (NIPS), 2005.
31. Doug Downey, Oren Etzioni, and Stephen Soderland. A probabilistic model of redundancy in information extraction. In Proceedings of the 19th International Joint Conference on Artificial Intelligence, Edinburgh, Scotand, August 2005.
32. Richard Duda and Peter Hart. Pattern Classification and Scene Analysis. John Wiley and Sons, 1973.
33. Susan Dumais and Hao Chen. Hierarchical classification of web content. In Proceedings of the 23rd ACM International Conference on Research and Development in Information Retrieval, pages 256-263, 2000.
34. Susan Dumais, John Piatt, David Heckerman, and Mehran Sahami. Inductive learning algorithms and representations for text categorization. In Proceedings of the 7th ACM International Conference on Information and Knowledge Management, pages 148-155, 1998.
35. Oren Etzioni, Michael Cafarella, Doug Downey, Stanley Kok, Ana-Maria Popescu, Tal Shaked, Stephen Soderland, Daniel Weld, and Alexander Yates. Webscale information extraction in know-itall (preliminary results). In Proceedings of the 13th International World Wide Web Conference (WWW'04), New York, USA, May 2004. ACM Press.
36. Tom Fawcett. Feature Discovery for Problem Solving Systems. PhD thesis, UMass, May 1993.
37. Christiane Fellbaum, editor. WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA, 1998.
38. Johannes Fuernkranz, Tom Mitchell, and Ellen Riloff. A case study in using linguistic phrases for text categorization on the WWW. In Mehran Sahami, editor, Learning for Text Categorization: Proceedings of the 1998 AAAI/ICML Workshop, pages 5-12. AAAI Press, Madison, Wisconsin, 1998.
39. Evgeniy Gabrilovich and Shaul Markovitch. Text categorization with many redundant features: Using aggressive feature selection to make SVMs competitive with C4.5. In Proceedings of the 21st International Conference on Machine Learning, pages 321-328, 2004.
40. Evgeniy Gabrilovich and Shaul Markovitch. Feature generation for text categorization using world knowledge. In Proceedings of the 19th International Joint Conference on Artificial Intelligence, pages 1048-1053, Edinburgh, Scotand, August 2005.
41. Evgeniy Gabrilovich and Shaul Markovitch. Overcoming the brittleness bottleneck using Wikipedia: Enhancing text categorization with encyclopedic knowledge. In Proceedings of the 21st National Conference on Artificial Intelligence, pages 1301-1306, July 2006.
42. Evgeniy Gabrilovich and Shaul Markovitch. Computing semantic relatedness using wikipedia-based explicit semantic analysis. In Proceedings of the 20th International Joint Conference on Artificial Intelligence, pages 1606-1611, January 2007.
43. Evgeniy Gabrilovich, Susan Dumais, and Eric Horvitz. Newsjunkie: Providing personalized news-feeds via analysis of information novelty. In Proceedings of the Thirteenth International World Wide Web Conference (WWW2004), pages 482-490, New York, NY, May 2004. ACM Press.
44. Andrew Goldberg and Xiaojin Zhu. Seeing stars when there aren't many stars: Graph-based semi-supervised learning for sentiment categorization. In Workshop on Textgraphs: Graph-based Algorithms for Natural Language Processing, HLT-NAACL 2006, 2006.
45. Eui-Hong (Sam) Han and George Karypis. Centroid-based document classification: Analysis and experimental results. In Proceedings of the Fourth European Conference on Principles and Practice of Knowledge Discovery in Databases, September 2000.
46. William Hersh, Chris Buckley, T.J. Leone, and David Hickam. OHSUMED: An interactive retrieval evaluation and new large test collection for research. In Proceedings of the 17th ACM International Conference on Research and Development in Information Retrieval, pages 192-201, 1994.
47. George Hripcsak, Carol Friedman, Philip O. Alderson, William DuMouchel, Stephen B. Johnson, and Paul D. Clayton. Unlocking clinical data from narrative reports: a study of natural language processing. Annals of Internal Medicine, 122(9):681-688, 1995.
48. Yuh-Jyh Hu and Dennis Kibler. A wrapper approach for constructive induction. In The Thirteenth National Conference on Artificial Intelligence, pages 47-52, 1996.
49. David A. Hull. Improving text retrieval for the routing problem using latent semantic indexing. In W Bruce Croft and Cornells J. Van Rijsbergen, editors, Proceedings of the 17th ACM International Conference on Research and Development in Information Retrieval, pages 282-289, Dublin, Ireland, 1994. Springer Verlag, Heidelberg, Germany. URL http://www.acm.org/pubs/articles/ proceedings/ir/188490/p282-hull/p282-hull.pdf.
50. Tony Jebara. Multi-task feature and kernel selection for svms. In Proceedings of the Twenty-First International Conference on Machine Learning, pages 55-63, 2004.
51. Thorsten Joachims. Text categorization with support vector machines: Learning with many relevant features. In Proceedings of the European Conference on Machine Learning, pages 137-142, 1998.
52. Thorsten Joachims. Making large-scale SVM learning practical. In B. Schoelkopf, C. Burges, and A. Smola, editors, Advances in Kernel Methods - Support Vector Learning, pages 169-184. The MIT Press, 1999a.
53. Thorsten Joachims. Transductive inference for text classification using support vector machines. In Proceedings of the 13th International Conference on Machine Learning, 1999b.
54. Daphne Koller and Mehran Sahami. Hierarchically classifying documents using very few words. In Proceedings of the 14th International Conference on Machine Learning, pages 170-178, 1997.
55. Daniel Kudenko and Haym Hirsh. Feature generation for sequence categorization. In Proceedings of the 15th Conference of the American Association for Artificial Intelligence, pages 733-738, 1998.
56. Giridhar Kumaran and James Allan. Text classification and named entities for new event detection. In Proceedings of the 27th ACM International Conference on Research and Development in Information Retrieval, pages 297-304, 2004.
57. Ken Lang. Newsweeder: Learning to filter netnews. In Proceedings of the 12th International Conference on Machine Learning, pages 331-339, 1995.
58. Douglas Lenat and Edward Feigenbaum. On the thresholds of knowledge. Artificial Intelligence, 47:185-250, 1990.
59. Edda Leopold and Joerg Kindermann. Text categorization with support vector machines: How to represent texts in input space. Machine Learning, 46:423-444, 2002.
60. David D. Lewis. An evaluation of phrasal and clustered representations on a text categorization task. In Proceedings of the 15th ACM International Conference on Research and Development in Information Retrieval, pages 37-50, 1992.
61. David D. Lewis and W. Bruce Croft. Term clustering of syntactic phrases. In Proceedings of the 13th ACM International Conference on Research and Development in Information Retrieval, pages 385-404, 1990.
62. David D. Lewis, Robert E. Schapire, James P. Callan, and Ron Papka. Training algorithms for linear text classifiers. In Proceedings of the 19th ACM International Conference on Research and Development in Information Retrieval, pages 298-306, 1996.
63. David D. Lewis, Yiming Yang, Tony Rose, and Fan Li. RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 5:361-397, 2004.
64. Shaul Markovitch and Danny Rosenstein. Feature generation using general constructor functions. Machine Learning, 49(1):59-98, 2002.
65. Christopher J. Matheus. The need for constructive induction. In L.A. Birnbaum and G.C. Collins, editors, Proceedings of the Eighth International Workshop on Machine Learning, pages 173-177, 1991.
66. Christopher J. Matheus and Larry A. Rendell. Constructive induction on decision trees. In Proceedings of the 11th International Conference on Artificial Intelligence, pages 645-650, 1989.
67. Ia Mcilwaine. The Universal Decimal Classification: Guide to its Use. UDC Consortium, 2000.
68. MeSH. Medical subject headings (MeSH). National Library of Medicine, 2003. http://www.nlm.nih.gov/mesh.
69. Rada Mihalcea. Turning wordnet into an information retrieval resource: Systematic polysemy and conversion to hierarchical codes. International Journal of Pattern Recognition and Artificial Intelligence (IJPRAI), 17(1):689-704, 2003.
70. Andrei Mikheev. Feature lattices and maximum entropy models. Information Retrieval, 1999.
71. Tom Mitchell, Richard Keller, and Smadar Kedar-Cabelli. Explanation-based generalization: A unifying view. Machine Learning, 1(1):47-80, 1986.
72. Mandar Mitra, Amit Singhal, and Chris Buckley. Improving automatic query expansion. In Proceedings of the 21st ACM International Conference on Research and Development in Information Retrieval, pages 206-214, 1998.
73. Dunja Mladenic. Feature subset selection in text learning. In Proceedings of ECML-98, 10th European Conference on Machine Learning, pages 95-100, 1998a.
74. Dunja Mladenic. Turning Yahoo into an automatic web-page classifier. In Proceedings of 13th European Conference on Artificial Intelligence, pages 473-474, 1998b.
75. Patrick M. Murphy and Michael J. Pazzani. ID2-of-3: Constructive induction of M-of-N concepts for discriminators in decision trees. In Proceedings of the 8th International Conference on Machine Learning, pages 183-188. Morgan Kaufmann, 1991.
76. Kamal Nigam, Andrew McCallum, Sebastian Thrun, and Tom Mitchell. Text classification from labeled and unlabeled documents using EM. Machine Learning, 39(2-3): 103-134, 2000.
77. Kamal Nigam, Andrew McCallum, and Tom Mitchell. Semi-supervised text classification using EM. In Olivier Chapelle, Bernhard Schoelkopf, and Alexander Zien, editors, Semi-Supervised Learning. MIT Press, Boston, MA, 2006.
78. Giulia Pagallo and David Haussler. Boolean feature discovery in empirical learning. Machine Learning, 5(1):71-99, 1990. ISSN 0885-6125.
79. Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 79-86, 2002.
80. Fuchun Peng and Dale Shuurmans. Combining naive Bayes and n-gram language models for text classification. In Proceedings of the 25th European Conference on Information Retrieval Research (ECIR-03), pages 335-350, 2003.
81. Fuchun Peng, Dale Schuurmans, and Shaojun Wang. Augmenting naive Bayes classifiers with statistical language models. Information Retrieval, 7(3-4):317-345, 2004.
82. Martin Porter. An algorithm for suffix stripping. Program, 14(3):130-137, 1980.
83. Rajat Raina, Andrew Ng, and Daphne Koller. Constructing informative priors using transfer learning. In Proceedings of the 23rd International Conference on Machine Learning (ICML), Pittsburgh, PA, 2006.
84. Rajat Raina, Alexis Battle, Honglak Lee, Benjamin Packer, and Andrew Y. Ng. Self-taught learning: Transfer learning from unlabeled data. In ICML '07: Proceedings of the 24th International Conference on Machine learning, 2007.
85. Bhavani Raskutti, Herman Ferra, and Adam Kowalczyk. Second order features for maximizing text classification performance. In L. De Raedt and P. Flach, editors, Proceedings of the European Conference on Machine Learning (ECML), Lecture notes in Artificial Intelligence (LNAI) 2167, pages 419-430. Springer-Verlag, 2001.
86. Reuters. Reuters-21578 text categorization test collection, Distribution 1.0. Reuters, 1997. daviddlewis.com/resources/testcollections/ reuters21578.
87. Joseph John Rocchio. Relevance feedback in information retrieval. In The SMART Retrieval System: Experiments in Automatic Document Processing, pages 313-323. Prentice Hall, 1971.
88. Monica Rogati and Yiming Yang. High-performing feature selection for text classification. In Proceedings of the International Conference on Information and Knowledge Management (CIKM'02), pages 659-661, 2002.
89. J.K. Rowling. Harry Potter and the Philosopher's Stone. Bloomsbury, 1997.
90. Miguel E. Ruiz and Padmini Srinivasan. Hierarchical text categorization using neural networks. Information Retrieval, 5:87-118, 2002.
91. Ian Ruthven and Mounia Lalmas. A survey on the use of relevance feedback for information access systems. Knowledge Engineering Review, 18(2):95-145, 2003.
92. Carl Sable, Kathleen McKeown, and Kenneth W. Church. NLP found helpful (at least for one text categorization task). In Conference on Empirical Methods in Natural Language Processing, pages 172-179, 2002.
93. Gerard Salton and Chris Buckley. Term weighting approaches in automatic text retrieval. Information Processing and Management, 24(5):513-523, 1988.
94. Gerard Salton and Michael McGiIl. An Introduction to Modern Information Retrieval. McGraw-Hill, 1983.
95. Sam Scott. Feature engineering for a symbolic approach to text classification. Master's thesis, U. Ottawa, 1998.
96. Fabrizio Sebastiani. Machine learning in automated text categorization. ACM Computing Surveys, 34(1): 1-47, 2002. URL http://faure.iei.pi.cnr.it/ fabrizio/Publications/ACMCS02.pdf.
97. Charles Sutton and Andrew McCallum. Composition of conditional random fields for transfer learning. In Emprical Methods in Natural Language Processing (HLT/EMNLP), 1998.
98. Kentaro Toyama and Eric Horvitz. Bayesian modality fusion: Probabilistic integration of multiple vision algorithms for head tracking. In Proceedings of the 4th Asian Conference on Computer Vision, 2000.
99. Alfonso Urena-Lopez, Manuel Buenaga, and Jose M. Gomez. Integrating linguistic resources in TC through WSD. Computers and the Humanities, 35:215-230, 2001.
100. Ellen M. Voorhees. Query expansion using lexical-semantic relations. In Proceedings of the 17th International Conference on Research and Development in Information Retrieval, pages 61-69, 1994.
101. Ellen M. Voorhees. Using wordnet for text retrieval. In Christiane Fellbaum, editor, WordNet, an Electronic Lexical Database. The MIT Press, 1998.
102. Bill B. Wang, R.I. McKay, Hussein A. Abbass, and Michael Barlow. A comparative study for domain ontology guided feature extraction. In Proceedings of the 26th Australian Computer Science Conference (ASCS-2003), pages 69-78, 2003.
103. Bernard Widrow and Samuel Stearns. Adaptive Signal Processing. Prentice Hall, 1985.
104. Michael Wong, Wojciech Ziarko, and Patrick C.N. Wong. Generalized vector spaces model in information retrieval. In Proceedings of the 8th ACM International Conference on Research and Development in Information Retrieval, pages 18-25, 1985.
105. Pengcheng Wu and Thomas G. Dietterich. Improving svm accuracy by training on auxiliary data sources. In Proceedings of the Twenty-First International Conference on Machine Learning, pages 871-878, New York, NY, USA, 2004. ACM Press.
106. Jinxi Xu and W. Bruce Croft. Query expansion using local and global document analysis. In Proceedings of the 19th International Conference on Research and Development in Information Retrieval, pages 4-11, 1996.
107. Jinxi Xu and W. Bruce Croft. Improving the effectiveness of information retrieval with local context analysis. ACM Transactions on Information Systems, 18(1):79-112, 2000.
108. Yiming Yang and Xin Liu. A re-examination of text categorization methods. In Proceedings of the 22nd International Conference on Research and Development in Information Retrieval, pages 42-49, 1999.
109. Yiming Yang and Jan Pedersen. A comparative study on feature selection in text categorization. In Proceedings of the 14th International Conference on Machine Learning, pages 412-420, 1997.
110. Yiming Yang, Sean Slattery, and Rayid Ghani. A study of approaches to hypertext categorization. Journal of Intelligent Information Systems, 18(2/3):219-241, 2002.
111. Shipeng Yu, Deng Cai, Ji-Rong Wen, and Wei-Ying Ma. Improving pseudo-relevance feedback in web information retrieval using web page segmentation. In Proceedings of the 12th International World Wide Web Conference (WWW'03), Budapest, Hungary, May 2003. ACM Press.
112. Sarah Zelikovitz and Haym Hirsh. Improving short-text classification using unlabeled background knowledge to assess document similarity. In Proceedings of the 17th International Conference on Machine Learning, pages 1183-1190, 2000.
113. Sarah Zelikovitz and Haym Hirsh. Using LSI for text classification in the presence of background text. In Proceedings of the Conference on Information and Knowledge Management, pages 113-118, 2001.
114. Justin Zobel and Alistair Moffat. Exploring the similarity space. ACM SIGIR Forum, 32(1):18-34, 1998.