RCV1: A new benchmark collection for text categorization research

Journal of Machine Learning Research

Volumn 5, Issue , 2004, Pages 361-397

  Lewis, David D ^a   Yang, Yiming ^b   Rose, Tony G ^c   Li, Fan ^b  

^a David D Lewis Consulting   (United States)

^b Carnegie Mellon University   (United States)

^c IMPERIAL CANCER RESEARCH FUND   (United Kingdom)

Author keywords

Applications; Automated indexing; Controlled vocabulary indexing; Effectiveness measures; Evaluation; Feature selection; K NN; Methodology; Multiclass; Multilabel; Nearest neighbor; News articles; Operational systems; Rocchio; SCut; SCutFBR; Support vector machines; SVMs; Term weighting; Test collection; Text classification; Thresholding

Indexed keywords

APPLICATIONS; CLASSIFICATION (OF INFORMATION); FEATURE EXTRACTION; INDEXING (OF INFORMATION); QUALITY CONTROL; SEMANTICS; SUPPORT VECTOR MACHINES; TAXONOMIES;

EFFECTIVENESS MEASURE; EVALUATION; METHODOLOGY; MULTI-LABEL; MULTICLASS; NEAREST NEIGHBORS; NEWS ARTICLES; OPERATIONAL SYSTEMS; ROCCHIO; SCUT; SCUTFBR; SVMS; TERM WEIGHTING; TEST COLLECTION; TEXT CLASSIFICATION; THRESHOLDING;

TEXT PROCESSING;

EID: 84876811202     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (45)

1. D. G. Altman. Practical Statistics for Medical Research. Chapman & Hall/CRC, 1991.
2. T. Ault and Y. Yang. kNN, Rocchio and metrics for information filtering at TREC-10. In The Tenth Text REtrieval Conference (TREC 2001), pages 84-93, Gaithersburg, MD 20899-0001, 2002. National Institute of Standards and Technology. http://trec.nist.gov/pubs/trec10/papers/cmucatcorrect.pdf.
3. C. Buckley, G. Salton, and J. Allan. The effect of adding relevance information in a relevance feedback environment. In Proceedings of the Seventeenth Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval (SIGIR 94), pages 292-300, 1994.
4. C. W. Cleverdon. The significance of the Cranfield tests of index languages. In Proceedings of the Fourteenth Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval (SIGIR 91), pages 3-12, 1991.
5. Great Britain Office for National Statistics. Indexes to UK Standard Industrial Classification of Economic Activities 1992 UKSIC (92). Office for National Statistics, London, 1997.
6. Great Britain Office for National Statistics. UK Standard Industrial Classification of Economic Activities UKSIC (92), December 20, 2002. http://www.statistics.gov. uk/methodsjquality/sic/contents.asp.
7. R. Grishman and B. Sundheim. Design of the MUC-6 evaluation. In Sixth Message Understanding Evaluation (MUC-6), pages 1-12. Defense Advanced Research Projects Agency, Morgan Kaufmann, 1995.
8. P. J. Hayes and S. P. Weinstein. CONSTRUE/TIS: A System for Content-Based Indexing of a Database of News Stories. In Second Annual Conference on Innovative Applications of Artificial Intelligence, pages 49-64, 1990.
9. W. Hersh, C. Buckley, T. J. Leone, and D. Hickman. OHSUMED: an interactive retrieval evaluation and new large text collection for research. In Proceedings of the Seventeenth Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval (SIGIR 94), pages 192-201, 1994.
10. D. J. Ittner, D. D. Lewis, and D. D. Ahn. Text Categorization of Low Quality Images. In Symposium on Document Analysis and Information Retrieval, pages 301-315, Las Vegas, 1995.
11. T. Joachims. Text categorization with support vector machines: Learning with many relevant features. In European Conference on Machine Learning (ECML'98), pages 137-142, Berlin, 1998.
12. T. Joachims. Transductive inference for text classification using support vector machines. In International Conference on Machine Learning (ICML'99), pages 200-209, San Francisco, CA, 1999.
13. T. Joachims. SVM Light: Support Vector Machine, May 13th, 2002. http://svmlight.joachims.org.
14. D. V. Khmelev and W. J. Teahan. A repetition based measure for verification of text collections and for text categorization. In Proceedings of the Twenty-Sixth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 03), pages 104-110, 2003.
15. D. Koller and M. Sahami. Hierarchically classifying documents using very few words. In International Conference on Machine Learning (ICML'97), pages 170-178, Nashville, 1997.
16. F. W. Lancaster. Indexing and Abstract ing in Theory and Practice. Second edition. University of Illinois, Champaign, IL, 1998.
17. D. D. Lewis. Evaluating text categorization. In Proceedings of Speech and Natural Language Workshop, pages 312-318. Defense Advanced Research Projects Agency, Morgan Kaufmann, 1991.
18. D. D. Lewis. An evaluation of phrasal and clustered representations on a text categorization task. In Proceedings of the Fifteenth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 92), pages 37-50, 1992.
19. D. D. Lewis. Evaluating and optimizing autonomous text classification systems. In Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 95), pages 246-254, 1995.
20. D. D. Lewis. Reuters-21578 text Categorization test collection. Distribution 1.0. README file (version 1.2). Manuscript, September 26, 1997. http://www.daviddlewis.com/resources/testcollections/reuters21578/readme.txt
21. D. D. Lewis. Applying support vector machines to the TREC-2001 batch filtering and routing tasks. In The Tenth Text REtrieval Conference (TREC 2001), pages 286-292, Gaithersburg, MD 20899-0001, 2002. National Institute of Standards and Technology. http://trec.nist.gov/pubs/trec10/papers/daviddlewis-trec2001-draft4.pdf
22. D. D. Lewis, R. E. Schapire, J. P. Callan, and R. Papka. Training algorithms for linear text classifiers. In Proceedings of the 19th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 96), pages 298-306, 1996.
23. D. D. Lewis and R. M. Tong. Text filtering in MUC-3 and MUC-4. In Proceedings of the Fourth Message Understanding Conference (MUC-4), pages 51-66. Defense Advanced Research Projects Agency, Morgan Kaufmann, 1992.
24. M. F. Porter. An algorithm for suffix stripping. Program, 14(3):130-137, 1980.
25. M. F. Porter. The Porter Stemming Algorithm, 2003. http://www.tartarus.org/~martin/PorterStemmer.
26. S. Robertson and I. Soboroff. The TREC 2001 filtering track report. In The Tenth Text REtrieval Conference (TREC 2001), pages 26-37, Gaithersburg, MD 20899-0001, 2002. National Institute of Standards and Technology. http://trec.nist.gov/pubs/trec10/papers/filtering2track.pdf
27. J. J. Rocchio, Jr.Relevance feedback in information retrieval. In G. Salton, editor, The SMART Retrieval System: Experiments in Automatic Document Processing, pages 313-323. Prentice-Hall, 1971.
28. M. Rogati and Y. Yang. High performing and scalable feature selection for text classification. In Proceedings of the Eleventh International Conference on Information and Knowledge Management, pages 659-661, 2002.
29. T. Rose, M. Stevenson, and M. Whitehead. The Reuters Corpus Volume 1 - from Yesterday's News to Tomorrow's Language Resources. In Proceedings of the Third International Conference on Language Resources and Evaluation, 2002. http://about.reuters.com/researchandstandards/corpus/LREC-cameraready.pdf
30. T. Rose. Electronic mail message to ReutersCorpora@yahoogroups.com, June 11, 2002. http://groups.yahoo.com/group/ReutersCorpora/message/70.
31. G. Salton and C. Buckley. Improving retrieval performance by relevance feedback. Journal of American Society for Information Sciences, 41:288-297, 1990.
32. G. Salton, editor. The SMART Retrieval System: Experiments in Automatic Document Processing. Prentice-Hall, 1971.
33. R. E. Schapire, Y. Singer, and A. Singhal. Boosting and Rocchio applied to text filtering. In Proceedings of the Twenty-First Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 98), pages 215-223, 1998.
34. F. Sebastiani. Machine learning in automated text categorization. ACM Computing Surveys, 34(1):1-47, 2002.
35. J. M. Tague. The pragmatics of information retrieval experimentation. In K. Sparck Jones, editor, Information Retrieval Experiment, chapter 5. Butterworths, 1981.
36. C. J. Van Rijsbergen. Automatic Information Structuring and Retrieval. PhD thesis, King's College, Cambridge, 1972.
37. C. J. Van Rijsbergen. Information Retrieval. Butterworths, 1979.
38. M. Whitehead. Electronic mail message to ReutersCorpora@yahoogroups.com, November 14, 2002. http://groups.yahoo.com/group/ReutersCorpora/message/106.
39. A. S. Weigend, E. D. Wiener, and J. O. Pedersen. Exploiting hierarchy in text categorization. Information Retrieval, 1(3):193-216, 1999.
40. Y. Yang. An evaluation of statistical approaches to text categorization. Information Retrieval, 1(1/2):67-88, 1999.
41. Y. Yang. A study on thresholding strategies for text categorization. In The Twenty-Fourth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 01), pages 137-145, 2001.
42. Y. Yang, T. Ault, T. Pierce, and C. W. Lattimer. Improving text categorization methods for event tracking. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 00), pages 65-72, 2000.
43. Y. Yang and X. Liu. A re-examination of text categorization methods. In Proceedings of the Twenty-Second Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 99), pages 42-49, 1999.
44. Y. Yang and J. O. Pedersen. A comparative study on feature selection in text categorization. In The Fourteenth International Conference on Machine Learning (ICML'97), pages 412-420. Morgan Kaufmann, 1997.
45. T. Zhang and F. J. Oles. Text categorization based on regularized linear classification methods. Information Retrieval, 4(1):5-31, 2001.