Automated subject classification of textual web documents

Journal of Documentation

Volumn 62, Issue 3, 2006, Pages 350-371

  Golub, Koraljka ^a  

^a LUND UNIVERSITY   (Sweden)

Author keywords

Automation; Classification; Controlled languages; Document management; Internet

Indexed keywords

EID: 33646369180     PISSN: 00220418     EISSN: None     Source Type: Journal    
DOI: 10.1108/00220410610666501     Document Type: Article

References (118)

1. 20 Newsgroups DataSet (1998), The 4 Universities Data Set, available at: www-2.cs.cmu.edu/afs/cs.cmu.edu/project/theo-20/www/data/news20.html (accessed 22 December 2004).
2. DDC (2005), "About DDC: research: a vital part of ongoing development", Dewey Services, available at: www.oclc.org/dewey/about/ research/ (accessed 8 August 2005).
3. Ardö, A. et al., (1994), "Improving resource discovery and retrieval on the internet: the Nordic WAIS/world wide web project summary report", NORDINFO Nytt, Vol. 17 No. 4, pp. 13-28.
4. Attardi, G., Gullì, A. and Sebastiani, F. (1999), "Automatic web page categorization by link and context analysis", in Hutchison, C. and Lanzarone, G. (Eds), Proceedings of THAI-99, European Symposium on Telematics, Hypermedia and Artificial Intelligence, pp. 105-19.
5. Bekkerman, R. et al., (2003), "Distributional word clusters vs words for text categorization", Journal of Machine Learning Research, Vol. 3, pp. 1183-208.
6. BINDEX (2001), "HLT Project Factsheet: BINDEX", HLTCentral, available at: www.hltcentral.org/projects/print.php?acronym=BINDEX (accessed 22 December 2004).
7. Blum, A. and Mitchell, T. (1998), "Combining labeled and unlabeled data with co-training", COLT: Proceedings of the Workshop on Computational Learning Theory, Morgan Kaufmann Publishers, San Mateo, CA.
8. Cai, L. and Hofmann, T. (2003), "Text categorization by boosting automatically extracted concepts", in Callan, J. et al. (Eds), Proceedings of SIGIR-03, 26th ACM International Conference on Research and Development in Information Retrieval, pp. 182-9.
9. CERES (2003), "CERES thesaurus effort", CERES The California Environmental Resources Evaluation System, available at: http://ceres.ca.gov/ thesaurus/ (accessed 22 December 2004).
10. Chakrabarti, S. et al. (1998a), "Automatic resource compilation by analyzing hyperlink structure and associated text", Proceedings of the Seventh International Conference on World Wide Web 7, Brisbane, Australia, pp. 65-74.
11. Chakrabarti, S., Dom, B. and Indyk, P. (1998b), "Scalable feature selection, classification and signature generation for organizing large text databases into hierarchical topic taxonomies", Journal of Very Large Data Bases, Vol. 7 No. 3, pp. 163-78.
12. Chan, L.M. (1994), Cataloging and Classification: An Introduction, 2nd ed., McGraw-Hill, New York, NY.
13. Chen, H. and Dumais, S.T. (2000), "Bringing order to the web: automatically categorizing search results", Proceedings of CHI-00, ACM International Conference on Human Factors in Computing Systems, Den Haag, pp. 145-52.
14. Chen, M., LaPaugh, A. and Singh, J.P. (2002), "Categorizing information objects from user access patterns", Proceedings of the Eleventh International Conference on Information and Knowledge Management, 4-9 November, pp. 365-72.
15. Clusty (2004), "Clusty the clustering engine", Vivsimo, available at: www.clusty.com (accessed 22 December 2004).
16. Cutting, D. et al. (1992), "Scatter/gather: a cluster-based approach to browsing large document collections", Proceedings of the 15th Annual International ACM/SIGIR Conference, Copenhagen, pp. 318-29.
17. DESIRE (2000), "DESIRE: development of a European service for information on research and education", DESIRE, available at: www.desire.org/ (accessed 22 December 2004).
18. DESIRE Project (1999), Lunds Universitets Bibliotek, available at: www.lub.lu.se/desire (accessed 22 December 2004).
19. Dittenbach, M., Berger, H. and Merkl, D. (2004), "Improving domain ontologies by mining semantics from text", Proceedings of the first Asian-Pacific Conference on Conceptual Modeling, Dunedin, New Zealand, Vol. 31, pp. 91-100.
20. Dumais, S.T. and Chen, H. (2000), "Hierarchical classification of web content", Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 24-28 July, Athens, Greece, pp. 256-63.
21. Dumais, S.T., Lewis, D.D. and Sebastiani, F. (2002), "Report on the workshop on operational text classification systems (OTC-02)", ACM SIGIR Forum, Vol. 35 No. 2, pp. 8-11.
22. EELS (2003), "'All' Engineering resources on the internet: a companion service to EELS", EELS, Engineering E-Library, Sweden, available at: http://eels.lub.lu.se/ae/ (accessed 22 December 2004).
23. Engine-e (2004), Lund University Libraries, available at: http://engine-e.lub.lu.se/ (accessed 22 December).
24. Engineering Electronic Library (2003), Lund University Libraries, available at: http://eels.lub.lu.se/ (accessed 22 December 2004).
25. FAST (2003), "FAST as a knowledge base for automated classification", OCLC projects, available at: www.oclc.org/research/ projects/fastac/ (accessed 7 August 2005).
26. FAST (2004), "FAST: faceted application of subject terminology", OCLC projects, available at: www.oclc.org/research/projects/ fast/ (accessed 22 December 2004).
27. Fasulo, D. (1999), "An analysis of recent work on clustering algorithms: technical report", University of Washington, available at: http://citeseer.nj.nec.com/fasulo99analysi.html (accessed 22 December 2004).
28. Fisher, M. and Everson, R. (2003), "When are links useful? Experiments in text classification", Proceedings of ECIR-03, 25th European Conference on Information Retrieval, Pisa, IT, pp. 41-56.
29. Frank, E. and Paynter, G.W. (2004), "Predicting library of congress classifications from library of congress subject headings", Journal of the American Society for Information Science and Technology, Vol. 55 No. 3, pp. 214-27.
30. Fürnkranz, J. (1999), "Exploiting structural information for text classification on the WWW", Proceedings of IDA-99, 3rd Symposium on Intelligent Data Analysis, pp. 487-97.
31. Fürnkranz, J. (2002), "Hyperlink ensembles: a case study in hypertext classification", Information Fusion, Vol. 3 No. 4, pp. 299-312.
32. Garfield, E., Malin, M.V. and Small, H. (1975), "A system for automatic classification of scientific literature", Journal of the Indian Institute of Science, Vol. 57 No. 2, pp. 61-74, (Reprinted in: Essays of an Information Scientist, Vol. 2, pp. 356-65).
33. GERHARD (1998), "GERHARD: German harvest automated retrieval and directory", GERHARD, available at: www.gerhard.de/ (accessed 22 December 2004).
34. GERHARD (1999), "GERHARD - navigating the web with the universal decimal classification system", GERHARD, available at: www.gerhard.de/info/ dokumente/vortraege/ecdl99/html/index.htm (accessed 22 December 2004).
35. Ghani, R., Slattery, S. and Yang, Y. (2001), "Hypertext categorization using hyperlink patterns and metadata", Proceedings of ICML-01, 18th International Conference on Machine Learning, pp. 178-85.
36. Glover, E.J. et al. (2002), "Using web structure for classifying and describing web pages", Proceedings of the Eleventh International Conference on World Wide Web Honolulu, Hawaii, USA, pp. 562-9.
37. Glover, E.J. et al. (2003), "Inferring hierarchical descriptions", Proceedings of the Eleventh International Conference on Information and Knowledge Management, CIKM 2002, November 4-9, pp. 507-14.
38. Godby, J. and Reighart, R. (1998), "The WordSmith indexing system", OCLC Digital Archive, available at: http://digitalarchive.oclc. org/da/ViewObject.jsp?fileid=0000003487: 000000090408&reqid=33836 (accessed 22 December 2004).
39. Golub, K. and Larsen, B. (2005), "Different approaches to automated classification: is there an exchange of ideas?", in Ingwersen, P. and Larsen, B. (Eds), Proceedings of ISSI 2005 - the 10th International Conference of the International Society for Scientometrics and Informetrics, Stockholm, Sweden, 24-28 July, Vol. 1, Karolinska University Press, Stockholm, pp. 270-4.
40. Goren-Bar, D. et al. (2000), "Supervised learning for automatic classification of documents using self-organizing maps", Proceedings of the First DELOS Network of Excellence Workshop on Information Seeking, Searching and Querying in Digital Libraries, Zrich, Switzerland, Vol. 11-12, p. 2000.
41. Gövert, N., Lalmas, M. and Fuhr, N. (1999), "A probabilistic description-oriented approach for categorising web documents", Proceedings of the Eighth International Conference on Information and Knowledge Management, pp. 475-82.
42. Hartigan, J.A. (1996), "Introduction", in Hubert, L. and De Soete, G. (Eds), Clustering and Classification Arabie, World Scientific, Singapore.
43. Hatzivassiloglou, V., Gravano, L. and Maganti, A. (2000), "An investigation of linguistic features and clustering algorithms for topical document clustering", Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Athens, Greece, pp. 224-31.
44. Haveliwala, T.H., Gionis, A. and Indyk, P. (2000), "Scalable techniques for clustering the web", Third International Workshop on the Web and Databases, May, pp. 129-34.
45. Hersh, W.R. (1994), "OHSUMED: an interactive retrieval evaluation and new large test collection for research", Proceedings of the 17th Annual ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 192-201.
46. Heuser, U., Babanine, A. and Rosenstiel, W. (1998), "HTML documents classification using (non-linear) principal component analysis and self-organizing maps", Proceedings of the Fourth International Conference on Neural Networks and their Applications (Neurap'98), 11-13 March 1998, Marseilles, France, pp. 291-5.
47. INitiative for the Evaluation of XML Retrieval (2004), DELOS Network of Excellence for Digital Libraries, available at: http://inex.is.informatik.uni- duisburg.de/ (accessed 22 December 2004).
48. Jain, A.K., Murty, M.N. and Flynn, P.J. (1999), "Data clustering: a review", ACM Computing Surveys, Vol. 31 No. 3, pp. 264-323.
49. Jenkins, C. et al., (1998), "Automatic classification of web resources using Java and Dewey decimal classification", Computer Networks & ISDN Systems, Vol. 30, pp. 646-8.
50. Kim, H.R. and Chan, P.K. (2003), "Learning implicit user interest hierarchy for context in personalization", Proceedings of the International Conference on Intelligent User Interfaces, pp. 101-8.
51. Koch, T. (1994), "Experiments with automatic classification of WAIS databases and indexing of WWW", Internet World & Document Delivery World International 94, London, May, pp. 112-5.
52. Koch, T. and Ardö, A. (2000), "Automatic classification", DESIRE II D3.6a, Overview of Results, available at: www.lub.lu.se/desire/ DESIRE36a-overview.html (accessed 22 December 2004).
53. Koch, T. and Day, M. (1997), "The role of classification schemes in internet resource description and discovery", EU Project DESIRE, Deliverable D3.2.3, available at: www.lub.lu.se/desire/radar/reports/D3.2.3/ (accessed 22 December 2004).
54. Koch, T. and Zettergren, A-S. (1999), "Provide browsing in subject gateways using classification schemes", EU Project DESIRE II, available at: www.lub.lu.se/desire/handbook/class.html (accessed 22 December 2004).
55. Kohonen, T. (2001), Self-Organizing Maps, 3rd ed., Springer-Verlag, Berlin.
56. Koller, D. and Sahami, M. (1997), "Hierarchically classifying documents using very few words", Proceedings of ICML-97, 14th International Conference on Machine Learning, pp. 170-8.
57. Labrou, Y. and Finin, T. (1999), "Yahoo As an ontology: using Yahoo Categories to describe documents", Proceedings of CIKM-99, 8th ACM International Conference on Information and Knowledge Management, pp. 180-7.
58. Larson, R.R. (1992), "Experiments in automatic library of congress classification", Journal of the American Society for Information Science, Vol. 43 No. 2, pp. 130-48.
59. Li, Y.H. and Jain, A.K. (1998), "Classification of text documents", The Computer Journal, Vol. 41 No. 8, pp. 537-46.
60. Liere, R. and Tadepalli, P. (1998), "Active learning with committees: preliminary results in comparing winnow and perception in text categorization", Proceedings of CONALD-98, 1st Conference on Automated Learning and Discovery.
61. Lindholm, J., Schönthal, T. and Jansson, K. (2003), "Experiences of harvesting web resources in engineering using automatic classification", Ariadne, No. 37, available at: www.ariadne.ac.uk/issue37/ lindholm/.
62. Liu, X. et al. (2002), "Document clustering with cluster refinement and model selection capabilities", Proceedings of the 25th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Tampere, Finland, pp. 191-8.
63. McCallum, A. et al. (1998), "Improving text classification by shrinkage in a hierarchy of classes", paper presented at ICML-98, 15th International Conference on Machine Learning, pp. 359-67.
64. McCallum, A. et al. (1999), "Building domain-specific search engines with machine learning techniques", paper presented at AAAI-99 Spring Symposium on Intelligent Agents in Cyberspace.
65. McCallum, A. et al., (2000), "Automating the construction of internet portals with machine learning", Information Retrieval Journal, Vol. 3, pp. 127-63.
66. Mandhani, B., Joshi, S. and Kummamuru, K. (2003), "A matrix density based algorithm to hierarchically co-cluster documents and words", Proceedings of the Twelfth International Conference on World Wide Web, Budapest, Hungary, pp. 511-8.
67. Manning, C. and Schütze, H. (1999), Foundations of Statistical Natural Language Processing, MIT Press, Cambridge, MA.
68. Merchkour, M., Harper, D.J. and Muresan, G. (1998), "The WebCluster project: using clustering for mediating access to the world wide web", Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Melbourne, Australia, pp. 357-8.
69. MetaCrawler Web Search (2005), available at: http://metacrawler.com (accessed 5 August 2005).
70. Mitchell, T. (1997), Machine Learning, McGraw-Hill, New York, NY.
71. Mladenic, D. (1998), "Turning Yahoo into an automatic web-page classifier", Proceedings of the 13th European Conference on Artificial Intelligence ECAI'98, pp. 473-4.
72. Mladenic, D. and Grobelnik, M. (2003), "Feature selection on hierarchy of web documents", Decision Support Systems, Vol. 35 No. 1, pp. 45-87.
73. Möller, G. et al. (1999), "Automatic classification of the WWW using the universal decimal classification", in McKenna, B. (Ed.), Proceedings of the 23rd International Online Information Meeting, London, 7-9 December, pp. 231-8.
74. Nordic WAIS/World Wide Web Project (1995), Lund University Libraries, available at: www.lub.lu.se/W4/ (accessed 22 December 2004).
75. Nübel, R. et al. (2002), "Bilingual indexing for information retrieval with AUTINDEX", LREC Proceedings, Las Palmas.
76. Olson, H.A. and Boll, J.J. (2001), Subject Analysis in Online Catalogs, 2nd ed., Libraries Unlimited, Englewood, CO.
77. Palmer, C.R. et al. (2001), "Demonstration of hierarchical document clustering of digital library retrieval results", Proceedings of the 1st ACM/IEEE-CS Joint Conference on Digital Libraries, Roanoke, Virginia, p. 451.
78. Pierre, J.M. (2001), "On the automated classification of web sites", Linköping Electronic Articles in Computer and Information Science, Vol. 6 No. 001.
79. Poincot, P., Lesteven, P.S. and Murtagh, F. (1998), "A spatial user interface to the astronomical literature", Astronomy & Astrophysics, 2 May, pp. 183-91.
80. Pratt, W. (1997), "Dynamic organization of search results using the UMLS", American Medical Informatics Association Fall Symposium, pp. 480-4.
81. Rasmussen, E. (1992), "Clustering algoritms", in Frakes, W.B. and Baeza-Yates, R. (Eds), Information Retrieval: Data Structures and Algorithms, Prentice-Hall, Engelwood Cliffs, NJ.
82. Rauber, A. and Merkl, D. (1999), "SOMLib: a digital library system based on neural networks", Proceedings of the Fourth ACM Conference on Digital Libraries, Berkeley, California, United States, pp. 240-1.
83. Reuters-21578 (2004), available at: www.daviddlewis.com/resources/ testcollections/reuters21578/ (accessed 3 August 2005).
84. Rocchio, J.J. (1971), "Relevance feedback in information retrieval", in Salton, G. (Ed.), The SMART Retrieval System: Experiments in Automatic Document Processing, Prentice-Hall, Englewood Cliffs, NJ, pp. 313-23.
85. Ruiz, M.E. and Srinivasan, P. (1999), "Hierarchical neural networks for text categorization", Proceedings of the 22nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 281-2.
86. Sahami, M., Yusufali, M. and Baldonado, M.Q. (1998), "SONIA: a service for organizing networked information autonomously", paper presented at 3rd ACM Conference on digital libraries, Pittsburgh, pp. 200-9.
87. Salton, G. (1991), "Developments in automatic text retrieval", Science, Vol. 253, pp. 974-9.
88. Schütze, H., Hull, D.A. and Pedersen, J.O. (1995), "A comparison of classifiers and document representations for the routing problem", Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Seattle, pp. 229-37.
89. Schwartz, C. (2001), Sorting Out the Web: Approaches to Subject Access, Ablex, Westport, CT.
90. Schweighofer, E., Rauber, A. and Dittenbach, M. (2001), "Automatic text representation, classification and labeling in European law", ICAIL 2001, pp. 78-87.
91. Scorpion (2004), OCLC software, available at: www.oclc.org/research/ software/scorpion/default.htm (accessed 22 December).
92. Sebastiani, F. (2002), "Machine learning in automated text categorization", ACM Computing Surveys, Vol. 34 No. 1, pp. 1-47.
93. Slattery, S. and Craven, M. (2000), "Discovering test set regularities in relational domains", Proceedings of ICML-00, 17th International Conference on Machine Learning, pp. 895-902.
94. Slonim, N., Friedman, N. and Tishby, N. (2003), "Unsupervised document classification using sequential information maximization", Proceedings of SIGIR'02, 25th ACM International Conference on Research and Development of Information Retireval, Tampere, Finland, 2002.
95. Soergel, D. et al., (2004), "Reengineering thesauri for new applications: the AGROVOC example", Journal of Digital Information, Vol. 4 No. 4, Article No. 257, available at: http://jodi.ecs.soton.ac.uk/Articles/v04/ i04/Soergel/.
96. Steinbach, M., Karypis, G. and Kumar, V. (2000), "A comparison of document clustering techniques", KDD Workshop on Text Mining, Boston, MA, 20-23 August.
97. Su, Z. et al. (2001), "Correlation-based document clustering using web logs", Proceedings of the 34th Annual Hawaii International Conference on System Sciences (HICSS-34), 3-6 January, Vol. 5, p. 5022.
98. Subramanian, S. and Shafer, K.E. (1998), "Clustering", OCLC Publications, available at: http://digitalarchive.oclc.org/da/ViewObject.jsp? objid=0000003409 (accessed 22 December 2004).
99. Sun, A., Lim, E-P. and Ng, W-K. (2001), "Hierarchical text classification and evaluation", ICDM 2001, IEEE International Conference on Data Mining.
100. Svenonius, E. (2000), The Intellectual Foundations of Information Organization, MIT Press, Cambridge, MA.
101. Thunderstone (2005), Thunderstone's Web Site Catalog, available at: http://search.thunderstone.com/texis/websearch (accessed 4 August 2005).
102. Tombros, A. and van Rijsbergen, C.J. (2001), "Query-sensitive similarity measures for the calculation of interdocument relationships", Proceedings of the Tenth International Conference on Information and Knowledge Management, Atlanta, Georgia, USA, pp. 17-24.
103. Toth, E. (2002), "Innovative solutions in automatic classification: a brief summary", Libri, Vol. 25 No. 1, pp. 48-53.
104. TREC (2004), "TREC: Text REtrieval Conference", National Institute of Standards and Technology, available at: http://trec.nist.gov/ (accessed 22 December 2004).
105. Vizine-Goetz, D. (1996), "Using library classification schemes for internet resources", OCLC Internet Cataloging Project Colloquium, available at: http://webdoc.sub.gwdg.de/ebook/aw/oclc/man/colloq/v-g.htm, (accessed 4 April 2006).
106. Wacholder, N., Evans, D.K. and Klavans, J.L. (2001), "Automatic identification and organization of index terms for interactive browsing", Proceedings of the ACM-IEEE Joint Conference on Digital Libraries, Roanoke, Virginia, June, pp. 128-34.
107. Wallis, J. and Burden, P. (1995), "Towards a classification-based approach to resource discovery on the web", University of Wolverhampton, Wolverhampton, available at: www.scit.wlv.ac.uk/wwlib/position.html (accessed 22 December 2004).
108. Wang, Y. and Kitsuregawa, M. (2002), "Evaluating contents-link coupled web page clustering for web search results", Proceedings of the Eleventh International Conference on Information and Knowledge Management, McLean, Virginia, USA, pp. 499-506.
109. WebKB (2001), CMU World Wide Knowledge Base, available at: www-2.cs.cmu.edu/ ∼ webkb/ (accessed 22 December 2004).
110. Weiss, R. et al. (1996), "HyPursuit: a hierarchical network search engine that exploits content-link hypertext clustering", Proceedings of the Seventh ACM Conference on Hypertext, Washington, DC, March, pp. 180-93.
111. Willet, P. (1988), "Recent trends in hierarchic document clustering: a critical review", Information Processing and Management, Vol. 24 No. 5, pp. 577-97.
112. Yahoo (2005), Yahoo Directory, available at: http://dir.yahoo.com/ (accessed 8 August 2005).
113. Yang, Y. (1999), "An evaluation of statistical approaches to text categorization", Journal of Information Retrieval, Vol. 1 Nos 1/2, pp. 67-88.
114. Yang, C., Chen, H. and Hong, K. (2003), "Visualization of large category map for internet browsing", Decision Support Systems (DSS), Vol. 35 No. 1, pp. 89-102.
115. Yang, Y., Slattery, S. and Ghani, R. (2002), "A study of approaches to hypertext categorization", Journal of Intelligent Information Systems, Vol. 8 Nos 2/3, pp. 219-41.
116. Zamir, O. and Etzioni, O. (1998), "Web document clustering: a feasibility demonstration", ACM SIGIR'98, Australia, pp. 46-54.
117. Zamir, O. et al. (1997), "Fast and intuitive clustering of web documents", Proceedings of the 3rd International Conference on Knowledge Discovery and Data Mining, pp. 287-90.
118. Zhao, Y. and Karypis, G. (2002), "Evaluation of hierarchical clustering algorithms for document dataset", Proceedings of the Eleventh International Conference on Information and Knowledge Management, McLean, Virginia, pp. 515-24.