How much noise is too much: A study in automatic text classification

Proceedings - IEEE International Conference on Data Mining, ICDM

Volumn , Issue , 2007, Pages 3-12

  Agarwal, Sumeet ^a   Godbole, Shantanu ^b   Roy, Shourya ^b   Punjani, Diwakar ^c  

^a INDIAN INSTITUTE OF TECHNOLOGY DELHI   (India)

^b IBM INDIA RESEARCH LAB   (India)

^c CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATIC TEXT CLASSIFICATION; BENCHMARK DATASETS; CONTACT CENTERS; DATA CLEANING; DATA-SETS; INFORMATION EXTRACTION; INTERNATIONAL CONFERENCES; MOBILE PHONES; NEWSGROUPS; PRE-PROCESSING; REAL-LIFE DATA; REUTERS-21578; RULE LEARNING; SIMULATED NOISE; TEXT CLASSIFICATION; TEXT CLASSIFIERS; TEXT-MINING;

ADMINISTRATIVE DATA PROCESSING; CLASSIFICATION (OF INFORMATION); CLEANING; DATA MINING; DATA PROCESSING; DECISION SUPPORT SYSTEMS; INFORMATION MANAGEMENT; INTERNET; MINING; SEARCH ENGINES; TELECOMMUNICATION; TELECOMMUNICATION EQUIPMENT;

TEXT PROCESSING;

EID: 49749119987     PISSN: 15504786     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDM.2007.21     Document Type: Conference Paper

References (27)

1. L. R. Bahl, S. Balakrishnan-Aiyer, J. Bellegarda, M. Franz, P. Gopalakrishnan, D. Nahamoo, M. Novak, M. Padmanabhan, M. Picheny, and S. Roukos. Performance of the IBM large vocabulary continuous speech recognition system on the ARPA wall street journal task. In Proc. ICASSP '95, pages 41-44, Detroit, MI, 1995.
2. T. Bayer, U. Kressel, H. Mogg-Schneider, , and I. Renz. Categorizing paper documents. In Computer Vision and Image Understanding, volume 70, pages 299-306, 1998.
3. D. Blei, A. Ng, and M. Jordan. Latent dirichlet allocation. In Proc. of NIPS 14, 2002.
4. M. Choudhury, R. Saraf, V. Jain, S. Sarkar, and A. Basu. Investigation and modeling of the structure of texting language. In Proc. of AND07 Workshop in conjunction with IJCAI, 2007.
5. F. J. Damerau. A technique for computer detection and correction of spelling errors. Commun. ACM, 7(3):171-176, 1964.
6. R. O. Duda, P. E. Hart, and D. G. Stork. Pattern Classification. John Wiley and Sons, Inc., New York, 2000.
7. Y. Freund and R. Schapire. A short introduction to boosting. In Japanese Society for AI 14(5), 771-780. 11, 1999.
8. P. Haffner, G. Tur, and J. Wright. Optimizing svms for complex call classification. In Proc. of ICASSP, 2003.
9. T. Hofmann. Probabilistic latent semantic analysis. In Proc. of UAI, 1999.
10. T. Joachims. Making large-scale support vector machine learning practical. In A. S. B. Schölkopf, C. Burges, editor, Advances in Kernel Methods: Support Vector Machines. MIT Press, Cambridge, MA, 1998.
11. K. Kukich. Technique for automatically correcting words in text. ACM Comput. Surv., 24(4):377-439, 1992.
12. V. I. Levenshtein. Binary codes capable of correcting deletions, insertions, and reversals. Technical Report 8, 1966.
13. D. D. Lewis. Evaluating text categorization. In HLT '91: Proceedings of the workshop on Speech and Natural Language, pages 312-318, Morristown, NJ, USA, 1991.
14. D. D. Lewis, R. Ghani, D. Mladenic, I. Moulinier, and M. Wasson. In 3rd Workshop on Operational Text Classification (OTC), in conjunction with SIGKDD, 2003.
15. A. K. McCallum. Bow: A toolkit for statistical language modeling, text retrieval, classification and clustering. http://www.cs.cmu.edu/mccallum/bow, 1996.
16. M. Michelson and C. A. Knoblock. Semantic annotation of unstructured and ungrammatical text. In Proc. of IJCAI, pages 1091-1098, 2005.
17. D. Miller, S. Boisen, R. Schwartz, R. Stone, and R. Weischedel. Named entity extraction from noisy input: speech and ocr. In Proc. of the sixth conference on Applied natural language processing, pages 316-324, San Francisco, CA, USA, 2000. Morgan Kaufmann Publishers Inc.
18. T. Nartker, K. Taghva, R. Young, J. Borsack, and A. Condit. Ocr correction based on document level knowledge. In Proc. of Intl. Symp. on Electronic Imaging Science and Technology, pages 103-110, Santa Clara, CA, 2003.
19. T. Nasukawa, D. Punjani, S. Roy, L. V. Subramaniam, and H. Takeuchi. Adding sentence boundaries to conversational speech transcriptions using noisily labeled examples. In Proc. of AND07 Workshop in conjunction with IJCAI, 2007.
20. K. Nigam, J. Lafferty, and A. McCallum. Using maximum entropy for text classification. In Proc. of IJCAI-99 Workshop on Machine Learning for Information Filtering, 1999.
21. L. R. Rabiner and B. H. Juang. Fundamentals of speech recognition. Prentice-Hall Inc., New Jersey, 1993.
22. S. Roy and L. V. Subramaniam. Automatic generation of domain models for call-centers from noisy transcriptions. In Proc. of ACL-COLING, 2006.
23. A. Sarma and D. Palmer. Context-based speech recognition error detection and correction. In Proc. of HLT-NAACL, 2004.
24. A. Vinciarelli. Noisy text categorization. In Proc. of ICPR'04 Volume 2, pages 554-557, Washington, DC, USA, 2004. IEEE Computer Society.
25. A. Vlachos. Active annotation. In Proc. of the EACL 2006 Workshop on Adaptive Text Extraction and Mining, 2006.
26. J. Zhang and Y. Yang. Robustness of regularized linear classification methods in text categorization. In Proc. of SIGIR, 2003.
27. T. Zhang and F. Oles. A probability analysis on the value of unlabeled data for classification problems. In Proc. of ICML, 2000.