Email data cleaning

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn , Issue , 2005, Pages 489-498

  Tang, Jie ^a   Li, Hang ^b   Cao, Yunbo ^b   Tang, Zhaohui ^c  

^a TSINGHUA UNIVERSITY   (China)

^b MICROSOFT RESEARCH ASIA   (China)

^c MICROSOFT   (United States)

Author keywords

Data Cleaning; Email Processing; Statistical Learning; Text Mining

Indexed keywords

DATA ACQUISITION; DATA MINING; LEARNING SYSTEMS; STATISTICAL METHODS; TEXT PROCESSING;

DATA CLEANING; EMAIL PROCESSING; STATISTICAL LEARNING; TEXT MINING;

ELECTRONIC MAIL;

EID: 32344435743     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1081870.1081926     Document Type: Conference Paper

References (35)

1. A. L. Berger, S. A. Della Pietra, and V. J. Della Pietra. A Maximum Entropy Approach to Natural Language Processing. Computational Linguistics, 22:39-71, 1996.
2. E. Brill and R. C. Moore. An Improved Error Model for Noisy Channel Spelling Correction. In Proc. of the 38th Annual Meeting of the ACL, 2000, pages 286-293.
3. H. L. Chieu and H. T. Ng. A Maximum Entropy Approach to Information Extraction from Semi-Structured and Free Text. In Proc. of Eighteenth National Conference on Artificial Intelligence, 2002.
4. K. Church and W. Gale. Probability Scoring for Spelling Correction. In Statistics and Computing, Vol. 1, 1991, pages 93-103.
5. A. Clark. Pre-processing Very Noisy Text. In Proc. of Workshop on Shallow Processing of Large Corpora. Corpus Linguistics 2003, Lancaster. 2003.
6. M. Collins. Discriminative Training Methods for Hidden Markov Models: Theory and Experiments with Perceptron Algorithms. In Proc. of Conference on Empirical Methods in NLP, 2002.
7. C. Cortes and V. Vapnik. Support-Vector Networks. Machine Learning 20:273-297, 1995
8. N. Ducheneaut and V. Bellotti. E-mail as Habitat: An Exploration of Embedded Personal Information Management. Interactions, 8, pages 30-38.
9. W. A. Gale, K. W. Church, and D. Yarowsky. 1994. Discrimination Decisions for 100,000-Dimensional Spaces. Current Issues in Computational Linguistics: In Honour of Don Walker. Kluwer Academic Publishers, pages 429-450.
10. H. Galhardas, D. Florescu, D. Shasha, and E. Simon. Declaratively Cleaning Your Data Using AJAX. In Journees Bases de Donnees, Oct. 2000. http://caravel.inria.fr/galharda/BDA.ps.
11. Z. Ghahramani and M. I. Jordan. Factorial Hidden Markov Models. Machine Learning, 29:245-273. 1997
12. A. R. Golding and D. Roth. Applying Winnow to Context-Sensitive Spelling Correction. In Proc. of the 13th International Conference on Machine Learning (ICML 1996).
13. M. A. Hernández and S. J. Stolfo. Real-world Data is Dirty: Data Cleansing and The Merge/Purge Problem. Publisher Kluwer Academic Publishers Hingham, MA, USA. Vol. 2, Issue 1, January 1998, pages 9-37.
14. N. Kushmerick. Learning to Remove Internet Advertisement. In Proc. of the Third International Conference on Autonomous Agents (Agents'99). ACM Press. Seattle, WA, USA. 1999, pages 175-181.
15. J. Lafferty, A. McCallum, and F. Pereira. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In Proc. of ICML 01, 2001.
16. S. H. Lin and J. M. Ho. Discovering Informative Content Blocks from Web Documents. In Proc. of SIGKDD-02, 2002.
17. L. V. Lita, A. Ittycheriah, S. Roukos, and N. Kambhatla. tRuEcasIng. In Proc. of the 41st Annual Meeting of the Association for Computational Linguistics (ACL 2003), July 7-12, Sapporo, Japan.
18. E. Mays, F. J. Damerau, and R. L. Mercer. Context Based Spelling Correction. Information Processing and Management: an International Journal, v.27 n.5, pages 517-522, 1991
19. A. McCallum, D. Freitag, and F. Pereira. Maximum Entropy Markov Models for Information Extraction and Segmentation, In Proc. of the ICML Coference, 2000.
20. A. Mikheev. Periods, Capitalized Words, etc. Computational Linguistics, 28(3):289-318, 2002.
21. H. T. Ng, C. Y. Lim, J. L. T. Koo. Learning to Recognize Tables in Free Text. In Proc. of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics (ACL'99). 1999, pages 443-450.
22. D. D. Palmer and M. A. Hearst. Adaptive Multilingual Sentence Boundary Disambiguation. Computational Linguistics. MIT Press Cambridge, MA, USA. Vol. 23, Issue 2 (June 1997), pages 241-267.
23. C. Parent, and S. Spaccapietra. Issues and Approaches of Database Integration. Comm. ACM 41(5):166-178, 1998.
24. D. Pinto, A. McCallum, X. Wei, and W. B. Croft. Table Extraction Using Conditional Random Fields. In Proc. of the 26th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 2003.
25. E. Rahm and H. H. Do. Data Cleaning: Problems and Current Approaches, IEEE Bulletin of the Technical Committee on Data Engineering, Vol. 23 No. 4, December 2000
26. A. Ratnaparkhi. Unsupervised Statistical Models for Prepositional Phrase Attachment. In Proc. of COLINGACL98. Montreal, Canada, 1998.
27. R. Sproat, A. Black, S. Chen, S. Kumar, M. Ostendorf, and C. Richards. Normalization of non-standard words. WS'99 Final Report.http://www.clsp.jhu.edu/ws99/projects/normal/.
28. V. Vapnik. Statistical Learning Theroy. Springer Verlage, New York, 1998.
29. Y. Wang and J. Hu. A Machine Learning based Approach for Table Detection on the Web. In Proc. of The Eleventh International World Wide Web Conference (WWW2002), Honolulu, Hawaii, USA, May 2002
30. E. Xun, C. Huang, and M. Zhou. A Unified Statistical Model for the Identification of English baseNP. In Proc. of The 38th Annual Meeting of the Association for Computational Linguistics (ACL), Hong Kong, 3-6 October 2000
31. L. Yi and B. Liu, and X. Li. Eliminating Noisy Information in Web Pages for Data Mining. In Proc. of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD-2003), Washington, DC, USA, August, 2003, pages 24-27.
32. L. Zhang, Y. Pan, and T. Zhang. Recognising and Using Named Entities: Focused Named Entity Recognition Using Machine Learning. In Proc. of SIGIR'04, 2004.
33. J. Decker. eClean 2000, http://www.jdsoftware.com/eClean2000/index.html.
34. ListCleaner Pro. http://www.WinPure.com/EmailCleaner.
35. Fuzzy Lookup and Fuzzy Grouping in Data Transformation Services for SQL Server 2005. http://msdn.microsoft.com/library/default.asp?url=/library/en -us/dnsql90/html/FzDTSSQL05.asp