Online supervised spam filter evaluation

ACM Transactions on Information Systems

Volumn 25, Issue 3, 2007, Pages

  Cormack, Gordon V ^a,b   Lynam, Thomas R ^a,b  

^a UNIVERSITY OF WATERLOO   (Canada)

^b UNIVERSITY OF WATERLOO   (Canada)

Author keywords

Email; Spam; Text classification

Indexed keywords

CLASSIFICATION (OF INFORMATION); COMPUTER AIDED SOFTWARE ENGINEERING; CONTENT BASED RETRIEVAL; ELECTRONIC MAIL; ENCAPSULATION; MESSAGE PASSING;

CHARACTERISTIC CURVE MEASUREMENTS; CONTENT-BASED FILTERS; E-MAIL MESSAGES; SPAM FILTERS;

ONLINE SYSTEMS;

EID: 34547480985     PISSN: 10468188     EISSN: 15582868     Source Type: Journal    
DOI: 10.1145/1247715.1247717     Document Type: Article

References (46)

1. AGRESTI, A. 1996. An Introduction to Categorical Data Analysis. Wiley, New York.
2. ANDROUTSOPOULOS, I., KOUTSIAS, J., CHANDRINOS, K., PALIOURAS, G., AND SPYROPOULOS, C. 2000a. An evaluation of naive Bayesian anti-spam filtering. In Proceedings of the Workshop on Machine Learning in the New Information Age, 11th European Conference on Machine Learning (ECML 2000), (Barcelona, Spain), G. Potamias, V. Moustakis, and M. van Someren, Eds. 9-17. Available: http://arXiv.org/abs/ cs.CL/0006013.
3. ANDROUTSOPOULOS, I., PALIOURAS, G., KARKALETSIS, V., SAKKIS, G., SPYROPOULOS, C., AND STAMATOPOULOS, P. 2000b. Learning to filter spam e-mail: A comparison of a naive Bayesian and a memory-based approach. In Proceedings of the Workshop on Machine Learning and Textual Information Access, 4th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD 2000), H. Zaragoza, P. Gallinari, and M. Rajman, Eds. (Lyon, France), 1-13. Available: http://arXiv.org/abs/cs/0009009.
4. ANDROUTSOPOULOS, I., PALIOURAS, G., AND MICHELAKIS, E. 2004. Learning to filter unsolicited commercial e-mail. Tech. Rep. 2004/2, NCSR Demokritos.
5. BURTON, B. 2002. Spamprobe - A fast Bayesian spam filter, http://spamprobe.sourceforge.net.
6. CORMACK, G. V. 2006. TREC 2006 Spam Track Overview. In Proceedings of the 15th Text REtrieval Conference (TREC-2005). NIST, Gaithersburg, MD.
7. CORMACK, G. V. AND BRATKO, A. 2006. Batch and online spam filter evaluation. In CEAS 2006 - The 3rd Conference on Email and Anti-Spam. Mountain View.
8. CORMACK, G. V. AND LYNAM, T. R. 2005a. Spam corpus creation for TREC. In Proceedings of CEAS 2005 - The Second Conference on Email and Anti-Spam.
9. CORMACK, G. V. AND LYNAM, T. R. 2005b. TREC 2005 Spam Track Overview. In Fourteenth Text REtrieval Conference (TREC-2005). NIST, Gaithersburg, MD.
10. DRUCKER, H., VAPNIK, V., AND WU, D. 1999. Support vector machines for spam categorization. IEEE Trans. Neural Netw. 10, 5, 1048-1054.
11. FAWCETT, T. 2003a. 'In Vivo' spam filtering: A challenge problem for data mining. KDD Explorations 5, 2 (December). Available: http://www.hpl.hp.com/personal/Tom_Fawcett/ papers/spam-KDDexp.pdf.
12. FAWCETT, T. 2003b. ROC graphs: Notes and practical considerations for data mining researchers. Tech. rep., HP Laboratories.
13. FISHER, R. A. 1925. Theory of statistical estimation. Proceedings of the Cambridge Philosophical Society 22, 700-725.
14. FLACH, P. A. 2004. The many faces of roc analysis in machine learning, http://www.cs.bris.ac.ukrflach/ICML04tutorial/.
15. GRAHAM, P. 2002. A plan for spam. http://www.paulgraham.com/ spam.html.
16. GRAHAM, P. 2004. Better Bayesian filtering. http://www. paulgraham.com/better.html.
17. GRAHAM-CUMMING, J. 2006. Does Bayesian poising exist? Virus Bulletin.
18. HIDALGO, J. M. G. 2002. Evaluating cost-sensitive unsolicited bulk email categorization. In Proceedings of SAC-02, 17th ACM Symposium on Applied Computing (Madrid, Spain) ACM, New York, 615-620.
19. HOLDEN, S. 2004. Spam filtering II. Hackin 9 (May/June), 68-77 http://sam.holden.id.au/ writings/spam2/.
20. HOLM, S. 1979. A simple sequentially rejective multiple test procedure. Scand. J. Stat. 6, 65-70.
21. KOHAVI, R. 1995. A study of cross-validation and bootstrap for accuracy estimation and model selection. In IJCAI. 1137-1145.
22. KOLCZ, A. AND ALSPECTOR, J. 2001. SVM-based filtering of e-mail spam with content-specific misclassification costs. In Proceedings of the TextDM'01 Workshop on Text Mining - held at the 2001 IEEE International Conference on Data Mining. IEEE Computer Society Press Los Alamitos, CA. (Available: http://www-ai.ijs.si/DunjaMladenic/TextDM01/papers/ Kolcz_TM.pdf.)
23. LENHARD, J. 2006. Models and statistical inference: The controversy between Fisher and Neyman-Pearson. Brit. J. Phil. Sci.
24. LEWIS, D. D. 1995. Evaluating and optimizing autonomous text classification systems. In SIGIR'95, Proceedings of the 18th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. (Seattle, WA, July 9-13) (Special Issue of the SIGIR Forum). E. A. Fox, P. Ingwersen, and R. Fidel, Eds. ACM, New York, 246-254.
25. LYNAM, T. AND CORMACK, G. 2005. TREC Spam Filter Evaluation Tool Kit. http://plg.uwaterloo. ca/~trlynam/spamjig.
26. MARTIN, A., DODDINGTON, G., KAMM, T., ORDOWSKI, M., AND PRZYBOCKI, M. 1997. The DET curve in assessment of detection task performance. In Proceedings of Eurospeech '97 (Rhodes, Greece), 1895-1898.
27. MEYER, T. AND WHATELEY, B. 2004. Spambayes: Effective open-source, bayesian based, email classification system. In Proceedings of the First Conference on Email and Anti-Spam, (CEAS). (Available: http://www.ceas.cc/papers-2004/136.pdf.)
28. PAMPAPATHI, R. M., MIRKIN, B., AND LEVENE, M. 2005. A suffix tree approach to email filtering. In Proceedings of the UK Workshop on Computational Intelligence (London, UK).
29. PARK, S. H., GOO, J. M., AND JO, C-H. 2004. Receiver Operating Characteristic (ROC) curve: Practical review for radiologists. Korean J. Radiology 5, 1, 11-18.
30. PETERS, T. 2004. Spambayes: Bayesian anti-spam classifier in python. http://spambayes. sourceforge.net/.
31. PROVOST, F., FAWCETT, T., AND KOHAVI, R. 1998. The case against accuracy estimation for comparing induction algorithms. In Proceedings of the 15th International Conference on Machine Learning. Morgan-Kaufmann, San Francisco, CA, 445-453.
32. RAYMOND, E. S., RELSON, D., ANDREE, M., AND LOUIS, G. 2004. Bogofilter. http://bogofilter.source- forge.net/.
33. ROBINSON, G. 2003. A statistical approach to the spam problem. Linux J. 107.
34. ROBINSON, G. 2004. Gary Robinson's spam rants. http://radio.weblogs.com/0101454/categories/ spam/.
35. ROTHMAN, K. J. AND GREENLAND, S. 1998. Modern Epidemiology. Lippinscott Williams and Wilkins.
36. SAHAMI, M., DUMAIS, S., HECKERMAN, D., AND HORVITZ, E. 1998. A Bayesian approach to filtering junk E-mail. In Learning for Text Categorization: Papers from the 1998 Workshop. AAAI Technical Report WS-98-05 (Madison, WI).
37. SAKKIS, G., ANDROUTSOPOULOS, I., PALIOURAS, G., KARKALETSIS, V., SPYROPOULOS, C. D., AND STAMATOPOULOS, P. 2001. Stacking classifiers for anti-spam filtering of e-mail. In Proceedings of EMNLP-01, 6th Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Morristown, US, Pittsburgh, US. Available: http://www.arxiv.org/abs/cs.CL/ 0106040.
38. SEBASTIANI, F. 2002. Machine learning in automated text categorization. ACM Comput. Surv. 34, 1, 1-47.
39. SPAMASSASSIN. 2004. The spamassassin public mail corpus. http://spamassassin.apache.org/ publiccorpus.
40. SPAMASSASSIN. 2005. The apache spamassassin project. http://spamassassin.apache.org.
41. STREINER, N. 1986. PDQ Statistics. B.C. Decker Inc.
42. TUTTLE, A., MILIOS, E., AND KALYANIWALLA, N. 2004. An evaluation of machine learning techniques for enterprise spam filters. Tech. Rep. CS-2004-03, Dalhousie University, Halifax, NS, Canada.
43. YERAZUNIS, B. 2004a. The spam-filtering plateau at 99.9In Proceedings of the Spam Conference. Available: http://crm114.sourceforge. net/Plateau-Paper.pdf.
44. YERAZUNIS, W. S. 2004b. CRM114 - the controllable regex mutilator. http://crm114.sourceforge.net/.
45. ZDZIARSKI, J. A. 2004. The DSPAM project. http://www. nuclearelephant.com/projects/dspam/.
46. ZHANG, L., ZHU, J., AND YAO, T. 2004. An evaluation of statistical spam filtering techniques. ACM Trans. Asian Lang. Inf Process. 3, 4, 243-269.