Chapter 3 Email Spam Filtering

Advances in Computers

Volumn 74, Issue , 2008, Pages 45-114

  Sanz, Enrique Puertas ^a,b,c   Gómez Hidalgo, José María ^a,b,c   Cortizo Pérez, José Carlos ^a,b,c  

^a UNIVERSIDAD EUROPEA DE MADRID   (Spain)

^b Optenet   (Spain)

^c AINet Solutions   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 48149108896     PISSN: 00652458     EISSN: None     Source Type: Book Series    
DOI: 10.1016/S0065-2458(08)00603-7     Document Type: Review

References (92)

1. Abadi M., Birrell A., Burrows M., Dabek F., and Wobber T. Bankable postage for network services. Proceedings of the 8th Asian Computing Science Conference, Mumbai, India (December 2003)
2. Ahn L.V., Blum M., and Langford J. How lazy cryptographers do AI. Communications of the ACM (February 2004)
3. Anderson R. Taking a bit out of spam. Network Computing (2004) Magazine Article, May 13
4. Androutsopoulos I., Koutsias J., Chandrinos K.V., Paliouras G., and Spyropoulos C.D. An evaluation of Naive Bayesian anti-spam filtering. Proceedings of the Workshop on Machine Learning in the New Information Age, 11th European Conference on Machine Learning (ECML), Barcelona, Spain (2000) 9-17
5. Androutsopoulos I., Paliouras G., Karkaletsis V., Sakkis G., Spyropoulos C.D., and Stamatopoulos P. Learning to filter spam e-mail: A comparison of a naive Bayesian and a memory-based approach. Proceedings of the Workshop on Machine Learning and Textual Information Access, 4th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD), Lyon, France (2000) 1-13
6. Androutsopoulos I., Koutsias J., Chandrinos K.V., and Spyropoulos C.D. An experimental comparison of naive Bayesian and keyword-based anti-spam filtering with encrypted personal e-mail messages. Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Athens, Greece, ACM Press, New York, US (2000) 160-167
7. Belkin N.J., and Croft W.B. Information filtering and information retrieval: Two sides of the same coin?. Communications of the ACM 35 12 (1992) 29-38
8. Bell S. Filters causing rash of false positives: TelstraClear's new virus and spam screening service gets mixed reviews (2003). http://computerworld.co.nz/news.nsf/news/CC256CED0016AD1ECC-256DAC000D90 D4?Opendocument
9. Bickel S. ECML/PKDD discovery challenge 2006 overview. Proceedings of the Discovery Challenge Workshop, 17th European Conference on Machine Learning (ECML) and 10th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD), Berlin, Germany (September 2006)
10. Biggio B., Giorgio F., Ignazio P., and Fabio R. Image spam filtering by content obscuring detection. Proceedings of the Fourth Conference on Email and Anti-Spam (CEAS 2007), August 23, Microsoft Research Silicon Valley, Mountain View, California (2007) 2-3
11. Bratko A., Cormack G.V., Filipic B., Lynam T.R., and Zupan B. Spam filtering using statistical data compression models. Journal of Machine Learning Research 7 (Dec 2006) 2699-2720
12. Byun B., Lee C.-H., Webb S., and Calton P. A discriminative classifier learning approach to image modeling and spam image identification. Proceedings of the Fourth Conference on Email and Anti-Spam (CEAS 2007), August 2-3, Microsoft Research Silicon Valley, Mountain View, California (2007)
13. Caropreso M.F., Matwin S., and Sebastiani F. A learner-independent evaluation of the usefulness of statistical phrases for automated text categorization. In: Chin A.G. (Ed). Text Databases and Document Management: Theory and Practice (2001), Idea Group Publishing, Hershey, US 78-102
14. Carreras X., and Márquez L. Boosting trees for anti-spam email filtering. Proceedings of RANLP-2001, 4th International Conference on Recent Advances in Natural Language Processing (2001)
15. Caruso J. Anti-spam law just a start, panel says. Networld World (December 2003). http://www.networkworld.com/news/2003/1218panel.html
16. Claburn T. Constant struggle: How spammers keep ahead of technology, Message Pipeline (2005). http://www.informationweek.com/software/messaging/57702892
17. Cleary J.G., and Teahan W.J. Unbounded length contexts for PPM. The Computer Journal 40 2/3 (1997) 67-75
18. Cohen W.W., and Hirsh H. Joins that generalize: Text classification using WHIRL. Proceedings of KDD-98, 4th International Conference on Knowledge Discovery and Data Mining, New York, NY (1998) 169-173
19. Cormack G.V., and Lynam T.R. TREC 2005 spam track overview. Proc. TREC 2005 - the Fourteenth Text REtrieval Conference, Gaithersburg (2005)
20. Cormack G.V., and Lynam T.R. Spam corpus creation for TREC. Proc. CEAS 2005 - The Second Conference on Email and Anti-spam, Palo Alto (July 2005)
21. Cormack G.V., and Bratko A. Batch and on-line spam filter evaluation. CEAS 2006 - Third Conference on Email and Anti-spam, Mountain View (July 2006)
22. Cormack G., Gómez Hidalgo J.M., and Puertas Sanz E. Spam filtering for short messages. ACM Sixteenth Conference on Information and Knowledge Management (CIKM 2007), Lisboa, Portugal (November 6-9, 2007)
23. Cosoi C.A. The medium or the message? Dealing with image spam. Virus Bulletin (December 2006). http://www.virusbtn.com/spambulletin/archive/2006/12/sb200612-image-spam .dkb
24. Cranor L.F., and LaMacchia B.A. Spam. Communications of the ACM 41 8 (1998) 74-83
25. Dalvi N., Domingos P., Sanghai M.S., and Verma D. Adversarial classification. Proceedings of the Tenth International Conference on Knowledge Discovery and Data Mining, ACM Press, Seattle, WA (2004) 99-108
26. Deerwester S., Dumais S.T., Furnas G.W., Landauer T.K., and Harshman R. Indexing by latent semantic indexing. Journal of the American Society for Information Science 41 6 (1990) 391-407
27. Domingos P., and Pazzani M.J. On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning 29 2-3 (1997) 103-130
28. Domingos P. MetaCost: A general method for making classifiers cost-sensitive. Proceedings of the Fifth International Conference on Knowledge Discovery and Data Mining, ACM Press, San Diego, CA (1999) 155-164
29. Drucker H., Vapnik V., and Wu D. Support vector machines for spam categorization. IEEE Transactions on Neural Networks 10 5 (1999) 1048-1054
30. Dwork C., Goldberg A., and Naor M. On memory-bound functions for fighting spam. Proceedings of the 23rd Annual International Cryptology Conference (CRYPTO 2003) (August 2003)
31. Fawcett T. "In vivo" spam filtering: A challenge problem for KDD. SIGKDD Explorations 5 2 (2003) 140-148
32. Fuhr N., Hartmann S., Knorz G., Lustig G., Schwantner M., and Tzeras K. AIR/X-a rule-based multistage indexing system for large subject fields. Proceedings of RIAO-91, 3rd International Conference "Recherche d'Information Assistee par Ordinateur", Barcelona, Spain (1991) 606-623
33. Garcia F.D., Hoepman J.-H., and van Nieuwenhuizen J. Spam filter analysis. In: Deswarte Y., Cuppens F., Jajodia S., and Wang L. (Eds). Security and Protection in Information Processing Systems. IFIP TC11 19th International Information Security Conference (SEC2004), Toulouse, France (2004) 395-410
34. Gee K., and Cook D.J. Using latent semantic Iidexing to filter spam. ACM Symposium on Applied Computing (2003) Data Mining Track
35. Gómez-Hidalgo J.M. Evaluating cost-sensitive unsolicited bulk email categorization. Proceedings of SAC-02, 17th ACM Symposium on Applied Computing, Madrid, ES (2002) 615-620
36. Gómez-Hidalgo J.M., Maña-López M., and Puertas-Sanz E. Evaluating cost-sensitive unsolicited bulk email categorization. Proceedings of JADT-02, 6th International Conference on the Statistical Analysis of Textual Data, Madrid, ES (2002)
37. Graham-Cumming J. The Spammer's Compendium. MIT Spam Conference (2003)
38. Graham-Cumming J. How to beat an adaptive spam filter. MIT Spam Conference (2004)
39. Graham-Cumming J. The rise and rise of image-based spam. Virus Bulletin (November 2006)
40. Graham P. A plan for spam. Hackers and Painters, Big Ideas from the Computer Age (2002)
41. O'Really. Available (2004). http://www.paulgraham.com/spam.html Reprinted in Paul Graham
42. Graham P. Better Bayesian filtering. Proceedings of the 2003 Spam Conference (January 2003). http://www.paulgraham.com/better.html Available
43. Gray A., and Haahr M. Personalised, collaborative spam filtering. Proceedings of the First Conference on Email and Anti-Spam (CEAS) (2004)
44. Hahn J. Image-based spam makes a comeback. Web Trends (2006). http://www.dmconfidential.com/blogs/column/Web_Trends/916/
45. Hall R.J. How to avoid unwanted email. Communications of the ACM (March 1998)
46. Hovold J. Naive Bayes spam filtering using word-position-based attributes. Proceedings of the Second Conference on Email and Anti-spam, CEAS, Stanford University (2005)
47. InfoWorld Test Center. Strong spam combatants: Brute anti-spam force takes on false-positive savvy (2004) Issue 22 May 31, 2004
48. Jeffrey E., and Friedl F. Mastering Regular Expressions. 3rd edn. (August 2006) O'Really
49. Joachims T. Text categorization with support vector machines: learning with many relevant features. Proceedings of ECML-98, 10th European Conference on Machine Learning, Chemnitz, Germany (1998) 137-142
50. Joachims T. Transductive inference for text classification using support vector machines. Proceedings of ICML-99, 16th International Conference on Machine Learning, Bled, Slovenia (1999) 200-209
51. Keogh E., Lonardi S., and Ratanamahatana C.A. Towards parameter-free data mining. Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, WA, USA, August 22-25, 2004). KDD '04, ACM, New York, NY (2004) 206-215
52. Kolcz A., Chowdhury A., and Alspector J. The impact of feature selection on signature-driven spam detection. Proceedings of the First Conference on Email and Anti-Spam (CEAS) (2004)
53. Larkey L.S., and Croft W.B. Combining classifiers in text categorization. In: Frei H.-P., Harman D., Schäuble P., and Wilkinson R. (Eds). Proceedings of SIGIR-96, 19th ACM International Conference on Research and Development in Information Retrieval, ACM Press, New York, US, Zurich, CH (1996) 289-297
54. Lewis D.D. Naive (Bayes) at forty: The independence assumption in information retrieval. In: Nédellec C., and Rouveirol C. (Eds). Proceedings of ECML-98, 10th European Conference on, Machine Learning, Springer Verlag, Heidelberg, DE, Chemnitz, DE. Lecture Notes in Computer Science 1398 (1998) 4-15
55. Li Y.H., and Jain A.K. Classification of text documents. The Computer Journal 41 8 (1998) 537-546
56. Lowd D., and Meek C. Adversarial Learning. Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), ACM Press, Chicago, IL (2005)
57. Lucas M.W. PGP & GPG: email for the Practical Paranoid. No Starch Press (2006)
58. McCallum A., and Nigam K. A comparison of event models for Naive Bayes text classification. Proceedings of the AAAI-98 Workshop on Learning for Text Categorization (1998)
59. Labs M. Message Labs Intelligence: 2006 Annual Security Report (2006). http://www.messagelabs.com/mlireport/2006_annual_security_report_5.pdf Available
60. Meyer T.A., and Whateley B. Spambayes: Effective open-source, bayesian based, email classification system. Proceedings of the First Conference on Email and Anti-spam (CEAS) (2004)
61. Mitchell T.M. Machine learning (1996), McGraw Hill, New York, US
62. O'Brien C., and Vogel C. Spam filters: Bayes vs. chi-squared; letters vs. words. Proceedings of the International Symposium on Information and Communication Technologies (September 2003)
63. Pantel P., and Lin D. Spamcop: A spam classification and organization program. Learning for Text Categorization: Papers from the 1998 Workshop, Madison, Wisconsin. AAAI Technical Report WS-98-05 (1998)
64. Platt J. Fast training of support vector machines using sequential minimal optimization. In: Schölkopf B., Burges C., and Smola A. (Eds). Advances in Kernel Methods - Support Vector Learning (1998)
65. Provost F., and Fawcett T. Analysis and visualization of classifier performance: Comparison under imprecise class and cost distributions. Proceedings of the Third International Conference on Knowledge Discovery and Data Mining (1997)
66. Provost J. Naive-bayes vs. rule-learning in classification of email. Technical report Department of Computer Sciences at the University of Texas Austin (1999)
67. Quinlan R. Induction of decision trees. Machine Learning 1 1 (1986) 81-106
68. Rigoutsos I., and Huynh T. Chung-kwei: A pattern-discovery-based system for the automatic identification of unsolicited e-mail messages (spam). Proceedings of the First Conference on Email and Anti-Spam (CEAS) (2004)
69. Saarinen J. Spammer ducks for cover as details published on the web, NZHerald (2003). http://www.nzherald.co.nz/section/1/story.cfm?c_id=1&objectid=351868 2
70. Sahami M., Dumais S., Heckerman D., and Horvitz E. A Bayesian approach to filtering junk e-mail. Proceedings of the AAAI-98 Workshop on Learning for Text Categorization, AAAI Press, Madison, WI (1998)
71. Sakkis G., Androutsopoulos I., Paliouras G., Karkaletsis V., Spyropoulos C.D., and Stamatopoulos P. Stacking classifiers for anti-spam filtering of e-mail. Proceedings of EMNLP-01, 6th Conference on Empirical Methods in Natural Language Processing, Pittsburgh, US (2001), Association for Computational Linguistics, Morristown, US
72. Salton G. A blueprint for automatic indexing. SIGIR Forum 16 2 (1981) 22-38
73. Salton G., and McGill M.J. Introduction to Modern Information Retrieval (1983), McGraw Hill, New York, US
74. Samosseiko D., and Thomas R. The game goes on: An analysis of modern spam techniques. Proceedings of the 16th Virus Bulletin International Conference (2006)
75. Sculley D., and Brodley C.E. Compression and machine learning: a new perspective on feature space vectors. Data Compression Conference (DCC'06) (2006) 332-341
76. Sebastiani F. Machine learning in automated text categorization. ACM Computing Surveys 34 1 (2002) 1-47
77. Seigneur J.-M., and Jensen C.D. Privacy recovery with disposable email addresses. IEEE Security and Privacy 1 6 (2004) 35-39
78. Sergeant M. Internet-level spam detection and SpamAssassin 2.50. Spam Conference (2003)
79. Stern H., Mason J., and Shepherd M. A linguistics-based attack on personalized statistical e-mail classifiers. Technical report CS-2004-06, Faculty of Computer Science (March 25, 2004), Dalhousie University, Canada
80. Taylor B. Sender reputation in a large webmail service. Proceedings of the Third Conference on Email and Anti-Spam (CEAS), Mountain View, California (2006)
81. Teahan W.J., and Harper D.J. Using compression based language models for text categorization. In: Croft W.B., and Laferty J. (Eds). Language Modeling for Information Retrieval, The Kluwer International Series on Information Retrieval, Kluwer Academic Publishers (2003)
82. Theo V.D. New and upcoming features in SpamAssassin v3, ApacheCon (2004)
83. Turing A.M. Computing machinery and intelligence. Mind 59: (1950) 433-460
84. Watson B. Beyond identity: Addressing problems that persist in an electronic mail system with reliable sender identification. Proceedings of the First Conference on Email and Anti-Spam (CEAS), Mountain View, CA (2004)
85. Wiener E.D., Pedersen J.O., and Weigend A.S. A neural network approach to topic spotting. Proceedings of SDAIR-95, 4th Annual Symposium on Document Analysis and Information Retrieval, Las Vegas, US (1995) 317-332
86. Wittel G.L., and Wu F. On attacking statistical spam filters. Proceedings of the Conference on Email and Anti-spam (CEAS) (2004)
87. Witten I.H., and Frank E. Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations (2000), Morgan Kaufmann, Los Altos, US
88. Yang Y., and Pedersen J.O. A comparative study on feature selection in text categorization. In: Fisher D.H. (Ed). Proceedings of ICML-97, 14th International Conference on Machine Learning (1997)
89. Yerazunis B. Sparse binary polynomial hash message filtering and the CRM114 discriminator. Proceedings of the Spam Conference (2003)
90. Yerazunis B. The plateau at 99.9. Proceedings of the Spam Conference (2004)
91. Zdziarski J. Advanced language classification using chained tokens. Proceedings of the Spam Conference (2004)
92. Zimmermann P.R. The Official PGP User's Guide (1995), MIT Press