An ontology enhanced parallel SVM for scalable spam filter training

Neurocomputing

Volumn 108, Issue , 2013, Pages 45-57

  Caruana, Godwin ^a   Li, Maozhen ^a,b   Liu, Yang ^a  

^a BRUNEL UNIVERSITY   (United Kingdom)

^b Ministry of Education   (China)

Author keywords

Classification; MapReduce; Parallel computing; Spam filtering; Support vector machine

Indexed keywords

ACCURACY LEVEL; COMPUTER NODES; MAP-REDUCE; ONTOLOGY SEMANTICS; ONTOLOGY-BASED; SPAM FILTERING; SUPPORT VECTOR MACHINE TECHNIQUES; SVM CLASSIFIERS;

CLASSIFICATION (OF INFORMATION); INTERNET; PARALLEL ARCHITECTURES; PARALLEL PROCESSING SYSTEMS; SEMANTICS;

SUPPORT VECTOR MACHINES;

ACCURACY; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATION; CLASSIFICATION; CLASSIFICATION ALGORITHM; CLASSIFIER; E-MAIL; MACHINE LEARNING; PRIORITY JOURNAL; SEMANTICS; SEQUENTIAL MINIMAL OPTIMIZATION ALGORITHM; SPAM FILTERING TRAINING; SUPPORT VECTOR MACHINE;

EID: 84875111175     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2012.12.001     Document Type: Article

References (69)

1. Blanzieri E., Bryl A. A survey of learning-based techniques of email spam filtering. Artif. Intell. Rev. 2008, 29(1):63-92.
2. Blanco Á., Ricket A., Martín-Merino M., Sandoval F., Prieto A., Cabestany J., Graña M. Computational and Ambient Intelligence 2007, 4507. Berlin, Heidelberg: Springer, Berlin Heidelberg, 903-910.
3. K.-L. K. Li, H.-K. Huang, S.-F. Tian, L. Kun-Lun, L. Kai, H. Hou-Kuan, and T. Sheng-Feng, Active learning with simplified SVMs for spam categorization, in: Proceedings. International Conference on Machine Learning and Cybernetics, vol. 3, pp. 1198-1202, 2002.
4. Mei L., Cheng-qing Y. Anti-spam technique with SVM and K-NN cooperation method. Comput. Eng. Appl. 2008, 44(4):135-137.
5. Zhang L., Zhu J., Yao T. An evaluation of statistical spam filtering techniques. ACM Trans. Asian Lang. Inf. Process. (TALIP) 2004, 3(4):243-269.
6. Graf H.P., Cosatto E., Bottou L.L., Durdanovic I., Vapnik V. Parallel support vector machines: the cascade svm. In Adv. Neural Inf. Process. Syst. 2005, 521-528.
7. Huang G., Mao K.Z., Siew C., Huang D. S. Member, Fast modular network implementation for support vector machines. IEEE Trans. Neural Networks 2005, 16:1651-1663.
8. T.-N. Do and F. Poulet, Classifying one billion data with a new distributed svm algorithm, in: Proceedings of the International Conference on Research, Innovation and Vision for the Future, pp. 59-66, 2006.
9. E.Y. Chang, K. Zhu, H. Wang, H. Bai, J. Li, and Z. Qiu, PSVM: parallelizing support vector machines on distributed computers, in: Proceedings of the 21st Annual Conference on Neural Information Processing Systems, 2007.
10. Cristianini N., Shaw-Taylor J. An Introduction to SVM's and Other Kernel-based Learning Methods 2000, Cambridge University Press.
11. Platt J.C. Fast training of support vector machines using sequential minimal optimization. Advances in Kernel Methods: Support Vector Machines 1999, 185-208. B. Schölkopf, C.J.C. Burges, A.J. Smola (Eds.).
12. F.R. Bach and M.I. Jordan, Predictive low-rank decomposition for kernel methods, in: Proceedings of the 22nd International Conference, pp. 33-40, 2005.
13. Suykens J.A.K., Vandewalle J. Least squares support vector machine classifiers. Neural Process. Lett. 1999, 9(3):293-300.
14. B. Catanzaro, N. Sundaram, and K. Keutzer, Fast support vector machine training and classification on graphics processors, in: Proceedings of the 25th International Conference on Machine Learning-ICML'08, pp. 104-111, 2008.
15. Do T.-N., Nguyen V.-H., Poulet F. Advanced Data Mining and Applications. Berlin Heidelberg 2008, vol. 5139. Berlin, Heidelberg: Springer, pp. 147-157.
16. H. Yu, J. Yang, and J. Han, Classifying large data sets using SVMs with hierarchical clusters, in: Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining-KDD'03, pp. 306-315, 2003.
17. Dean J., Ghemawat S. MapReduce. Commun. ACM 2008, 51(1):107. Jan.
18. N. A. Syed, S. Huan, L. Kah, and K. Sung, Incremental Learning with Support Vector Machines, 1999. [Online]. Available: (accessed: 03-Aug-2010). http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.46.6367.
19. G. Caruana, M. Li, and H. Qi, SpamCloud: A MapReduce based anti-spam architecture, in Proceedings of the Seventh International Conference on Fuzzy Systems and Knowledge Discovery, vol. 6, pp. 3003-3006, 2010.
20. G. Caruana, M. Li, and M. Qi, A MapReduce based parallel SVM for large scale spam filtering, in: Proceedings of the Eighth International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), pp. 2659-2662, 2011.
21. E. Prud'hommeaux and A. Seaborne, SPARQL Query Language for RDF (Working Draft), , 2005. (accessed:07-Oct-2012). http://www.w3.org/TR/2005/WD-rdf-sparql-query-20051123.
22. S.C. for B. I. Research, Protege [Online]. Available: (accessed: 01-Oct-2011). http://protege.stanford.edu/.
23. Caruana G., Li M. A survey of emerging approaches to spam filtering. ACM Comput. Surv. 2012, 44(2):1-27.
24. PoweredBy-Hadoop Wiki. [Online]. Available: http://wiki.apache.org/hadoop/PoweredBy.
25. RSA Conference Connect HT1-301: Yokai vs. the Elephant: Hadoop and the Fight Against Shape-Shifting Spam (PK Session). [Online]. Available: (accessed: 28-Jul-2012). http://365.rsaconference.com/community/connect/blog/2010/02/12/ht1-301-yokai-vs-the-elephant-hadoop-and-the-fight-against-shape-shifting-spam-pk-session.
26. A.S. Das, M. Datar, A. Garg, and S. Rajaram, Google news personalization, in: Proceedings of the 16th International Conference on World Wide Web-WWW '07, pp. 271-280, 2007.
27. M.G. Noll and C. Meinel, Building a scalable collaborative web filter with free and open source software, in: Proceedings of the IEEE International Conference on Signal Image Technology and Internet Based Systems, pp. 563-571, 2008.
28. Colas F., Brazdil P., Bramer M. Artificial Intelligence in Theory and Practice 2006, 217. Springer, US, 169-178.
29. M. Ye, T. Tao, F.-J. Mai, and X.-H. Cheng, A spam discrimination based on mail header feature and SVM, in: 2008 Fourth International Conference on Wireless Communications, Networking and Mobile Computing, pp. 1-4, 2008.
30. B. Scholkopf and A.J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond, 2001.
31. Zanghirati G., Zanni L. A parallel solver for large quadratic programs in training support vector machines. Parallel Comput. 2003, 29(4):535-551.
32. Cao L.J., Keerthi S.S., Ong C.-J.J., Zhang J.Q., Periyathamby U., Fu X.J.J., Lee H.P. Parallel sequential minimal optimization for the training of support vector machines. IEEE Trans. Neural Networks 2006, 17(4):1039-1049.
33. Woodsend K., Gondzio J. Hybrid MPI/OpenMP parallel linear support vector machine training. J. Mach. Learn. Res. 2009, 10:1937-1953.
34. B. C. Kuszmaul, Cilk provides the 'best overall productivity' for high performance computing, in: Proceedings of the 19th Annual ACM Symposium on Parallel Algorithms and Architectures-SPAA'07, pp. 299-300, 2007.
35. J. Yang, X. Yang, and J. Zhang, A parallel multi-class classification support vector machine based on sequential minimal optimization, in: Proceedings of the 1st International Multi-Symposiums on Computer and Computational Sciences (IMSCCS'06), pp. 443-446, 2006.
36. C.-T. T. Chu, S. K. Kim, Y.-A. A. Lin, Y. Yu, G. R. Bradski, A. Y. Ng, and K. Olukotun, Map-reduce for machine learning on multicore, in: Proceedings of the 20th Annual Conference on Neural Information Processing Systems (NIPS), pp. 281-288, 2006.
37. Kim J., Dou D., Liu H., Kwak D. Advances in Artificial Intelligence 2007, 4509. Springer, Berlin, Heidelberg, pp. 272-283.
38. I.H. Witten, E. Frank, and M.A. Hall, Data Mining: Practical Machine Learning Tools and Techniques, 2011.
39. S. Youn and D. McLeod, Efficient spam email filtering using adaptive ontology, in: Proceedings of the Fourth International Conference on Information Technology, pp. 249-254, 2007.
40. M. Balakumar and V. Vaidehi, Ontology based classification and categorization of email, in: Proceedings of the International Conference on Signal Processing, Communications and Networking, pp. 199-202, 2008.
41. A. Tsymbal, The Problem of Concept Drift: Definitions and Related Work, Citeseer, Ireland, Apr. 2004.
42. A. Han, H.-J. Kim, I. Ha, and G.-S. Jo, Semantic analysis of user behaviors for detecting spam mail, in: Proceedings of the IEEE International Workshop on Semantic Computing and Applications, pp. 91-95, 2008.
43. Blei D.M., Ng A.Y., Jordan M.I. Latent dirichlet allocation. J. Mach. Learn. Res. 2003, 3:993-1022.
44. Gansterer W.N., Janecek A.G.K., Neumayer R., Berry M.W., Castellanos M. Survey of Text Mining II 2008, Springer, London, pp. 165-183.
45. G. Gargiulo, A. Picariello, and C. Sansone, Using Visual and Semantic features for anti-spam filtering, in: Proceedings of 21st Neural Information Processing Systems, 2007.
46. J.-H. Hsia and M.-S. Chen, Language-model-based detection cascade for efficient classification of image-based spam e-mail, in: Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), pp. 1182-1185, 2009.
47. S. Youn and D. McLeod, Improved spam filtering by extraction of information from text embedded image e-mail, in: Proceedings of the 2009 ACM Symposium on Applied Computing (SAC), pp. 1754-1755, 2009.
48. B. He, W. Fang, Q. Luo, N.K. Govindaraju, and T. Wang, Mars: a MapReduce framework on graphics processors, in: Proceedings of the 17th International Conference on Parallel Architectures and Compilation Techniques (PACT), pp. 260-269, 2008.
49. Taura K., Kaneda K., Endo T., Yonezawa A. Phoenix, a parallel programming model for accommodating dynamically joining/leaving resources. ACM SIGPLAN Not. 2003, 38(10):216-229.
50. Welcome to Apache Hadoop! [Online]. Available: (accessed: 07-Oct-2012). http://hadoop.apache.org/.
51. T. Aarnio, Parallel Data Processing with Mapreduce, TKK T-110.5190, Seminar on Internetworking, 2009. [Online]. Available: (accessed: 07-Oct-2012). http://www.cse.tkk.fi/en/publications/B/5/papers/Aarnio_final.pdf.
52. A. Anand, Apache Hadoop Wins Terabyte Sort Benchmark. [Online]. Available: (accessed: 07-Oct-2012). http://developer.yahoo.net/blogs/hadoop/2008/07/apache_hadoop_wins_terabyte_sort_benchmark.html.
53. O. O'Malley, Terabyte Sort on Apache Hadoop, Available: (accessed: 07-Oct-2012). http://sortbenchmark.org/YahooHadoop.pdf.
54. Vapnik V., Lerner A. Pattern recognition using generalized portrait method,. Autom. Remote Control 1963, 24.
55. Kearns M. Efficient noise-tolerant learning from statistical queries. J. ACM 1998, 45(6):983-1006.
56. Hall M., Frank E., Holmes G., Pfahringer B., Reutemann P., Witten I.H. The WEKA data mining software,. ACM SIGKDD Explorations Newsletter 2009, 11(1):10-18.
57. B.E. Boser, I.M. Guyon, and V.N. Vapnik, A training algorithm for optimal margin classifiers, in: Proceedings of the Fifth Annual Workshop on Computational Learning Theory (COLT), pp. 144-152, 1992.
58. A. Asuncion and D.J. Newman, UCI Machine Learning Repository, available online: http://archive.ics.uci.edu/ml/ (accessed: 07-Oct-2012).
59. Matthew Horridge and P.F. Patel-Schneider, Manchester Syntax OWL 2 Web Ontology Language, W3C, 2011. [Online]. Available: (accessed: 07-Oct-2012). http://www.w3.org/2007/OWL/wiki/ManchesterSyntax.
60. U. Rebbapragada and C.E. Brodley, Class noise mitigation through instance weighting, in: Proceedings of the 18th European Conference on Machine Learning (ECML), vol. 4701, pp. 708-715, 2007.
61. A. Frank and A. Asuncion, UCI Machine Learning Repository-Adult Data Set, [Online] (accessed: 07-Oct-2012). http://archive.ics.uci.edu/ml/datasets/Adult.
62. A. Pavlo, E. Paulson, A. Rasin, D.J. Abadi, D.J. DeWitt, S. Madden, and M. Stonebraker, A comparison of approaches to large-scale data analysis, in: Proceedings of the ACM SIGMOD International Conference on Management of Data, pp. 165-178, 2009.
63. 5 Common Questions About Hadoop «Cloudera» Apache Hadoop for the Enterprise. [Online]. Available: (accessed: 07-Oct-2012). http://www.cloudera.com/blog/2009/05/5-common-questions-about-hadoop/.
64. Bauer E., Kohavi R. An empirical comparison of voting classification algorithms: bagging, boosting, and variants,. Mach. Learn. 1999, 36(1-2):105-139.
65. W.N. Street and Y. Kim, A streaming ensemble algorithm (SEA) for large-scale classification, in: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 377-382, 2001.
66. Wang S., Mathew A., Chen Y., Xi L., Ma L., Lee J. Empirical analysis of support vector machine ensemble classifiers,. Expert Syst. Appl. 2009, 36(3):6466-6476.
67. Breiman L. Bagging predictors. Mach. Learn. 1996, 24(2):123-140.
68. Chawla N.V., Bowyer K.W., Hall L.O., Kegelmeyer W.P. SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 2002, 16(1):321-357.
69. E. M. Voorhees and L. P. Buckland, Overview of TREC 2007, in: Proceedings of the 16th Text REtrieval Conference (TREC), 2007.