Pattern recognition systems under attack: Design issues and research challenges

International Journal of Pattern Recognition and Artificial Intelligence

Volumn 28, Issue 7, 2014, Pages

  Biggio, Battista ^a   Fumera, Giorgio ^a   Roli, Fabio ^a  

^a UNIVERSITY OF CAGLIARI   (Italy)

Author keywords

Adversarial learning; Robust classification; Secure pattern recognition

Indexed keywords

LEARNING ALGORITHMS; MALWARE; PATTERN RECOGNITION;

ADVERSARIAL LEARNING; MALWARE DETECTION; POTENTIAL ATTACK; PRE-PROCESSING STEP; PROACTIVE SECURITY; RESEARCH CHALLENGES; ROBUST CLASSIFICATION; SENSITIVE APPLICATION;

PATTERN RECOGNITION SYSTEMS;

EID: 84988423255     PISSN: 02180014     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0218001414600027     Document Type: Article

References (74)

1. A. Adler, Vulnerabilities in biometric encryption systems, 5th Int. Conf. Audio- and Video-Based Biometric Person Authentication, eds. T. Kanade, A. K. Jain and N. K. Ratha, LNCS, Vol. 3546(Springer, 2005), pp. 1100-1109.
2. M. Antonakakis, R. Perdisci, Y. Nadji, N. Vasiloglou, S. Abu-Nimeh, W. Lee and D. Dagon, From throw-away traffic to bots: Detecting the rise of DGA-based malware, in 21st USENIX Security Symp. (USENIX, 2012), pp. 491-506.
3. I. Arce, The weakest link revisited, IEEE Security Privacy 1 (2) (2003) 72-76.
4. A. Attar, R. M. Rad and R. E. Atani, A survey of image spamming and -ltering techniques, Artif. Intell. Rev. 40 (1) (2013) 71-105.
5. M. Barreno, P. L. Bartlett, F. J. Chi, A. D. Joseph, B. Nelson, B. I. Rubinstein, U. Saini and J. D. Tygar, Open problems in the security of learning, Proc. 1st ACM Workshop on Arti-cial Intell. Sec., AISec'08(ACM, 2008), pp. 19-26.
6. M. Barreno, B. Nelson, R. Sears, A. D. Joseph and J. D. Tygar, Can machine learning be secure? Proc. ACM Symp. Information, Computer and Comm. Sec., ASIACCS'06(ACM, 2006), pp. 16-25.
7. A. Barth, B. I. Rubinstein, M. Sundararajan, J. C. Mitchell, D. Song and P. L. Bartlett, A learning-based approach to reactive security, IEEE Trans. Dependable Secure Comput. 9 (4) (2012) 482-493.
8. B. Biggio, Z. Akhtar, G. Fumera, G. L. Marcialis and F. Roli, Security evaluation of biometric authentication systems under real spoo-ng attacks, IET Biometrics 1 (1) (2012) 11-24.
9. B. Biggio, I. Corona, G. Fumera, G. Giacinto and F. Roli, Bagging classiers for-ghting poisoning attacks in adversarial environments, in 10th Int. Workshop on MCSs, eds. C. Sansone et al., LNCS, Vol. 6713(Springer, 2011), pp. 350-359.
10. B. Biggio, I. Corona, D. Maiorca, B. Nelson, N. Šrndiæ, P. Laskov, G. Giacinto and F. Roli, Evasion attacks against machine learning at test time, in European Conf. Machine Learning and Principles and Practice Knowl. Discovery in Databases, Part III, eds. H. Blockeel et al., LNCS, Vol. 8190(Springer, 2013), pp. 387-402.
11. B. Biggio, I. Corona, B. Nelson, B. Rubinstein, D. Maiorca, G. Fumera, G. Giacinto and F. Roli, Security evaluation of support vector machines in adversarial environments, in Support Vector Machines Applications, eds. Y. Ma and G. Guo (Springer International Publishing, 2014), pp. 105-153.
12. B. Biggio, L. Didaci, G. Fumera and F. Roli, Poisoning attacks to compromise face templates, 6th IAPR Int. Conf. Biometrics (2013), pp. 1-7.
13. B. Biggio, G. Fumera, I. Pillai and F. Roli, A survey and experimental evaluation of image spamltering techniques, Pattern Recogn. Lett. 32 (10) (2011) 1436-1446.
14. B. Biggio, G. Fumera and F. Roli, Adversarial pattern classi-cation using multiple classi-ers and randomisation, 12th Joint IAPR Int. Workshop on Structural and Syntactic Pattern Rec., LNCS, Vol. 5342(Springer-Verlag, 2008), pp. 500-509.
15. B. Biggio, G. Fumera and F. Roli, Evade hard multiple classi-er systems, in Supervised and Unsupervised Ensemble Methods and their Applications, eds. O. Okun and G. Valentini, Vol. 245, Studies in Computational Intell. (Springer, 2009), pp. 15-38.
16. B. Biggio, G. Fumera and F. Roli, Multiple classifier systems for adversarial classification tasks, Proc. 8th Int. Workshop on MCSs, eds. J. A. Benediktsson, J. Kittler and F. Roli, LNCS, Vol. 5519(Springer, 2009), pp. 132-141.
17. B. Biggio, G. Fumera and F. Roli, Multiple classifier systems for robust classifier design in adversarial environments, Int. J. Mach. Learn. Cybern. 1 (1) (2010) 27-41.
18. B. Biggio, G. Fumera and F. Roli, Multiple classifier systems under attack, in 9th Int. Workshop on MCSs, eds. N. E. Gayar et al., LNCS, Vol. 5997(Springer, 2010), pp. 74-83.
19. B. Biggio, G. Fumera and F. Roli, Design of robust classifiers for adversarial environments, IEEE Int. Conf. Systems, Man and Cybern. (2011), pp. 977-982.
20. B. Biggio, G. Fumera and F. Roli, Security evaluation of pattern classifiers under attack, IEEE Trans. Knowl. Data Eng. 26 (4) (2014) 984-996.
21. B. Biggio, G. Fumera, F. Roli and L. Didaci, Poisoning adaptive biometric systems, in Structural, Syntactic and Statistical Pattern Recognition, eds. G. Gimel'farb et al., LNCS, Vol. 7626(Springer, 2012), pp. 417-425.
22. B. Biggio, B. Nelson and P. Laskov, Poisoning attacks against support vector machines, 29th Int. Conf. Mach. Learn., eds. J. Langford et al. (Omnipress, 2012), pp. 1807-1814.
23. B. Biggio, I. Pillai, S. R. Bulò, D. Ariu, M. Pelillo and F. Roli, Is data clustering in adversarial settings secure? Proc. Workshop on Artificial Intell. Security, AISec'13(ACM, 2013), pp. 87-98.
24. M. Brückner, C. Kanzow and T. Scheffer, Static prediction games for adversarial learning problems, J. Mach. Learn. Res. 13 (2012) 2617-2654.
25. M. Brückner and T. Scheffer, Nash equilibria of static prediction games, in Advances in Neural Information Processing Systems 22, eds. Y. Bengio et al. (MIT Press, 2009), pp. 171-179.
26. M. Brückner and T. Scheffer, Stackelberg games for adversarial prediction problems, 17th Int. Conf. Knowl. Disc. Data Mining, KDD'11(ACM, 2011), pp. 547-555.
27. A. Christmann and I. Steinwart, On robust properties of convex risk minimization methods for pattern recognition, J. Mach. Learn. Res. 5 (2004) 1007-1034.
28. G. F. Cretu, A. Stavrou, M. E. Locasto, S. J. Stolfo and A. D. Keromytis, Casting out demons: Sanitizing training data for anomaly sensors, IEEE Symp. Security Privacy (IEEE CS, 2008), pp. 81-95.
29. C. Croux, P. Filzmoser and M. R. Oliveira, Algorithms for projection - pursuit robust principal component analysis, Chemometr. Intell. Lab. Syst. 87 (2) (2007) 218-225.
30. G. Cybenko and C. E. Landwehr, Security analytics and measurements, IEEE Security Privacy 10 (3) (2012) 5-8.
31. N. Dalvi, P. Domingos, Mausam, S. Sanghai and D. Verma, Adversarial classification, 10th Int. Conf. Knowl. Disc. Data Mining (2004), pp. 99-108.
32. O. Dekel, O. Shamir and L. Xiao, Learning to classify with missing and corrupted features, Mach. Learn. 81 (2010) 149-178.
33. R. O. Duda, P. E. Hart and D. G. Stork, Pattern Classification (Wiley-Interscience Publication, 2000).
34. C. Dwork, Differential privacy, Proc. 33rd Int. Colloquium on Automata, Languages and Programming (ICALP) (Springer, 2006), pp. 1-12.
35. P. Fogla, M. Sharif, R. Perdisci, O. Kolesnikov and W. Lee, Polymorphic blending attacks, Proc. 15th Conf. USENIX Security Symp. (USENIX Association, Berkeley, CA, USA, 2006), pp. 241-256.
36. J. Galbally, C. McCool, J. Fierrez, S. Marcel and J. Ortega-Garcia, On the vulnerability of face verification systems to hill-climbing attacks, Pattern Recogn. 43 (3) (2010) 1027-1038.
37. A. Globerson and S. T. Roweis, Nightmare at test time: Robust learning by feature deletion, Proc. 23rd Int. Conf. Mach. Learn., eds. W. W. Cohen and A. Moore, Vol. 148(ACM, 2006), pp. 353-360.
38. M. Großhans, C. Sawade, M. Brückner and T. Scheffer, Bayesian games for adversarial regression problems, Proc. 30th Int. Conf. Mach. Learn., Vol. 28(3) (Atlanta, USA, 2013), pp. 55-63.
39. P. Haider, L. Chiarandini and U. Brefeld, Discriminative clustering for market segmentation, Proc. 18th Int. Conf. Knowl. Disc. and Data Mining, KDD'12(ACM, 2012), pp. 417-425.
40. F. R. Hampel, E. M. Ronchetti, P. J. Rousseeuw and W. A. Stahel, Robust Statistics: The Approach Based on Influence Functions, Probability and Mathematical Statistics (John Wiley and Sons, NY, USA, 1986).
41. L. Huang, A. D. Joseph, B. Nelson, B. Rubinstein and J. D. Tygar, Adversarial machine learning, in 4th Workshop on Artificial Intell. Sec. (2011), pp. 43-57.
42. A. K. Jain, A. Ross, S. Pankanti and S. Member, Biometrics: A tool for information security, IEEE Trans. Inf. Forensics Security 1 (2006) 125-143.
43. S. Jana and V. Shmatikov, Abusing file processing in malware detectors for fun and profit, Proc. 33rd IEEE Symp. Sec. & Privacy (2012), pp. 80-94.
44. A. D. Joseph, P. Laskov, F. Roli, J. D. Tygar and B. Nelson, Machine learning methods for computer security (Dagstuhl Perspectives Workshop 12371), Dagstuhl Manifestos 3 (1) (2013) 1-30.
45. M. Kantarcioglu, B. Xi and C. Clifton, Classifier evaluation and attribute selection against active adversaries, Data Mining Knowl. Discov. (2010), pp. 1-45.
46. M. Kloft and P. Laskov, Online anomaly detection under adversarial impact, Proc. 13th Int. Conf. Artificial Intell. and Statistics (2010), pp. 405-412.
47. A. Kolcz and C. H. Teo, Feature weighting for improved classifier robustness, in 6th Conf. Email and Anti-Spam (Mountain View, CA, USA, 2009), pp. 1-8.
48. L. I. Kuncheva, Classifier ensembles for detecting concept change in streaming data: Overview and perspectives, in Workshop on Supervised and Unsupervised Ensemble Methods and their Applications, eds. O. Okun and G. Valentini (Patras, Greece, 2008), pp. 5-10.
49. W. Liu and S. Chawla, Mining adversarial patterns via regularized loss minimization, Mach. Learn. 81 (1) (2010) 69-83.
50. D. Lowd and C. Meek, Adversarial learning, Proc. 11th Int. Conf. Knowl. Disc. and Data Mining, ed. A. Press (2005), pp. 641-647.
51. D. Lowd and C. Meek, Good word attacks on statistical spam filters, 2nd Conf. Email and Anti-Spam (Mountain View, CA, USA, 2005), pp. 1-8.
52. D. Maiorca, I. Corona and G. Giacinto, Looking at the bag is not enough to find the bomb: An evasion of structural methods for malicious pdf files detection, Proc. 8th Symp. Inform., Comp. and Comm. Sec., ASIACCS'13(ACM, 2013), pp. 119-130.
53. R. A. Maronna, R. D. Martin and V. J. Yohai, Robust Statistics: Theory and Methods, Probability and Mathematical Statistics (John Wiley and Sons, NY, USA, 2006).
54. M. Martinez-Diaz, J. Fierrez, J. Galbally and J. Ortega-Garcia, An evaluation of indirect attacks and countermeasures in fingerprint verification systems, Pattern Recogn. Lett. 32 (12) (2011) 1643-1651.
55. B. Nelson, M. Barreno, F. J. Chi, A. D. Joseph, B. I. P. Rubinstein, U. Saini, C. Sutton, J. D. Tygar and K. Xia, Exploiting machine learning to subvert your spam filter, Proc. 1st Workshop on Large-Scale Expl. and Em. Threats (USENIX, 2008), pp. 1-9.
56. B. Nelson, B. Biggio and P. Laskov, Understanding the risk factors of learning in adversarial environments, 4th Workshop on Artificial Intell. and Sec., AISec'11(ACM, 2011), pp. 87-92.
57. B. Nelson, B. I. Rubinstein, L. Huang, A. D. Joseph, S. J. Lee, S. Rao and J. D. Tygar, Query strategies for evading convex-inducing classifiers, J. Mach. Learn. Res. 13 (2012) 1293-1332.
58. B. Nelson, B. I. P. Rubinstein, L. Huang, A. D. Joseph, S. Hon Lau, S. Lee, S. Rao, A. Tran and J. D. Tygar, Near-optimal evasion of convex-inducing classifiers, in J. Mach. Learn. Res. Proc. 13th Int. Conf. Artificial Intell. and Statistics (AISTATS), Vol. 9. (Chia, Sardinia, Italy, 2010), pp. 549-556.
59. J. Newsome, B. Karp and D. Song, Paragraph: Thwarting signature learning by training maliciously, in RAID, LNCS (Springer, 2006), pp. 81-105.
60. S. Pastrana, A. Orfila, J. E. Tapiador and P. Peris-Lopez, Randomized anagram revisited, J. Netw. Comput. Appl. 41 (2014) 182-196.
61. R. Perdisci, D. Ariu and G. Giacinto, Scalable fine-grained behavioral clustering of httpbased malware, Comput. Netw. 57 (2) (2013) 487-500.
62. N. K. Ratha, J. H. Connell and R. M. Bolle, An analysis of minutiae matching strength, in Int. Conf. Audio- and Video-Based Biometric Person Authentication, eds. J. Bigün and F. Smeraldi, LNCS, Vol. 2091(Springer, 2001), pp. 223-228.
63. K. Rieck, P. Trinius, C. Willems and T. Holz, Automatic analysis of malware behavior using machine learning, J. Comput. Secur. 19 (4) (2011) 639-668.
64. R. N. Rodrigues, L. L. Ling and V. Govindaraju, Robustness of multimodal biometric fusion methods against spoof attacks, J. Vis. Lang. Comput. 20 (3) (2009) 169-179.
65. F. Roli, B. Biggio and G. Fumera, Pattern recognition systems under attack, in Progress in Pattern Recognition Image Analysis, Computer Vision, and Applications, eds. J. Ruiz-Shulcloper and G. S. di Baja, LNCS, Vol. 8258(Springer, 2013), pp. 1-8.
66. B. I. Rubinstein, B. Nelson, L. Huang, A. D. Joseph, S.-H. Lau, S. Rao, N. Taft and J. D. Tygar, Antidote: Understanding and defending against poisoning of anomaly detectors, Proc. 9th Int. Measurement Conf., IMC'09(ACM, 2009), pp. 1-14.
67. B. I. P. Rubinstein, P. L. Bartlett, L. Huang and N. Taft, Learning in a large function space: Privacy-preserving mechanisms for SVM learning, J. Privacy Confidentiality 4 (1) (2012) 65-100.
68. C. H. Teo, A. Globerson, S. Roweis and A. Smola, Convex learning with invariances, in Advances in Neural Information Processing Systems 20, eds. J. Platt, D. Koller, Y. Singer and S. Roweis (MIT Press, Cambridge, MA, 2008), pp. 1489-1496.
69. M. Torkamani and D. Lowd, Convex adversarial collective classification, Proc. 30th Int. Conf. Mach. Learning, JMLR, Vol. 28(2013), pp. 642-650.
70. G. L. Wittel and S. F. Wu, On attacking statistical spam filters, First Conf. Email and Anti-Spam (Mountain View, CA, USA, 2004), pp. 1-7.
71. M. Wooldridge. Does game theory work? IEEE Intell. Syst. 27 (6) (2012) 76-80.
72. H. Xu, C. Caramanis and S. Mannor, Robustness and regularization of support vector machines, J. Mach. Learn. Res. 10 (2009) 1485-1510.
73. Y. Zhou, M. Kantarcioglu and B. Thuraisingham, Sparse bayesian adversarial learning using relevance vector machine ensembles, IEEE 12th Int. Conf. Data Mining (2012), pp. 1206-1211.
74. Y. Zhou, M. Kantarcioglu, B. Thuraisingham and B. Xi, Adversarial support vector machine learning, Proc. 18th Int. Conf. Knowl. Disc. and Data Mining, KDD'12(ACM, 2012), pp. 1059-1067.