The limitations of deep learning in adversarial settings

Proceedings - 2016 IEEE European Symposium on Security and Privacy, EURO S and P 2016

Volumn , Issue , 2016, Pages 372-387

  Papernot, Nicolas ^a   Mcdaniel, Patrick ^a   Jha, Somesh ^b   Fredrikson, Matt ^c   Celik, Z Berkay ^a   Swami, Ananthram ^d  

^a PENNSYLVANIA STATE UNIVERSITY   (United States)

^b University of Wisconsin Madison   (United States)

^c Carnegie Mellon University   (United States)

^d U S ARMY RESEARCH LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; LEARNING SYSTEMS;

COMPUTATIONALLY EFFICIENT; DEEP NEURAL NETWORKS; HARDNESS MEASURES; INPUT FEATURES; LARGE DATASETS; TARGET CLASSIFICATION; TRAINING ALGORITHMS; TRAINING PHASE;

COMPUTER VISION;

EID: 84978047763     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/EuroSP.2016.36     Document Type: Conference Paper

References (37)

1. M. Barreno, B. Nelson, A. D. Joseph, and J. Tygar. The security of machine learning. Machine Learning, 81(2):121-148, 2010
2. M. Barreno, B. Nelson, R. Sears, A. D. Joseph, and J. D. Tygar. Can machine learning be secure? In Proceedings of the 2006 ACM Symposium on Information, computer and communications security, pages 16-25. ACM, 2006
3. Y. Bengio. Learning deep architectures for AI. Foundations and trends in Machine Learning, 2(1):1-127, 2009
4. J. Bergstra, O. Breuleux, F. Bastien, P. Lamblin, R. Pascanu, G. Desjardins, J. Turian, D. Warde-Farley, and Y. Bengio. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for scientific computing conference (SciPy), volume 4, page 3. Austin, TX, 2010
5. B. Biggio, I. Corona, D. Maiorca, B. Nelson, N. Šrndíc, P. Laskov, G. Giacinto, and F. Roli. Evasion attacks against machine learning at test time. In Machine Learning and Knowledge Discovery in Databases, pages 387-402. Springer, 2013
6. B. Biggio, G. Fumera, and F. Roli. Pattern recognition systems under attack: Design issues and research challenges. International Journal of Pattern Recognition and Artificial Intelligence, 28(07):1460002, 2014
7. B. Biggio, G. Fumera, and F. Roli. Security evaluation of pattern classifiers under attack. IEEE Transactions on Knowledge and Data Engineering, 26(4):984-996, 2014
8. B. Biggio, B. Nelson, and P. Laskov. Support vector machines under adversarial label noise. In ACML, pages 97-112, 2011
9. B. Biggio, B. Nelson, and L. Pavel. Poisoning attacks against support vector machines. In Proceedings of the 29th International Conference on Machine Learning, 2012
10. B. Biggio, K. Rieck, D. Ariu, C. Wressnegger, I. Corona, G. Giacinto, and F. Roli. Poisoning behavioral malware clustering. In Proceedings of the 2014 Workshop on Artificial Intelligent and Security Workshop, pages 27-36. ACM, 2014
11. D. Cirešan, U. Meier, J. Masci, et al. Multi-column deep neural network for traffic sign classification. Neural Networks, 32:333-338, 2012
12. R. Collobert and J. Weston. A unified architecture for natural language processing: Deep neural networks with task learning. In Proceedings of the 25th international conference on Machine learning, pages 160-167. ACM, 2008
13. G. E. Dahl, J. W. Stokes, L. Deng, and D. Yu. Large-scale malware classification using random projections and neural networks. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3422-3426. IEEE, 2013
14. G. E. Dahl, D. Yu, et al. Context-dependent pre-Trained deep neural networks for large-vocabulary speech recognition. IEEE Transactions on Audio, Speech, and Language Processing, 20(1):30-42, 2012
15. P. Fogla and W. Lee. Evading network anomaly detection systems: formal reasoning and practical techniques. In Proceedings of the 13th ACM conference on Computer and communications security, pages 59-68. ACM, 2006
16. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, et al. Generative adversarial nets. In Advances in Neural Information Processing Systems, pages 2672-2680, 2014
17. I. J. Goodfellow, J. Shlens, and C. Szegedy. Explaining and harnessing adversarial examples. In Proceedings of the 2015 International Conference on Learning Representations. Computational and Biological Learning Society, 2015
18. S. Gu and L. Rigazio. Towards deep neural network architectures robust to adversarial examples. In Proceedings of the 2015 International Conference on Learning Representations. Computational and Biological Learning Society, 2015
19. G. Hinton, S. Osindero, and Y.-W. Teh. A fast learning algorithm for deep belief nets. Neural computation, 18(7):1527-1554, 2006
20. K. Hornik, M. Stinchcombe, et al. Multilayer feedforward networks are universal approximators. Neural networks, 2(5):359-366, 1989
21. L. Huang, A. D. Joseph, B. Nelson, B. I. Rubinstein, and J. Tygar. Adversarial machine learning. In Proceedings of the 4th ACM workshop on security and artificial intelligence, pages 43-58. ACM, 2011
22. C. Kaufman, R. Perlman, and M. Speciner. Network security: private communication in a public world. Prentice Hall Press, 2002
23. E. Knorr. How paypal beats the bad guys with machine learning. http://www.infoworld.com/article/2907877/machine-learning/howpaypal-reduces-fraud-with-machine-learning.html, 2015
24. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097-1105, 2012
25. H. Larochelle, Y. Bengio, J. Louradour, and P. Lamblin. Exploring strategies for training deep neural networks. The Journal of Machine Learning Research, 10:1-40, 2009
26. Y. LeCun, L. Bottou, et al. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998
27. Y. LeCun and C. Cortes. Mnist handwritten digit database. AT&T Labs [Online]. Available: http://yann. lecun. com/exdb/mnist, 1998
28. LISA lab. http://deeplearning.net/tutorial/lenet.html, 2010
29. K. P. Murphy. Machine learning: a probabilistic perspective. MIT 2012
30. A. Nguyen, J. Yosinski, and J. Clune. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In IEEE Computer Vision and Pattern Recognition, 2015
31. D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning representations by back-propagating errors. Cognitive modeling, 5, 1988
32. H. Sak, A. Senior, and F. Beaufays. Long short-Term memory recurrent neural network architectures for large scale acoustic modeling. In Proceedings of the Annual Conference of International Speech Communication Association (INTERSPEECH), 2014
33. K. Simonyan, A. Vedaldi, and A. Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013
34. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. arXiv preprint arXiv:1409.4842, 2014
35. C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks. In Proceedings of the 2014 International Conference on Learning Representations. Computational and Biological Learning Society, 2014
36. Y. Taigman, M. Yang, et al. Deepface: Closing the gap to human-level performance in face verification. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1701-1708, 2014
37. J. Yosinski, J. Clune, Y. Bengio, and H. Lipson. How transferable are features in deep neural networks? In Advances in Neural Information Processing Systems, pages 3320-3328, 2014