High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning

Pattern Recognition

Volumn 58, Issue , 2016, Pages 121-134

  Erfani, Sarah M ^a   Rajasegarar, Sutharshan ^a   Karunasekera, Shanika ^a   Leckie, Christopher ^a  

^a UNIVERSITY OF MELBOURNE   (Australia)

Author keywords

Anomaly detection; Deep belief net; Deep learning; Feature extraction; High dimensional data; One class SVM; Outlier detection

Indexed keywords

CLUSTERING ALGORITHMS; DATA MINING; SIGNAL DETECTION; SUPPORT VECTOR MACHINES;

ANOMALY DETECTION; DEEP BELIEF NET; DEEP LEARNING; HIGH DIMENSIONAL DATA; ONE CLASS-SVM; OUTLIER DETECTION;

FEATURE EXTRACTION;

EID: 84992311617     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2016.03.028     Document Type: Article

References (61)

1. ISSNIP, Internet of Things (IoT) for Creating Smart Cities. (http://issnip.unimelb.edu.au/research-program/Internet-of-Things), 2013
2. J. Gubbi, R. Buyya, S. Marusic, and M. Palaniswami Internet of Things (IoT): a vision, architectural elements, and future directions Future Gener. Comput. Syst. 29 7 2013 1645 1660
3. R. Kassab, and F. Alexandre Incremental data-driven learning of a novelty detection model for one-class classification with application to high-dimensional noisy data Mach. Learn. 74 2 2009 191 234
4. S.M. Erfani, M. Baktashmotlagh, S. Rajasegarar, S. Karunasekera, C. Leckie, R1SVM: a randomised nonlinear approach to large-scale anomaly detection, in: Twenty-Ninth AAAI Conference on Artificial Intelligence, 2015, pp. 432-438.
5. Sarah M Erfani, Mahsa Baktashmotlagh, Sutharshan Rajasegarar, Vinh Nguyen, Christopher Leckie, James Bailey, and Kotagiri Ramamohanarao. R1STM: One-class Support Tensor Machine with Randomised Kernel. In Proceedings of SIAM International Conference on Data Mining (SDM), 2016
6. Sarah M. Erfani, Mahsa Baktashmotlagh, Masud Moshtahgi, Vinh Nguyen, Christopher Leckie, James Bailey, and Kotagiri Ramamohanarao. Robust Domain Generalisation by Enforcing Distribution Invariance. In Proceedings of International Joint Conference on Artificial Intelligence(IJCAI), 2016.
7. A. Zimek, E. Schubert, and H.-P. Kriegel A survey on unsupervised outlier detection in high-dimensional numerical data Stat. Anal. Data Min. 5 5 2012 363 387
8. D.M. Tax, and R.P. Duin Support vector data description Mach. Learn. 54 2004 45 66
9. D. Wang, D.S. Yeung, and E.C. Tsang Structured one-class classification IEEE Trans. Syst. Man Cybern. Part B: Cybern. 36 6 2006 1283 1295
10. B. Schölkopf, and A.J. Smola Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond 2001 MIT Press Cambridge
11. S. Abe Support Vector Machines for Pattern Classification 2005 Springer New York
12. L. Auria, R.A. Moro, Support Vector Machines (SVM) as a Technique for Solvency Analysis, Technical Report, Discussion Paper of DIW Berlin, German Institute for Economic Research, 2008.
13. S. Vempati, A. Vedaldi, A. Zisserman, C. Jawahar, Generalized RBF feature maps for efficient detection, in: 21st British Machine Vision Conference, 2010, pp. 1-11.
14. F.J. Huang, Y. LeCun, Large-scale learning with SVM and convolutional for generic object categorization, in: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, 2006, pp. 284-291.
15. D. Liu, H. Qian, G. Dai, and Z. Zhang An iterative SVM approach to feature selection and classification in high-dimensional datasets Pattern Recognit. 46 9 2013 2531 2537
16. Y. Bengio, and Y. LeCun Scaling learning algorithms towards AI L. Bottou, Large Scale Kernel Machines 2007 1 41
17. Y. Bengio Learning deep architectures for AI Found. Trends Mach. Learn. 2 1 2009 1 127
18. G.E. Hinton, and R.R. Salakhutdinov Reducing the dimensionality of data with neural networks Science 313 5786 2006 504 507
19. G.E. Hinton, S. Osindero, and Y.-W. Teh A fast learning algorithm for deep belief nets Neural Comput. 18 7 2006 1527 1554
20. Y. Liu, S. Zhou, and Q. Chen Discriminative deep belief networks for visual data classification Pattern Recognit. 44 10 2011 2287 2296
21. L. Cao, K. Chua, W. Chong, H. Lee, and Q. Gu A comparison of PCA, KPCA and ICA for dimensionality reduction in support vector machine Neurocomputing 55 1 2003 321 336
22. J. Neumann, C. Schnörr, and G. Steidl Combined SVM-based feature selection and classification Mach. Learn. 61 1-3 2005 129 150
23. A. Widodo, B.-S. Yang, and T. Han Combination of independent component analysis and support vector machines for intelligent faults diagnosis of induction motors Expert Syst. Appl. 32 2 2007 299 312
24. A. Widodo, and B.-S. Yang Application of nonlinear feature extraction and support vector machines for fault diagnosis of induction motors Expert Syst. Appl. 33 1 2007 241 250
25. K.-Q. Shen, C.-J. Ong, X.-P. Li, and E.P. Wilder-Smith Feature selection via sensitivity analysis of SVM probabilistic outputs Mach. Learn. 70 1 2008 1 20
26. I. Guyon Feature Extraction: Foundations and Applications 2006 Springer New York
27. T. Liu, A novel text classification approach based on deep belief network, in: 17th International Conference on Neural Information Processing: Theory and Algorithms (ICONIP), 2010, pp. 314-321.
28. N. Bashah, I.B. Shanmugam, A.M. Ahmed, Hybrid intelligent intrusion detection system, in: World Academy of Science, Engineering and Technology, vol. 6, 2005, pp. 291-294.
29. P. Hamel, D. Eck, Learning features from music audio with deep belief networks, in: International Society for Music Information Retrieval Conference (ISMIR), 2010, pp. 339-344.
30. V.N. Vapnik Statistical Learning Theory 1998 Wiley New York
31. B. Schölkopf, J.C. Platt, J. Shawe-Taylor, A.J. Smola, and R.C. Williamson Estimating the support of a high-dimensional distribution Neural Comput. 13 7 2001 1443 1471
32. P. Laskov, C. Schäfer, I. Kotenko, and K.-R. Müller Intrusion detection in unlabeled data with quarter-sphere support vector machines Prax. Inf. Kommun. 27 4 2004 228 236
33. S. Rajasegarar, C. Leckie, J.C. Bezdek, and M. Palaniswami Centered hyperspherical and hyperellipsoidal one-class support vector machines for anomaly detection in sensor networks IEEE Trans. Inf. Forensics Secur. 5 3 2010 518 533
34. S. Rajasegarar, C. Leckie, M. Palaniswami, CESVM: Centered hyperellipsoidal support vector machine based anomaly detection, in: IEEE International Conference on Communications, 2008, pp. 1610-1614.
35. S. Rajasegarar, C. Leckie, M. Palaniswami, J.C. Bezdek, Quarter sphere based distributed anomaly detection in wireless sensor networks, in: IEEE International Conference on Communications, 2007, pp. 3864-3869.
36. C. Campbell, K. Bennett, A linear programming approach to novelty detection, in: Advances in Neural Information Processing Systems (NIPS), vol. 13, 2001, pp. 395-401.
37. J.H. Ward Hierarchical grouping to optimize an objective function J. Am. Stat. Assoc. 58 1-301 1963 236 244
38. Y. Bengio, P. Lamblin, D. Popovici, H. Larochelle, Greedy layer-wise training of deep networks, in: Advances in Neural Information Processing Systems (NIPS), vol. 19, 2007, pp. 153-160.
39. D. Wulsin, J. Gupta, R. Mani, J. Blanco, and B. Litt Modeling electroencephalography waveforms with semi-supervised deep belief nets: fast classification and anomaly measurement J. Neural Eng. 8 3 2011 036015
40. V. Nair, G.E. Hinton, 3D object recognition with deep belief nets, in: Advances in Neural Information Processing Systems (NIPS), vol. 21, 2009, pp. 1339-1347.
41. Y. Tang, Deep learning using support vector machines, in: ICML Workshop on Challenges in Representation Learning, 2013.
42. Y. Wang, and D. Wang Towards scaling up classification-based speech separation IEEE Trans. Audio Speech Lang. Process. 21 7 2013 1381 1390
43. G.E. Hinton Training products of experts by minimizing contrastive divergence Neural Comput. 14 8 2002 1771 1800
44. H. Larochelle, Y. Bengio, J. Louradour, and P. Lamblin Exploring strategies for training deep neural networks J. Mach. Learn. Res. 10 2009 1 40
45. L.M. Manevitz, and M. Yousef One-class SVMs for document classification J. Mach. Learn. Res. 2 2002 139 154
46. J.A. Lee, and M. Verleysen Nonlinear Dimensionality Reduction 2007 Springer New York
47. K. Brügge, A. Fischer, and C. Igel The flip-the-state transition operator for restricted Boltzmann machines Mach. Learn. 93 1 2013 53 69
48. C.C. Chang, and C.J. Lin LIBSVM: a library for support vector machines ACM Trans. Intell. Syst. Technol. 2 3 2011 27
49. L. Wang, U.T. Nguyen, J.C. Bezdek, C. Leckie, K. Ramamohanarao, iVAT and aVAT: enhanced visual analysis for cluster tendency assessment, in: Advances in Knowledge Discovery and Data Mining, 2010, pp. 16-27.
50. S. Subramaniam, T. Palpanas, D. Papadopoulos, V. Kalogeraki, D. Gunopulos, Online outlier detection in sensor data using non-parametric models, in: Proceedings of the 32nd International Conference on Very Large Data Bases (VLDB), 2006, pp. 187-198.
51. G. Hinton, A practical guide to training restricted Boltzmann machines, in: Neural Networks: Tricks of the Trade, 2012, pp. 599-619.
52. C.W. Hsu, C.-C. Chang, C.-J. Lin, et al., A practical guide to support vector classification. (https://www.cs.sfu.ca/people/Faculty/teaching/726/spring11/svmguide.pdf)
53. J. Demšar Statistical comparisons of classifiers over multiple data sets J. Mach. Learn. Res. 7 2006 1 30
54. S. García, F. Herrera, and J. Shawe-taylor An extension on statistical comparisons of classifiers over multiple data sets for all pairwise comparisons J. Mach. Learn. Res. 9 2008 2677 2694
55. S. García, A. Fernández, J. Luengo, and F. Herrera Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: experimental analysis of power Inf. Sci. 180 10 2010 2044 2064
56. M. Galar, A. Fernández, E. Barrenechea, H. Bustince, and F. Herrera An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes Pattern Recognit. 44 8 2011 1761 1776
57. F. Wilcoxon Individual comparisons by ranking methods Biom. Bull. 1 6 1945 80 83
58. D.J. Sheskin Handbook of Parametric and Nonparametric Statistical Procedures 4th Edition 2007 CRC Press London
59. J.P. Shaffer Modified sequentially rejective multiple test procedures J. Am. Stat. Assoc. 81 395 1986 826 831
60. M. Friedman The use of ranks to avoid the assumption of normality implicit in the analysis of variance J. Am. Stat. Assoc. 32 200 1937 675 701
61. Y. Bengio, A. Courville, and P. Vincent Representation learning: a review and new perspectives IEEE Trans. Pattern Anal. Mach. Intell. 35 8 2013 1798 1828