An experimental evaluation of novelty detection methods

Neurocomputing

Volumn 135, Issue , 2014, Pages 313-327

  Ding, Xuemei ^a,b   Li, Yuhua ^a   Belatreche, Ammar ^a   Maguire, Liam P ^a  

^a UNIVERSITY OF ULSTER   (United Kingdom)

^b FUJIAN NORMAL UNIVERSITY   (China)

Author keywords

Gaussian mixture; K Nearest neighbours; K Means clustering; Novelty detection; Support vector data description

Indexed keywords

DATA DESCRIPTION;

GAUSSIAN MIXTURES; K-MEANS CLUSTERING; K-NEAREST NEIGHBOURS; NOVELTY DETECTION; SUPPORT VECTOR DATA DESCRIPTION;

DETECTORS;

AREA UNDER THE CURVE; ARTICLE; ARTIFICIAL NEURAL NETWORK; CLASSIFICATION ALGORITHM; CLUSTER ANALYSIS; CONTROLLED STUDY; DATA ANALYSIS; EVALUATION STUDY; EXPERIMENTAL STUDY; INTERMETHOD COMPARISON; K NEAREST NEIGHBOR; MACHINE LEARNING; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; NOVELTY DETECTION SYSTEM; PARAMETRIC TEST; PERFORMANCE MEASUREMENT SYSTEM; PRIORITY JOURNAL; PROBABILITY; QUALITY CONTROL; SUPPORT VECTOR MACHINE; KERNEL METHOD; NOVELTY DETECTION; QUALITATIVE ANALYSIS; QUANTITATIVE ANALYSIS; RECEIVER OPERATING CHARACTERISTIC;

EID: 84897912404     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2013.12.002     Document Type: Article

References (89)

1. Chandola V., Banerjee A., Kumar V. Anomaly detection: a survey. ACM Comput. Surv. 2009, 41(3):1-58.
2. C.M. Bishop, Novelty detection and neural network validation, IEE Proc. Vision, Image Signal Process., 1994, pp. 217-222.
3. Ritter G., Gallegos M.T. Outliers in statistical pattern recognition and an application to automatic chromosome classification. Pattern Recognit. Lett. 1997, 18(6):525-539.
4. N. Japkowicz, Concept-learning in the Absence of Counter-examples: An Autoassociation-based Approach to Classification, Ph.D. Thesis, 1999.
5. Schölkopf B., Platt J.C., Shawe-Taylor J., Smola A.J., Williamson R.C. Estimating the support of a high-dimensional distribution. Neural Comput. 2001, 13(7):1443-1471.
6. Gornitz N., Kloft M., Rieck K., Brefeld U. Toward supervised anomaly detection. J. Artif. Intell. Res. 2013, 46:235-262.
7. Tax D.M.J., Duin R.P.W. Support vector data description. Mach. Learn. 2004, 54(1):45-66.
8. EI-Yaniv R., Nisenson M. Optimal single-class classification strategies. Adv. Neural Inf. Process. Syst. 2007, 19:377-384.
9. Park C., Huang J.Z., Ding Y. A computable plug-in estimator of minimum volume sets for noveltydetection. Oper. Res. 2010, 58(5):1469-1480.
10. Juszczak P., Tax D.M.J., Pekalska E., Duin R.P.W. Minimum spanning tree based one-class classifier. Neurocomputing 2009, 72(7-9):1859-1869.
11. D.M.J. Tax, One-class Classification: Concept-learning in the Absence of Counter-examples, Ph.D. Thesis, 2001.
12. Markou M., Singh S. Novelty detection: a review-Part 1&2: Statistical approaches & neural network based approaches. Signal Process 2003, 83(12):2481-2521.
13. Hodge V.J., Austin J. A survey of outlier detection methodologies. Artif. Intell. Rev. 2004, 22(2):85-126.
14. Patcha A., Park J.M. An overview of anomaly detection techniques: existing solutions and latest technological trends. Comput. Netw. 2007, 51(12):3448-3470.
15. Gogoi P., Bhattacharyya D.K., Borah B., Kalita J.K. A survey of outlier detection methods in network anomaly identification. Comput. J. 2011, 54(4):570-588.
16. M. Gupta, J. Gao, C.C. Aggarwal, J. Han (2013, Jan.) Outlier Detection for Temporal Data: A Survey. [Online]. http://dais.cs.uiuc.edu/manish.
17. Surace C., Worden K. Novelty detection in a changing environment: a negative selection approach. Machan. Syst. Signal Process. 2010, 24(4):1114-1128.
18. Wu M., Ye J. A small sphere and large margin approach for novelty detection using training data with outliers. IEEE Trans. Pattern Anal. Mach. Intell. 2009, 31(11):2088-2092.
19. T. Le, D. Tran, W. Ma, D. Sharma, An optimal sphere and two large margins approach for novelty detection, in: International Joint Conference on Neural Networks, Canberra, Australia, 2010, pp. 1-6.
20. Smola A.J., Song L., Teo C.H. Relative novelty detection. J. Mach. Learn. Res. 2009, 5:536-543.
21. Blanchard G., Lee G., Scott C. Semi-supervised novelty detection. J. Mach. Learn. Res. 2010, 11:2973-3009.
22. Chen Y., Dang X., Peng H., Jr H.L.B. Outlier detection with the kernelized spatial depth function. IEEE Trans. Pattern Anal. Mach. Intell. 2009, 31(2):288-305.
23. Catterson V.M., McArthur S.D.J., Moss G. Online conditional anomaly detection in multivariate data for transformer monitoring. IEEE Trans. Power Delivery 2010, 25(4):2556-2564.
24. Khreich W., Granger E., Miri A., Sabourin R. Iterative Boolean combination of classifiers in the ROC space: an application to anomaly detection with HMMs. Pattern Recogn. 2010, 43(8):2732-2752.
25. Xiao Y., Wang H., Xu W., Zhou J. L1 norm based KPCA for novelty detection. Pattern Recognit. 2013, 46(1):389-396.
26. Jiang F., Sui Y., Cao C. A hybrid approach to outlier detection based on boundary region. Pattern Recognit. Lett. 2011, 32(14):1860-1870.
27. Garces H., Sbarbaro D. Outliers detection in environmental monitoring databases. Eng. Appl. Artif. Intell. 2011, 24(2):341-349.
28. Barreto G.A., Frota R.A. A unifying methodology for the evaluation of neural network models on novelty detection tasks. Pattern Anal. Appl. 2013, 16(1):83-97.
29. T. Vatanen et al., Semi-supervised detection of collective anomalies with an application in high energy particle physics, in: International Joint Conference on Neural Networks, Brisbane, Australia, 2012, pp. 1-8.
30. Li W., Guo Q., Elkan C. A positive and unlabeled learning algorithm for one-class classification of remote-sensing data. IEEE Trans. Geosci. Remote Sens. 2011, 49(2):717-725.
31. Yong S.P., Deng J.D., Purvis M.K. Novelty detection in wildlife scenes through semantic context modelling. Pattern Recognit. 2012, 45(9):3439-3450.
32. Papatheou E., Manson G., Barthorpe R.J., Worden K. The use of pseudo-faults for novelty detection in SHM. J. Sound Vib. 2010, 329(12):2349-2366.
33. Schölkopf B., Williamson R.C., Smola A.J., Shawe-Taylor J., Platt J. Support vector method for novelty detection. Adv. Neural Inf. Process. Syst. 2000, 12(3):582-588.
34. Li Y. Selecting training points for one-class support vector machines. Pattern Recognit. Lett. 2011, 32(11):1517-1522.
35. Lian H. On feature selection with principal component analysis for one-class SVM. Pattern Recognit. Lett. 2012, 33(9):1027-1031.
36. Wei Z., Xun J., Wang X. One-class classification based finance news story recommendation. Comput. Inf. Syst. 2009, 5(6):1625-1631.
37. Verdejo V.G., Garcia J.A., Gredilla M.L., Vazquez A.N. Adaptive one-class support vector machine. IEEE Trans. Signal Process. 2011, 59(6):2975-2981.
38. Clifton D.A., et al. Novelty detection for identifying deterioration in emergency department patients. Lect. Notes Comput. Sci. 2011, 6936:220-227.
39. Guo S.M., Chan L.C., Tsai J.S.H. A boundary method for outlier detection based on support vector d omain description. Pattern Recognit. 2009, 42(1):77-83.
40. Wang X., Chung F., Wang S. Theoretical analysis for solution of support vector data description. Neural Netw. 2011, 24(4):360-369.
41. Liu Y.H., Liu Y.C., Chen Y.J. Fast support vector data descriptions for novelty detection. IEEE Trans. Neural Netw. 2010, 21(8):1296-1313.
42. Peng X., Xu D. Efficient support vector data descriptions for novelty detection. Neural Comput. Appl. 2012, 21(8):2023-2032.
43. Z. Noumir, P. Honeine, C. Richard, On simple one-class classification methods, in: IEEE International Symposium on Information Theory Proceedings, 2012, pp. 2022-2026.
44. Wang S., Yu J., Lapira E., Lee J. A modified support vector data description based novelty detection approach for machinery components. Appl. Software Comput. 2013, 13(2):1193-1205.
45. McBain J., Timusk M. Feature extraction for novelty detection as applied to fault detection in machinery. Pattern Recognit. Lett. 2011, 32(7):1054-1061.
46. Mariam M., Delb W., Schick B., Strauss D.J. Feasibility of an objective electrophysiological loudness scaling: a kernel-based novelty detection approach. Artif. Intell. Med. 2012, 55(3):185-195.
47. Zhang K., Hutter M., Jin H. A new local disctance-based outlier detection approach for scattered real-world data. Advances in Knowledge Discovery and Data Mining 2009, 5476:813-822.
48. Cabral G.G., Oliveira A.L.I., Cahu C.B.G. Combining nearest neighbor data description and structural risk minimization for one-class classification. Neural Comput. Appl. 2009, 18(2):175-183.
49. D.A. Clifton et al., Bayesian extreme value statistics for novelty detection in gas-turbine engines, in: IEEE Aerospace Conference, Montana, USA, 2008, pp. 1-11.
50. Roweis Sam T., Saul Lawrence K. Nonlinear dimensionality reduction by locally linear embedding. Science 2000, 290(22):2323-2326.
51. Angiulli F. Prototype-based domain description for one-class classification. IEEE Trans. Pattern Anal. Mach. Intell. 2012, 34(6):1131-1144.
52. Tzizkos I., Cavallaro A., Xu L. Event monitoring via local motion abnormality detection in non-linear subspace. Neurocomputing 2010, 73(10-12):1881-1891.
53. Masud M.M., Gao J., Khan L., Han J., Thuraisingham B. Classification and novel class detection in concept-drifting data streams under time constraints. IEEE Trans. Knowl. Data Eng. 2011, 23(6):859-874.
54. G.G. Cabral, A.L.I. Oliveira, A hybrid method for novelty detection in time series based on states transitions and swarm intelligence, in: International Conference on Neural Networks, Spain, 2010, pp. 1-8.
55. Filev D.P., Chinnam R.B., Tseng F., Baruah P. An industrial strength novelty detection framework for autonomous equipment monitoring and diagnostics. IEEE Trans. Ind. Inf. 2010, 6(4):767-779.
56. Moshtaghi M., et al. Clustering ellipses for anomaly detection. Pattern Recognit. 2011, 44(1):55-69.
57. X. Ding, Y. Li, A. Belatreche, L. Maguire, Constructing the minimum volume surfaces using level set methods for novelty detection, in: International Joint Conference on Neural Networks, Brisbane, Australia, 2012, pp. 3158-3163.
58. X. Ding, Y. Li, A. Belatreche, L. Maguire, Novelty detection using level set methods with adaptive boundaries, in: IEEE International Conference on SMC, Manchester, UK, 2013, pp. 3020-3025.
59. Ntalampiras S., Potamitis I., Fakotakis N. Probabilistic novelty detection for acoustic suiveillance under real-world conditions. IEEE Trans. Multimedia 2011, 13(4):713-719.
60. Clifton D.A., Hugueny S., Tarassenko L. Novelty detection with multivariate extreme value statistics. Signal Process. Syst. 2011, 65(3):371-389.
61. Filippone M., Sanguinetti G. Information theoretic novelty detection. Pattern Recognit. 2010, 43(3):805-814.
62. Benezeth Y., Jodoin P.M., Saligrama V. Abnormality detection using low-level co-occurring events. Pattern Recognit. Lett. 2011, 32(3):423-431.
63. Angiulli F., Ben-Eliyahu-Zohary R., Palopoli L. Outlier detection for simple default theories. Artif. Intell. 2010, 174(15):1247-1253.
64. Breaban M., Luchian H. Outlier detection with nonlinear projection pursuit. Int. J. Comput. Commun. Control 2013, 8(1):30-36.
65. Weinshall D., et al. Beyond novelty detection: incongruent events, when general and specific classifiers disagree. IEEE Trans. Pattern Anal. Mach. Intell. 2012, 34(10):1886-1901.
66. Miller David J., Browning John A mixture model and EM-based algorithm for class discovery, robust classification, and outlier rejection in mixed labeled/unlabeled data sets. IEEE Trans. Pattern Anal. Mach. Intell. 2003, 25(11):1468-1483.
67. Jeong Byeungwoo, Landgrebe D.A. Partially supervised classification using weighted unsupervised clustering. IEEE Trans. Geosci. Remote Sensing 1999, 37(2):1073-1079.
68. David J. Miller, H. Uyar, A mixture of experts classifier with learning based on both labelled and unlabelled data, in: Neural Information Processing Systems Conference, 1997, pp. 571-577.
69. Haggett S.J., Chu D.F. Evolving novelty detectors for specific applications. Neurocomputing 2009, 72(10-12):2392-2405.
70. Li Y., Pont M.J., Jones N.B., Twiddle J.A. Applying MLP and RBF classifiers in embedded condition monitoring and fault diagnosis systems. Trans. Inst. Meas. Control 2001, 23(5):315-343.
71. Tax D.M.J., Duin R.P.W. Support vector domain description. Pattern Recoginit. Lett. 1999, 20(11-13):1191-1199.
72. Lee K.Y., et al. Density-induced support vector data description. IEEE Trans. Neural Netw. 2007, 18(1):284-289.
73. Jian Luo, Bo Li, Changqing Wu, Yinghui Pan, A fast SVDD algorithm based on decomposition and combinaiton for fault detection, in: The Eighth IEEE International Conference on Control and Automation, Xiamen, China, 2010, pp. 1924-1928.
74. C.S. Chu, I.W. Tsang, J.T. Kwok, Scaling up support vector data description by using core-sets, in: IEEE International Joint Conference on Neural Networks, Budapest, 2004, pp. 425-430.
75. Li Y., Maguire L. Selecting critical patterns based on local geometrical and statistical information. IEEE Trans. Pattern Anal. Mach. Intell. 2011, 33(6):1189-1201.
76. Leiva-Murillo J.M., Artés-Rodríguez A. Algorithms for maximum-likelihood bandwidth selection in kernel density estimators. Pattern Recognit. Lett. 2012, 33(13):1717-1724.
77. Friedman J.H., Bentley J.L., Finkel R.A. An algorithm for finding best matches in logarithmic expected time. ACM Trans. Math. Softw. 1997, 3(3):209-226.
78. Samet Hanan K-nearest neighbor finding using MaxNearestDist. IEEE Trans. Pattern Anal. Mach. Intell. 2008, 30(2):243-252.
79. Varshney K.R., Willsky A.S. Classification using geometric level sets. J. Mach. Learn. Res. 2010, 11:491-516.
80. Hartigan J.A., Wong M.A. A k-means clustering algorithm. J. R. Stat. Soc.: Ser. C (Appl. Stat. 1979, 28(1):100-108.
81. Bishop C.M. Pattern Recognition and Machine Learning 2006, Springer, New York.
82. [Online]. ftp://ftp.dice.ucl.ac.be/pub/neural-net/ELENA/databases/REAL/phoneme.
83. C.C. Chang, C.J. Lin (2011) LIBSVM: A Library for Support Vector Machines. [Online]. http://www.csie.ntu.edu.tw/~cjlin/libsvm.
84. D.J. Newman, S. Hettich, C.L. Blake, C.J. Merz (1998) UCI Repository of Machine Learning Databases. [Online]. http://archive.ics.uci.edu/ml.
85. Harmeling S., Dornhege G., Tax D.M.J., Meinecke F., Muller K.R. From outliers to prototypes: ordering dat. Neurocomputing 2006, 69(13-15):1608-1618.
86. D.M.J. Tax. (2012) DDtools, the Data Description Toolbox for Matlab. [Online]. http://prlab.tudelft.nl/david-tax/dd_tools.html.
87. Fawcett T. An introduction to ROC analysis. Pattern Recoginit. Lett. 2006, 27(8):861-874.
88. R.P.W. Duin et al. (2010) PRTools: A Matlab Toolbox for Pattern Recognition. [Online]. http://www.prtools.org/index.html.
89. Demsar J. Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 2006, 7:1-30.