One class random forests

Pattern Recognition

Volumn 46, Issue 12, 2013, Pages 3490-3506

  Désir, Chesner ^a   Bernard, Simon ^a   Petitjean, Caroline ^a   Heutte, Laurent ^a  

^a UNIVERSITÉ DE ROUEN   (France)

Author keywords

Decision trees; Ensemble methods; One class classification; Outlier detection; Outlier generation; Random forests; Supervised learning

Indexed keywords

ENSEMBLE METHODS; ONE-CLASS CLASSIFICATION; OUTLIER DETECTION; OUTLIER GENERATIONS; RANDOM FORESTS;

ALGORITHMS; RANDOM PROCESSES; STATISTICS; SUPERVISED LEARNING;

DECISION TREES;

EID: 84881077165     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2013.05.022     Document Type: Article

References (92)

1. M. Moya, and D. Hush Network constraints and multi-objective optimization for one-class classification Neural Networks 9 3 1996 463 474
2. M. Koppel, J. Schler, Authorship verification as a one-class classification problem, in: Proceedings of the Twenty-First International Conference on Machine Learning, ACM, 2004, p. 62.
3. K. Hempstalk, E. Frank, and I. Witten One-class classification by combining density and class probability estimation Machine Learning and Knowledge Discovery in Databases 2008 505 519
4. A. Brew, M. Grimaldi, and P. Cunningham An evaluation of one-class classification techniques for speaker verification Artificial Intelligence Review 27 4 2007 295 307
5. O. Mazhelis One-class classifiers a review and analysis of suitability in the context of mobile-masquerader detection South African Computer Journal (SACJ), ARIMA & SACJ Joint Special Issue on Advances in End-User Data-Mining Techniques 36 2006 29 48
6. W. Fan, M. Miller, S. Stolfo, W. Lee, and P. Chan Using artificial anomalies to detect unknown and known network intrusions Knowledge and Information Systems 6 5 2004 507 527
7. K. Wang, S. Stolfo, One-class training for masquerade detection, in: Workshop on Data Mining for Computer Security, Citeseer, 2003, pp. 19-22.
8. C. Désir, S. Bernard, C. Petitjean, L. Heutte. A random forest based approach for one class classification in medical imaging. 3rd MICCAI International Workshop on Machine Learning in Medical Imaging (MLMI), Nice, France, LNCS 7588, 2012, pp. 250-257
9. V. Hodge, and J. Austin A survey of outlier detection methodologies Artificial Intelligence Review 22 2 2004 85 126
10. V. Chandola, A. Banerjee, and V. Kumar Anomaly detection a survey ACM Computing Surveys (CSUR) 41 3 2009 1 58
11. R. Duda, and P. Hart Pattern Classification and Scene Analysis 1973 John Wiley and Sons
12. D. Tax, and R. Duin Combining one-class classifiers J. Kittler, F. Roli, Multiple Classifier Systems, Lecture Notes in Computer Science vol. 2096 2001 Springer 299 308
13. S. Khan, and M. Madden A survey of recent trends in one class classification Artificial Intelligence and Cognitive Science 2010 188 197
14. C. Desir, S. Bernard, C. Petitjean, L. Heutte, A new random forest method for one-class classification, in: IAPR International Workshop on Statistical Techniques in Pattern Recognition, SPR 2012, Hiroshima, Japan, Lecture Notes in Computer Science, vol. 7626, 2012, pp. 282-290.
15. L. Breiman Random forests Machine Learning 45 1 2001 5 32
16. B. Ng, Survey of Anomaly Detection Methods, 2006.
17. V. Chandola, A. Banerjee, and V. Kumar Outlier detection a survey ACM Computing Surveys 2009 1 72
18. S. Marsland Novelty detection in learning systems Neural Computing Surveys 3 2003 157 195
19. M. Markou, and S. Singh Novelty detection a review-part 1: statistical approaches Signal Processing 83 12 2003 2481 2497
20. N. Abe, B. Zadrozny, J. Langford, Outlier detection by active learning, in: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, ACM, 2006, pp. 504-509.
21. D. Tax, R. Duin, Outlier detection using classifier instability, in: Lecture Notes in Computer Science, 1998, pp. 593-601.
22. A. Nairac, T. Corbett-Clark, R. Ripley, N. Townsend, L. Tarassenko, Choosing an appropriate model for novelty detection, in: Fifth International Conference on Artificial Neural Networks (Conference Publication No. 440), IET, 1997, pp. 117-122.
23. J. Toivola, M. Prada, and J. Hollmén Novelty detection in projected spaces for structural health monitoring Advances in Intelligent Data Analysis IX 2010 208 219
24. L. Tarassenko, D. Clifton, P. Bannister, S. King, D. King, Novelty detection, Encyclopedia of Structural Health Monitoring.
25. G. Cohen, H. Sax, A. Geissbuhler, Novelty detection using one-class Parzen density estimator. An application to surveillance of nosocomial infections, in: EHealth Beyond the Horizon: Get It There: Proceedings of MIE2008 the XXIst International Congress of the European Federation for Medical Informatics, Ios Pr Inc, 2008, p. 21.
26. L. Tarassenko, P. Hayton, N. Cerneaz, M. Brady, Novelty detection for the identification of masses in mammograms, in: Fourth International Conference on Artificial Neural Networks, 1995, pp. 442-447.
27. C. Bishop Novelty detection and neural network validation IEE Proceedings - Vision, Image and Signal Processing 141 4 1994 217 222
28. R. Duin On the choice of smoothing parameters for Parzen estimators of probability density functions IEEE Transactions on Computers 100 11 1976 1175 1179
29. M. Kraaijveld, R. Duin, A Criterion for the Smoothing Parameter for Parzen-Estimators of Probability Density Functions, Technical Report, Delft University of Technology, 1991.
30. C.M. Bishop Neural Networks for Pattern Recognition 1995 Clarendon Press Oxford
31. T. Hastie, R. Tibshirani, and J.H. Friedman The Elements of Statistical Learning 2001 Springer Verlag
32. A.P. Dempster, N.M. Laird, and D.B. Rubin Maximum likelihood from incomplete data via the em algorithm Journal of Royal Statistical Society 39 1 1977 1 38
33. S. Roberts, and L. Tarassenko A probabilistic resource allocating network for novelty detection Neural Computation 6 2 1994 270 284
34. D. Tax, and R. Duin Support vector data description Machine Learning 54 1 2004 45 66
35. K. Beyer, J. Goldstein, R. Ramakrishnan, and U. Shaft When is "nearest neighbor" meaningful? Database Theory - ICDT 99 1999 217 235
36. M. Yousef, N. Najami and W. Khaliefa, A Comparison Study Between One-Class and Two-Class Machine Learning for MicroRNA Target Detection, Journal of Biomedical Science and Engineering 3, 2010, p. 247-252
37. F. Ratle, M. Kanevski, A. Terrettaz-Zufferey, P. Esseiva, and O. Ribaux A comparison of one-class classifiers for novelty detection in forensic case data Intelligent Data Engineering and Automated Learning - IDEAL 2007 2007 67 76
38. M. Markou, and S. Singh Novelty detection: a review. Part 2: neural network based approaches Signal Processing 83 12 2003 2499 2521
39. M. Desforges, P. Jacob, and J. Cooper Applications of probability density estimation to the detection of abnormal conditions in engineering Proceedings of the Institution of Mechanical Engineers, Part C Journal of Mechanical Engineering Science 212 8 1988 687 703
40. N. Japkowicz, C. Myers, M. Gluck, A novelty detection approach to classification, in: International Joint Conference on Artificial Intelligence, vol. 14, Lawrence Erlbaum Associates Ltd, 1995, pp. 518-523.
41. V. Vapnik Statistical Learning Theory 1998 Wiley Interscience
42. B. Liu, Y. Xia, P. Yu, Clustering through decision tree construction, in: Proceedings of the Ninth International Conference on Information and Knowledge Management, ACM New York, NY, USA, 2000, pp. 20-29.
43. C. Aggarwal, P. Yu, Outlier detection for high dimensional data, in: Proceedings of the 2001 ACM SIGMOD International Conference on Management of Data, ACM, 2001, p. 46.
44. B. Scholkopf, J. Platt, J. Shawe-Taylor, A. Smola, and R. Williamson Estimating the support of a high-dimensional distribution Neural computation 13 7 2001 1443 1471
45. A. Shieh, and D. Kamm Ensembles of one class support vector machines Multiple Classifier Systems 2009 181 190
46. P. Evangelista, M. Embrechts, and B. Szymanski Taming the curse of dimensionality in kernels and novelty detection Applied Soft Computing Technologies The Challenge of Complexity 2006 425 438
47. T. Dietterich Ensemble methods in machine learning J. Kittler, F. Roli, Multiple Classifier Systems, Lecture Notes in Computer Science vol. 1857 2000 Springer 1 15 URL http://dblp.uni-trier.de/db/conf/mcs/mcs2000.html#Dietterich00
48. L. Kuncheva Combining pattern classifiers: methods and algorithms (Kuncheva, li; 2004)[book review] IEEE Transactions on Neural Networks 18 3 2007 964
49. R. Genuer, J. Poggi, C. Tuleau, Random Forests: Some Methodological Insights, Arxiv Preprint arXiv:0811.3619.
50. G. Biau, Analysis of a random forests model, Arxiv preprint arXiv:1005.0208.
51. M. Robnik-Sikonja Improving random forests Machine Learning ECML 2004 2004 359 370
52. L. Kuncheva, and J. Rodríguez An experimental study on rotation forest ensembles Multiple Classifier Systems 2007 459 468
53. S. Bernard, L. Heutte, and S. Adam Influence of hyperparameters on random forest accuracy Multiple Classifier Systems 2009 171 180
54. L. Breiman Bagging predictors Machine Learning 26 2 1996 123 140
55. P. Geurts, D. Ernst, and L. Wehenkel Extremely randomized trees Machine Learning 63 1 2006 3 42
56. S. Bernard, L. Heutte, and S. Adam Forest-rk a new random forest induction method Advanced Intelligent Computing Theories and Applications. With Aspects of Artificial Intelligence 2008 430 437
57. D. Tax, and R. Duin Uniform object generation for optimizing one-class classifiers The Journal of Machine Learning Research 2 2002 155 173
58. T. Ho The random subspace method for constructing decision forests IEEE Transactions on Pattern Analysis and Machine Intelligence 20 8 1998 832 844
59. D. Tax, A. Ypma, R. Duin, Support vector data description applied to machine vibration analysis, in: Proceedings of the Fifth Annual Conference of the Advanced School for Computing and Imaging, Heijen, NL, Citeseer, 1999.
60. C. Blake, C. Merz, Uci Repository of Machine Learning Databases, University of California, Department of Information and Computer Science 55, Irvine, CA http://www.ics.uci.edu/mlearn/mlrepository.html.
61. P. Baldi, S. Brunak, Y. Chauvin, C. Andersen, and H. Nielsen Assessing the accuracy of prediction algorithms for classification an overview Bioinformatics 16 5 2000 412 424
62. B. Matthews Comparison of the predicted and observed secondary structure of t4 phage lysozyme Biochimica et Biophysica Acta (BBA) - Protein Structure 405 2 1975 442 451
63. T. Hastie, R. Tibshirani, J. Friedman, and J. Franklin The elements of statistical learning data mining, inference and prediction The Mathematical Intelligencer 27 2 2005 83 85
64. R. Kohavi, Wrappers for Performance Enhancement and Oblivious Decision Graphs, 1996.
65. Z. Zheng, A benchmark for classifier learning, in: Proceedings of the Sixth Australian Joint Conference on Artificial Intelligence, Citeseer, 1993, pp. 281-286.
66. R. Duin A note on comparing classifiers Pattern Recognition Letters 17 5 1996 529 536
67. D. Hand Classifier technology and the illusion of progress Statistical Science 21 1 2006 1 14
68. A. Jamain, and D.J. Hand Mining supervised classification performance studies a meta-analytic investigation Journal of Classification 25 2008 87 112 10.1007/s00357-008-9003-y
69. J. Demsar Statistical comparisons of classifiers over multiple data sets The Journal of Machine Learning Research 7 2006 1 30
70. T.G. Dietterich Approximate statistical tests for comparing supervised classification learning algorithms Neural Computation 10 7 1998 1895 1924
71. C. Nadeau, and Y. Bengio Inference for the generalization error Machine Learning 52 3 2003 239 281
72. P. Brazdil, and C. Soares A comparison of ranking methods for classification algorithm selection Machine Learning ECML 2000 2000 63 75
73. J. Menke, T. Martinez, Using permutations instead of Student's t distribution for p-values in paired-difference algorithm comparisons, in: 2004 IEEE International Joint Conference on Neural Networks, Proceedings, vol. 2, IEEE, 2004, pp. 1331-1335.
74. R. Banfield, L. Hall, K. Bowyer, and W. Kegelmeyer A comparison of decision tree ensemble creation techniques IEEE Transactions on Pattern Analysis and Machine Intelligence 29 1 2007 173 180
75. S. Fisher Statistical Methods and Scientific Inference vol. 1959 1959 Oliver and Boyd
76. M. Friedman The use of ranks to avoid the assumption of normality implicit in the analysis of variance Journal of the American Statistical Association 32 200 1937 675 701
77. M. Friedman A comparison of alternative tests of significance for the problem of m rankings The Annals of Mathematical Statistics 11 1 1940 86 92
78. P. Nemenyi, Distribution-Free Multiple Comparisons, PhD Thesis, Princeton University, 1963.
79. R. Iman, J. Davenport, Approximations of the Critical Region of the Friedman Statistic, Technical Report, Sandia Labs., Albuquerque, NM, USA; Texas Tech University, Lubbock, USA, 1979.
80. O. Dunn Multiple comparisons among means Journal of the American Statistical Association 1961 52 64
81. R. Duin, PRTools Version 3.0: A Matlab Toolbox for Pattern Recognition, in: Proceedings of the SPIE, Citeseer, 2000.
82. D. Tax, Ddtools, The Data Description Toolbox for Matlab, Version 1.7.3, December 2009.
83. L.M. Manevitz, and M. Yousef One-class svms for document classification Journal of Machine Learning Research 2 2002 139 154
84. K. Mardia Applications of some measures of multivariate skewness and kurtosis in testing normality and robustness studies Sankhya The Indian Journal of Statistics, Series B 1974 115 128
85. A. Von Eye, and G. Bogat Testing the assumption of multivariate normality Psychology Science 46 2004 243 258
86. H. Holgersson, and R. Mansoor Assessing normality of high-dimensional data Communications in Statistics - Simulation and Computation 42 2 2013 360 369
87. J. Liang, M.-L. Tang, and P.S. Chan A generalized Shapiro-Wilk w statistic for testing high-dimensional normality Computational Statistics & Data Analysis 53 11 2009 3883 3891
88. J.-N. Hwang, S.-R. Lay, and A. Lippman Nonparametric multivariate density estimation a comparative study IEEE Transactions on Signal Processing 42 10 1994 2795 2810
89. S.P. Awate, Adaptive, Nonparametric Markov Models and Information - Theoretic Methods for Image Restoration and Segmentation, PhD Thesis, Salt Lake City, UT, USA, aAI3239672, 2006.
90. K. Popat, and R.W. Picard Cluster-based probability model and its application to image and texture processing IEEE Transactions on Image Processing 6 2 1997 268 284
91. M. Verleysen Learning high-dimensional data Nato Science Series III, Computer And Systems Sciences 186 2003 141 162
92. D. Donoho, Aide-memoire. High-Dimensional Data Analysis: The Curses and Blessings of Dimensionality, American Math. Society Lecture-Math Challenges of the 21st Century.