Deep learning with support vector data description

Neurocomputing

Volumn 165, Issue , 2015, Pages 111-117

  Kim, Sangwook ^a   Choi, Yonghwa ^a   Lee, Minho ^a  

^a Kyungpook National University   (South Korea)

Author keywords

Deep learning; Generalization; Pattern recognition; Support vector data description

Indexed keywords

DATA DESCRIPTION; DEEP NEURAL NETWORKS; LEARNING SYSTEMS; PATTERN RECOGNITION; SUPPORT VECTOR MACHINES; VECTORS;

CRITICAL PROBLEMS; GENERALIZATION; GENERALIZATION PERFORMANCE; LEARNING NEURAL NETWORKS; MACHINE LEARNING METHODS; ONE-CLASS CLASSIFICATION; OVER FITTING PROBLEM; SUPPORT VECTOR DATA DESCRIPTION;

DEEP LEARNING;

ALGORITHM; ARTICLE; ARTIFICIAL NEURAL NETWORK; INFORMATION PROCESSING; LEARNING ALGORITHM; MATHEMATICAL COMPUTING; MATHEMATICAL PARAMETERS; MEASUREMENT ACCURACY; MODEL; PRIORITY JOURNAL; SIMULATION; SUPPORT VECTOR DATA DESCRIPTION; SUPPORT VECTOR MACHINE;

EID: 84929944640     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2014.09.086     Document Type: Article

References (41)

1. Arel I., Rose D.C., Karnowski T.P. Deep machine learning-a new frontier in artificial intelligence research [research frontier]. Comput. Intell. Mag. IEEE 2010, 5:13-18.
2. ® Mach. Learn. 2009, 2:1-127.
3. G. Tesauro, Practical issues in temporal difference learning, in: Reinforcement Learning, Springer, 1992, pp. 33-53.
4. Utgoff P.E., Stracuzzi D.J. Many-layered learning. Neural Comput. 2002, 14:2497-2529.
5. Fukushima K. Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern. 1980, 36:193-202.
6. Ackley D.H., Hinton G.E., Sejnowski T.J. A learning algorithm for Boltzmann machines. Cogn. Sci. 1985, 9:147-169.
7. P. Smolensky, Information Processing in Dynamical Systems: Foundations of Harmony Theory, 1986.
8. Hinton G.E., Osindero S., Teh Y.-W. A fast learning algorithm for deep belief nets. Neural Comput. 2006, 18:1527-1554.
9. Le Roux N., Bengio Y. Representational power of restricted Boltzmann machines and deep belief networks. Neural Comput. 2008, 20:1631-1649.
10. Vincent P., Larochelle H., Lajoie I., Bengio Y., Manzagol P.-A. Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res. 2010, 11:3371-3408.
11. Hinton G. A practical guide to training restricted Boltzmann machines. Momentum 2010, 9:926.
12. Y. LeCun, Y. Bengio, Convolutional networks for images, speech, and time series, in: The Handbook of Brain Theory and Neural Networks, 3361, 1995.
13. Golub G.H., Heath M., Wahba G. Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics 1979, 21:215-223.
14. G.E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, R.R. Salakhutdinov, Improving neural networks by preventing co-adaptation of feature detectors, arXiv preprint, 2012. arxiv:1207.0580.
15. I.J. Goodfellow, D. Warde-Farley, M. Mirza, A. Courville, Y. Bengio, Maxout networks, arXiv preprint, 2013. arxiv:1302.4389.
16. L. Wan, M. Zeiler, S. Zhang, Y.L. Cun, R. Fergus, Regularization of neural networks using dropconnect, in: Proceedings of the 30th International Conference on Machine Learning (ICML-13), 2013, pp. 1058-1066.
17. Cortes C., Vapnik V. Support-vector networks. Mach. Learn. 1995, 20:273-297.
18. Bishop C.M. Pattern Recognition and Machine Learning 2006, Springer, New York.
19. Shawe-Taylor J., Bartlett P.L., Williamson R.C., Anthony M. Structural risk minimization over data-dependent hierarchies. Inf. Theory IEEE Trans. 1998, 44:1926-1940.
20. Tax D.M., Duin R.P. Support vector data description. Mach. Learn. 2004, 54:45-66.
21. Kim S., Kavuri S., Lee M. Deep network with support vector machines. Neural Information Processing 2013, 458-465. Springer, Berlin Heidelberg. M. Lee, A. Hirose, Z.-G. Hou, R. Kil (Eds.).
22. Schölkopf B., Burgest C., Vapnik V. Extracting support data for a given task, Proceedings, First International Conference on Knowledge Discovery & Data Mining 1995, AAAI Press, Menlo Park, CA.
23. Tax D.M., Duin R.P. Uniform object generation for optimizing one-class classifiers. J. Mach. Learn. Res. 2002, 2:155-173.
24. Lee S.-W., Park J., Lee S.-W. Low resolution face recognition based on support vector data description. Pattern Recognit. 2006, 39:1809-1812.
25. A. Banerjee, P. Burlina, R. Meth, Fast hyperspectral anomaly detection via SVDD, in: Proceedings of the IEEE International Conference on Image Processing, ICIP, 2007, pp. IV-101-IV-104.
26. Ge Z., Gao F., Song Z. Batch process monitoring based on support vector data description method. J. Process Control 2011, 21:949-959.
27. Vapnik V.N., Chervonenkis A.J. Theory of Pattern Recognition 1974, Nauka, Moscow.
28. Moody J., Hanson S., Krogh A., Hertz J.A. A simple weight decay can improve generalization. Advan. Neural Inf. Process. Syst. 1995, 4:950-957.
29. LeCun Y. Generalization and network design strategies. Connections in Perspective 1989, Elsevier, Zurich, Switzerland. R. Pfeifer, Z. Schreter, F. Fogelman, L. Steels (Eds.).
30. Nowlan S.J., Hinton G.E. Simplifying neural networks by soft weight-sharing. Neural Comput. 1992, 4:473-493.
31. Amari S.-I., Murata N., Muller K.-R., Finke M., Yang H.H. Asymptotic statistical theory of overtraining and cross-validation. Neural Netw. IEEE Trans. 1997, 8:985-996.
32. Y. Bengio, Y. LeCun, Scaling learning algorithms towards AI, in: Large-Scale Kernel Machines, vol. no. 34, 2007, pp. 1-41.
33. Erhan D., Bengio Y., Courville A., Manzagol P.-A., Vincent P., Bengio S. Why does unsupervised pre-training help deep learning?. J. Mach. Learn. Res. 2010, 11:625-660.
34. Valiant L.G. A theory of the learnable. Commun. ACM 1984, 27:1134-1142.
35. P. Bartlett, J. Shawe-Taylor, Generalization performance of support vector machines and other pattern classifiers, in: Advances in Kernel Methods-Support Vector Learning, 1999, pp. 43-54.
36. Mangasarian O.L., Street W.N., Wolberg W.H. Breast cancer diagnosis and prognosis via linear programming. Oper. Res. 1995, 43:570-577.
37. Sigillito V. Pima Indians Diabetes Database 1990, 9. The Johns Hopkins University Applied Physics Laboratory, Laurel, MD.
38. Lucas D., Klein R., Tannahill J., Ivanova D., Brandon S., Domyancic D., Zhang Y. Failure analysis of parameter-induced simulation crashes in climate models. Geosci. Model Dev. Discuss. 2013, 6:1157-1171.
39. M. Lichman, UCI Machine Learning Repository []. Irvine, CA: University of California. http://archive.ics.uci.edu/ml.
40. D. Tax, DDtools, the Data Description Toolbox for Matlab, 2014 (version 2.1.1).
41. J. Weston, C. Watkins, Multi-Class Support Vector Machines, Technical Report CSD-TR-98-04, Department of Computer Science, Royal Holloway, University of London, 1998.