Improving classification performance of Support Vector Machine by genetically optimising kernel shape and hyper-parameters

Applied Intelligence

Volumn 36, Issue 2, 2012, Pages 280-294

  Dio̧san, Laura ^a,b   Rogozan, Alexandrina ^b   Pecuchet, Jean Pierre ^b  

^a UNIVERSITÉ ET INSA DE ROUEN   (France)

^b BABE BOLYAI UNIVERSITY   (Romania)

Author keywords

Classification problems; Genetic programming; Hybrid model; Hyper parameters optimization; Kernel of kernels; Multiple kernel; SVM

Indexed keywords

CLASSIFICATION ACCURACY; CLASSIFICATION PERFORMANCE; HETEROGENEOUS DATA; HYBRID MODEL; KERNEL OF KERNELS; KERNEL PARAMETER; MULTIPLE KERNELS; NUMERICAL EXPERIMENTS; REAL-WORLD APPLICATION; STANDARD TOOLS; STATE OF THE ART; SVM; SVM ALGORITHM; UNIFIED FRAMEWORK;

GENETIC ALGORITHMS; GENETIC PROGRAMMING; OPTIMIZATION;

SUPPORT VECTOR MACHINES;

EID: 84862152671     PISSN: 0924669X     EISSN: None     Source Type: Journal    
DOI: 10.1007/s10489-010-0260-1     Document Type: Article

References (63)

1. Bach FR, Thibaux R, Jordan MI (2004) Computing regularization paths for learning multiple kernels. In: NIPS, pp 1-10.
2. Banzhaf W (1998) Genetic programming: an introduction: on the automatic evolution of computer programs and its applications.
3. Bennett K, Hu J, Ji X, Kunapuli G, Pang J-S (2006) Model selection via bilevel optimization. In: IJCNN'06. International joint conference on neural networks. IEEE Computer Society, Los Alamitos, pp 1922-1929. (Pubitemid 351369560)
4. Boardman M, Trappenberg T (2006) A heuristic for free parameter optimization with SVM. In: IJCNN 2006. IEEE, New York, pp 1337-1344.
5. Boser BE, Guyon I, Vapnik V (1992) A training algorithm for optimal margin classifiers. In: COLT, pp 144-152. (Pubitemid 23615470)
6. Bousquet O, Herrmann DJL (2002) On the complexity of learning the kernel matrix. In: Becker S et al (eds) NIPS. MIT Press, Cambridge, pp 399-406.
7. Chang BR, Tsai H-F (2007) Composite of adaptive support vector regression and nonlinear conditional heteroscedasticity tuned by quantum minimization for forecasts. Appl Intell 27(3):277-289. (Pubitemid 47518991)
8. Chang C-C, Lin C-J (2001) LIBSVM a library for SVM. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
9. Chapelle O (2004) Support vector machines: induction principle, adaptive tuning and prior knowledge. PhD thesis, UPMC.
10. Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for Support Vector Machines.Mach Learn 46(1/3):131-159. (Pubitemid 34129966)
11. Cho S-B, Shimohara K (1998) Evolutionary learning of modular neural networks withgenetic programming. Appl Intell 9(3):191- 200. (Pubitemid 128512700)
12. Chung K-M, Kao W-C, Sun C-L, Wang L-L, Lin C-J (2003) Radius margin bounds for Support Vector Machines with the RBF kernel. Neural Comput 15(11):2643-2681. (Pubitemid 37206930)
13. Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273-297.
14. Crammer K, Singer Y (2002) On the algorithmic implementation of multiclass kernel-based vector machines. J Mach Learn Res 2:265-292.
15. Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines. Cambridge University Press, Cambridge.
16. Cristianini N, Shawe-Taylor J, Elisseeff A, Kandola JS (2001) On kernel-target alignment. In: Dietterich TG, Becker S, Ghahramani Z (eds) NIPS 2001. MIT Press, Cambridge, pp 367-373.
17. Dio̧san L, Oltean M, Rogozan A, Pécuchet JP (2007) Improving SVM performance using a linear combination of kernels. In: ICANNGA'07. LNCS, vol 4432, pp 218-227.
18. Dio̧san L, Rogozan A, Pécuchet J-P (2007) Evolving kernel functions for SVMs by genetic programming. In: ICMLA'07, Ohio, USA.
19. Frank A, Asuncion A (2010) UCI machine learning repository.
20. Friedrichs F, Igel C (2005) Evolutionary tuning of multiple SVM parameters. Neurocomputing 64:107-117. (Pubitemid 40425314)
21. Fröhlich H, Chapelle O, Schölkopf B (2003) Feature selection for SVM by means of GAs. In: ICTAI. IEEE, New York, pp 142-148.
22. Gagne C et al (2006) Genetic programming for kernel-based learning with co-evolving subsets selection. In: Runarsson TP et al (eds) 9th PPSN'06. Springer, Berlin, pp 1008-1017. (Pubitemid 44609477)
23. Girolami M, Rogers S (2005) Hierarchic Bayesian models for kernel learning. In: ICML, pp 241-248. (Pubitemid 43183339)
24. Gold C, Sollich P (2003) Model selection for Support Vector Machine classification. Neurocomputing 55(1-2):221-249. (Pubitemid 37336677)
25. Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison Wesley, Reading.
26. Gunn S, Kandola J (2002) Structural modelling with sparse kernels. Mach Learn 48:137-163. (Pubitemid 34247576)
27. Hastie T, Rosset S, Tibshirani R, Zhu J (2003/2004) The entire regularization path for the SVM. J Mach Learn Res 5:1391-1415.
28. Hooke R, Jeeves TA (1961) Direct search solution of numerical and statistical problems. J ACM 8:212-229.
29. Howley T, Madden MG (2005) The genetic kernel Support Vector Machine: description and evaluation. Artif Intell Rev 24(3- 4):379-395. (Pubitemid 41813391)
30. Huang Y (2009) Advances in artificial neural networks-methodological development and application. Algorithms 2(3):973- 1007.
31. Igel C (2005) Multi-objective model selection for SVM. In: Coello Coello CA et al (eds) EMO 2005. LNCS, vol 3410. Springer, Berlin, pp 534-546.
32. Imbault F, Lebart K (2004) A stochastic optimization approach for parameter tuning of SVM. In: ICPR (4), pp 597-600.
33. Joachims T (2001) The maximum-margin approach to learning text classifiers. Künstl Intell 15(3):63-65.
34. Keerthi S, Sindhwani V, Chapelle O (2006) An efficient method for gradient-based adaptation of hyperparameters in SVM models. In: NIPS'06. IEEE Computer Society, Los Alamitos, pp 1-10.
35. King RD (1992) Statlog databases.
36. Kirkpatrick S, Gelatt CD Jr, Vecchi MP (1983) Optimization by simulated annealing. Science 220:671-680.
37. Koza JR (1992) Genetic programming: on the programming of computers by means of natural selection. MIT Press, Cambridge.
38. Lacerda E, Carvalho AC, Braga AP, Ludermir TB (2005) Evolutionary radial basis functions for credit assessment. Appl Intell 22(3):167-181. (Pubitemid 40573951)
39. Lanckriet GRG et al (2004) Learning the kernel matrix with Semidefinite Programming. J Mach Learn Res 5:27-72.
40. Mallick BK, Ghosh D, Ghosh M (2005) Bayesian classification of tumours by using gene expression data. J R Stat Soc Ser B 67(2):219-234. (Pubitemid 40465872)
41. Mercer J (1909) Functions of positive and negative type and their connection with the theory of integral equations. Philos Trans R Soc 209:415-446.
42. Momma M, Bennett KP (2002) A pattern search method for model selection of SV Regression. In: Grossman RL et al (eds) SIAM 2002. SIAM, Philadelphia, pp 2-16.
43. Ohn S-Y, Nguyen H-N, Chi S-D (2004) Evolutionary parameter estimation algorithm for combined kernel function in SVM. In: Content computing, AWCC 2004. Springer, Berlin, pp 481-486. (Pubitemid 39744451)
44. Ong CS, Smola A, Williamson B (2005) Learning the kernel with hyperkernels. J Mach Learn Res 6:1043-1071.
45. Rakotomamonjy A, Bach FR, Canu S, Grandvalet Y (2007) More efficiency in multiple kernel learning. In: ICML, pp 775-782.
46. Lessmann RS, Crone S (2005) Genetically constructed kernels for SVM. In: Proc. of GOR. Springer, Berlin, pp 257-262.
47. Schölkopf B (2000) The kernel trick for distances. In: Leen TK, Dietterich TG, Tresp V (eds) NIPS. MIT Press, Cambridge, pp 301-307.
48. Schölkopf B, Smola AJ (2002) Learning with kernels. MIT Press, Cambridge.
49. Simon HA (2001) The sciences of the artificial, 3rd edn. MIT Press, Cambridge.
50. Sonnenburg S et al (2006) Large scale multiple kernel learning. J Mach Learn Res 7:1531-1565. (Pubitemid 44373694)
51. Staelin C (2003) Parameter selection for Support Vector Machines. Tech Rep HPL-2002-354R1, Hewlett Packard Laboratories.
52. Sullivan K, Luke S (2007) Evolving kernels for SVM classification. In: Lipson H (ed) GECCO 2007. ACM, New York, pp 1702- 1707.
53. Syswerda G (1991) A study of reproduction in generational and steady state Genetic Algorithms. In: Rawlins GJE (ed) FOGA. Morgan Kaufmann, San Mateo, pp 94-101.
54. Taylor JS, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, Cambridge.
55. Tsuda K, Rätsch G, Mika S, Müller K-R (2001) Learning to predict the leave-one-out error of kernel based classifiers. In: LNCS, vol 2130, pp 331-338. (Pubitemid 33316980)
56. Vapnik V (1995) The nature of statistical learning theory. Springer, Berlin.
57. Vapnik V, Chapelle O (2000) Bounds on error expectation for SVM. Neural Comput 12(9):2013-2036.
58. Verma B, Hassan S (2009) Hybrid ensemble approach for classification. Appl Intell, 1-21.
59. Wahba G, Lin Y, Zhang H (1999) GACV for support vector machines. In: Smola B, SchRolkopf S (eds) Advances in large margin classifiers. MIT Press, Cambridge.
60. Wang G, Yeung D-Y, Lochovsky FH (2007) A kernel path algorithm for SVM. In: ICML 07. ACMPress, New York, pp 951-958.
61. Xiong H, Swamy M, Ahmad M (2005) Optimizing the kernel in the empirical feature space. IEEE Trans Neural Netw 16(2):460- 474. (Pubitemid 40390828)
62. Zhang Z, Jordan MI (2006) Bayesian multicategory support vector machines. In: The twenty-second conference on uncertainty in artificial intelligence (UAI), 2006.
63. Zhang Z, Kwok JT, Yeung D-Y (2006) Model-based transductive learning of the kernel matrix. Mach Learn 63(1):69-101.