Learning SVM with complex multiple kernels evolved by genetic programming

International Journal on Artificial Intelligence Tools

Volumn 19, Issue 5, 2010, Pages 647-677

  Dioşan, Laura ^a,b   Rogozan, Alexandrina ^a   Pecuchet, Jean Pierre ^a  

^a University of Rouen   (France)

^b BABE BOLYAI UNIVERSITY   (Romania)

Author keywords

genetic programming; hybrid model; Multiple kernel learning; SVM

Indexed keywords

CLASSIFICATION ACCURACY; DATA SETS; EVOLUTIONARY SEARCH; FITNESS FUNCTIONS; HYBRID MODEL; KERNEL BASED CLASSIFIERS; MATHEMATICAL EXPRESSIONS; MULTIPLE KERNEL LEARNING; MULTIPLE KERNELS; NUMERICAL EXPERIMENTS; REAL WORLD DATA; REAL-WORLD APPLICATION; SIMPLE KERNEL; SINGLE KERNEL; SVM; SVM ALGORITHM; UCI REPOSITORY;

CLASSIFIERS; EVOLUTIONARY ALGORITHMS; FUNCTION EVALUATION; SUPPORT VECTOR MACHINES; TREES (MATHEMATICS);

GENETIC PROGRAMMING;

EID: 77958120029     PISSN: 02182130     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0218213010000352     Document Type: Article

References (59)

1. V. Vapnik, The Nature of Statistical Learning Theory (Springer, 1995).
2. B. Schoelkopf and A. J. Smola, Learning with Kernels (The MIT Press, 2002).
3. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee, Choosing multiple parameters for support vector machines, Machine Learning 46(1/3) (2002) 131-159. (Pubitemid 34129966)
4. G. R. G. Lanckriet, N. Cristianini, P. Bartlett, L. E. Ghaoui, and M. I. Jordan, Learning the kernel matrix with semidefinite programming, Journal of Machine Learning Research 5 (2004) 27-72.
5. V. N. Vapnik, Estimation of Dependences Based on Empirical Data (Springer, 1982).
6. B. Schölkopf, K. Tsuda, and J. P. Vert, Kernel Methods in Computational Biology (MIT Press, 2004).
7. H. W. Mewes, D. Frishman, C. Gruber, B. Geier, D. Haase, A. Kaps, K. Lemcke, G. Mannhaupt, F. Pfeiffer, C. Schüller, S. Stocker, and B. Weil, MIPS: A database for genomes and protein sequences (2002).
8. H. Lodhi, C. Saunders, J. Shawe-Taylor, N. Cristianini, and C. Watkins, Text classification using string kernels, Journal of Machine Learning Research 2 (2002) 419-444.
9. B. Vanschoenwinkel, B. Manderick, Appropriate kernel functions for support vector machine learning with sequences of symbolic data, in: J. Winkler, M. Niranjan, and N. D. Lawrence (Eds.), Deterministic and Statistical Methods in Machine Learning, Vol.3635 of LNCS (Springer, 2004), pp. 256-280.
10. S. Sonnenburg, G. Rtsch, C. Schafer, and B. Scholkopf, Large scale multiple kernel learning, Journal of Machine Learning Research 7 (2006) 1531-1565.
11. F. R. Bach, G. R. G. Lanckriet, and M. I. Jordan, Multiple kernel learning, conic duality, and the SMO algorithm, in ICML, ed. C. E. Brodley (ACM, 2004), pp. 41- 48.
12. A. Rakotomamonjy, F. R. Bach, S. Canu, and Y. Grandvalet, More efficiency in multiple kernel learning, in Proceedings of the Twenty-fourth International Conference on Machine Learning (ICML), 2007, accepted.
13. L. Diosan, M. Oltean, A. Rogozan, and J. P. Pecuchet, Improving svm performance using a linear combination of kernels, in Adaptive and Natural Computing Algorithms, ICANNGA'07, Vol.4432 of LNCS (Springer, 2007), pp. 218-227.
14. H.-N. Nguyen, S.-Y. Ohn, and W.-J. Choi, Combined kernel function for support vector machine and learning method based on evolutionary algorithm, in: N. R. Pal, N. Kasabov, R. K. Mudi, S. Pal, and S. K. Parui (Eds.), Neural Information Pro- cessing, 11th International Conference, ICONIP 2004, Vol.3316 of LNCS (Springer, 2004), pp. 1273-1278.
15. S.-Y. Ohn, H.-N. Nguyen, and S.-D. Chi, Evolutionary parameter estimation algorithm for combined kernel function in support vector machine, in: C.-H. Chi and K.-Y. Lam (Eds.), Content Computing, Advanced Workshop on Content Computing, AWCC 2004, Vol.3309 of LNCS (Springer, 2004), pp. 481-486.
16. S.-Y. Ohn, H.-N. Nguyen, D. S. Kim, and J. S. Park, Determining optimal decision model for support vector machine by genetic algorithm, in: J. Zhang, J.-H. He, and Y. Fu (Eds.), Computational and Information Science, First International Sympo- sium, CIS 2004, Vol.3314 of LNCS (Springer, 2004), pp. 895-902.
17. R. S. S. Lessmann and S. Crone, Genetically constructed kernels for support vector machines, in Proc. of German Operations Research (Springer, 2005), pp. 257-262.
18. C. S. Ong, A. Smola, and B. Williamson, Learning the kernel with hyperkernels, Journal of Machine Learning Research 6 (2005) 1043-1071.
19. N. Cristianini, J. Shawe-Taylor, A. Elisseeff, and J. S. Kandola, On kernel-target alignment, in: T. G. Dietterich, S. Becker, and Z. Ghahramani (Eds.), Advances in Neural Information Processing Systems 14 [Neural Information Processing Systems: Natural and Synthetic, NIPS 2001, December 3-8, 2001, Vancouver, British Columbia, Canada] (MIT Press, 2001), pp. 367-373.
20. D. H. Wolpert and W. G. Macready, No free lunch theorems for search, Tech. Rep. SFI-TR-95-02-010, Santa Fe Institute (1995).
21. J. R. Koza, Genetic Programming: On the Programming of Computers by Means of Natural Selection (MIT Press, 1992).
22. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines (Cambridge University Press, 2000).
23. T. Joachims, Making large-scale SVM learning practical, in Advances in Kernel Method - Support Vector Learning, eds. B. Scholkopf, C. J. C. Burges, and A. J. Smola (MIT Press, 1999), pp. 169-184.
24. J. C. Burges, A tutorial on support vector machines for pattern recognition, in Knowl- edge Discovery and Data Mining, Vol.2 (Kluwer Academic, 1998), pp. 121-167.
25. V. Vapnik, Statistical Learning Theory (Wiley, 1998).
26. M. A. Aizerman, E. M. Braverman, and L. I. Rozonóer, Theoretical foundations of the potential function method in pattern recognition learning, Automation and Remote Control 25 (1964) 821-837.
27. B. E. Boser, I. Guyon, and V. Vapnik, A training algorithm for optimal margin classifiers, in COLT, 1992, pp. 144-152.
28. B. Schölkopf, The kernel trick for distances in T. K. Leen, T. G. Dietterich, and V. Tresp (Eds.) NIPS (MIT Press, 2000), pp. 301-307.
29. C. Cortes and V. Vapnik, Support-vector networks, Machine Learning 20 (1995) 273-297.
30. H. Lin and C. Lin, A study on sigmoid kernels for SVM and the training of non-PSD kernels by SMO-type methods (2003).
31. T. Howley and M. G. Madden, The genetic kernel support vector machine: Description and evaluation, Artif. Intell. Rev. 24(3-4) (2005) 379-395.
32. T. Howley and M. G. Madden, An evolutionary approach to automatic kernel construction, in: S. D. Kollias, A. Stafylopatis, W. Duch, and E. Oja (Eds.), Artificial Neural Networks - ICANN 2006, Vol.4132 of LNCS (Springer, 2006), pp. 417-426.
33. L. Dio̧san, A. Rogozan, J.-P. Pcuchet, Evolving kernel functions for svms by genetic programming, in The 2007 International Conference on Machine Learning and Ap- plications (ICMLA'07 ), Ohio, USA, 2007, accepted.
34. C. Gagne, M. Schoenauer, M. Sebag, and M. Tomassini, Genetic programming for kernel-based learning with co-evolving subsets selection, in: T. P. Runarsson, H.-G. Beyer, E. K. Burke, J. J. M. Guervos, L. D.Whitley, and X. Yao (Eds.), Parallel Prob- lem Solving from Nature - PPSN IX, (9th PPSN'06), Vol.4193 of LNCS (Springer, 2006), pp. 1008-1017.
35. C. Berg, J. P. R. Christensen, and P. Ressel, Harmonic Analysis on Semigroups (Springer, 1984).
36. M. R. Hestenes and E. Stiefel, Methods of conjugate gradients for solving linear systems, Journal of Research of the National Bureau of Standards 49(6) (1952) 409-436.
37. D. E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning (Addison Wesley, 1989).
38. F. R. Bach, G. R. G. Lanckriet, and M. I. Jordan, Multiple kernel learning, conic duality, and the SMO algorithm, in Machine Learning, Proceedings of ICML 2004 (ACM, 2004), p. 6.
39. M. Girolami and S. Rogers, Hierarchic bayesian models for kernel learning, in ICML, 2005, pp. 241-248.
40. J. Platt, Fast training of support vector machines using sequential minimal optimization, in: B. Schölkopf, C. J. C. Burges, A. J. Smola (Eds.), Advances in Kernel Methods - Support Vector Learning (MIT Press, 1999), pp. 185-208.
41. W. Banzhaf, Genetic programming: An introduction: On the automatic evolution of computer programs and its applications (1998).
42. G. Syswerda, Uniform crossover in genetic algorithms, in ICGA, 1989, pp. 2-9.
43. G. Syswerda, A study of reproduction in generational and steady state genetic algorithms, in: G. J. E. Rawlins (Ed.), Proceedings of FOGA Conference (Morgan Kaufmann, 1991), pp. 94-101.
44. C.-C. Chang and, C.-J. Lin, LIBSVM: a library for support vector machines, software available at http://www.csie.ntu.edu.tw/?cjlin/libsvm (2001).
45. C. B. D.J. Newman, S. Hettich, and C. Merz, UCI repository of machine learning databases (1998), http://www.ics.uci.edu/?mlearn/MLRepository.html
46. R. D. King, Statlog databases (1992), http://www.liacc.up.pt/ML/old/ statlog/ datasets.html
47. J. J. Grefenstette, Optimization of control parameters for genetic algorithms, IEEE Transactions on Systems, Man, and Cybernetics SMC 16(1) (1986) 122-128.
48. T. Bäck, Parallel optimization of evolutionary algorithms, in: Y. Davidor, H.-P. Schwefel (Eds.), Parallel Problem Solving From Nature - PPSN III, Vol.866 of Lecture Notes in Computer Science (Springer Verlag, 1994), pp. 418-427.
49. W. Banzhaf, F. D. Francone, and P. Nordin, The effect of extensive use of the mutation operator on generalization in genetic programming using sparse data sets, Lecture Notes in Computer Science 1141 (1996) 300-309.
50. R. Poli and W. B. Langdon, Genetic programming with one-point crossover, in: P. K. Chawdhry, R. Roy, R. K. Pant (Eds.), Soft Computing in Engineering Design and Manufacturing (Springer-Verlag, London, 1997), pp. 180-189.
51. A. Piszcz and T. Soule, Genetic programming: Analysis of optimal mutation rates in a problem with varying difficulty, in: G. C. J. Sutcliffe, R. G. Goebel (Eds.), Proceed- ings of the Nineteenth International Florida Artificial Intelligence Research Society Conference (American Association for Artificial Intelligence, 2006), pp. 451-456.
52. G. Wang, D.-Y. Yeung, and F. H. Lochovsky, A kernel path algorithm for support vector machines, in ICML '07: Proceedings of the 24th International Conference on Machine Learning (ACM Press, 2007), pp. 951-958.
53. M. R. Hestenes and E. Stiefel, Machines noyaux pour lapprentissage statistique, Technologies de l'information 4(1) (2007) 1-20.
54. O. Chapelle, Support vector machines: Induction principle, adaptive tuning and prior knowledge, Ph.D. thesis, UPMC (2004).
55. P. J. Angeline, An investigation into the sensitivity of genetic programming to the frequency of leaf selection during subtree crossover, in: J. R. Koza, D. E. Goldberg, D. B. Fogel, R. L. Riolo (Eds.), Genetic Programming 1996: Proceedings of the First Annual Conference (MIT Press, 1996), pp. 21-29.
56. T. Bäck, Evolutionary Algorithms in Theory and Practice (Oxford University Press, New York, 1996).
57. D. B. Fogel, L. J. Fogel, and J. W. Atmar, Meta-evolutionary programming, in: R. R. Chen (Ed.), Proc. of the 25th Asilomar Conf. on Signals, Systems, and Computers (Maple, 1991), pp. 540-545.
58. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, Gradient-based learning applied to document recognition, Proceedings of the IEEE 86(11) (1998) 2278-2324.
59. S. Knerr, L. Personnaz, and G. Dreyfus, Single-layer learning revisited: A stepwise procedure for building and training a neural network, Neurocomputing: Algorithms, Architectures and Applications F68 (1990) 41-50.