Efficient multi-criteria optimization on noisy machine learning problems

Applied Soft Computing Journal

Volumn 29, Issue , 2015, Pages 357-370

  Koch, Patrick ^a   Wagner, Tobias ^b   Emmerich, Michael T M ^c   Bäck, Thomas ^c   Konen, Wolfgang ^a  

^a TH Köln University of Applied Sciences   (Germany)

^b TU DORTMUND UNIVERSITY   (Germany)

^c LEIDEN UNIVERSITY   (Netherlands)

Author keywords

Efficient Global Optimization; Hypervolume indicator; Kriging; Machine learning; Multi criteria optimization

Indexed keywords

ARTIFICIAL INTELLIGENCE; BUDGET CONTROL; GLOBAL OPTIMIZATION; INTERPOLATION; SUPERVISED LEARNING;

EFFICIENT GLOBAL OPTIMIZATION; HYPERVOLUME INDICATORS; KRIGING; LATIN HYPERCUBE SAMPLING; MACHINE LEARNING PROBLEM; MULTICRITERIA OPTIMIZATION; SUPERVISED MACHINE LEARNING; TRADITIONAL APPROACHES;

LEARNING SYSTEMS;

EID: 84954468789     PISSN: 15684946     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asoc.2015.01.005     Document Type: Article

References (70)

1. P. Koch, B. Bischl, O. Flasch, T. Bartz-Beielstein, C. Weihs, and W. Konen Tuning and evolution of support vector kernels Evol. Intell. 2011 1 18 (Special Issue on Evolutionary Kernel Machines)
2. P. Koch, and W. Konen Efficient sampling and handling of variance in tuning data mining models V. Cutello, M. Pavone, PPSN 2012: 12th International Conference on Parallel Problem Solving from Nature 2012 Springer Berlin, Taormina, Italy 195 205
3. W. Konen, P. Koch, O. Flasch, T. Bartz-Beielstein, M. Friese, and B. Naujoks Tuned data mining: a benchmark study on different tuners Proceedings of Genetic and Evolutionary Computation Conference (GECCO) 2011 ACM, Dublin, Ireland 2011 1995 2002
4. D. Jones, M. Schonlau, and W. Welch Efficient global optimization of expensive black-box functions J. Global Optim. 13 4 1998 455 492
5. T. Wagner, M. Emmerich, A. Deutz, and W. Ponweiser On expected-improvement criteria for model-based multi-objective optimization Proceedings of the International Conference on Parallel Problem Solving from Nature (PPSN XI) 2010 Springer 718 727
6. V. Vapnik Statistical Learning Theory 1998 John Wiley and Sons Inc. New York
7. B. Schölkopf, and A. Smola Learning with Kernels: Support Vector Machines, Regularization, Optimization and Beyond 2002 The MIT Press Cambridge, MA, USA
8. K. Burnham, and D. Anderson Model Selection and Multi-model Inference: A Practical Information-theoretic Approach 2002 Springer New York
9. W. Zucchini An introduction to model selection J. Math. Psychol. 44 1 2000 41 61
10. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee Choosing multiple parameters for support vector machines Mach. Learn. 46 1 2002 131 159
11. S. Keerthi Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms IEEE Trans. Neural Netw. 13 5 2002 1225 1229
12. S. Keerthi, V. Sindhwani, and O. Chapelle An efficient method for gradient-based adaptation of hyperparameters in SVM models Schölkopf B., Advances in Neural Information Processing Systems 19 2007 673 680
13. G. Cohen, M. Hilario, and A. Geissbuhler Model selection for support vector classifiers via genetic algorithms. An application to medical decision support Biol. Med. Data Anal. 2004 200 211
14. F. Friedrichs, and C. Igel Evolutionary tuning of multiple SVM parameters Neurocomputing 64 2005 107 117
15. N. Hansen, and A. Ostermeier Completely derandomized self-adaptation in evolution strategies Evol. Comput. 9 2001 159 195
16. N. Hansen The CMA evolution strategy: a comparing review J. Lozano, P. Larranaga, I. Inza, E. Bengoetxea, Towards a New Evolutionary Computation. Advances on Estimation of Distribution Algorithms 2006 Springer Berlin, Heidelberg 75 102
17. T. Glasmachers, and C. Igel Gradient-based adaptation of general Gaussian kernels Neural Comput. 17 10 2005 2099 2105
18. T. Glasmachers, and C. Igel Uncertainty handling in model selection for support vector machines Parallel Problem Solving from Nature - PPSN X 2008 185 194
19. G. Box, and K. Wilson On the experimental attainment of optimum conditions J. R. Stat. Soc. 13 1 1951 1 45
20. J. Sacks, W. Welch, T. Mitchell, and H. Wynn Design and analysis of computer experiments Stat. Sci. 4 4 1989 409 423
21. Y. Jin A comprehensive survey of fitness approximation in evolutionary computation Soft Comput. 9 1 2005 3 12
22. A. Ratle Kriging as a surrogate fitness landscape in evolutionary optimization Artif. Intell. Eng. Des. Manuf. 15 1 2001 37 49
23. M. Emmerich, K. Giannakoglou, and B. Naujoks Single-and multiobjective evolutionary optimization assisted by Gaussian random field metamodels IEEE Trans. Evol. Comput. 10 4 2006 421 439
24. Z. Zhou, Y.S. Ong, P.B. Nair, A.J. Keane, and K.Y. Lum Combining global and local surrogate models to accelerate evolutionary optimization IEEE Trans. Syst. Man Cybern. C: Appl. Rev. 37 1 2007 66 76
25. D. Lim, Y. Jin, Y.-S. Ong, and B. Sendhoff Generalizing surrogate-assisted evolutionary computation IEEE Trans. Evol. Comput. 14 3 2010 329 355
26. A. Forrester, A. Keane, and N. Bressloff Design and analysis of "noisy" computer experiments AIAA J. 44 10 2006 2331
27. D. Huang, T.T. Allen, W.I. Notz, and N. Zeng Global optimization of stochastic black-box systems via sequential Kriging meta-models J. Global Optim. 34 3 2006 441 466
28. V. Picheny, D. Ginsbourger, and Y. Richet Noisy expected improvement and on-line computation time allocation for the optimization of simulators with tunable fidelity 2nd International Conference on Engineering Optimization 2010 1 10
29. V. Picheny, T. Wagner, and D. Ginsbourger A benchmark of Kriging-based infill criteria for noisy optimization Struct. Multidis. Optim. 48 3 2013 607 626
30. G. Liu, and V. Kadirkamanathan Learning with multi-objective criteria Proceedings of the 4th International Conference on Artificial Neural Networks 1995 IET 53 58
31. A. Freitas A critical review of multi-objective optimization in data mining: a position paper ACM SIGKDD Explor. Newsl. 6 2 2004 77 86
32. Y. Jin, and B. Sendhoff Pareto-based multiobjective machine learning: an overview and case studies IEEE Trans. Syst. Man Cybern. C: Appl. Rev. 38 3 2008 397 415
33. Y. Jin Pareto-based multi-objective machine learning Proceedings of the 7th International Conference on Hybrid Intelligent Systems (HIS 2007) IEEE 2007 2
34. J. Knowles, and H. Nakayama Meta-modeling in multiobjective optimization Multiobjective Optimization 2008 245 284
35. K. Giannakoglou, A. Giotis, and M. Karakasis Low-cost genetic optimization based on inexact pre-evaluations and the sensitivity analysis of design parameters Inverse Probl. Eng. 9 4 2001 389 412
36. M. Emmerich, and B. Naujoks Metamodel assisted multiobjective optimisation strategies and their application in airfoil design Adaptive Computing in Design and Manufacture VI 2004 249 260
37. G. Ascia, V. Catania, A. Di Nuovo, M. Palesi, and D. Patti Performance evaluation of efficient multi-objective evolutionary algorithms for design space exploration of embedded computer systems Appl. Soft Comput. 11 1 2011 382 398
38. J. Knowles ParEGO: a hybrid algorithm with on-line landscape approximation for expensive multiobjective optimization problems IEEE Trans. Evol. Comput. 10 1 2006 50 66
39. W. Ponweiser, T. Wagner, D. Biermann, and M. Vincze Multiobjective optimization on a limited budget of evaluations using model-assisted S-metric selection Proceedings of the International Conference on Parallel Problem Solving from Nature (PPSN X) 2008 Springer 784 794
40. M. Emmerich, A. Deutz, and J. Klinkenberg Hypervolume-based expected improvement: monotonicity properties and exact computation Proceedings of the IEEE Congress on Evolutionary Computation (CEC) IEEE 2011 2147 2154
41. Q. Zhang, W. Liu, E. Tsang, and B. Virginas Expensive multiobjective optimization by MOEA/D with Gaussian process model IEEE Trans. Evol. Comput. 14 3 2010 456 474
42. M. Zaefferer, B. Naujoks, T. Bartz-Beielstein, M. Emmerich, and T. Wagner A case study on multi-criteria optimization of an event detection software under limited budgets Proceedings of the 7th Evolutionary Multi-Criterion Optimization Conference (EMO 2013) 2013 756 770
43. F. Corona, M. Mulas, R. Baratti, and J. Romagnoli On the topological analysis of industrial process data using the SOM C.O.d.N.R. de Brito Alves, E. Biscaia, Computer Aided Chemical Engineering: Proceedings of PSE 2009 International Symposium on Process Systems Engineering Salvador Bahia, Brazil, vol. 27 of Computer Aided Chemical Engineering 2009 Elsevier 1173 1178
44. J. Knowles, D. Corne, and A. Reynolds Noisy multiobjective optimization on a budget of 250 evaluations Evolutionary Multi-Criterion Optimization 2009 Springer 36 50
45. D. Krige A statistical approach to some basic mine valuation problems on the Witwatersrand J. Chem. Metal Mining Soc. South Africa 52 1951 119 139
46. G. Matheron Traité de géostatistique appliquée, vol. 1 1962 Editions Technip Paris
47. M. Stein Interpolation of Spatial Data: Some Theory for Kriging 1999 Springer New York
48. M. Emmerich, A. Giotis, M. Özdemir, T. Bäck, and K. Giannakoglou Metamodel-assisted evolution strategies Proceedings of the Conference on Parallel Problem Solving from Nature (PPSN VII) 2002 Springer 361 370
49. J. Svenson Computer experiments: multiobjective optimization and sensitivity analysis (Ph.D. Thesis) 2011 Ohio State University
50. S. Sakata, and F. Ashida Ns-kriging based microstructural optimization applied to minimizing stochastic variation of homogenized elasticity of fiber reinforced composites Struct. Multidis. Optim. 38 5 2009 443 453
51. J. Mockus, V. Tiesis, and A. Zilinskas The application of Bayesian methods for seeking the extremum Towards Global Optim. 2 1978 117 129
52. E. Zitzler, L. Thiele, and J. Bader On set-based multiobjective optimization IEEE Trans. Evol. Comput. 14 1 2010 58 79
53. K. Shimoyama, K. Sato, S. Jeong, and S. Obayashi Comparison of the criteria for updating Kriging response surface models in multi-objective optimization IEEE Congress on Evolutionary Computation (CEC) IEEE 2012 1 8
54. K. Shimoyama, K. Sato, S. Jeong, and S. Obayashi Updating kriging surrogate models based on the hypervolume indicator in multi-objective optimization J. Mech. Des. 135 9 2013
55. M. Fleischer The measure of Pareto optima applications to multi-objective metaheuristics Evolutionary Multi-Criterion Optimization (EMO) 2003 Springer 519 533
56. A. Auger, J. Bader, D. Brockhoff, and E. Zitzler Theory of the hypervolume indicator: optimal μ-distributions and the choice of the reference point Proceedings of the 10th ACM SIGEVO Workshop on Foundations of Genetic Algorithms ACM 2009 87 102
57. W. Konen, P. Koch, The TDMR package: tuned data mining in R, CIOP Technical Report 02/2012, Cologne University of Applied Sciences (November 2012). http://gociop.de/ciop-reports.
58. M. Hansen, and A. Jaszkiewicz Evaluating the Quality of Approximations to the Non-dominated Set 1998 IMM, Department of Mathematical Modelling, Technical University of Denmark Lyngby, Denmark
59. E. Zitzler, K. Deb, and T. Thiele Comparison of multiobjective evolutionary algorithms: empirical results Evol. Comput. 8 2 2000 173 195
60. C. Wolf, D. Gaida, A. Stuhlsatz, T. Ludwig, S. McLoone, and M. Bongards Predicting organic acid concentration from UV/vis spectrometry measurements - a comparison of machine learning techniques Trans. Inst. Meas. Control 19 2011 1 11
61. P. Domingos Metacost: a general method for making classifiers cost-sensitive Proceedings of the fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining ACM 1999 155 164
62. T. Bartz-Beielstein, C. Lasarczyk, and M. Preuss The sequential parameter optimization toolbox T. Bartz-Beielstein, Experimental Methods for the Analysis of Optimization Algorithms 2010 Springer Berlin, Heidelberg, New York 337 360
63. I. Hupkens Complexity reduction and validation of computing the expected hypervolume improvement (Master's thesis) 2013 University of Leiden The Netherlands
64. I. Couckuyt, D. Deschrijver, and T. Dhaene Fast calculation of multiobjective probability of improvement and expected improvement criteria for Pareto optimization J. Global Optim. 60 3 2014 575 594
65. L. Breiman Random forests Mach. Learn. 45 1 2001 5 32
66. C. Cortes, and V. Vapnik Support-vector networks Mach. Learn. 20 3 1995 273 297
67. E. Dimitriadou, K. Hornik, F. Leisch, D. Meyer, and A. Weingessel e1071: Misc functions of the department of statistics (e1071) 2006 TU Wien 1 60 e1071 R package (2006)
68. A. Karatzoglou, A. Smola, K. Hornik, and A. Zeileis kernlab - an s4 package for kernel methods in R J. Stat. Softw. 11 2004 1 20
69. G. Bo, and H. Xianwu SVM multi-class classification J. Data Acquisit. Process. 21 3 2006 334 339
70. O. Schütze, X. Esquivel, A. Lara, and C. Coello Using the averaged Hausdorff distance as a performance measure in evolutionary multiobjective optimization IEEE Trans. Evol. Comput. 16 4 2012 504 522