Consistencies and contradictions of performance metrics in multiobjective optimization

IEEE Transactions on Cybernetics

Volumn 44, Issue 12, 2014, Pages 2391-2404

  Jiang, Siwei ^a   Ong, Yew Soon ^a   Zhang, Jie ^a   Feng, Liang ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Capacity; Convergence; Diversity; Hypervolume; jMetal; Multiobjective optimization (MOO); Performance metrics

Indexed keywords

CAPACITY; CONVERGENCE; DIVERSITY; HYPERVOLUME; JMETAL; PERFORMANCE METRICS;

MULTIOBJECTIVE OPTIMIZATION;

EID: 84911866044     PISSN: 21682267     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCYB.2014.2307319     Document Type: Article

References (70)

1. K. Deb, Multi-Objective Optimization Using Evolutionary Algorithms, vol. 16. New York, NY, USA: Wiley, 2001.
2. C. A. C. Coello, "Evolutionary multi-objective optimization: A historical view of the field," IEEE Comput. Intell. Mag., vol. 1, no. 1, pp. 28-36, Feb. 2006.
3. H. A. Abbass, A. Bender, S. Gaidow, and P. Whitbread, "Computational red teaming: Past, present and future," IEEE Comput. Intell. Mag., vol. 6, no. 1, pp. 30-42, Feb. 2011.
4. J. J. Durillo, A. J. Nebro, and E. Alba, "The jMetal framework for multiobjective optimization: Design and architecture," in Proc. IEEE Cong. Evol. Comput., 2010, pp. 1-8.
5. J. J. Durillo and A. J. Nebro, "jMetal: A java framework for multiobjective optimization," Adv. Eng. Softw., vol. 42, no. 10, pp. 760-771, 2011.
6. K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, "A fast and elitist multiobjective genetic algorithm: NSGA-II," IEEE Trans. Evol. Comput., vol. 6, no. 2, pp. 182-197, Apr. 2002.
7. E. Zitzler, M. Laumanns, and L. Thiele, "SPEA2: Improving the strength Pareto evolutionary algorithm," in Proc. Comput. Eng. Netw. Lab., vol. 103. 2001.
8. Q. Zhang and H. Li, "MOEA/D: A multiobjective evolutionary algorithm based on decomposition," IEEE Trans. Evol. Comput., vol. 11, no. 6, pp. 712-731, Dec. 2007.
9. H. Li and Q. Zhang, "Multiobjective optimization problems with complicated Pareto sets, MOEA/D and NSGA-II," IEEE Trans. Evol. Comput., vol. 13, no. 2, pp. 284-302, Apr. 2009.
10. E. Zitzler and S. Kúnzli, "Indicator-based selection in multiobjective search," in Proc. PPSN, 2004, pp. 832-842.
11. N. Beume, B. Naujoks, and M. Emmerich, "SMS-EMOA: Multiobjective selection based on dominated hypervolume," Eur. J. Oper. Res., vol. 181, no. 3, pp. 1653-1669, 2007.
12. S. Kukkonen and J. Lampinen, "GDE3: The third evolution step of generalized differential evolution," in Proc. IEEE Cong. Evol. Comput., vol. 1. 2005, pp. 443-450.
13. A. J. Nebro, J. J. Durillo, J. Garcia-Nieto, C. A. C. Coello, F. Luna, and E. Alba, "SMPSO: A new PSO-based metaheuristic for multi-objective optimization," in Proc. IEEE Symp. Comput. Intell. Miulticriteria Decision Making, 2009, pp. 66-73.
14. S. Jiang, J. Zhang, and Y. S. Ong, "Asymmetric Pareto-adaptive scheme for multiobjective optimization," in Proc. Conf. Artif. Intell., 2011, pp. 351-360.
15. I. W. Tsang, C. W. Seah, Y. S. Ong, and S. Jiang, "Pareto rank learning in multi-objective evolutionary algorithms," in Proc. IEEE Cong. Evol. Comput., 2012, pp. 1-8.
16. S. Jiang, Z. Cai, J. Zhang, and Y. S. Ong, "Multiobjective optimization by decomposition with Pareto-adaptive weight vectors," in Proc. Conf. Natural Comput., vol. 3. 2011, pp. 1260-1264.
17. S. Jiang, J. Zhang, and Y. S. Ong, "A multiagent evolutionary framework based on trust for multiobjective optimization," in Proc. Conf. Auton. Agents Multiagent Syst., 2012, pp. 299-306.
18. M. N. Le, Y. S. Ong, S. Menzel, C. W. Seah, and B. Sendhoff, "Multi co-objective evolutionary optimization: Cross surrogate augmentation for computationally expensive problems," in Proc. IEEE Cong. Evol. Comput., 2012, pp. 1-8.
19. C. K. Goh, K. C. Tan, C. Y. Cheong, and Y. S. Ong, "Noise-induced features in robust multi-objective optimization problems," in Proc. IEEE Cong. Evol. Comput., 2007, pp. 568-575.
20. Y. Wang, Z. Cai, G. Guo, and Y. Zhou, "Multiobjective optimization and hybrid evolutionary algorithm to solve constrained optimization problems," IEEE Trans. Sys., Man, Cybern. B, Cybern., vol. 37, no. 3, pp. 560-575, Jun. 2007.
21. S. Mostaghim and J. Teich, "Strategies for finding good local guides in multi-objective particle swarm optimization (MOPSO)," in Proc. IEEE Swarm Intell. Symp., 2003, pp. 26-33.
22. D. Liu, K. C. Tan, C. K. Goh, and W. K. Ho, "A multiobjective memetic algorithm based on particle swarm optimization," IEEE Trans. Sys., Man, Cybern. B, Cybern., vol. 37, no. 1, pp. 42-50, Feb. 2007.
23. W. F. Leong and G. G. Yen, "PSO-based multiobjective optimization with dynamic population size and adaptive local archives," IEEE Trans. Sys., Man, Cybern. B, Cybern., vol. 38, no. 5, pp. 1270-1293, Oct. 2008.
24. S. Bandyopadhyay, S. K. Pal, and B. Aruna, "Multiobjective GAs, quantitative indices, and pattern classification," IEEE Trans. Sys., Man, Cybern. B, Cybern., vol. 34, no. 5, pp. 2088-2099, Oct. 2004.
25. X. Zou, Y. Chen, M. Liu, and L. Kang, "A new evolutionary algorithm for solving many-objective optimization problems," IEEE Trans. Sys., Man, Cybern. B, Cybern., vol. 38, no. 5, pp. 1402-1412, Oct. 2008.
26. M. Daneshyari and G. G. Yen, "Cultural-based multiobjective particle swarm optimization," IEEE Trans. Sys., Man, Cybern. B, Cybern., vol. 41, no. 2, pp. 553-567, Apr. 2011.
27. E. Masazade, R. Rajagopalan, P. K. Varshney, C. K. Mohan, G. K. Sendur, and M. Keskinoz, "A multiobjective optimization approach to obtain decision thresholds for distributed detection in wireless sensor networks," IEEE Trans. Sys., Man, Cybern. B, Cybern., vol. 40, no. 2, pp. 444-457, Apr. 2010.
28. G. G. Yen and W. F. Leong, "Dynamic multiple swarms in multiobjective particle swarm optimization," IEEE Trans. Sys., Man, Cybern. A, Syst., Humans, vol. 39, no. 4, pp. 890-911, Jul. 2009.
29. R. V. Kulkarni and G. K. Venayagamoorthy, "Particle swarm optimization in wireless-sensor networks: A brief survey," IEEE Trans. Sys., Man, Cybern. C, Appl. Rev., vol. 41, no. 2, pp. 262-267, Mar. 2011.
30. S. Bandyopadhyay, "Multiobjective simulated annealing for fuzzy clustering with stability and validity," IEEE Trans. Sys., Man, Cybern. C, Appl. Rev., vol. 41, no. 5, pp. 682-691, Sep. 2011.
31. G. Avigad and A. Moshaiov, "Interactive evolutionary multiobjective search and optimization of set-based concepts," IEEE Trans. Sys., Man, Cybern. B, Cybern., vol. 39, no. 4, pp. 1013-1027, Aug. 2009.
32. C. K. Goh and K. C. Tan, "A competitive-cooperative coevolutionary paradigm for dynamic multiobjective optimization," IEEE Trans. Evol. Comput., vol. 13, no. 1, pp. 103-127, Feb. 2009.
33. C. K. Goh and K. C. Tan, "An investigation on noisy environments in evolutionary multiobjective optimization," IEEE Trans. Evol. Comput., vol. 11, no. 3, pp. 354-381, Jun. 2007.
34. K. Tang, K. C. Tan, and H. Ishibuchi, "Guest editorial: Memetic algorithms for evolutionary multi-objective optimization," Memetic Comput., vol. 2, no. 1, p. 1, 2010.
35. H. Ishibuchi, Y. Sakane, N. Tsukamoto, and Y. Nojima, "Simultaneous use of different scalarizing functions in moea/d," in Proc. Genet. Evol. Comput., 2010, pp. 519-526.
36. Z. Wang, K. Tang, and X. Yao, "Multi-objective approaches to optimal testing resource allocation in modular software systems," IEEE Trans. Reliab., vol. 59, no. 3, pp. 563-575, Sep. 2010.
37. Y. Mei, K. Tang, and X. Yao, "Decomposition-based memetic algorithm for multiobjective capacitated arc routing problem," IEEE Trans. Evol. Comput., vol. 15, no. 2, pp. 151-165, Apr. 2011.
38. Z. Wang, T. Chen, K. Tang, and X. Yao, "A multi-objective approach to redundancy allocation problem in parallel-series systems," in Proc. IEEE Cong. Evol. Comput., 2009, pp. 582-589.
39. K. Tang, Z. Wang, X. Cao, and J. Zhang, "A multi-objective evolutionary approach to aircraft landing scheduling problems," in Proc. IEEE Cong. Evol. Comput., 2008, pp. 3650-3656.
40. A. J. Nebro, J. J. Durillo, and C. A. Coello Coello, "Analysis of leader selection strategies in a multi-objective particle swarm optimizer," in Proc. IEEE Cong. Evol. Comput., Jun. 2013, pp. 3153-3160.
41. D. A. Van Veldhuizen and G. B. Lamont, "On measuring multiobjective evolutionary algorithm performance," in Proc. IEEE Cong. Evol. Comput., 2000, pp. 204-211.
42. D. A. Van Veldhuizen and G. B. Lamont, "Multiobjective evolutionary algorithm test suites," in Proc. ACM Symp. Appl. Comput., 1999, pp. 351-357.
43. A. Zhou, Y. Jin, Q. Zhang, B. Sendhoff, and E. Tsang, "Combining model-based and genetics-based offspring generation for multi-objective optimization using a convergence criterion," in Proc. IEEE Cong. Evol. Comput., 2006, pp. 892-899.
44. E. Zitzler and L. Thiele, "Multiobjective optimization using evolutionary algorithms-A comparative case study," in Proc. Parall. Problem Solv. Nature, 1998, pp. 292-301.
45. E. Zitzler and L. Thiele, "Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach," IEEE Trans. Evol. Comput., vol. 3, no. 4, pp. 257-271, Nov. 1999.
46. T. Okabe, Y. Jin, and B. Sendhoff, "A critical survey of performance indices for multi-objective optimisation," in Proc. IEEE Cong. Evol. Comput., vol. 2. 2003, pp. 878-885.
47. E. Zitzler, L. Thiele, M. Laumanns, C. M. Fonseca, and V. G. da Fonseca, "Performance assessment of multiobjective optimizers: An analysis and review," IEEE Trans. Evol. Comput., vol. 7, no. 2, pp. 117-132, Apr. 2003.
48. K. C. Tan, T. H. Lee, and E. F. Khor, "Evolutionary algorithms for multiobjective optimization: Performance assessments and comparisons," Artif. Intell. Rev., vol. 17, no. 4, pp. 251-290, 2002.
49. J. Knowles and D. Corne, "On metrics for comparing nondominated sets," in Proc. IEEE Cong. Evol. Comput., vol. 1. 2002, pp. 711-716.
50. C. Grosan, M. Oltean, and D. Dumitrescu, "Performance metrics for multiobjective optimization evolutionary algorithms," in Proc. Appl. Ind. Math., 2003.
51. J. Wu and S. Azarm, "Metrics for quality assessment of a multiobjective design optimization solution set," J. Mech. Design, vol. 123, no. 1, pp. 18-25, 2001.
52. K. Deb, S. Agrawal, A. Pratap, and T. Meyarivan, "A fast elitist nondominated sorting genetic algorithm for multi-objective optimization: NSGA-II," in Proc. PPSN, 2000, pp. 849-858.
53. D. A. Van Veldhuizen, "Multiobjective evolutionary algorithms: Classifications, analyses, and new innovations," Evol. Comput., 1999.
54. P. Czyzẓak and A. Jaszkiewicz, "Pareto simulated annealing: A meta-heuristic technique for multiple-objective combinatorial optimization," J. Multi-Criteria Decision Anal., vol. 7, no. 1, pp. 34-47, 1998.
55. M. P. Hansen and A. Jaszkiewicz, "Evaluating the quality of approximations to the non-dominated set," Tech. Univ. Denmark, Kongens Lyngby, Denmark, IMM Tech. Rep. IMM-REP-1998-7, 1998.
56. E. Zitzler, K. Deb, and L. Thiele, "Comparison of multiobjective evolutionary algorithms: Empirical results," Evol. Comput., vol. 8, no. 2, pp. 173-195, 2000.
57. J. R. Schott, "Fault tolerant design using single and multicriteria genetic algorithm optimization," Tech. Rep. DTIC Doc., 1995.
58. A. Farhang-Mehr and S. Azarm, "Diversity assessment of Pareto optimal solution sets: An entropy approach," in Proc. IEEE Cong. Evol. Comput., vol. 1. 2002, pp. 723-728.
59. S. Mostaghim and J. Teich, "A new approach on many objective diversity measurement," in Proc. Pract. Appro. MultiObj. Optimi., 2005, p. 254.
60. J. Bader and E. Zitzler, "HypE: An algorithm for fast hypervolumebased many-objective optimization," Evol. Comput., vol. 19, no. 1, pp. 45-76, 2011.
61. L. While and L. Bradstreet, "Applying the WFG algorithm to calculate incremental hypervolumes," in Proc. IEEE Cong. Evol. Comput., 2012, pp. 1-8.
62. E. Zitzler, "Evolutionary algorithms for multiobjective optimization: Methods and applications," Ph.D. dissertation, 1999.
63. K. Bringmann and T. Friedrich, "Approximating the least hypervolume contributor: NP-hard in general, but fast in practice," Theor. Comput. Sci., vol. 425, pp. 104-116, 2012.
64. B. Naujoks, N. Beume, and M. Emmerich, "Multi-objective optimisation using S-metric selection: Application to three-dimensional solution spaces," in Proc. IEEE Cong. Evol. Comput., vol. 2. 2005, pp. 1282-1289.
65. C. M. Fonseca, L. Paquete, and M. López-Ibánez, "An improved dimension-sweep algorithm for the hypervolume indicator," in Proc. IEEE Cong. Evol. Comput., 2006, pp. 1157-1163.
66. N. Beume, C. M. Fonseca, M. López-Ibáñez, L. Paquete, and J. Vahrenhold, "On the complexity of computing the hypervolume indicator," IEEE Trans. Evol. Comput., vol. 13, no. 5, pp. 1075-1082, Oct. 2009.
67. N. Beume, "S-metric calculation by considering dominated hypervolume as Klee's measure problem," Evol. Comput., vol. 17, no. 4, pp. 477-492, 2009.
68. L. Bradstreet, "The hypervolume indicator for multi-objective optimisation: Calculation and use," Ph.D. dissertation, 2010.
69. L. While, L. Bradstreet, and L. Barone, "A fast way of calculating exact hypervolumes," IEEE Trans. Evol. Comput., vol. 16, no. 1, pp. 86-95, Feb. 2012.
70. O. Schutze, X. Esquivel, A. Lara, and C. A. C. Coello, "Using the averaged hausdorff distance as a performance measure in evolutionary multiobjective optimization," IEEE Trans. Evol. Comput., vol. 16, no. 4, pp. 504-522, Aug. 2012.