Analyzing probabilistic models in hierarchical BOA

IEEE Transactions on Evolutionary Computation

Volumn 13, Issue 6, 2009, Pages 1199-1217

  Hauschild, Mark ^a   Pelikan, Martin ^a   Sastry, Kumara ^b   Lima, Claudio ^c  

^a UNIVERSITY OF MISSOURI ST LOUIS   (United States)

^b UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^c UNIVERSITY OF ALGARVE   (Portugal)

Author keywords

Estimation of distribution algorithms; Hierarchical BOA; Model complexity; Model structure; Probabilistic model

Indexed keywords

DECOMPOSABLE PROBLEMS; ESTIMATION OF DISTRIBUTION ALGORITHMS; HIERARCHICAL BAYESIAN; HIERARCHICAL BOA; ISING SPIN GLASS; MODEL COMPLEXITY; NEARLY DECOMPOSABLE; OPTIMIZATION PROBLEMS; PERIODIC BOUNDARY CONDITIONS; PROBABILISTIC MODELS; TEST PROBLEM;

EVOLUTIONARY ALGORITHMS; ISING MODEL; SPIN GLASS; STRUCTURAL OPTIMIZATION;

PROBABILITY DISTRIBUTIONS;

EID: 71549169964     PISSN: 1089778X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TEVC.2008.2004423     Document Type: Article

References (81)

1. M. Pelikan and D. E. Goldberg, "Escaping hierarchical traps with competent genetic algorithms," in Proc. Genetic Evol. Comput. Conf. (GECCO-2001), 2001, pp. 511-518.
2. M. Pelikan, Hierarchical Bayesian Optimization Algorithm: Toward a New Generation of Evolutionary Algorithms. Berlin, Germany: Springer-Verlag, 2005.
3. M. Pelikan and D. E. Goldberg, "Hierarchical Bayesian optimization algorithm," in Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications, M. Pelikan, K. Sastry, and E. Cantú-Paz, Eds. New York: Springer, 2006, pp. 63-90.
4. K. Sastry, D. E. Goldberg, and M. Pelikan, "Don't evaluate, inherit," in Proc. Genetic Evol. Comput. Conf. (GECCO-2001), 2001, pp. 551-558.
5. M. Pelikan and K. Sastry, "Fitness inheritance in the Bayesian optimization algorithm," in Proc. Genetic Evol. Comput. Conf. (GECCO-2004), 2004, vol. 2, pp. 48-59.
6. C. F. Lima, M. Pelikan, K. Sastry, M. V. Butz, D. E. Goldberg, and F. G. Lobo, "Substructural neighborhoods for local search inthe Bayesian optimization algorithm," Parallel Problem Solving From Nature, pp. 232-241, 2006.
7. M. Pelikan, K. Sastry, and D. E. Goldberg, "Sporadic model building for efficiency enhancement of hierarchical BOA," in Proc. Genetic Evol. Comput. Conf. (GECCO-2006), 2006, pp. 405-412.
8. K. Sastry, M. Pelikan, and D. E. Goldberg, "Efficiency enhancement of estimation of distribution algorithms," in Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications, M. Pelikan, K. Sastry, and E. Cantú-Paz, Eds. New York: Springer, 2006, pp. 161-185.
9. H. Wu and J. L. Shapiro, "Does overfitting affect performance in estimation of distribution algorithms," in Proc. Genetic Evol. Comput. Conf. (GECCO-2006), 2006, pp. 433-434.
10. C. F. Lima, M. Pelikan, D. E. Goldberg, F. G. Lobo, K. Sastry, and M. Hauschild, "Influence of selection and replacement strategies on linkage learning in BOA," Missouri Estimation of Distribution Algorithms Laboratory, University of Missouri-St. Louis, St. Louis, MO, MEDAL Rep., 2007.
11. D. H. Ackley, "An empirical study of bit vector function optimization," Genetic Algorithms and Simulated Annealing, pp. 170-204, 1987.
12. K. Deb and D. E. Goldberg, "Sufficient conditions for deceptive and easy binary functions," Ann. Math. Artif. Intell., vol. 10, pp. 385-408, 1994.
13. K. Deb and D. E. Goldberg, "Analyzing deception in trap functions," in Foundations of Genetic Algorithms, 1992, pp. 93-108.
14. M.Pelikan, K. Sastry, M.V. Butz, andD.E.Goldberg, "Performance of evolutionary algorithms on random decomposable problems," in Proc. PPSN2006, 2006, pp. 788-797.
15. K. Sastry, M. Pelikan, and D. E. Goldberg, "Empirical analysis of ideal recombination on random decomposable problems," in Proc. Genetic Evol. Comput. Conf. (GECCO-2007), 2007, pp. 1388-1395.
16. D. E. Goldberg, The Design of Innovation: Lessons From and for Competent Genetic Algorithms. Norwell, MA: Kluwer, 2002.
17. K. Binder and A. Young, "Spin-glasses: Experimental facts, theoretical concepts and open questions," Rev. Mod. Phys, vol. 58, pp. 801-801, 1986.
18. M. Mezard, G. Parisi, and M. Virasoro, Spin Glass Theory and Beyond. Singapore: World Scientific, 1987.
19. K. Fischer and J. Hertz, Spin Glasses. Cambridge, U.K.: Cambridge Univ. Press, 1991.
20. A. Young, Ed., Spin Glasses and Random Fields. Singapore: World Scientific, 1998.
21. H. Mühlenbein, T. Mahnig, and A. Ochoa-Rodriguez, "Schemata, distributions and graphical models in evolutionary optimization," J. Heuristics, vol. 5, pp. 215-247, 1999.
22. M. Pelikan and D. E. Goldberg, "Hierarchical BOA solves Ising spin glasses and maxsat," in Proc. Genetic Evol. Comput. Conf. (GECCO-2003), 2003, vol. II, pp. 1275-1286.
23. F. Barahona, "On the computational complexity of Ising spin glass models," J. Physics A: Mathematical, Nuclear and General, vol. 15, no. 10, pp. 3241-3253, 1982.
24. P. Dayal, S. Trebst, S. Wessel, D. Würtz, M. Troyer, S. Sabhapandit, and S. Coppersmith, "Performance limitations of flat histogram methods and optimality of Wang-Langdau sampling," Phys. Rev. Lett., vol. 92, no. 9, pp. 097201-097201, 2004.
25. M. Pelikan and A. K. Hartmann, "Searching for ground states of Ising spin glasses with hierarchical BOA and cluster exact approximation," in Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications, M. Pelikan, K. Sastry, and E. Cantú-Paz, Eds. New York: Springer, 2006, pp. 333-349.
26. S. Baluja, "Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning," Carnegie Mellon Univ., Pittsburgh, PA, Tech. Rep. CMU-CS-94-163, 1994.
27. H. Mühlenbein and G. Paaß, "From recombination of genes to the estimation of distributions I. Binary parameters," Parallel Problem Solving from Nature, pp. 178-187, 1996.
28. P. Larrañaga and J. A. Lozano, Eds., Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. Boston, MA: Kluwer, 2002.
29. M. Pelikan, D. E. Goldberg, and F. Lobo, "A survey of optimization by building and using probabilistic models," Comput. Opt. Appl., vol. 21, no. 1, pp. 5-20, 2002.
30. M. Pelikan, K. Sastry, and E. Cantú-Paz, Eds., Scalable Optimization via Probabilistic Modeling: From Algorithms to Applications. Berlin, Germany: Springer-Verlag, 2006.
31. J. A. Lozano, P. Larrañaga, I. Inza, and E. Bengoetxea, Eds., Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation. New York: Springer, 2006.
32. P. A. N. Bosman and D. Thierens, "Continuous iterated density estimation evolutionary algorithms within the idea framework," in Proc. Workshop Genetic Evol. Comput. Conf. (GECCO-2000), 2000, pp. 197-200.
33. M. Pelikan and D. E. Goldberg, "A hierarchy machine: Learning to optimize from nature and humans," Complexity, vol. 8,no. 5, pp. 36-45, 2003.
34. G.R. Harik, "Finding multimodal solutions using restricted tournament selection," in Proc. Int. Conf. Genetic Algorithms (ICGA-95), 1995, pp. 24-31.
35. R. Etxeberria and P. Larrañaga, "Global optimization using Bayesian networks," in Proc. 2nd Symp. Artif. Intell. (CIMAF-99), 1999, pp. 332-339.
36. H. Mühlenbein and T. Mahnig, "The factorized distribution algorithm for additively decomposed functions," in Proc. Congr. Evol. Comput. (CEC-99), 1999, pp. 752-759.
37. D. M. Chickering, D. Heckerman, and C. Meek, A Bayesian approach to learning Bayesian networks with local structure Microsoft Research, Redmond, WA, Tech. Rep. MSR-TR-97-107, 1997.
38. N. Friedman and M. Goldszmidt, "Learning Bayesian networks with local structure," in Graphical models, M. I. Jordan, Ed. Cambridge, MA: MIT Press, 1999, pp. 421-459.
39. K. A. De Jong, "An analysis of the behavior of a class of genetic adaptive systems," Ph.D. dissertation, Univ. Michigan, Ann Arbor, MI, 1975.
40. S. W. Mahfoud, Niching Methods for Genetic Algorithms. Urbana, IL: Univ. Illinois at Urbana-Champaign, 1995.
41. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA: Morgan Kaufmann, 1988.
42. R. A. Howard and J. E. Matheson, "Influence diagrams," in Readings on the Principles and Applications of Decision Analysis, R. A. Howard and J. E. Matheson, Eds. Menlo Park, CA: Strategic Decisions Group, 1981, vol. II, pp. 721-762.
43. H. A. Simon, The Sciences of the Artificial. Cambridge, MA: The MIT Press, 1968.
44. M. Pelikan, K. Sastry, and D. E. Goldberg, "Scalability of the Bayesian optimization algorithm," Int. J. Approximate Reasoning, vol. 31, no. 3, pp. 221-258, 2002.
45. D. Thierens, "Scalability problems of simple genetic algorithms," Evol. Comput., vol. 7, no. 4, pp. 331-352, 1999.
46. H. Mühlenbein and T. Mahnig, "FDA-A scalable evolutionary algorithm for the optimization of additively decomposed functions," Evol. Comput., vol. 7, no. 4, pp. 353-376, 1999.
47. D. Thierens and D. E. Goldberg, "Mixing in genetic algorithms," in Proc. Int. Conf. Genetic Algorithms (ICGA-93), 1993, pp. 38-45.
48. L. Kallel, B. Naudts, and R.Reeves, "Properties of fitness functions and search landscapes," in Theoretical Aspects of Evolutionary Computing, L. Kallel, B. Naudts, and A. Rogers, Eds. Berlin, Germany: Springer-Verlag, 2000, pp. 177-208.
49. R. Santana, P. Larranaga, and J. Lozano, "Interactions and dependencies in estimation of distribution algorithms," in Proc. IEEE Congr. Evol. Comput., Sep. 2-5, 2005, vol. 2, pp. 1418-1425. (Pubitemid 41496017)
50. K. Sastry, "Evaluation-relaxation schemes for genetic and evolutionary algorithms," M.S. thesis, University of Illinois at Urbana-Champaign, Dept. General Eng., Urbana, IL, 2001.
51. T.-L. Yu, K. Sastry, D. E. Goldberg, and M. Pelikan, "Population sizing for entropy-based model building in discrete estimation of distribution algorithms," in Proc. Genetic Evol. Comput. Conf. (GECCO-2007), 2007, pp. 601-608.
52. D. Heckerman, D. Geiger, and D. M. Chickering, "Learning Bayesian networks: The combination of knowledge and statistical data," Microsoft Research, Redmond, WA, Tech. Rep. MSR-TR-94-109, 1994.
53. G. F. Cooper and E. H. Herskovits, "A Bayesian method for the induction of probabilistic networks from data," Mach. Learn., vol. 9, pp. 309-347, 1992.
54. M. Pelikan, D. E. Goldberg, and E. Cantú-Paz, "BOA: The Bayesian optimization algorithm," in Proc. Genetic Evol. Comput. Conf. (GECCO-99), 1999, vol. I, pp. 525-532.
55. K. Sastry and D. E. Goldberg, "Designing competent mutation operators via probabilistic model building of neighborhoods," in Proc. Genetic Evol. Comput. Conf. (GECCO-2004), 26-30, 2004, pp. 114-125.
56. M. Pelikan, K. Sastry, M. V. Butz, and D. E. Goldberg, Hierarchical BOA on random decomposable problems Missouri Estimation of Distribution Algorithms Laboratory, University of Missouri-St. Louis, St. Louis, MO, Medical Rep., 2006.
57. M. Pelikan, K. Sastry, M. V. Butz, and D. E. Goldberg, "Generator and interface for random decomposable problems in C," Missouri Estimation of Distribution Algorithms Laboratory, Univ. Missouri-St. Louis, St. Louis, Mo, MEDICAL Rep., 2006.
58. D. Thierens, D. E. Goldberg, and A. G. Pereira, "Domino convergence, drift, and the temporal-salience structure of problems," in Proc. Int. Conf. Evol. Comput. (ICEC-98), 1998, pp. 535-540.
59. M. Pelikan, J. Ocenasek, S. Trebst, M. Troyer, and F. Alet, "Computational complexity and simulation of rare events of ising spin glasses," in Proc. Genetic Evol. Comput. Conf. (GECCO-2004), 2004, vol. 2, pp. 36-47.
60. R. Santana, "Estimation of distribution algorithms with Kikuchi approximations," Evol. Comput., vol. 13, no. 1, pp. 67-97, 2005. (Pubitemid 40402991)
61. S. K. Shakya, J. A. McCall, and D. F. Brown, "Solving the Ising spin glass problem using a bivariate EDA based on Markov random fields," Proc. IEEE Congr. Evol. Comput. (CEC 2006), pp. 908-915, 2006.
62. Spin Glass Ground State Server Univ. Köln, Köln, Germany, 2004. [Online]. Available: http://www.informatik.uni-koeln.de/ls-juenger/ research/sgs/sgs.html
63. T.-L. Yu, "A Matrix approach for finding extreme: Problems with modularity, hierarchy, and overlap," Ph.D. dissertation, Univ. Illinois at Ur-bana-Champaign, Urbana, IL, 2006.
64. Q. Zhang, J. Sun, and E. Tsang, "Evolutionary algorithm with guided mutation for the maximum clique problem," IEEE Trans. Evol. Comput., vol. 9, pp. 192-200, 2005.
65. M. Pelikan, R. Kalapala, and A. K. Hartmann, "Hybrid evolutionary algorithms on minimum vertex cover for random graphs," in Proc. Genetic Evol. Comput. Conf. (GECCO-2007), 2007, pp. 547-554.
66. A. Brownlee, M. Pelikan, J. McCall, and A. Petrovski, "An application of a multivariate estimation of distribution algorithm to cancer chemotherapy," Missouri Estimation of Distribution Algorithms Laboratory, Univ. Missouri-St. Louis, St. Louis, MO, Medical Rep., 2008.
67. H. Mühlenbein and T. Mahnig, "Evolutionary optimization and the estimation of search distributions with applications to graph biparti-tioning," Int. J. Approximate Reasoning, vol. 31, no. 3, pp. 157-192, 2002.
68. G. Schwarz, "Estimating the dimension of a model," The Annals of Statistics, vol. 6, pp. 461-464, 1978.
69. H. Mühlenbein and R. Höns, "The estimation of distributions and the minimum relative entropy principle," Evol. Comput., vol. 13, no. 1, pp. 1-27, 2005.
70. E. Bengoetxea, "Inexact graph matching using estimation of distribution algorithms," Ph.D. dissertation, Ecole Nationale Supérieure des Télécommunications, Paris, France, 2003.
71. R. Etxeberria and P. Larrañaga, A. A. O. Rodriguez, M. R. S. Ortiz, and R. S. Hermida, Eds., "Global optimization using Bayesian networks," in Proc. 2nd Symp. Artif. Intell. (CIMAF-99), Habana, Cuba, 1999, pp. 332-339.
72. P. Larrañaga, R. Etxeberria, J. A. Lozano, and J. M. Pena, "Optimization in continuous domains by learning and simulation of Gaussian networks," in Proc. Workshop Genetic Evol. Comput. Conf. (GECCO-2000), 2000, pp. 201-204.
73. R. Santana, P. Larrañaga, and J. A. Lozano, "Protein folding in simplified models with estimation of distribution algorithms," IEEE Trans. Evol. Comput., pp. 418-438, 2007.
74. Q. Zhang, "On the convergence of a factorized distribution algorithm with truncation selection: Research articles," Complex, vol. 9, no. 4, pp. 17-23, 2004.
75. Q. Zhang, "On stability of fixed points of limit models of univariate marginal distribution algorithm and factorized distribution algorithm," IEEE Trans. Evol. Comput., vol. 8, pp. 80-93, Feb. 2004.
76. Q. Zhang and H. Muhlenbein, "On the convergence of a class of estimation of distribution algorithms," IEEE Trans. Evol. Comput., vol. 8, pp. 127-136, Apr. 2004.
77. C. Echegoyen, R. Santana, J. A. Lozano, and P. Larrañaga, "Exact Bayesian network learning in estimation of distribution algorithms," in Proc. 2007 Congr. Evol. Comput. (CEC 2007), Singapore, 2007.
78. R. A. Watson, G. S. Hornby, and J. B. Pollack, "Modeling building-block interdependency," Parallel Problem Solving from Nature, pp. 97-106, 1998.
79. R. A. Watson, "Analysis of recombinative algorithms on a non-separable building-block problem," Foundations of Genetic Algorithms, 2001, to be published.
80. S. A. Kauffman, The Origins of Order: Self-Organization and Selection in Evolution. Oxford, U.K.: Oxford Univ. Press, 1993.
81. C. F. Lima, F. G. Lobo, and M. Pelikan, "From mating pool distributions to model overfitting," in Proc. Genetic Evol. Comput. Conf. (GECCO-2008), 2008, pp. 431-438.