Learning factorizations in estimation of distribution algorithms using affinity propagation

Evolutionary Computation

Volumn 18, Issue 4, 2010, Pages 515-546

  Santana, Roberto ^a   Larrãnaga, Pedro ^b   Lozano, Jośea ^c  

^a UNIVERSIDAD POLITÉCNICA DE MADRID   (Spain)

^b UNIVERSIDAD POLITÉCNICA DE MADRID   (Spain)

^c UNIVERSITY OF THE BASQUE COUNTRY UPV EHU   (Spain)

Author keywords

Affinity propagation; Estimation of distribution algorithms; Extended compact genetic algorithm; Factorizations; Factorized distribution algorithms; Marginal product models; Message passing algorithms

Indexed keywords

BINARY CODES; FACTORIZATION; GENETIC ALGORITHMS; LEARNING ALGORITHMS; MESSAGE PASSING;

AFFINITY PROPAGATION; COMPACT GENETIC ALGORITHM; ESTIMATION OF DISTRIBUTION ALGORITHMS; FACTORIZED DISTRIBUTION ALGORITHM; MESSAGE PASSING ALGORITHM; PRODUCT MODELS;

CLUSTERING ALGORITHMS;

EID: 78649779395     PISSN: 10636560     EISSN: 15309304     Source Type: Journal    
DOI: 10.1162/EVCO_a_00002     Document Type: Article

References (58)

1. Ackley, D. H. (1987). An empirical study of bit vector function optimization. Genetic Algorithms and Simulated Annealing, pp. 170-204.
2. Babel, L., Kellerer, H., and Kotov, V. (1998). The k-partitioning problem. Mathematical Methods of Operations Research, 47(1):59-82.
3. Braunstein, A., Ḿezard, M., and Zecchina, R. (2005). Survey propagation: An algorithm for satisfiability. Random Structures and Algorithms, 27(2):201-226.
4. Bron, C., and Kerbosch, J. (1973). Algorithm 457-Finding all cliques of an undirected graph. Communications of the ACM, 16(6):575-577.
5. Crescenzi, P., Goldman, D., Papadimitriou, C. H., Piccolboni, A., and Yannakakis, M. (1998). On the complexity of protein folding. Journal of Computational Biology, 5(3):423-466.
6. Cutello, V., Nicosia, G., Pavone, M., and Timmis, J. (2007).An immune algorithm for protein structure prediction on lattice models. IEEE Transactions on Evolutionary Computation, 11(1):101-117.
7. de la Ossa, L., Sastry, K., and Lobo, F. G. (2006). X-ary extended compact genetic algorithm in C++. IlliGAL Report 2006013, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL.
8. Deb, K., and Goldberg, D. E. (1991). Analyzing deception in trap functions. IlliGAL Report 91009, University of Illinois at Urbana-Champaign, Illinois GeneticAlgorithms Laboratory, Urbana, IL.
9. Deming, W. E., and Stephan, F. F. (1940). On a least squares adjustment of a sampled frequency table when the expected marginal totals are known. Annals of Mathematical Statistics, 11(4):427-444.
10. Dill, K. A. (1985). Theory for the folding and stability of globular proteins. Biochemistry, 24(6):1501-1509.
11. Duque, T. S. P. C., Goldberg, D. E., and Sastry, K. (2008). Enhancing the efficiency of the ECGA. In Rudolph, G., Jansen, T., Lucas, S., Poloni, C., and Beume, N., editors, Parallel Problem Solving from Nature, PPSN X, Vol. 5199 of Lecture Notes in Computer Science, pp. 165-174. Berlin: Springer.
12. Echegoyen, C., Lozano, J. A., Santana, R., and Larra ̃naga, P. (2007). Exact Bayesian network learning in estimation of distribution algorithms. In Proceedings of the 2007 Congress on Evolutionary Computation CEC-2007, pp. 1051-1058.
13. Etxeberria, R., and Larrãnaga, P. (1999). Global optimization using Bayesian networks. In A. Ochoa, M. R. Soto, and R. Santana (Eds.), Proceedings of the Second Symposium on Artificial Intelligence (CIMAF-99), pp. 151-173.
14. Frey, B. J., and Dueck, D. (2006). Mixture modeling by affinity propagation. In Proceedings of the 2005 Conference Advances in Neural Information Processing Systems 18, NIPS, pp. 379-386.
15. Frey, B. J., and Dueck, D. (2007). Clustering by passing messages between data points. Science, 315:972-976.
16. Ǵamez, J. A., Mateo, J. L., and Puerta, J. M. (2007). EDNA: Estimation of dependency networks algorithm. In J. Mira and J. R. A ́ lvarez (Eds.), Bio-inspired Modeling of Cognitive Tasks, Second International Work-Conference on the Interplay Between Natural and Artificial Computation, IWINAC 2007, Vol. 4527 of Lecture Notes in Computer Science (pp. 427-436). Berlin: Springer.
17. Ǵamez, J. A., Mateo, J. L., and Puerta, J. M. (2008). Improved EDNA (estimation of dependency networks algorithm) using combining function with bivariate probability distributions. In Proceedings of the 10th Annual Conference on Genetic and Evolutionary Computation GECCO-2008, pp. 407-414.
18. Ginalski, K., Grishin, N. V., Godzik, A., and Rychlewski, L. (2005). Practical lessons from protein structure prediction. Nucleid Acids Research, 33(6):1874-1891.
19. Goldberg, D. E. (1987). Simple genetic algorithms and the minimal, deceptive problem. In L. Davis (Ed.), Genetic Algorithms and Simulated Annealing (pp. 74-88). London: Pitman Publishing.
20. Goldberg, D. E. (1989). Genetic algorithms in search, optimization, and machine learning. Reading, MA: Addison-Wesley.
21. Greenwood, G. W., and Shin, J.-M. (2002). On the evolutionary search for solutions to the protein folding problem. In G. B. Fogel and D. W. Corne (Eds.), Evolutionary computation in bioinformatics (pp. 115-136). San Mateo, CA: Morgan Kaufmann.
22. Harik, G. (1999). Linkage learning via probabilisticmodeling in the ECGA. IlliGAL Report 99010, University of Illinois at Urbana-Champaign, Illinois GeneticAlgorithms Laboratory, Urbana, IL.
23. Harik, G. R. (1997). Learning gene linkage to efficiently solve problems of bounded difficulty using genetic algorithms. PhD thesis, University of Michigan, Ann Arbor. Also IlliGAL Report No. 97005.
24. Harik, G. R., Lobo, F. G., and Sastry, K. (2006). Linkage learning via probabilistic modeling in the ECGA. In M. Pelikan, K. Sastry, and E. Cant ́ u-Paz (Eds.), Scalable optimization via probabilistic modeling: From algorithms to applications (pp. 39-62). Berlin: Springer.
25. Hart, W. E., and Istrail, S. C. (1996). Fast protein folding in the hydrophobic-hydrophilic model within three-eights of optimal. Journal of Computational Biology, 3(1):53-96.
26. Hartmann, A. K., and Weigt, M. (2005). Phase transitions in combinatorial optimization problems: Basics, algorithms and statistical mechanics. New York: JohnWiley & Sons.
27. Heckerman, D., Chickering, D. M., Meek, C., Rounthwaite, R., and Kadie, C. M. (2000). Dependency networks for inference, collaborative filtering, and data visualization. Journal of Machine Learning Research, 1:49-75.
28. Heckerman, D., Geiger, D., and Chickering, D. M. (1995). Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning, 20:197-243.
29. Ḧons, R. (2006). Estimation of distribution algorithms and minimum relative entropy. PhD thesis, University of Bonn, Bonn, Germany.
30. Ḧons, R., Santana, R., Larrǎ;naga, P., and Lozano, J. A. (2007). Optimization by max-propagation using Kikuchi approximations. Tech. Rep. EHU-KZAA-IK-2/07, Department of Computer Science and Artificial Intelligence, University of the Basque Country.
31. Hsu, H.-P., Mehra, V., Nadler, W., and Grassberger, P. (2003). Growth algorithms for lattice heteropolymers at low temperatures. Journal of Chemical Physics, 118(1):444-451.
32. Krasnogor, N., Hart, W. E., Smith, J., and Pelta, D. A. (1999). Protein structure prediction with evolutionary algorithms. In W. Banzhaf, J. Daida, A. E. Eiben, M. H. Garzon, V. Honavar, M. Jakiela, and R. E. Smith (Eds.), Proceedings of the Genetic and Evolutionary Computation Conference, Vol. 2, pp. 1596-1601.
33. Larrãnaga, P., and Lozano, J. A. (Eds.). (2002). Estimation of distribution algorithms. A new tool for evolutionary computation. Dordrecht, The Netherlands: Kluwer Academic Publishers.
34. Leone, M., Sumedha, and Weigt, M. (2007). Clustering by soft-constraint affinity propagation: Applications to gene- xpression data. Bioinformatics, 23(20):2708-2715.
35. Mendiburu, A., Santana, R., and Lozano, J. A. (2007). Introducing belief propagation in estimation of distribution algorithms: A parallel framework. Tech. Rep. EHU-KAT-IK-11/07, Department of Computer Science and Artificial Intelligence, University of the Basque Country.
36. Ḿezard, M. (2007). Where are the good exemplars? Science, 315:949-951.
37. Ḿezard, M., Parisi, G., and Zechina, R. (2002). Analytic and algorithmic solution of random satisfiability problems. Science, 297:812.
38. M̈uhlenbein, H., and Ḧons, R. (2006). The factorized distributions and the minimum relative entropy principle. In M. Pelikan, K. Sastry, and E. Cant ́ u-Paz (Eds.), Scalable optimization via probabilistic modeling: From algorithms to applications (pp. 11-38). Berlin: Springer.
39. M̈uhlenbein, H., and Mahnig, T. (2001). Evolutionary synthesis of Bayesian networks for optimization. In M. Patel, V. Honavar, and K. Balakrishnan (Eds.), Advances in Evolutionary Synthesis of Intelligent Agents (pp. 429-455). Cambridge, MA: MIT Press.
40. M̈ uhlenbein, H., and Paaß, G. (1996). From recombination of genes to the estimation of distributions. I. Binary parameters. In H.-M. Voigt, W. Ebeling, I. Rechenberg, and H.-P. Schwefel (Eds.), Parallel Problem Solving from Nature, PPSN IV, Vol. 1141 of Lecture Notes in Computer Science (pp. 178-187). Berlin: Springer.
41. Neapolitan, R. E. (2003). Learning Bayesian networks. Upper Saddle River, NJ: Prentice Hall.
42. Ochoa, A., Ḧons, R., Soto, M. R., and M̈uhlenbein, H. (2003). A maximum entropy approach to sampling in EDA-The single connected case. In Progress in Pattern Recognition, Speech and Image Analysis, Vol. 2905 of Lecture Notes in Computer Science (pp. 683-690). Berlin: Springer.
43. Pearl, J. (1988). Probabilistic reasoning in intelligent systems:Networks of plausible inference. SanMateo, CA: Morgan Kaufmann.
44. Pelikan, M. (2005). Hierarchical Bayesian optimization algorithm. Toward a new generation of evolutionary algorithms. Berlin: Springer.
45. Pelikan, M., Goldberg, D. E., and Lobo, F. (2002). A survey of optimization by building and using probabilistic models. Computational Optimization and Applications, 21(1):5-20.
46. Pelikan, M., Sastry, K., and Goldberg, D. E. (2008). Sporadic model building for efficiency enhancement of the hierarchical BOA. Genetic Programming and Evolvable Machines, 9(1):53-84.
47. Rissanen, J. J. (1978). Modelling by shortest data description. Automatica, (14):465-471.
48. Santana, R. (2003). A Markov network based factorized distribution algorithm for optimization. In Proceedings of the 14th European Conference on Machine Learning (ECML-PKDD 2003), Vol. 2837 of Lecture Notes in Artificial Intelligence (pp. 337-348). Berlin: Springer.
49. Santana, R. (2005). Estimation of distribution algorithms with Kikuchi approximations. Evolutionary Computation, 13(1):67-97.
50. Santana, R., Echegoyen, C., Mendiburu, A., Bielza, C., Lozano, J. A., Larrãnaga, P., Armãnanzas, R., and Shakya, S. (2009). MATEDA: A suite of EDA programs in Matlab. Tech. Rep. EHUKZAA-I2/09, Department of Computer Science and Artificial Intelligence, University of the Basque Country.
51. Santana, R., Larrǎnaga, P., and Lozano, J. A. (2008). Protein folding in simplified models with estimation of distribution algorithms. IEEE Transactions on Evolutionary Computation, 12(4):418-438.
52. Santana, R., Ochoa, A., and Soto, M. R. (2002). Solving problems with integer representation using a tree based factorized distribution algorithm. In Electronic Proceedings of the First International NAISO Congress on Neuro Fuzzy Technologies.
53. Sastry, K., and Orriols-Puig, A. (2007). Extended compact genetic algorithm in Matlab. IlliGAL Report 2007009, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL.
54. Soto, M. R. (2003). A single connected factorized distribution algorithm and its cost of evaluation (In Spanish). PhD thesis, University of Havana, Havana, Cuba.
55. Soto, M. R., Ochoa, A., Acid, S., and Campos, L. M. (1999). Bayesian evolutionary algorithms based on simplified models. In A. Ochoa, M. R. Soto, and R. Santana (Eds.), Proceedings of the Second Symposium on Artificial Intelligence (CIMAF-99), pp. 360-367.
56. Yedidia, J. S., Freeman, W. T., and Weiss, Y. (2005). Constructing free energy approximations and generalized belief propagation algorithms. IEEE Transactions on Information Theory, 51(7):2282-2312.
57. Yu, T.-L. (2006). Amatrix approach for finding extrema: Problems with modularity, hierarchy and overlap. PhD thesis, University of Illinois at Urbana-Champaign, Urbana, Illinois.
58. Yu, T.-L., Goldberg, D. E., and Chen, Y.-P. (2003). A genetic algorithm design inspired by organizational theory: A pilot study of a dependency structure matrix driven genetic algorithm. IlliGAL Report 2003007, University of Illinois at Urbana-Champaign, Illinois Genetic Algorithms Laboratory, Urbana, IL.