The factorized distribution algorithm and the minimum relative entropy principle

Studies in Computational Intelligence

Volumn 33, Issue , 2007, Pages 11-37

  Mühlenbein, Heinz ^a   Höns, Robin ^a  

^a FRAUNHOFER IAIS   (Germany)

Author keywords

Bethe Kikuchi approximation; Boltzmann distribution; Estimation of distributions; Factorization of distributions; Maximum entropy principle; Maximum log likelihood; Minimum relative entropy

Indexed keywords

EID: 33846326856     PISSN: 1860949X     EISSN: None     Source Type: Book Series    
DOI: 10.1007/978-3-540-34954-9_2     Document Type: Article

References (27)

1. S. M. Aji and R. J. McEliece. The generalized distributive law. IEEE Transactions of Information Theory, 46(2):325-343, March 2000
2. S. M. Aji and R. J. McEliece. The generalized distributive law and free energy minimization. In Proceedings of the 39th Annual Allerton Conference on Communication, Control, and Computing, pages 672-681, 2001
3. R. G. Almond. Graphical Belief Modelling. Chapman and Hall, London, 1995
4. T. M. Cover and J. Thomas. Elements of Information Theory. Wiley, New York, 1989
5. J. Darroch and D. Ratcliff. Generalized iterative scaling for log-linear models. Ann.Math. Statistics, 43:1470-1480, 1972
6. Y. Gao and J. Culberson. Space complexity of estimation of distribution algorithms. Evolutionary Computation, 13(1):125-143, 2005
7. R. Höns. Estimation of Distribution Algorithms and Minimum Relative Entropy. PhD thesis, University of Bonn, 2005
8. E. T. Jaynes. Information theory and statistical mechanics. Physical Review, 6:620-643, 1957
9. E. T. Jaynes. Where do we stand on maximum entropy? In R. D. Levine and M. Tribus, editors, The Maximum Entropy Formalism. MIT Press, Cambridge, 1978
10. F. V. Jensen and F. Jensen. Optimal junction trees. In Proceedings of the 10th Conference on Uncertainty in Artificial Intelligence, pages 360-366, Seattle, 1994
11. M. I. Jordan, editor. Learning in Graphical Models. MIT Press, Cambrigde, 1999
12. B. W. Kernighan and S. Lin. An efficient heuristic for partitioning graphs. Bells Systems Technical Journal, 2:291-307, 1970
13. S. L. Lauritzen. Graphical Models. Clarendon Press, Oxford, 1996
14. R. J. McEliece and M. Yildirim. Belief propagation on partially ordered sets. In Proceedings of the 15th Internatonal Symposium on Mathematical Theory of Networks and Systems (MTNS 2002), 2002
15. H. Mühlenbein and R. Höns. The estimation of distributions and the minimum relative entropy principle. Evolutionary Computation, 13(1): 1-27, 2005
16. H. Mühlenbein and T. Mahnig. FDA - a scalable evolutionary algorithm for the optimization of additively decomposed functions. Evolutionary Computation, 7(4):353-376, 1999
17. H. Mühlenbein and T. Mahnig. Evolutionary optimization and the estimation of search distributions with applications to graph bipartitioning. Journal of Approximate Reasoning, 31(3):157-192, 2002
18. H. Mühlenbein and T. Mahnig. Mathematical analysis of evolutionary algorithms. In C. C. Ribeiro and P. Hansen, editors, Essays and Surveys in Metaheuristics, Operations Research/Computer Science Interface Series, pages 525-556. Kluwer, Norwell, 2002
19. H. Mühlenbein and T. Mahnig. Evolutionary algorithms and the Boltzmann distribution. In K. D. Jong, R. Poli, and J. C. Rowe, editors, Foundations of Genetic Algorithms 7, pages 525-556. Morgan Kaufmann Publishers, San Francisco, 2003
20. H. Mühlenbein, T. Mahnig, and A. Ochoa. Schemata, distributions and graphical models in evolutionary optimization. Journal of Heuristics, 5(2):213-247, 1999
21. H. Mühlenbein and G. Paaß. From recombination of genes to the estimation of distributions I. binary parameters. In H.-M. Voigt, W. Ebeling, I. Rechenberg, and H.-P. Schwefel, editors, Lecture Notes in Computer Science 1141: Parallel Problem Solving from Nature - PPSN IV), pages 178-187, Berlin, 1996. Springer-Verlag
22. M. Pelikan and D. Goldberg. Hierarchical BOA solves Ising spin glasses and MAXSAT. In Genetic and Evolutionary Computation Conference 2003, volume 2724 of Lecture Notes in Computer Science, pages 1271-1282. Springer, Berlin Heidelberg New York 2003. Also IlliGAL Report No. 2003001
23. R. Santana. Estimation of distribution algorithms with Kikuchi approximations. Evolutionary Computation, 13(1):67-97, 2005
24. Y. W. Teh and M. Welling. On improving the efficiency of the iterative proportional fitting procedure. In Proceedings of the International Workshop on Artificial Intelligence and Statistics, volume 9, 2003
25. C. Yanover and Y. Weiss. Finding the M most probable configurations using loopy belief propagation. In S. Thrun, L. Saul, and B. Schölkopf, editors, Advances in Neural Information Processing Systems 16. MIT Press, Cambridge, MA, 2004
26. J. S. Yedidia, W. T. Freeman, and Y. Weiss. Constructing free energy approximations and generalized belief propagation algorithms. Technical Report 2004-040, Mitsubishi Electric Research Laboratories, May 2004
27. A. L. Yuille. CCCP algorithms to minimize the Bethe and Kikuchi free energies: Convergent alternatives to belief propagation. Neural Computation, 14(7):1691-1722, 2002