The estimation of distributions and the minimum relative entropy principle

Evolutionary Computation

Volumn 13, Issue 1, 2005, Pages 1-27

  Mühlenbein, Heinz ^a   Höns, Robin ^a  

^a FRAUNHOFER IAIS   (Germany)

Author keywords

Bayesian information criterion; Bethe approximation; Boltzmann distribution; Estimation of distributions; Factorization of distributions; Maximum entropy principle; Minimum log likelihood ratio; Minimum relative entropy

Indexed keywords

BAYESIAN INFORMATION CRITERION; BETHE APPROXIMATION; BOLTZMANN DISTRIBUTION; ESTIMATION OF DISTRIBUTIONS; FACTORIZATION OF DISTRIBUTIONS; MAXIMUM ENTROPY PRINCIPLE; MINIMUM LOG-LIKELIHOOD RATIO; MINIMUM RELATIVE ENTROPY;

APPROXIMATION THEORY; COMPUTER NETWORKS; DATA REDUCTION; ENTROPY; LEARNING ALGORITHMS; MATHEMATICAL MODELS; PROBLEM SOLVING; STATISTICAL METHODS;

GENETIC ALGORITHMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; BAYES THEOREM; BIOLOGY; COMPUTER SIMULATION; ENTROPY; EVOLUTION; METHODOLOGY; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BAYES THEOREM; COMPUTATIONAL BIOLOGY; COMPUTER SIMULATION; ENTROPY; EVOLUTION; LIKELIHOOD FUNCTIONS; MODELS, STATISTICAL;

EID: 15544390134     PISSN: 10636560     EISSN: None     Source Type: Journal    
DOI: 10.1162/1063656053583469     Document Type: Article

References (29)

1. Almond, R. G. (1995). Graphical Belief Modelling. Chapman & Hall, London.
2. Bertelé, U. and Brioschi, F. (1972). Nonserial Dynamic Programming. Academic Press, New York.
3. Bethe, H. (1935). Statistical theory of superlattices. Proc. Roy. Soc. London A, 150:552-558.
4. Cover, T. M. and Thomas, J. (1989). Elements of Information Theory. Wiley, New York.
5. Csiszár, I. (1975). I-divergence geometry of probability distributions and minimization problems. Annals of Probability, 3:146-158.
6. Huang, C. and Darwiche, A. (1996). Inference in belief networks: A procedural guide. International Journal of Approximate Reasoning, 15(3):225-263.
7. Jaynes, E. T. (1957). Information theory and statistical mechanics. Phys. Rev, 6:620-643.
8. Jaynes, E. T. (1978). Where do we stand on maximum entropy? In Levine, R. D. and Tribus, M., editors, The Maximum Entropy Formalism. MIT Press, Cambridge.
9. Jensen, F. V. and Jensen, F. (1994). Optimal junction trees. In Proceedings of the 10th Conference on Uncertainty in Artificial Intelligence, pages 360-366, Seattle.
10. Jordan, M. I. (1999). Learning in Graphical Models. MIT Press, Cambrigde.
11. Kikuchi, R. (1951). A theory of cooperative phenomena. Phys.Review, 115:988-1003.
12. Kullback, S. (1968). Probability densities with given marginals. Annals of Mathematical Statistics, 39(4):1236-1243.
13. Larrañaga, P. and Lozano, J. (2002). Estimation of Distribution Algorithms: A New Tool for Evolutionary Optimization. Kluwer Academic Press, Boston.
14. Lauritzen, S. L. (1996). Graphical Models. Clarendon Press, Oxford.
15. Lin, S. and Kernighan, B. (1973). An effective heuristic algorithm for the traveling-salesman. Operations Research, 21:498-516.
16. Martelli, A. and Montari, U. (1972). Nonserial dynamic programming: On the optimal strategy of variable elimination for the rectangular lattice. Jour. Math. Analyis and Applications, 40:226-242.
17. Mühlenbein, H. and Mahnig, T. (1999). FDA - a scalable evolutionary algorithm for the optimization of additively decomposed functions. Evolutionary Computation, 7(4):353-376.
18. Mühlenbein, H. and Mahnig, T. (2000). Evolutionary algorithms: From recombination to search distributions. In Kallel, L., Naudts, B., and Rogers, A., editors, Theoretical Aspects of Evolutionary Computing, Natural Computing, pages 137-176. Springer Verlag, Berlin.
19. Mühlenbein, H. and Mahnig, T. (2001). Evolutionary computation and beyond. In Uesaka, Y., Kanerva, P., and Asoh, H., editors, Foundations of Real-World Intelligence, pages 123-188. CSLI Publications, Stanford, California.
20. Mühlenbein, H. and Mahnig, T. (2002a). Evolutionary optimization and the estimation of search distributions with applications to graph bipartitioning. Journal of Approximate Reasoning, 31(3):157-192.
21. Mühlenbein, H. and Mahnig, T. (2002b). Mathematical analysis of evolutionary algorithms. In Ribeiro, C. C. and Hansen, P., editors, Essays and Surveys in Metaheuristics, Operations Research/Computer Science Interface Series, pages 525-556. Kluwer Academic Publisher, Norwell.
22. Mühlenbein, H. and Mahnig, T. (2003). Evolutionary algorithms and the Boltzmann distribution. In Jong, K. D., Poli, R., and Rowe, J. C., editors, Foundations of Genetic Algorithms 7, pages 525-556. Morgan Kaufmann Publishers, San Francisco.
23. Mühlenbein, H., Mahnig, T., and Ochoa, A. (1999). Schemata, distributions and graphical models in evolutionary optimization. Journal of Heuristics, 5(2):213-247.
24. Mühlenbein, H. and Paaß, G. (1996). From recombination of genes to the estimation of distributions I. binary parameters. In Voigt, H.-M., Ebeling, W., Rechenberg, I., and Schwefel, H.-P., editors, Lecture Notes in Computer Science 1141: Parallel Problem Solving from Nature - PPSN IV), pages 178-187, Springer Verlag, Berlin.
25. Ochoa, A., Höns, R., Soto, M., and Mühlenbein, H. (2003). A maximum entropy approach to sampling in EDA - the singly connected case. In Proceedings of the 8th Iberoamerican Congress on Pattern Recognition (CIARP 2003), volume 2905 of Lecture Notes in Computer Science, pages 683-690. Springer Verlag, Berlin.
26. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 7:461-464.
27. Xiang, Y., Wong, S., and Cereone, N. (1997). A 'microscopic' study of minimum entropy search in learning decomposable markov networks. Machine Learning, 26:65-92.
28. Yedidia, J. S., Freeman, W. T., and Weiss, Y. (2001). Understanding belief propagation and its generalizations. Technical Report 2001-22, Mitsubishi Electric Research Laboratories.
29. Yedidia, J. S., Freeman, W. T., and Weiss, Y. (2004). Constructing free energy approximations and generalized belief propagation algorithms. Technical Report 2004-40, Mitsubishi Electric Research Laboratories.