A tutorial on the cross-entropy method

Annals of Operations Research

Volumn 134, Issue 1, 2005, Pages 19-67

  De Boer, Pieter Tjerk ^a   Kroese, Dirk P ^b   Mannor, Shie ^c   Rubinstein, Reuven Y ^d  

^a UNIVERSITY OF TWENTE   (Netherlands)

^b UNIVERSITY OF QUEENSLAND   (Australia)

^c MCGILL UNIVERSITY   (Canada)

^d TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Cross entropy method; Machine learning; Monte Carlo simulation; Randomized optimization; Rare events

Indexed keywords

EID: 17444409624     PISSN: 02545330     EISSN: 15729338     Source Type: Journal    
DOI: 10.1007/s10479-005-5724-z     Document Type: Article

References (50)

1. Aarts, E.H.L. and J.H.M Korst. (1989). Simulated Annealing and Boltzmann Machines. John Wiley & Sons.
2. Alon, G., D.P. Kroese, T. Raviv, and R.Y. Rubinstein. (2005). "Application of the Cross-Entropy Method to the Buffer Allocation Problem in a Simulation-Based Environment." Annals of Operations Research 134, 19-67.
3. Asmussen, S., D.P. Kroese, and R.Y. Rubinstein. (2005). "Heavy Tails, Importance Sampling and Cross-Entropy." Stochastic Models 21(1). To appear.
4. Barto, A., R. Sutton, and C. Anderson. (1983). "Neuron-Like Adaptive Elements that can Solve Difficult Learning Control Problems." IEEE Transactions on Systems, Man, and Cybernetics 13, 834-846.
5. Barto, A.G. and R.S. Sutton. (1998). Reinforcement Learning. MIT Press.
6. Baxter, J., P.L. Bartlett, and L. Weaver. (2001). Experiments with Infinite-Horizon, Policy-Gradient Estimation." Journal of Artificial Intelligence Research 15, 351-381.
7. Bertsekas, D.P. (1995). Dynamic Programming and Optimal Control. Athena Scientific.
8. Bertsekas, D.P. and J.N. Tsitsiklis. (1995). Neuro-Dynamic Programming. Athena Scientific.
9. Chepuri, K. and T. Homem-de-Mello. (2005). "Solving the Vehicle Routing Problem with Stochastic Demands using the Cross-Entropy Method." Annals of Operations Research 134, 153-181.
10. Cohen, I., B. Golany, and A. Shtub. (2005). "Managing Stochastic Finite Capacity Multi-Project Systems Through the Cross-Entropy Method." Annals of Operations Research 134, 183-199.
11. Colorni, A., M. Dorigo, F. Maffioli, V. Maniezzo, G. Righini, and M. Trubian. (1996). "Heuristics from Nature for Hard Combinatorial Problems." International Transactions in Operational Research 3(1), 1-21.
12. Dayan, P. and C. Watkins (1992). "Q-Learning." Machine Learning 8, 279-292.
13. de Boer, P.T. (2000). "Analysis and Efficient Simulation of Queueing Models of Telecommunication Systems." Ph.D. thesis, University of Twente.
14. de Boer, P.T., D.P. Kroese, and R.Y. Rubinstein. (2002). "Estimating Buffer Overflows in Three Stages using Cross-Entropy." In Proceedings of the 2002 Winter Simulation Conference. San Diego, pp. 301-309.
15. de Boer, P.T., D.P. Kroese, and R.Y. Rubinstein. (2004). "A Fast Cross-Entropy Method for Estimating Buffer Overflows in Queueing Networks." Management Science 50(7), 883-895.
16. Dorigo, M., G. Di Caro, and L.M. Gambardella. (1999). "Ant Algorithms for Discrete Optimization." Artificial Life 5(2), 137-172.
17. Dubin, U. (2002). "Application of the Cross-Entropy Method to Neural Computation." Master's thesis, Technion, Electrical Engineering.
18. Dubin, U. (2004). "Application of the Cross-Entropy Method for Image Segmentation." Unpublished.
19. Garey, M.R. and D.S. Johnson. (1979). Computers and Intractability: A Guide to the Theory of NP-Completeness. San Francisco: W.H. Freeman and Company.
20. Glover, F. and M. Laguna. (1993). Modern Heuristic Techniques for Combinatorial Optimization, chapter 3: Tabu search. Blackwell Scientific Publications.
21. Goldberg, D. (1989). Genetic Algorithms in Search, Optimization and Machine Learning. Addison Wesley.
22. Gutjahr, W.J. (2000). "A Graph-Based Ant System and Its Convergence." Future Generations Computing 16, 873-888.
23. Helvik, B.E. and O. Wittner. (2001). "Using the Cross-Entropy Method to Guide/Govern Mobile Agent's Path Finding in Networks." In 3rd International Workshop on Mobile Agents for Telecommunication Applications - MATA'01.
24. Homem-de-Mello, T. and R.Y. Rubinstein. (2002). "Rare Event Estimation for Static Models via Cross-Entropy and Importance Sampling." (submitted for publication).
25. Hui, K.-P., N. Bean, M. Kraetzl, and D.P. Kroese. (2005). "The Cross-Entropy Method for Network Reliability Estimation." Annals of Operations Research 134, 101-118.
26. Kaelbling, L.P., M. Liftman, and A.W. Moore. (1996). "Reinforcement Learning - A Survey." Journal of Artificial Intelligence Research 4, 237-285.
27. Keith, J. and D.P. Kroese. (2002). "SABRES: Sequence Alignment By Rare Event Simulation." In Proceedings of the 2002 Winter Simulation Conference, San Diego, pp. 320-327.
28. Konda, V.R. and J.N. Tsitsiklis. (2003). "Actor-Critic Algorithms." SIAM Journal on Control and Optimization 42(4), 1134-1166.
29. Kroese, D.P. and R.Y. Rubinstein. (2004). "The Transform Likelihood Ratio Method for Rare Event Simulation with Heavy Tails." Queueing Systems 46, 317-351.
30. Lieber, D. (1998). "Rare-Events Estimation via Cross-Entropy and Importance Sampling." Ph.D. thesis, William Davidson Faculty of Industrial Engineering and Management, Technion, Haifa, Israel.
31. Mannor, S., R.Y. Rubinstein, and Y. Gat. (2003). "The Cross-Entropy Method for Fast Policy Search." In Proceedings of the Twentieth International Conference on Machine Learning. Morgan Kaufmann, pp. 512-519.
32. Margolin, L. (2002). "Application of the Cross-Entropy Method to Scheduling Problems." Master's thesis, Technion, Industrial Engineering.
33. Margolin, L. (2004). "The Cross-Entropy Method for the Single Machine Total Weighted Tardiness Problem." Unpublished.
34. Margolin, L. (2005). "On the Convergence of the Cross-Entropy Method." Annals of Operations Research 134, 201-214.
35. Menache, I., S. Mannor, and N. Shimkin. (2005). "Basis Function Adaption in Temporal Difference Reinforcement Learning." Annals of Operations Research 134, 215-238.
36. Papadimitriou, C.H. and M. Yannakakis. (1991). "Optimization, Approximation, and Complexity Classes." J. Comput. System Sci. 43, 425-440.
37. Puterman, M. (1994). Markov Decision Processes. Wiley-Interscience.
38. Ridder, A. (2005). "Importance Sampling Simulations of Markovian Reliability Systems Using Cross-Entropy." Annals of Operations Research 134, 119-136.
39. Rosenstein, M.T. and A.G. Barto. (2001). "Robot Weightlifting by Direct Policy Search." In B. Nebel (ed.). Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence. Morgan Kaufmann, pp. 839-846.
40. Rubinstein, R.Y. (1997). "Optimization of Computer Simulation Models with Rare Events." European Journal of Operations Research 99, 89-112.
41. Rubinstein, R.Y. (1999). "The Simulated Entropy Method for Combinatorial and Continuous Optimization." Methodology and Computing in Applied Probability 2, 127-190.
42. Rubinstein, R.Y. (2001). "Combinatorial Optimization, Cross-Entropy, Ants and Rare Events." In S. Uryasev and P.M. Pardalos (eds.). Stochastic Optimization: Algorithms and Applications. Kluwer, pp. 304-358.
43. Rubinstein, R. Y. (2002). "Cross-Entropy and Rare-Events for Maximal Cut and Bipartition Problems." ACM Transactions on Modeling and Computer Simulation, 27-53.
44. Rubinstein, R.Y. and D.P. Kroese. (2004). The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning. Springer-Verlag, New York.
45. Rubinstein, R.Y. and B. Melamed. (1998). Modern Simulation and Modeling. Wiley series in probability and Statistics.
46. Rubinstein, R.Y. and A. Shapiro. (1993). Discrete Event Systems: Sensitivity Analysis and Stochastic Optimization via the Score Function Method. Wiley.
47. Shi, L. and S. Olafsson. (2000). "Nested Partitioning Method for Global Optimization." Operations Research 48(3), 390-407.
48. Sutton, R.S., D. McAllester, S. Singh, and Y. Mansour. (2000). "Policy Gradient Methods for Reinforcement Learning with Function Approximation." In Advances in Neural Information Processing Systems 12. MIT Press, pp. 1057-1063.
49. Voudouris, C. (2003). "Guided Local Search - An Illustrative Example in Function Optimisation." BT Technology Journal 16(3), 46-50.
50. Webb, A. (1999). Statistical Pattern Recognition. Arnold.