Variational information maximisation for intrinsically motivated reinforcement learning

Advances in Neural Information Processing Systems

Volumn 2015-January, Issue , 2015, Pages 2125-2133

  Mohamed, Shakir ^a   Rezende, Danilo J ^a  

^a DEEPMIND   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPLEX NETWORKS; COMPUTATIONAL COMPLEXITY; DIGITAL STORAGE; INFORMATION SCIENCE; LEARNING SYSTEMS; OPTIMIZATION; REINFORCEMENT LEARNING; STOCHASTIC SYSTEMS; VARIATIONAL TECHNIQUES;

BLAHUT-ARIMOTO ALGORITHM; CONVOLUTIONAL NETWORKS; ENUMERATIVE ALGORITHMS; EXPONENTIAL COMPLEXITY; NOISY TRANSMISSION CHANNELS; STATISTICAL QUANTITY; STOCHASTIC OPTIMISATION; VARIATIONAL INFERENCE;

LEARNING ALGORITHMS;

EID: 84965128263     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (29)

1. Barber, D. and Agakov, F. The IM algorithm: a variational approach to information maximization. In NIPS, volume 16, pp. 201, 2004.
2. Brunel, N. and Nadal, J. Mutual information, Fisher information, and population coding. Neural Computation, 10(7):1731-1757, 1998.
3. Buhmann, J. M., Chehreghani, M. H., Frank, M., and Streich, A. P. Information theoretic model selection for pattern analysis. Workshop on Unsupervised and Transfer Learning, 2012.
4. Cover, T. M. and Thomas, J. A. Elements of information theory. John Wiley & Sons, 1991.
5. Duchi, J., Hazan, E., and Singer, Y. Adaptive subgradient methods for online learning and stochastic optimization. The Journal of Machine Learning Research, 12:2121-2159, 2011.
6. Gao, S., Steeg, G. V., and Galstyan, A. Efficient estimation of mutual information for strongly dependent variables. arXiv:1411.2003, 2014.
7. Gretton, A., Herbrich, R., and Smola, A. J. The kernel mutual information. In ICASP, volume 4, pp. IV-880, 2003.
8. Itti, L. and Baldi, P. F. Bayesian surprise attracts human attention. In NIPS, pp. 547-554, 2005.
9. Jaakkola, T. S. and Jordan, M. I. Improving the mean field approximation via the use of mixture distributions. In Learning in graphical models, pp. 163-173. 1998.
10. Jung, T., Polani, D., and Stone, P. Empowerment for continuous agent-environment systems. Adaptive Behavior, 19(1):16-39, 2011.
11. Klyubin, A. S., Polani, D., and Nehaniv, C. L. Empowerment: A universal agent-centric measure of control. In IEEE Congress on Evolutionary Computation, pp. 128-135, 2005.
12. Koutník, J., Schmidhuber, J., and Gomez, F. Evolving deep unsupervised convolutional networks for vision-based reinforcement learning. In GECCO, pp. 541-548, 2014.
13. LeCun, Y. and Bengio, Y. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361:310, 1995.
14. Little, D. Y. and Sommer, F. T. Learning and exploration in action-perception loops. Frontiers in neural circuits, 7, 2013.
15. Mnih, V., Kavukcuoglu, K., and Silver, D., et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529-533, 2015.
16. Nelson, J. D. Finding useful questions: on Bayesian diagnosticity, probability, impact, and information gain. Psychological review, 112(4):979, 2005.
17. Ng, Andrew Y, Harada, Daishi, and Russell, Stuart. Policy invariance under reward transformations: Theory and application to reward shaping. In ICML, 1999.
18. Oudeyer, P. and Kaplan, F. How can we define intrinsic motivation? In International conference on epigenetic robotics, 2008.
19. Rubin, J., Shamir, O., and Tishby, N. Trading value and information in MDPs. In Decision Making with Imperfect Decision Makers, pp. 57-74. 2012.
20. Salge, C., Glackin, C., and Polani, D. Changing the environment based on empowerment as intrinsic motivation. Entropy, 16(5):2789-2819, 2014.
21. Salge, C., Glackin, C., and Polani, D. Empowerment-an introduction. In Guided Self-Organization: Inception, pp. 67-114. 2014.
22. Schmidhuber, J. Formal theory of creativity, fun, and intrinsic motivation (1990-2010). IEEE Trans. Autonomous Mental Development, 2(3):230-247, 2010.
23. Singh, S. P., Barto, A. G., and Chentanez, N. Intrinsically motivated reinforcement learning. In NIPS, 2005.
24. Still, S. and Precup, D. An information-theoretic approach to curiosity-driven reinforcement learning. Theory in Biosciences, 131(3):139-148, 2012.
25. Sutton, R. S. and Barto, A. G. Introduction to reinforcement learning. MIT Press, 1998.
26. Tishby, N., Pereira, F. C., and Bialek, W. The information bottleneck method. In Allerton Conference on Communication, Control, and Computing, 1999.
27. Todorov, E., Erez, T., and Tassa, Y. Mujoco: A physics engine for model-based control. In Intelligent Robots and Systems (IROS), pp. 5026-5033, 2012.
28. Wissner-Gross, A. D. and Freer, C. E. Causal entropic forces. Phys. Rev. Let., 110(16), 2013.
29. Yeung, R. W. The Blahut-Arimoto algorithms. In Information Theory and Network Coding, pp. 211-228. 2008.