What do you want to do today? Relevant-information bookkeeping in goal-oriented behaviour

Artificial Life XII: Proceedings of the 12th International Conference on the Synthesis and Simulation of Living Systems, ALIFE 2010

Volumn , Issue , 2010, Pages 176-183

  Van Dijk, Sander G ^a   Polani, Daniel ^a   Nehaniv, Chrystopher L ^a  

^a UNIVERSITY OF HERTFORDSHIRE   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

AMOUNT OF INFORMATION; GOAL-ORIENTED; PERCEPTION-ACTION LOOPS;

ARTIFICIAL INTELLIGENCE; CHEMISTRY; CIRCUIT SIMULATION; GENETIC ENGINEERING; SOFTWARE ENGINEERING; SPEECH SYNTHESIS;

BIOLOGICAL SYSTEMS;

EID: 80051618668     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (46)

1. Asadi, M. and Huber, M. (2005). Accelerating Action Dependent Hierarchical Reinforcement Learning Through Autonomous Subgoal Discovery. In Proceedings of the ICML 2005 Workshop on Rich Representations for Reinforcement Learning.
2. Bakker, B. and Schmidhuber, J. (2004). Hierarchical Reinforcement Learning Based on Subgoal Discovery and Subpolicy Specialization. In Proceedings of the 8-th Conference on Intelligent Autonomous Systems, IAS-8, pages 438-445.
3. Barlow, H. B. (1961). Possible Principles Underlying the Transformations of Sensory Messages. In Rosenblith, W., editor, Sensory Communication, chapter 13, pages 217-234. MIT Press, Cambridge, MA.
4. Barto, A. G. and Mahadevan, S. (2003). Recent Advances in Hierarchical Reinforcement Learning. Discrete Event Dynamic Systems, 13(1-2): 41-77.
5. Bell, W. J. (1990). Searching Behavior Patterns in Insects. Annual Review of Entomology, 35: 447-467.
6. Blahut, R. E. (1972). Computation of Channel Capacity and RateDistortion Functions. IEEE Transactions on Information Theory, 18: 460-473.
7. Brenner, N., Bialek, W., and de Ruyter van Steveninck, R. (2000). Adaptive Rescaling Maximizes Information Transmission. Neuron, 26(3): 695-702.
8. Cover, T. M. and Thomas, J. A. (1991). Elements of information theory. Wiley-Interscience, New York, NY, USA.
9. Digney, B. (1996). Emergent Hierarchical Control Structures: Learning Reactive / Hierarchical Relationships in Reinforcement Environments. In Proceedings of the Fourth Conference on the Simulation of Adaptive Behavior: SAB 98, pages 363-372. MIT Press.
10. Girgin, S., Polat, F., and Alhajj, R. (2006). Learning by Automatic Option Discovery from Conditionally Terminating Sequences. In ECAI, pages 494-498.
11. Hamilton, A. F. d. C. and Grafton, S. T. (2006). Goal Representation in Human Anterior Intraparietal Sulcus. Journal of Neuroscience, 26(4): 1133-1137.
12. Hengst, B. (2002). Discovering Hierarchy in Reinforcement Learning with HEXQ. In ICML '02: Proceedings of the Nineteenth International Conference on Machine Learning, pages 243-250, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
13. Hommel, B., Müsseler, J., Aschersleben, G., and Prinz, W. (2001). The Theory of Event Coding (TEC): A framework for perception and action planning. Behavioral and Brain Sciences, 24(05): 849-878.
14. Jeffery, W. (2001). Cavefish as a Model System in Evolutionary Developmental Biology. Developmental Biology, 231(1):1-12.
15. Jonsson, A. and Barto, A. (2006). Causal Graph Based Decomposition of Factored MDPs. Journal Of Machine Learning Research, 7: 2259-2301.
16. Kazemitabar, S. J. and Beigy, H. (2009). Automatic Discovery of Subgoals in Reinforcement Learning Using Strongly Connected Components. Advances in Neuro-Information Processing, 5506: 829-834.
17. Kheradmandian, G. and Rahmati, M. (2009). Automatic Abstraction in Reinforcement Learning Using Data Mining Techniques. Robotics and Autonomous Systems, 57(11):1119-1128.
18. Klyubin, A. S., Polani, D., and Nehaniv, C. L. (2004). Tracking Information Flow through the Environment: Simple Cases of Stigmergy. In Pollack, J., Bedau, M., Husbands, P., Ikegami, T., and Watson, R. A., editors, Artificial Life IX: Proceedings of the Ninth International Conference on the Simulation and Synthesis of Living Systems, pages 563-568. The MIT Press.
19. Klyubin, A. S., Polani, D., and Nehaniv, C. L. (2008). Keep Your Options Open: An Information-Based Driving Principle for Sensorimotor Systems. PLoS ONE, 3(12):e4018.
20. Konidaris, G. and Barto, A. (2007). Building Portable Options: Skill Transfer in Reinforcement Learning. In IJCAI'07: Proceedings of the 20th International Joint Conference on Artifical intelligence, pages 895-900, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
21. Kozlova, O., Sigaud, O., and Meyer, C. (2009). Automated Discovery of Options in Factored Reinforcement Learning. In Proceedings of the ICML/UAI/COLT Workshop on Abstraction in Reinforcement Learning, pages 24-29, Montreal, Canada.
22. Kretchmar, R., Feil, T., and Bansal, R. (2003). Improved Automatic Discovery of Subgoals for Options in Hierarchical Reinforcement Learning. Journal of Computer Science & Technology, 3: 9-14.
23. Laughlin, S. B., de Ruyter van Steveninck, R. R., and Anderson, J. C. (1998). The Metabolic Cost of Neural Information. Nature Neuroscience, 1: 36-41.
24. Mannor, S., Menache, I., Hoze, A., and Klein, U. (2004). Dynamic Abstraction in Reinforcement Learning via Clustering. In ICML '04: Proceedings of the twenty-first International Conference on Machine Learning, page 71, New York, NY, USA. ACM.
25. McGovern, A. and Barto, A. G. (2001). Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density. In ICML '01: Proceedings of the Eighteenth International Conference on Machine Learning, pages 361-368, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
26. Montague, P. R., Hyman, S. E., and Cohen, J. D. (2004). Computational Roles for Dopamine in Behavioural Control. Nature, 431(7010): 760-7.
27. Nehaniv, C. L., Polani, D., Olsson, L., and Klyubin, A. S. (2007). Information-Theoretic Modeling of Sensory Ecology: Channels of Organism-Specific Meaningful Information. In Laubichler, M. D. and Müller, G. B., editors, Modeling Biology: Structures, Behaviors, Evolution (The Vienna Series in Theoretical Biology), pages 241-282. MIT Press.
28. Perkins, T. and Precup, D. (1999). Using Options for Knowledge Transfer in Reinforcement Learning. Technical report, University of Massachusetts, Amherst, MA, USA. UM-CS-1999-034.
29. Pfeifer, R., Lungarella, M., Sporns, O., and Kuniyoshi, Y. (2007). On the Information-Theoretic Implications of Embodiment - Principles and Methods. In Proc. of the 50th Anniversary Summit of Artificial Intelligence, volume 4850, pages 76-86. Springer-Verlag.
30. Pickett, M. and Barto, A. G. (2002). PolicyBlocks: An Algorithm for Creating Useful Macro-Actions in Reinforcement Learning. In ICML '02: Proceedings of the Nineteenth International Conference on Machine Learning, pages 506-513, San Francisco, CA, USA. Morgan Kaufmann Publishers Inc.
31. Polani, D. (2009). Information: Currency of Life? HFSP Journal, 3: 307-316.
32. Polani, D., Nehaniv, C. L., Martinetz, T., and Kim, J. T. (2006). Relevant Information in Optimized Persistence vs. Progeny Strategies. In Artificial Life X: Proceedings of The 10th International Conference on the Simulation and Synthesis of Living Systems, Bloomington IN.
33. Saito, N., Mushiake, H., Sakamoto, K., Itoyama, Y., and Tanji, J. (2005). Representation of Immediate and Final Behavioral Goals in the Monkey Prefrontal Cortex During an Instructed Delay Period. Cerebral Cortex, 15(10): 1535-46.
34. Salge, C. and Polani, D. (2010). Digested Information as an Information Theoretic Motivation for Social Interaction. Journal of Artificial Societies and Social Simulation (JASSS). accepted.
35. Shannon, C. E. (1948). A Mathematical Theory of Communication. Bell System Technical Journal, 27: 379-423 and 623-656.
36. Şimşek, Ö. and Barto, A. G. (2004). Using Relative Novelty to Identify Useful Temporal Abstractions in Reinforcement Learning. In ICML '04: Proceedings of the Twenty-First International Conference on Machine Learning, page 95, New York, NY, USA. ACM.
37. Şimşek, Ö. and Barto, A. G. (2009). Skill Characterization Based on Betweenness. In Koller, D., Schuurmans, D., Bengio, Y., and Bottou, L., editors, Advances in Neural Information Processing Systems 21, pages 1497-1504.
38. Şimşek, Ö., Wolfe, A. P., and Barto, A. G. (2005). Identifying Useful Subgoals in Reinforcement Learning by Local Graph Partitioning. In Proceedings of the Twenty-Second International Conference on Machine Learning, pages 816-823.
39. Spiers, H. J. and Maguire, E. A. (2006). Thoughts, Behaviour, and Brain Dynamics During Navigation in the Real World. Neuroimage, 31(4): 1826-40.
40. Sporns, O. and Lungarella, M. (2006). Evolving Coordinated Behavior by Maximizing Information Structure. In Artificial Life X, pages 323-329. MIT Press.
41. Sun, R. and Sessions, C. (2000). Self-Segmentation of Sequences: Automatic Formation of Hierarchies of Sequential Behaviors. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 30(3): 403-418.
42. Sutton, R. S. and Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press.
43. Sutton, R. S., Precup, D., and Singh, S. (1999). Between MDPs and Semi-MDPs: a Framework for Temporal Abstraction in Reinforcement Learning. Artificial Intelligence, 112(1-2):181-211.
44. van Dijk, S. G., Polani, D., and Nehaniv, C. L. (2009). Hierarchical Behaviours: Getting the Most Bang for your Bit. In Proceedings of the 10th European Conference on Artificial Life. Springer (In Press).
45. Vergassola, M., Villermaux, E., and Shraiman, B. I. (2007). 'Info-taxis' as a Strategy for Searching Without Gradients. Nature, 445(7126): 406-9.
46. Weld, D. S. (1994). An Introduction to Least Commitment Planning. AI Magazine, 15(4): 27-61.