Information-theoretic objective functions for lifelong learning

AAAI Spring Symposium - Technical Report

Volumn SS-13-05, Issue , 2013, Pages 62-69

  Zhang, Byoung Tak ^a  

^a Seoul National University   (South Korea)

Author keywords

[No Author keywords available]

Indexed keywords

BAYESIAN FRAMEWORKS; INFORMATION GAIN; INTERNAL MODELING; KNOWLEDGE TRANSFER; LIFE LONG LEARNING; MODEL COMPLEXITY; OBJECTIVE FUNCTIONS; SELECTIVE SAMPLING;

INFORMATION THEORY; KNOWLEDGE MANAGEMENT;

LEARNING SYSTEMS;

EID: 84883288598     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (26)

1. Ay, N., Bertschinger, N., Der, R., Guetter, F., and Olbrich, E. 2008. Predictive Information and Explorative Behavior in Autonomous Robots. European Physical Journal B 63:329-339.
2. Barber, D., Cemgeil, A. T., and Chiappa, S. eds. 2011. Bayesian Time Series Model. Cambridge University Press.
3. Bialek, W., Nemenman, I., and Tishby, N. 2001. Predictability, Complexity, and Learning. Neural Computation 13:2409-2463.
4. Cohn, D., Atlas, L., and Ladner, R. 1990. Training Connectionist Networks with Queries and Selective Sampling. In: D. Touretzky (ed.) Advances in Neural Information Processing Systems 2, Morgan Kaufmann.
5. Cohn, D., Atlas, L., and Ladner, R. 1994. Improving Generalization with Active Learning. Machine Learning 15(2):201-221.
6. Creutzig, F., Globerson, A., and Tishby, N. 2009. Past-Future Information Bottleneck in Dynamical Systems. Physical Review E 79(4):042519.
7. Eaton, E. and desJardins, M. 2011. Selective Transfer between Learning Tasks using Task-based Boosting. In Proceedings of the Twenty-Fifth Conference on Artificial Intelligence (AAAI-11), 337-342. AAAI Press.
8. Freund, Y., Seung, H. S., Shamir, E., and Tishby, N. 1993. Information, Prediction, and Query by Committee. In: S. Hanson et al. (ed.) Advances in Neural Information Processing Systems 5, Morgan Kaufmann.
9. Friston, K. 2009. The Free-energy Principle: A Unified Brain Theory? Nature Reviews Neuroscience 11:127-138.
10. Jung, T., Polani, D., and Stone, P. 2011. Empowerment for Continuous Agent-Environment Systems. Adaptive Behavior 19(1): 16-39.
11. Polani, D. 2011. An Information Perspective on How the Embodiment Can Relieve Cognitive Burden. In Proceedings of IEEE Symposium Series in Computational Intelligence: Artificial Life, 78-85. IEEE Press.
12. Schmidhuber, J. 1991. Curious Model-Building Control Systems. In Proceedings of the International Joint Conference on Neural Networks, 1458-1463.
13. Still, S. 2009. Information-Theoretic Approach to Interactive Learning. European Physical Journal 85:28005.
14. Still, S. and Precup, D. 2012. An Information-Theoretic Approach to Curiosity-driven Reinforcement learning. Theory in Biosciences 131(3):139-148.
15. Sutton, R. S. and Barto, A. G. 1998. Reinforcement Learning: An Introduction. MIT Press.
16. Tishby, N., Pereira, F. C., and Bialek, W. 1999. The Information Bottleneck Method, In Proceedings of the 37th Annual Allerton Conference on Communication, Control and Computing.
17. Tishby, N. and Polani, D. 2010. Information Theory of Decisions and Actions. In: Perception-Reason-Action Cycle: Models, Algorithm and Systems. Springer.
18. Thrun, S. and Moeller, K. 1992. Active Exploration in Dynamic Environments, In: J. Moody et al., (ed.) Advances in Neural Information Processing Systems 4, Morgan Kaufmann.
19. Valiant, L. G. 1984. A Theory of the Learnable, Communications of the ACM 27(11):1134-1342.
20. Yi, S.-J., Zhang, B.-T., and Lee, D. D. 2012. Online Learning of Uneven Terrain for Humanoid Bipedal Walking, In Proceedings of the Twenty-Fourth Conference on Artificial Intelligence (AAAI-10), 1639-1644. AAAI Press.
21. Zahedi, K., Ay, N., and Der, R. 2010. Higher Coordination with Less Control - A Result of Information Maximization in the Sensorimotor Loop. Adaptive Behavior 18(3-4):338-355.
22. Zhang, B.-T., Ha, J.-W., and Kang, M. 2012. Sparse Population Code Models of Word Learning in Concept Drift, In Proceedings of the Thirty-Fourth Annual Conference of the Cognitive Science Society (CogSci 2012), 1221-1226.
23. Zhang, B.-T. 2008. Hypernetworks: A Molecular Evolutionary Architecture for Cognitive Learning and Memory, IEEE Computational Intelligence Magazine 3(3):49-63.
24. Zhang, B.-T. 1994. Accelerated Learning by Active Example Selection. International Journal of Neural Systems 5(1):67-75.
25. Zhang, B.-T. and Veenker, G. 1991a. Focused Incremental Learning for Improved Generalization with Reduced Training Sets. In Proceedings of the International Conference on Artificial Neural Networks (ICANN 1991), 227-232.
26. Zhang, B.-T. and Veenker, G. 1991b. Neural Networks that Teach Themselves Through Genetic Discovery of Novel Examples, In Proceedings of the International Joint Conference on Neural Networks (IJCNN 1991), 690-695.