Optimal Behavioral Hierarchy

PLoS Computational Biology

Volumn 10, Issue 8, 2014, Pages

  Solway, Alec ^a   Diuk, Carlos ^a,c   Córdova, Natalia ^a   Yee, Debbie ^a   Barto, Andrew G ^b   Niv, Yael ^a   Botvinick, Matthew M ^a  

^a PRINCETON UNIVERSITY   (United States)

^b UNIVERSITY OF MASSACHUSETTS   (United States)

^c FACEBOOK   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ACTION HIERARCHY; BEHAVIORAL EXPERIMENT; GOAL-DIRECTED; HIERARCHICAL STRUCTURES; HUMAN BEHAVIORS; OPTIMAL ACTIONS; OPTIMAL HIERARCHY; SUBTASK;

BEHAVIORAL RESEARCH;

BEHAVIOR; BRAIN; HUMAN; HUMAN EXPERIMENT; ADOLESCENT; ADULT; BIOLOGICAL MODEL; BIOLOGY; FEMALE; LEARNING; MALE; MIDDLE AGED; MOTIVATION; PHYSIOLOGY; YOUNG ADULT;

ADOLESCENT; ADULT; BEHAVIOR; COMPUTATIONAL BIOLOGY; FEMALE; GOALS; HUMANS; LEARNING; MALE; MIDDLE AGED; MODELS, NEUROLOGICAL; YOUNG ADULT;

EID: 84923228672     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1003779     Document Type: Article

References (47)

1. Hebb DO (1949) The organization of behavior: A neuropsychological theory. New York, NY: John Wiley & Sons.
2. Miller GA, Galanter E, Pribram KH (1960) Plans and the structure of behavior. New York, NY: Holt, Rinehart & Winston.
3. Tinbergen N (1951) The study of instinct. Oxford, England: Clarendon Press.
4. Tolman EC (1932) Purposive behavior in animals and men. New York, NY: Century.
5. Botvinick MM, (2008) Hierarchical models of behavior and prefrontal function. Trends in Cognitive Sciences 12: 201–208.
6. Shallice T, Cooper RP (2011) The organization of mind. Oxford, England: Oxford University Press.
7. Whiten A, Flynn E, Brown K, Lee T, (2006) Imitation of hierarchical action structure by young children. Developmental Science 9: 574–582.
8. Zacks JM, Kurby CA, Eisenberg ML, Haroutunian N, (2011) Prediction error associated with the perceptual segmentation of naturalistic events. Journal of Cognitive Neuroscience 23: 4057–4066.
9. Graybiel AM, (1998) The basal ganglia and chunking of action repertoires. Neurobiology of Learning and Memory 70: 119–136.
10. Barto AG, Konidaris GD, Vigorito CM (2013) Behavioral Hierarchy: Exploration and Representation, Springer. pp. 13–46.
11. Botvinick MM, Niv Y, Barto AC, (2009) Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective. Cognition 113: 262–280.
12. Foster D, Dayan P, (2002) Structure in the space of value functions. Machine Learning 49: 325–346.
13. Anderson JR, Bothell D, Byrne MD, Douglass S, Lebiere C, et al. (2004) An integrated theory of the mind. Psychological Review 111: 1036–1060.
14. Boutilier C, Dean T, Hanks S, (1999) Decision-theoretic planning: structural assumptions and computational leverage. Journal of Artificial Intelligence Research 11: 1–94.
15. Laird JE (2012) The Soar cognitive architecture. Cambridge, MA: The MIT Press.
16. Newell A, Simon HA (1972) Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.
17. Sutton RS, Barto AG (1998) Reinforcement learning: An introduction. The MIT Press.
18. Sutton RS, Precup D, Singh S, (1999) Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence 112: 181–211.
19. Sutton RS (1990) Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In: Proceedings of the 7th International Conference on Machine Learning. pp. 216–224.
20. Simsek Ö, Barto AG (2008) Skill characterization based on betweenness. In: Koller D, Schuurmans D, Bengio Y, Bottou L, editors, Advances in Neural Information Processing Systems. volume 21 , pp. 1497–1504.
21. van Dijk SG, Polani D (2011) Grounding subgoals in information transitions. In: IEEE Symposium on Adaptive Dynamic Programming And Reinforcement Learning (ADPRL). IEEE, pp. 105–111.
22. Vigorito CM, Barto AG, (2010) Intrinsically motivated hierarchical skill learning in structured environments. IEEE Transactions on Autonomous Mental Development 2: 132–143.
23. MacKay DJC (2003) Information theory, inference and learning algorithms. Cambridge University Press.
24. Hauskrecht M, Meuleau N, Kaelbling LP, Dean T, Boutilier C (1998) Hierarchical solution of Markov decision processes using macro-actions. In: Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence. San Francisco, CA: Morgan Kaufmann, pp. 220–229.
25. Rosvall M, Bergstrom CT, (2008) Maps of random walks on complex networks reveal community structure. Proceedings of the National Academy of Sciences 105: 1118–1123.
26. Cover TM, Thomas JA (2012) Elements of information theory. Hoboken, NJ: John Wiley & Sons.
27. Feldman J, (2000) Minimization of boolean complexity in human concept learning. Nature 407: 630–633.
28. Kemp C, (2012) Exploring the conceptual universe. Psychological Review 119: 685–722.
29. Schmidhuber J, (1992) Learning complex, extended sequences using the principle of history compression. Neural Computation 4: 234–242.
30. Thrun S, Schwartz A (1995) Finding structure in reinforcement learning. In: Tesauro G, Touretzky DS, Leen TK, editors, Advances in Neural Information Processing Systems, volume 7 . pp. 385–392.
31. van Dijk SG, Polani D, Nehaniv CL (2011) Hierarchical behaviours: getting the most bang for your bit. In: Kampis G, Karsai I, Szathmáry E, editors, Advances in Artificial Life: Darwin Meets von Neumann, Springer. pp. 342–349.
32. Kazemitabar SJ, Beigy H (2009) Automatic discovery of subgoals in reinforcement learning using strongly connected components. In: Köppen M, Kasabov N, Coghill G, editors, Advances in Neuro-Information Processing. Springer, volume 5506 , pp.829–834.
33. Moradi P, Shiri ME, Rad AA, Khadivi A, Hasler M, (2012) Automatic skill acquisition in reinforcement learning using graph centrality measures. Intelligent Data Analysis 16: 113–135.
34. Feldman J, (2009) Bayes and the simplicity principle in perception. Psychological Review 116: 875–887.
35. Orbán G, Fiser J, Aslin RN, Lengyel M, (2008) Bayesian learning of visual chunks by human observers. Proceedings of the National Academy of Sciences 105: 2745–2750.
36. Brady TF, Konkle T, Alvarez GA, (2009) Compression in visual working memory: Using statistical regularities to form more efficient memory representations. Journal of Experimental Psychology: General 138: 487–502.
37. Mathy F, Feldman J, (2012) What's magic about magic numbers? Chunking and data compression in short-term memory. Cognition 122: 346–362.
38. Finley S, Newport EL (2010) Morpheme segmentation from distributional information. In: Boston University Conference on Language Development (BUCLD) Online Proceedings Supplement.
39. Chater N, Vitányi P, (2003) Simplicity: A unifying principle in cognitive science? Trends in Cognitive Sciences 7: 19–22.
40. Robinet V, Lemaire B, Gordon MB, (2011) MDLChunker: A MDL-based cognitive model of inductive learning. Cognitive Science 35: 1352–1389.
41. Schapiro AC, Rogers TT, Cordova NI, Turk-Browne NB, Botvinick MM, (2013) Neural representations of events arise from temporal community structure. Nature Neuroscience 16: 486–492.
42. Wiener JM, Mallot HA, (2003) ‘Fine-to-coarse’ route planning and navigation in regionalized environments. Spatial Cognition & Computation 3: 331–358.
43. Anderson JR, Douglass S, (2001) Tower of Hanoi: Evidence for the cost of goal retrieval. Journal of Experimental Psychology: Learning, Memory, and Cognition 27: 1331–1346.
44. Barto AG, Mahadevan S, (2003) Recent advances in hierarchical reinforcement learning. Discrete Event Dynamic Systems 13: 341–379.
45. van Dijk SG, Polani D, (2013) Informational constraints-driven organization in goal-directed behavior. Advances in Complex Systems 16: 1350016.
46. Brandes U, Delling D, Gaertler M, Gorke R, Hoefer M, et al. (2008) On modularity clustering. IEEE Transactions on Knowledge and Data Engineering 20: 172–188.
47. Scrucca L, (2013) GA: a package for genetic algorithms in R. Journal of Statistical Software. 53: 1–37.