Self-segmentation of sequences: automatic formation of hierarchies of sequential behaviors

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 30, Issue 3, 2000, Pages 403-418

  Sun, Ron ^a,b   Sessions, Chad ^c  

^a IEEE   (United States)

^b UNIVERSITY OF MISSOURI   (United States)

^c UNIVERSITY OF ALABAMA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

HIERARCHICAL LEARNING; REINFORCEMENT LEARNING; SELF SEGMENTATION OF SEQUENCES; SEQUENTIAL DECISION MAKING;

ALGORITHMS; COMPUTATIONAL COMPLEXITY; DECISION THEORY; DYNAMIC PROGRAMMING; KNOWLEDGE BASED SYSTEMS; MARKOV PROCESSES; PROBABILITY DISTRIBUTIONS; RECURRENT NEURAL NETWORKS;

LEARNING SYSTEMS;

EID: 0033720075     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/3477.846230     Document Type: Article

References (45)

1. F. Bacchus and Q. Yang, "Downward refinement and the efficiency of hierarchical problem solving," Artif. Intell., vol. 71, no. 1, pp. 43-100, 1994.
2. D. Bertsekas and J. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, 1996.
3. A. Cassandra, L. Kaelbling, and M. Littman, "Acting optimally in partially observable stochastic domains," presented at the Proc. 12th Nat. Conf. Artificial Intelligence, San Mateo, CA, 1994.
4. L. Chrisman, "Reinforcement learning with perceptual aliasing: The perceptual distinction approach," in Proc. AAAI. San Mateo, CA, 1993, pp. 183-188.
5. P. Dayan and G. Hinton, "Feudal reinforcement learning," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 1993.
6. T. Dietterich, "Hierarchical Reinforcement Learning with MAXQ Value Function Decomposition," , http://www.engr.orst.edu/∼tgd/cv/pubs.html, 1997.
7. J. Elman, "Finding structure in time," Cognitive Sci., vol. 14, pp. 179-212, 1990.
8. P. Frasconi, M. Gori, and G. Soda, "Recurrent neural networks and prior knowledge for sequence processing," Knowledge Based Syst., vol. 8, no. 6, pp. 313-332, 1995.
9. C. L. Giles, B. G. Home, and T. Lin, "Learning a class of large finite state machines with a recurrent neural network," Neural Networks, vol. 8, no. 9, pp. 1359-1365, 1995.
10. M. Humphrys, "W-Learning: A Simple RL-Based Society of Mind," Univ. Cambridge, Computer Laboratory, Tech. Rep. 362, 1996.
11. L. Kaelbling, "Hierarchical learning in stochastic domains: Preliminary results," in Proc. ICML. San Francisco, CA, 1993, pp. 167-173.
12. L. Kaelbling, M. Littman, and A. Moore, "Reinforcement learning: A survey," J. Artif. Intell. Res., vol. 4, pp. 237-285, 1996.
13. C. Knoblock, J. Tenenberg, and Q. Yang, "Characterizing abstraction hierarchies for planning," in Proc. AAAI. San Mateo, CA, 1994, pp. 692-697.
14. Y. Kuniyoshi, M. Inaba, and H. Inoue, "Learning by watching: Extracting reusable task knowledge from visual observation of human performance," IEEE Trans. Robot. Automat., 1991.
15. L. Lin, "Reinforcement Learning for Robots Using Neural Networks," Ph.D. dissertation, Carnegie Mellon, Pittsburgh, PA, 1993.
16. S. Mahadevan and J. Connell, "Automatic programming of behavior-based robot with reinforcement learning," Artif. Intell., vol. 55, pp. 311-365, 1992.
17. A. McCallum, "Learning to use selective attention and short-term memory in sequential tasks," in Proc. Conf. Simulation of Adaptive Behavior. Cambridge, MA, 1996, pp. 315-324.
18. _, "Reinforcement Learning with Selective Perception and Hidden State," Ph.D. dissertation, Dept. Computer Science, University of Rochester, Rochester, NY, 1996.
19. G. Monohan, "A survey of partially observable Markov decision processes: Theory, models, and algorithms," Manage. Sci., vol. 28, no. 1, pp. 1-16, 1982.
20. A. Moore and C. Atkeson, "The parti-game algorithm for variable resolution reinforcement learning in multidimensional state spaces," Mach. Learn., 1994.
21. C. Nevill-Manning and I. Witten, "Identifying hierarchical structure in sequences: A linear-time algorithm," J. Artif. Intell. Res., vol. 7, pp. 67-82, 1997.
22. R. Parr and S. Russell, "Approximating optimal policies for partially observable stochastic domains," in Proc. IJCAI'95. San Mateo, CA, 1995, pp. 1088-1094.
23. _, "Reinforcement learning with hierarchies of machines," in Advances in Neural Information Processing Systems 9. Cambridge, MA: MIT Press, 1997.
24. D. Precup, R. Sutton, and S. Singh, "Multi-time models for temporary abstract planning," in Advances in Neural Information Processing Systems 10. Cambridge, MA: MIT Press, 1998.
25. K. Pfleger and B. Hayes-Roth, "Plan should abstractly describe intended behavior," in Proc. JCIS, vol. 1. Durham, NC, 1997, pp. 29-33.
26. M. Puterman, Markov Decision Processes. New York: Wiley, 1994.
27. M. Ring, "Incremental development of complex behaviors through automatic construction of sensory-motor hierarchies," in Proc. ICML. San Francisco, CA, 1991, pp. 343-347.
28. J. Rosca and D. Ballard, "Evolution-based discovery of hierarchical behavior," in Proc. AAAI. Cambridge, MA, 1996.
29. A. Roy, S. Govil, and R. Miranda, A neural network learning theory and a polynomial time RBF algorithm, , 1996. Manuscript.
30. E. Sacerdoti, "Planning in a hierarchy of abstraction spaces," Artif. Intell., vol. 5, pp. 115-135, 1974.
31. J. Schmidhuber, "Learning complex, extended sequences using the principle of history compression," Neural Computation, vol. 4, no. 2, pp. 234-242, 1992.
32. _, "Learning unambiguous reduced sequence descriptions," in Advances in Neural Information Processing Systems, 1993, pp. 291-298.
33. S. Singh, "Learning to Solve Markovian Decision Processes," Ph.D. dissertation, Univ. Massachusetts, Amherst, MA, 1994.
34. S. Singh, T. Jaakkola, and M. Jordan, "Reinforcement learning with soft state aggregation," in Advances in Neural Information Processing Systems 7, S. J. Hanson, J. Cowan, and C. L. Giles, Eds. San Mateo, CA: Morgan Kaufmann, 1994.
35. E. Sondik, "The optimal control of partially observable Markov processes over the infinite horizon: Discounted costs," Oper. Res., vol. 26, no. 2, 1978.
36. R. Sun and T. Peterson, "Multi-agent reinforcement learning: Weighting and partitioning," Neural Networks, vol. 12, no. 4-5, pp. 127-153, 1999.
37. R. Sun and C. Sessions, Learning plans without a priori knowledge, in Proc. WCCI-IJCNN'98, IEEE Press, Piscataway, NJ, vol. 1, no. 1-6, 1998.
38. R. Sutton, "TD Models: Modeling the world at a mixture of time scales," in Proc. ICML. San Francisco, CA, 1995.
39. P. Tadepalli and T. Dietterich, "Hierarchical explanation-based reinforcement learning," in Proc. Int. Conf. Machine Learning. San Francisco, CA, 1997, pp. 358-366.
40. C. Tham, "Reinforcement learning of multiple tasks using a hierarchical CMAC architecture," Robotics and Autonomous Systems, vol. 15, pp. 247-274, 1995.
41. S. Thrun and A. Schwartz, "Finding structure in reinforcement learning," in Neural Information Processing Systems. Cambridge, MA: MIT Press, 1995.
42. C. Watkins, "Learning with Delayed Rewards," Ph.D. dissertation, Cambridge Univ., Cambridge, U.K., 1989.
43. G. Weiss, "Distributed reinforcement learning," Robotics and Autonomous Systems, vol. 15, no. 1-2, pp. 135-142, 1995.
44. S. Whitehead and L. Lin, "Reinforcement learning of non-Markov decision processes," Artif. Intell., vol. 73, no. 1-2, pp. 271-306, 1995.
45. M. Wiering and J. Schmidhuber, "HQ-learning," Adaptive Behavior, vol. 6, no. 2, pp. 219-246, 1998.