Incremental acquisition of behaviors and signs based on a reinforcement learning schemata model and a spike timing-dependent plasticity network

Advanced Robotics

Volumn 21, Issue 10, 2007, Pages 1177-1199

  Taniguchi, Tadahiro ^a   Sawaragi, Tetsuo ^a  

^a KYOTO UNIVERSITY   (Japan)

Author keywords

Modular learning; Operant conditioning; Reinforcement learning; Spike timing dependent plasticity; Symbol emergence

Indexed keywords

BRAIN; COMPUTER ARCHITECTURE; MATHEMATICAL MODELS; NEUROLOGY;

MODULAR LEARNING; OPERANT CONDITIONING; SPIKE TIMING-DEPENDENT PLASTICITY; SYMBOL EMERGENCE;

REINFORCEMENT LEARNING;

EID: 34547521549     PISSN: 01691864     EISSN: 15685535     Source Type: Journal    
DOI: 10.1163/156855307781389374     Document Type: Article

References (46)

1. G. S. Reynolds, A Primer of Operant Conditioning. Scott, Foresman, Glenview, IL (1975).
2. R. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA (1998).
3. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Athena Scientific, Nashua, NH (1996).
4. C. Watkins and P. Dayan, Technical note: Q-learning, Machine Learn. 8, 279-292 (1992).
5. K. Doya, Reinforcement learning in continous time and space, Neural Comput.12, 219-245 (2000).
6. S. P. Singh, Transfer of learning by composing solutions of elemental sequential tasks, Machine Learning Arch.8, 323-339 (1992).
7. A. G. Barto, Adaptive critics and the basal ganglia, in: Models of Information Processing in the Basal Ganglia(J. C. Houk, J. L. Davis and D. G. Beiser, Eds), pp. 215-232. MIT Press, Cambridge, MA (1995).
8. K. Doya, What are the computations of the cerebllum, the basal ganglia, and the cerebral cortex?, Neural Networks12, 961-974 (1999).
9. S. C. Tanaka, K. Doya, G. Okada, K. Ueda, Y. Okamoto and S. Yamawaki, Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops, Nat. Neurosci.7, 887-893 (2004).
10. T. Aosaki, H. Tsubokawa, A. Ishida, K. Watanabe, A. M. Graybiel and M. Kimura, Responses of tonically active neurons in the primate's striatum undergo systematic changes during behavioral sensorimotor conditioning, J. Neurosci.14, 3969-3984 (1994).
11. W. Schultz, Predictive reward signal of dopamine neurons, J. Neurophysiol.80, 1-27 (1998).
12. W. Schultz, P. Dayan and P. R. Montague, A neural substrate of prediction and reward, Annu. Rev. Neurosci.15, 353 (1992).
13. A. R. Cassandra, L. P. Kaelbling and M. L. Littman, Acting optimally in partially observable stochastic domains, in: Proc. 12th Natl. Conf. on Artificial Intelligence, Seattle, WA, vol. 2, pp. 1023-1028 (1994).
14. T. Taniguchi and T. Sawaragi, Incremental acquisition of behavioral concepts through social interactions with a caregiver, in: Proc. Artificial Life and Robotics(2006).
15. D. O. Hebb, The Organization of Behavior. Wiley, New York, NY (1949).
16. Aibo offcial site: http://www.jp.aibo.com/.
17. M. Asada, K. F. MacDorman, H. Ishiguro and Y Kuniyoshi, Cognitive developmental robotics as a new paradigm for the design of humanoid robots, Robotics Autonomous Syst.37, 185-193 (2001).
18. R. S. Sutton, D. Precup and S. Singh, Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning, Artif. Intell.112, 181-211 (1999).
19. A. G. Barto, S. Singh and N. Chentanez, Intrinsically motivated, learning of hierarchical collections of skills, in: Proc. Int. Conf. on Development and Learning(2004).
20. V. Soni and S. Singh, Reinforcement learning of hierarchical skills on the sony Aibo robot, in: Proc. Int. Conf. on Development and Learning(2006).
21. Y. Takahashi et al., Modular learning syatem. and scheduling for behavior acquisition in multiagent environment, in: RoboCup 2004 Symp. Papers and Team Description Papers, CD-ROM (2004).
22. H. Markram et al., Regulation of synaptic efficacy by coincidence of postsynaptic aps and epsps, Science275, 213-215 (1997).
23. L. F. Abbott and S. B. Nelson, Synaptic plasticity: taming the beast, Nat. Neurosci. Suppl.3, 1178-1182 (2000).
24. Y. Sakai, K. Nakano and S. Yoshizawa, Synaptic regulation on various stdp rules, Neurocomputing58-60, 351-357 (2004).
25. D. M. Wolpert and M. Kawato, Multiple paired forward and inverse models for motor control, Neural Networks11, 1317-1329 (1998).
26. J. Tani, M. Ito and Y Sugita, Seif-organization of distributedly represented multiple behavior schemata in a mirror system: reviews of robots using RNNPB, Neural Networks17, 1273-1289 (2004).
27. R. A. Jacobs, M. I. Jordan et al., Adaptive mixtures of local experts, Neural Comput.3, 79-87 (1991).
28. T. Taniguchi and T. Sawaragi, Design and performance of symbols self-organized within an autonomous agent interacting with varied environments, in: Proc. IEEE Int. Workshop on ROMAN, CD-ROM (2004).
29. T. Taniguchi and T. Sawaragi, Self-organization of inner symbols for chase: symbol organization and embodiment, in: Proc. IEEE Int. Conf. on SMC, CD-ROM (2004).
30. M. Haruno, D. M. Wolpert and M. Kawato, Mosaic model for sensorimotor learning and control, Neural Comput.13, 2201-2220 (2001).
31. K. Doya et al., Multiple model-based reinforcement learning, Neural Comput.14, 1347-1369 (2000).
32. J. Tani and S. Nolfi, Learning to perceive the world as articulated: an approach for hierarchical learning in sensory-motor systems, Neural Networks12, 1131-1141 (1999).
33. R. Dearden et al., Bayesian Q-learning, in: Proc. AAAI-98, pp. 761-768 (1998).
34. R. Legenstein, C. Naeger and W. Maass, What can a neuron learn with spike-timing-dependent plasticity?, Neural Comput.17, 2337-2382 (2005).
35. H. D. I. Abarbanel, R. Huerta and M. I. Rabinovich, Dynamical model of long-term synaptic plasticity, Proc. Natl. Acad. Sci. USA99, 10132-10137 (2002).
36. H. Cateau and T. Fukai, A stochastic method to predict the consequence of arbitrary forms of spike-timing-dependent plasticity, Neural Comput.15, 597-620 (2003).
37. S. M. Bohte and M. C. Mozer, Reducing spike train variability: a computational theory of spike-timing dependent plasticity, Adv. Neural Inform. Process. Syst.17, 201-208 (2005).
38. G. Q. Bi and M. M. Poo, Synaptic modification in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type, J. Neurosci.18, 10464-10472 (1998).
39. S. Song et al., Competitive hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience3, 919-926 (2000).
40. M. Haruno, D. M. Wolpert and M. Kawato, Hierarchical mosaic for movement generation, Int. Congr. Ser.1250, 575-590 (2003).
41. S. R. Waterhouse and A. J. Robinson, Constructive algorithms for hierarchical mixtures of experts, Adv. Neural Inform. Process. Syst.8, 584-590 (1996).
42. S. Takamuku, Y Takahashi and M. Asada. Lexicon acquisition based on behavior learning, in: Proc. 4th IEEE Int. Conf. on Development and Learning(2005).
43. Webots, http://www.cybe.rbotics.com. (commercial mobile robot simulation software).
44. S. Hamad, The symbol grounding problem, Physica D42, 35-346 (1990).
45. T. Taniguchi and T. Sawaragi, Symbol emergence by combining a reinforcement learning schema model with asymmetric synaptic plasticity, in: Proc. 5th Int. Conf. on Development and Learning(2006).
46. S. Song and L. F. Abbott, Cortical development and remapping through spike timing-dependent plasticity, Neuron32, 339-350 (2001).