Hierarchical curiosity loops and active sensing

Neural Networks

Volumn 32, Issue , 2012, Pages 119-129

  Gordon, Goren ^a   Ahissar, Ehud ^a  

^a WEIZMANN INSTITUTE OF SCIENCE   (Israel)

Author keywords

Active sensing; Internal models; Intrinsic reward; Object localization; Reinforcement learning; Touch; Vibrissa; Whisker

Indexed keywords

ACTIVE SENSING; INTERNAL MODELS; INTRINSIC REWARDS; OBJECT LOCALIZATION; TOUCH; VIBRISSA; WHISKER;

MAMMALS; OBJECT RECOGNITION; OPTIMIZATION; REINFORCEMENT LEARNING;

AUTONOMOUS AGENTS;

ACTIVE SENSING; ARTICLE; ARTIFICIAL NEURAL NETWORK; CONCEPT MAPPING; DEVELOPMENTAL STAGE; FEEDBACK SYSTEM; INFORMATION PROCESSING; LEARNING ALGORITHM; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; PREDICTION; PRIORITY JOURNAL; PROBABILITY; PROCESS DEVELOPMENT; PROCESS OPTIMIZATION; REINFORCEMENT; REWARD; SENSORIMOTOR FUNCTION; SIGNAL DETECTION; VIBRISSA;

ALGORITHMS; ANIMALS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; EXPLORATORY BEHAVIOR; NEURAL NETWORKS (COMPUTER); PHYSICAL STIMULATION; RATS; RATS, WISTAR; REINFORCEMENT (PSYCHOLOGY); SPACE PERCEPTION; TOUCH; VIBRISSAE;

EID: 84861775137     PISSN: 08936080     EISSN: 18792782     Source Type: Journal    
DOI: 10.1016/j.neunet.2012.02.024     Document Type: Article

References (36)

1. Ahissar E., Abeles M., Ahissar M., Haidarliu S., Vaadia E. Hebbian-like functional plasticity in the auditory cortex of the behaving monkey. Neuropharmacology 1998, 37:633-655.
2. Ahissar E., Arieli A. Figuring space by time. Neuron 2001, 32:185-201.
3. Ahissar E., Kleinfeld D. Closed-loop neuronal computations: focus on vibrissa somatosensation in rat. Cereb Cortex 2003, 13:53-62.
4. Barto, A.G., Singh, S., & Chentanez, N. (2004) Intrinsically motivated learning of hierarchical collections of skills. International conference on developmental learning (ICDL).
5. Behera, L., Gopal, M., & Chaudhury, S. (1995) Self-organizing neural networks for learning inverse dynamics of robot manipulator. IEEE/IAS international conference on industrial automation and control (I A & C'95) pp. 457-460.
6. Bhatnagar, S., Sutton, R., Ghavamzadeh, M., & Lee, M. 2007 Incremental natural actor-critic algorithms. Twenty-first annual conference on advances in neural information processing systems pp. 105-112.
7. Birdwell J.A., Solomon J.H., Thajchayapong M., Taylor M.A., Cheely M., Towal R.B., Conradt J., Hartmann M.J.Z. Biomechanical models for radial distance determination by the rat vibrissal system. Journal of Neurophysiology 2007, 98:2439-2455.
8. Cheah C.C., Liu C., Slotine J.J. Adaptive tracking control for robots with unknown kinematic and dynamic properties. The International Journal of Robotics Research 2006, 25:283-296.
9. Diamond M.E., von Heimendahl M., Knutsen P.M., Kleinfeld D., Ahissar E. 'Where' and 'what' in the whisker sensorimotor system. Nature Reviews Neuroscience 2008, 9:601-612.
10. Ego-Stengel V., Shulz D.E., Haidarliu S., Sosnik R., Ahissar E. Acetylcholine-dependent induction and expression of functional plasticity in the barrel cortex of the adult rat. Journal of Neurophysiology 2001, 86:422-437.
11. Gordon, G., & Ahissar, E. (2011) Reinforcement active learning hierarchical loops. International joint conference on neural networks (IJCNN) pp. 3008-3015.
12. Grant R.A., Mitchinson B., Fox C.W., Prescott T.J. Active touch sensing in the rat: anticipatory and regulatory control of whisker movements during surface exploration. Journal of Neurophysiology 2009, 101:862-874.
13. Grant R.A., Mitchinson B., Prescott T.J. The development of whisker control in rats in relation to locomotion. Developmental Psychobiology 2011.
14. Hill D.N., Bermejo R., Zeigler H.P., Kleinfeld D. Biomechanics of the vibrissa motor plant in rat: rhythmic whisking consists of triphasic neuromuscular activity. Journal of Neuroscience 2008, 28:3438-3455.
15. Jordan M.I. Forward models: supervised learning with a distal teacher. Cognitive Science 1992, 16:307-354.
16. Kawato M.M. Internal models for motor control and trajectory planning. Current Opinion in Neurobiology 1999, 9:718-727.
17. Kleinfeld D., Berg R., O'Conner S. Anatomical loops and their electrical dynamics in relation to whisking by rat. Somatosensory and Motor Research 1999, 16:69-88.
18. Knutsen P.M., Ahissar E. Orthogonal coding of object location. Trends in Neurosciences 2009, 32:101-109.
19. Knutsen P.M., Pietr M., Ahissar E. Haptic object localization in the vibrissal system: behavior and performance. Journal of Neuroscience 2006, 26:8451-8464.
20. Lalazar H., Vaadia E. Neural basis of sensorimotor learning: modifying internal models. Current Opinion in Neurobiology 2008, 18:573-581.
21. Nguyen-Tuong, D., Peters, J., Seeger, M., & Scholkopf, B. (2008). Learning inverse dynamics: a comparison. European symposium on artificial neural networks (ESANN) pp. 13-18.
22. O'Connor D.H., Clack N.G., Huber D., Komiyama T., Myers E.W., Svoboda K. Vibrissa-based object localization in head-fixed mice. Journal of Neuroscience 2010, 30:1947-1967.
23. Oudeyer P.Y., Kaplan F., Hafner V.V. Intrinsic motivation systems for autonomous mental development. Evolutionary Computation, IEEE Transactions on 2007, 11:265-286.
24. Ouyang P.R., Zhang W.J., Gupta M.M. An adaptive switching learning control method for trajectory tracking of robot manipulators. Mechatronics 2006, 16:51-61.
25. Peters J., Schaal S. Natural actor-critic. Neurocomputing 2005, 71:1180-1190. 1352986.
26. Schmidhuber J. Formal theory of creativity, fun, and intrinsic motivation (1990-2010). IEEE Transactions on Autonomous Mental Development 2010, 2:230-247.
27. Shadmehr R., Krakauer J.W. A computational neuroanatomy for motor control. Experimental Brain Research 2008, 185:359-381.
28. Simony E., Bagdasarian K., Herfst L., Brecht M., Ahissar E., Golomb D. Temporal and spatial characteristics of vibrissa responses to motor commands. Journal of Neuroscience 2010, 30:8935-8952.
29. Simsek, O., & Barto, A.G. (2006) An intrinsic reward mechanism for efficient exploration. 23rd international conference on Machine learning pp. 833-840.
30. Szwed M., Bagdasarian K., Ahissar E. Encoding of vibrissal active touch. Neuron 2003, 40:621-630.
31. Szwed M., Bagdasarian K., Blumenfeld B., Barak O., Derdikman D., Ahissar E. Responses of trigeminal ganglion neurons to the radial distance of contact during active vibrissal touch. Journal of Neurophysiology 2006, 95:791-802.
32. Towal R.B., Hartmann M.J. Variability in velocity profiles during free-air whisking behavior of unrestrained rats. Journal of Neurophysiology 2008, 100:740-752.
33. Venkatraman S., Carmena J.M. Active sensing of target location encoded by cortical microstimulation. IEEE Transactions on Neural Systems and Rehabilitation Engineering 2011, 19:317-324.
34. Wainscott S.K., Donchin O., Shadmehr R. Internal models and contextual cues: encoding serial order and direction of movement. Journal of Neurophysiology 2005, 93:786-800.
35. Weng J. Developmental robotics: theory and experiments. International Journal of Humanoid Robotics 2004, 1:199-236.
36. Wolpert D.M., Kawato M. Multiple paired forward and inverse models for motor control. Neural Networks 1998, 11:1317-1329.