Learning Obstacle Avoidance with an Operant Behavior Model

Artificial Life

Volumn 10, Issue 1, 2004, Pages 65-81

  Gutnisky, D A ^a,b   Zanutto, B S ^a,b  

^a UNIVERSIDAD DE BUENOS AIRES   (Argentina)

^b Facultad de Ciencias Naturales e Instituto M Lillo   (Argentina)

Author keywords

Animals; Artificial neural networks; Neural networks; Operant learning; Q Learning; Reinforcement learning

Indexed keywords

ALGORITHMS; BEHAVIORAL RESEARCH; MATHEMATICAL MODELS; NEURAL NETWORKS; NEUROLOGY; ROBOT LEARNING; ROBOTS; ROBUSTNESS (CONTROL SYSTEMS);

ANIMATS; OPERANT LEARNING; Q-LEARNING; REINFORCEMENT LEARNING;

ARTIFICIAL INTELLIGENCE;

ARTICLE; ARTIFICIAL NEURAL NETWORK; AVOIDANCE BEHAVIOR; INSTRUMENTAL CONDITIONING; PHYSIOLOGY; PSYCHOLOGICAL MODEL;

AVOIDANCE LEARNING; CONDITIONING, OPERANT; MODELS, PSYCHOLOGICAL; NEURAL NETWORKS (COMPUTER);

EID: 1542334432     PISSN: 10645462     EISSN: None     Source Type: Journal    
DOI: 10.1162/106454604322875913     Document Type: Article

References (35)

1. Amsel, A. (1958). The role of frustrative nonreward in continuous reward situations. Psychological Bulletin, 55, 102-119.
2. Baloch, A., & Waxman, A. (1991). Visual learning, adaptive expectations, and behavioral conditioning of the mobile robot Mavin. Neural Networks, 4, 271-302.
3. Barto, A. G., Sutton, R. S., & Anderson, C. W. (1983). Neuronlike elements that can solve difficult learning problems. IEEE Transactions on Systems, Man, and Cybernetics, 13, 835-846.
4. Bates, J. F., & Goldman-Rakic, P. S. (1993). Prefrontal connections of medial motor areas in the rhesus monkey. Journal of Comparative Neurolobiology, 336, 211-228.
5. Chang, C., & Gaudiano, P. (1998). Application of biological learning theories to mobile robot avoidance and approach behaviors. Journal of Complex Systems, 1, 79-114.
6. Crites, R. H., & Barto, A. G. (1996). Improving elevator performance using reinforcement learning. In D. Touretzky, M. Mozer, & M. Hasselmo (Eds.), Neural Information Processing Systems, Vol. 8.
7. Daly, H. B., & Daly, J. T. (1982). A mathematical model of reward and aversive nonreward: Its application in over 30 appetitive learning situations. Journal of Experimental Psychology: General, 111, 441-480.
8. Daly, H. B., & Daly, J. T. (1984). DMOD - A mathematical model of reward and aversive nonreward in appetitive learning situations: Program and instruction manual. Behavior Research Methods, Instruments, & Computers, 16, 38-52.
9. Damper, R. I., French, R. L. B., & Scutt, T. W. (2000). ARBIB: An autonomous robot based on inspiration from biology. Robotics and Autonomous Systems, 31(4), 247-274.
10. Dean, J. (1998). Animats and what they can tell us. Trends in Cognitive Sciences, 2(2), 60-67.
11. Donahoe, J. W., Burgos, J. E., & Palmer, D. C. (1993). A selectionist approach to reinforcement. Journal of the Experimental Analysis of Behavior, 60, 17-40.
12. Donahoe, J. W., & Palmer, D. C. (1994). Learning and complex behavior. Boston: Allyn & Bacon.
13. Donahoe, J. W., Palmer, D. C., & Burgos, J. E. (1997). The S-R issue: Its status in behavior analysis and in Donahoe & Palmer's Learning and Complex Behavior. Journal of the Experimental Analysis of Behavior, 67, 193-211.
14. Gaudiano, P., & Chang, C. (1997). Adaptive obstacle avoidance with a neural network for operant conditioning: Experiments with real robots. In Proceedings of the 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA) (pp. 13-18). Monterey, CA.
15. Goldman-Rakic, P. S. (1987). Circuitry of primate prefrontal cortex and regulation of behavior by representational memory. In F. Plum (Ed.), Handbook of Physiology: The Nervous System (pp. 373-417). Bethesda, MD: American Physiological Society.
16. Hebb, D. O. (1949). The organization of behavior; A neuropsychological theory. New York: Wiley.
17. Jaakola, T., Singh, S. P., & Jordan, M. I. (1995). Reinforcement learning algorithm for partially observable Markov decision problems. In G. Tesauro, D. Touretzky, & T. Leen (Eds.), Advances in neural information processing systems, Vol. 7 (pp. 345-352). Cambridge, MA: MIT Press.
18. Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237-285.
19. Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101, 99-134.
20. Lew, S. E., Wedemeyer, C., & Zanutto, B. S. (2001). Role of unconditioned stimulus prediction in the operant learning: A neural network model. In Proceedings of IEEE Conference on Neural Networks (pp. 331-336).
21. Mahadevan, S., & Connell, J. (1991). Automatic programming of behavior-based robots using reinforcement learning. In Proceedings of the Ninth National Conference on Artificial Intelligence, Anaheim, CA.
22. McFarland, D., & Bösser, T. (1993). Intelligent behavior in animals and robots. Cambridge, MA: Bradford Books, MIT Press.
23. Rescorla, R. A., & Wagner, A. R. (1972), A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Current research and theory. New York: Appleton-Century-Crofts.
24. Schmajuk, N., & Zanutto, B. S. (1997). Escape, avoidance and imitation: A neural network approach. Adaptive Behavior, 6, 63-129.
25. Schultz, W., Dayan P., & Montague, R. (1997). A neural substrate of prediction and reward. Science, 275, 1593-1598.
26. Singh, S. P, Jaakola, T., & Jordan, M. I. (1994). Learning without state-estimation in partially observable Markovian decision processes. In W. W. Cohen & H. Hirsh (Eds.), Proceedings of the Eleventh International Conference on Machine Learning (pp. 284-292). San Francisco, CA: Morgan Kaufmann.
27. Sutton, R. S. (1991). Reinforcement learning architectures for animats, In J.-A. Meyer & S. W. Wilson (Eds.), From animals to animals: Proceedings of the First International Conference on Simulation of Adaptative Behavior (pp. 288-296). Cambridge, MA: MIT Press.
28. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press.
29. Staddon, J. E. R. (1983). Adaptive behavior and learning. Cambridge, UK: Cambridge University Press.
30. Steels, L. (1991). Towards a theory of emergent functionality. In J. A. Meyer & S. W. Wilson (Eds.), From animals to animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior (pp. 451-461). Cambridge, MA: Bradford.
31. Touretzky, D. S., & Saksida, M. L. (1997). Operant conditioning in skinnerbots. Adaptive Behavior, 5(3/4), 219-247.
32. Verschure, P. F., & Voegtlin, T. (1998). A bottom up approach towards the acquisition and expression of sequential representations applied to a behaving real-world device: Distributed adaptive control III. Neural Networks, 11, 1531-1549.
33. Waelti, P., Dickinson, A., & Schultz, W. (2001). Dopamine responses comply with basic assumptions of formal learning theory. Nature, 412, 43-48.
34. Watkins, C. J. C. H. (1989). Learning with delayed rewards. Ph.D. dissertation, Psychology Department, Cambridge University.
35. Zanutto, B. S., & Lew, S. (2000). A neural network model of aversive behavior. In M. H. Hamza (Ed.), Proceedings of the LASTED Neural Networks NN'2000 (pp. 118-123). Zürich; IASTED/ACTA Press.