Operant conditioning in skinnerbots

Adaptive Behavior

Volumn 5, Issue 3-4, 1997, Pages 219-247

  Touretzky, David S ^a   Saksida, Lisa M ^b  

^a Carnegie Mellon University   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Chaining; Instrumental learning; Learning robots; Operant conditioning; Shaping

Indexed keywords

ANIMALIA;

EID: 0031498145     PISSN: 10597123     EISSN: None     Source Type: Journal    
DOI: 10.1177/105971239700500302     Document Type: Article

References (51)

1. Asada, M., Uchibe, E., Noda, S., Tawaratsumida, S., & Hosoda, K. (1994). Coordination of multiple behaviors acquired by a vision-based reinforcement learning. In Proceedings of the International Conference on Intelligent Robots and Systems, Munich, Germany. Los Alamitos, CA: IEEE Press.
2. Barnett, S. (1981). Modern ethology. London: Oxford University Press.
3. Barto, A. G., & Sutton, R. S. (1990). Time-derivative models of Pavlovian conditioning. In M. Gabriel & J. Moore (Eds.), Learning and computational neuroscience: Foundations of adaptive networks. Cambridge, MA: MIT Press.
4. Baxter, D. A., Buonomano, D. V., Raymond, J. L., Cook, D. G., Kuenzi, F. M., Carew, T. J., & Byrne, J. H. (1991). Empirically derived adaptive elements and networks simulate associative learning. In M. L. Commons, S. Grossberg & J. E. R. Staddon (Eds.), Neural Network Models of Conditioning and Action. Hillsdale, NJ: Lawrence Erlbaum Associates.
5. Berridge, K., Fentress, J. C., & Parr, H. (1987). Natural syntax rules control action sequence of rats. Behavioural Brain Research, 23, 59-68.
6. Blough, D. (1959). Delayed matching in the pigeon. Journal of the Experimental Analysis of Behavior, 2, 151-160.
7. Breland, K., & Breland, M. (1961). The misbehavior of organisms. American Psychologist, 16, 681-684.
8. Brown, P., & Jenkins, H. (1968). Auto-shaping of the pigeon's keypeck. Journal of the Experimental Analysis of Behavior, 11, 1-8.
9. Bussey, T. J., Muir, J. L., & Robbins, T. W. (1994). A novel automated touch-screen procedure for assessing learning in the rat using computer graphic stimuli. Neuroscience Research Communications, 15(2), 103-109.
10. Catania, A. C., & Harnad, S. (Eds.). (1988). The selection of behavior. London: Cambridge University Press.
11. Canine Companions for Independence (1995). The CCI program. Canine Companions for Independence, Santa Rosa, CA. Informational page available at http://grunt.berkeley.edu/cci/cci.html.
12. Chomsky, N. (1959). Review of Skinner's verbal behavior. Language, 35, 26-58.
13. Colombetti, M., Dorigo, M., & Borghi, G. (1996). Behavior analysis and training: A methodology for behavior engineering. IEEE Transactions on Systems, Man, and Cybernetics [B], 26(3), 365-380.
14. Dickinson, A. (1995). Instrumental conditioning. In N. J. Mackintosh (Ed.), Handbook of perception and cognition (Vol. 9.) Orlando, FL: Academic Press.
15. Dorigo, M., & Columbetti, M. (1994). Robot shaping: Developing autonomous agents through learning. Artificial Intelligence, 70(2), 321-370.
16. Drescher, G. L. (1991). Made-up minds. Cambridge, MA: MIT Press.
17. Gallistel, C. R. (1990). The organization of learning. Cambridge, MA: MIT Press.
18. Gollub, L. (1977). Conditioned reinforcement: Schedule effects. In W. Honig & J. Staddon (Eds.), Handbook of operant behavior. Mt. Kisco, NY: Prentice Hall.
19. Graham, J., Alloway, T., and Krames, L. (1994). Sniffy, the virtual rat: Simulated operant conditioning. Behavior Research Methods, Instruments, & Computers, 26(2), 134-141.
20. Grossberg, S. (1972). A neural theory of punishment and avoidance. II. Quantitative theory. Mathematical Biosciences, 15, 253-285.
21. Gutnikov, S., Barnes, J., & Rawlins, J. (1994). Working memory tasks in five-choice operant chambers: Use of relative and absolute spatial memories. Behavioral Neuroscience, 108(5), 899-910.
22. Hammer, M. (1993). An identified neuron mediates the unconditioned stimulus in associative olfactory learning in honeybees. Nature, 366, 59-63.
23. Hampson, R. E., Heyser, C. J., & Deadwyler, S. A. (1993). Hippocampal cell firing correlates of delayed-match-to-sample performance in the rat. Behavioral Neuroscience, 107(5), 715-739.
24. Holland, P. C. (1993). Cognitive aspects of classical conditioning. Current Opinion in Neurobiology, 3, 230-236.
25. Klopf, A. H. (1988). A neuronal model of classical conditioning. Psychobiology, 16, 85-125.
26. Krames, L., Graham, J., & Alloway, T. (1995). Sniffy, the virtual rat. Pacific Grove, CA: Brooks/Cole. [Includes software diskette.]
27. Lashley, K. (1951). The problem of serial order in behavior. In L. Jeffries (Ed.), Cerebral mechanisms in behavior. New York: Wiley.
28. Mahadevan, S., & Connell, J. (1992). Automatic programming of behavior-based robots using reinforcement learning. Artificial Intelligence, 55, 311-365.
29. Maki, W. S., & Abunawass, A. M. (1991). A connectionist approach to conditional discriminations: Learning, short-term memory, and attention. In M. L. Commons, S. Grossberg & J. E. R. Staddon (Eds.), Neural Network Models of Conditioning and Action. Hillsdale, NJ: Lawrence Erlbaum Associates.
30. Matarić, M. J. (1995). Designing and understanding adaptive group behavior. Adaptive Behavior, 4(1), 50-81.
31. Matarić, M. J. (1996). Reinforcement learning in multi-robot domains. Autonomous Robots, 4(1), 73-83.
32. Millán, J. d. R. (1994). Learning efficient reactive behavioral sequences from basic reflexes in a goal-directed autonomous robot. In D. Cliff, P. Husbands, J.-A. Meyer & S. W. Wilson (Eds.), From animals to animals 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior. Cambridge, MA: MIT Press.
33. Millán, J. d. R. (1996). Rapid, safe, and incremental learning of navigation strategies. IEEE Transactions on Systems, Man, and Cybernetics [B], 26(3), 408-420.
34. Montague, P. R., Dayan, P., Person, C., & Sejnowski, T. J. (1995). Bee foraging in uncertain environments using predictive Hebbian learning. Nature, 377, 725-728.
35. Pavlov, I. P. (1927). Conditioned reflexes. London: Oxford University Press.
36. Pryor, K. (1975). Lads before the wind. New York: Harper & Row.
37. Raymond, J. L., Baxter, D. A., Buonomano, D. V. & Byrne, J. H. (1992). A learning rule based on empirically derived activity-dependent neuromodulation supports operant conditioning in a small network. Neural Networks, 5(5), 789-803.
38. Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical conditioning II: Theory and research. New York: Appleton-Century-Crofts.
39. Reynolds, G. S. (1968). A primer of operant conditioning. Glenview, IL: Scott, Foresman.
40. Schmajuk, N. A., & Urry, D. W. (1995). The frightening complexity of avoidance: An adaptive neural network. In C. Wynne & J. Staddon (Eds.), Models of action: Mechanisms for adaptive behavior. Hillsdale, NJ: Lawrence Erlbaum Associates.
41. Schultz, W., Romo, R., Ljungberg, T., Mirenowicz, J., Hollerman, J. R., & Dickinson, A. (1995). Reward-related signals carried by dopamine neurons. In J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia. Cambridge, MA: MIT Press.
42. Schwartz, B. (1989). Psychology of learning and behavior. New York: Norton.
43. Simmons, R. (1994). Structured control for autonomous robots. IEEE Transactions on Robotics and Automation, 10(1), 34-43.
44. Singh, S. (1992). Transfer of learning across sequential tasks. Machine Learning, 8, 323-339.
45. Skinner, B. (1938). Behavior of organisms. New York: Appleton-Century-Crofts.
46. Skinner, B. (1948). "Superstition" in the pigeon. Journal of Experimental Psychology, 38, 168-172.
47. Staddon, J., & Simmelhag, V. (1971). The "superstition" experiment: A reexamination of its implications for the principle of adaptive behavior. Psychological Review, 78, 3-43.
48. Sutton, R. S., & Barto, A. G. (1981). Toward a modern theory of adaptive networks: Expectation and prediction. Psychological Review, 88, 135-170.
49. Touretzky, D. S., & Saksida, L. (1996). Skinnerbots. In P. Maes, M. Matarić, J.-A. Meyer, J. Pollack, & S. W. Wilson (Eds.), From animals to animals 4: Proceedings of the Fourth International Conference on Simulation of Adaptive Behavior. Cambridge, MA: MIT Press.
50. Verschure, P. F. M. J., Wray, J., Sporns, O., Tononi, G., & Edelman, G. M. (1995). Multilevel analysis of classical conditioning in a real world artifact. Robotics and Autonomous Systems, 16(2-4), 247-265.
51. Watkins, C. J. C. H. (1989). Learning from delayed rewards. Unpublished doctoral thesis, Cambridge University, Cambridge, England.