Anticipations control behavior: Animal behavior in an anticipatory learning classifier system

Adaptive Behavior

Volumn 10, Issue 2, 2002, Pages 75-96

  Butz, Martin V ^a   Hoffmann, Joachim ^a  

^a UNIVERSITY OF WÜRZBURG   (Germany)

Author keywords

ACS2; Anticipations; Anticipatory behavioral control; Anticipatory learning systems; Predictive environmental model; Reinforcement learning

Indexed keywords

ANIMALIA;

EID: 0038477059     PISSN: 10597123     EISSN: None     Source Type: Journal    
DOI: 10.1177/1059712302010002001     Document Type: Article

References (65)

1. Adams, C., & Dickinson, A. (1981). Instrumental responding following reinforcer devaluation. Quarterly Journal of Experimental Psychology B, 33, 109-121.
2. Butz, M. V. (2002). Anticipatory learning classifier systems. Boston, MA: Kluwer Academic.
3. Butz, M. V., Goldberg, D. E., & Stolzmann, W. (2000). Introducing a genetic generalization pressure to the anticipatory classifier system: Part 2 - Performance analysis. In D. Whitely, D. E., Goldberg. E. Cantu-Paz, L. Spector, I. Parmee, & H.-G. Beyer (Eds.), Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2000) (pp. 42-49). San Francisco: Morgan Kaufmann.
4. Colwill, R. M., & Rescorla, R. A. (1985). Postconditioning devaluation of a reinforcer affects instrumental learning. Journal of Experimental Psychology: Animal Behavior Processes, 11(1), 120-132.
5. Colwill, R. M., & Rescorla, R. A. (1990). Evidence for the hierarchical structure of instrumental learning. Animal Learning & Behavior, 18(1), 71-82.
6. Dickinson, A. (1994). Instrumental conditioning. In N. Mackintosh (Ed.), Animal learning and cognition (pp. 45-79). San Diego, CA: Academic Press.
7. Donnart, J.-Y., & Meyer, J.-A. (1994). A hierarchical classifier system implementing a motivationally autonomous animal. In D. Cliff, P. Husbands, J.-A. Meyer, & S. W. Wilson (Eds.), From animals to animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior (pp. 144 153). Cambridge, MA: MIT Press.
8. Dorigo, M., & Colombetti, M. (1997). Robot shaping: An experiment in behavior engineering. Cambridge, MA: MIT Press.
9. Drescher, G. L. (1991). Made-up minds: A constructivist approach to artificial intelligence. Cambridge, MA: MIT Press.
10. Elsner, B., & Hommel, B. (2001). Effect anticipation and action control. Journal of Experimental Psychology: Human Perception and Performance, 27, 229-240.
11. Gérard, P., & Sigaud, O. (2001). Adding a generalization mechanism to YACS. In L. Spector, E. D. Goodman, A. Wu, W. B. Longdon, H.-M. Voigt, M. Gen, S. Sen, M. Dorigo, S. Pezeshk, M. H. Garzon, & E. Burke (Eds.), Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2001) (pp. 951 -957). San Francisco: Morgan Kaufmann.
12. Gérard, P., & Sigaud, O. (2001b). YACS: Combining dynamic programming with generalization in classifier systems. In P. L. Lanzi, W. Stolzmann, & S. W. Wilson (Eds.), Advances in learning classifier systems: Third International Workshop, IWLCS 2000 (pp. 52 69). Berlin: Springer.
13. Greenwald, A. (1970). Sensory feedback mechanisms in performance control: With special reference to the ideo-motor mechanism. Psychological Review, 77, 73-99.
14. Harle, E. (1861). Der Apparat des Willens. Zeitschrift für Philosophie und philosophische Kritik, 38, 50-73.
15. Hoffmann, J. (1993). Vorhersage und Erkenntnis: Die Funktion von Antizipationen in der menschlichen Verhaltenssteue-rung und Wahrnehmung [Anticipation and cognition: The function of anticipations in human behavioral control and perception]. Goettingen, Germany: Hogrefe.
16. Hoffmann, J., Sebald, A., & Stöcker, C. (2001). Irrelevant response effects improve serial learning in serial reaction time tasks. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27, 470-482.
17. Holland, J. H. (1976). Adaptation. In R. Rosen & F. Snell (Eds.), Progress in theoretical biologv (Vol. 4, pp. 263-293). New York: Academic Press.
18. Holland, J. H. (1990). Concerning the emergence of tag-mediated lookahead in classifier systems. In S. Forrest (Ed.), Emergent computation. Proceedings of the Ninth Annual International Conference of the Center for Nonlinear Studies on Self-organizing, Collective, and Cooperative Phenomena in Natural and Artificial Computing Networks. A special issue of Physica D (Vol. 42. pp. 188-201). New York: North-Holland.
19. Hommel, B. (1996). The cognitive representation of action: Automatic integration of perceived action effects. Psychological Research. 59, 176-186.
20. Hommel, B. (1998). Perceiving one's own action - and what it leads to. In J. S. Jordan (Ed.), Systems theory and a priori aspects of perception (pp. 143-179). Amsterdam: North Holland.
21. James, W. (1981). The principles of psychology (Vol. 2). Cambridge, MA: Harvard University Press. Original work published 1890.
22. Kaelbling, L. P. (1993). Learning in embedded systems. Cambridge, MA: MIT Press.
23. Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237-258.
24. Kunde, W. (2001). Response-effect compatibility in manual choice reaction tasks. Journal of Experimental Psychology: Human Perception and Performance, 27, 387-394.
25. Kuvayev, L., & Sulton, R. S. (1996). Model-based reinforcement learning with an approximate, learned model. InProceedings of the Ninth Yale Workshop on Adaptive and Learning Systems (pp. 101-105). New Haven, CT: Yale University Press.
26. Lanzi, P. L., Stolzmann, W., & Wilson, S. W. (Eds.), (2000). Learning classifier systems: From foundations to applications, Berlin: Springer.
27. Lotze, H. (1852). Medizinische Psychologie oder Physiologie der Seele. Leipzig: Weidmann'sche Buchhandlung.
28. Moore, A. W., & Atkeson, C. (1993). Prioritizes sweeping: Reinforcement learning with less data and less real time. Machine Learning, 13, 103-130.
29. Münsterberg, H. (1889). Beiträge zur Experimentalpsychologie, Vol. 1. Greiburg: J.C.B. Mohr.
30. Pashler, H., Johnston, J. C., & Ruthruff, E. (2001). Attention and performance. Annual Review of Psychology, 52, 629-651.
31. Pearce, J. M. (1997). Animal learning and cognition (2nd ed.). Hove: Psychology Press.
32. Prinz, W. (1990). A common coding approach to perception and action. In O. Neumann & W. Prinz (Eds.), Relationships between perception and action (pp. 167-201). Berlin: Springer.
33. Prinz, W. (1997). Perception and action planning. European Journal of Cognitive Psychology, 9, 129-154.
34. Rescorla, R. A. (1990). Evidence for an association between the discriminative stimulus and the response-outcome association in instrumental learning. Journal of Experimental Psychology: Adaptive Behavior Processes, 16(4), 326-334.
35. Rescorla, R. A. (1991). Associative relations in instrumental learning: The eighteenth Bartlett memorial lecture. Quarterly Journal of Experimental Psychology B, 43, 1-23.
36. Rescorla, R. A. (1995). Full preservation of a response-out-come association through training with a second outcome. Quarterly Journal of Experimental Psychology B, 48, 252-261.
37. Riolo, R. L. (1991). Lookahead planning and latent learning in a classifier system. In J.-A. Meyer & S.W. Wilson (Eds.), From animals to animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior (pp. 316-326). Cambridge, MA: MIT Press.
38. Roitblat, H. L. (1994). Mechanism and process in animal behavior: Models of animals, animals as models. In D. Cliff, P. Husbands, J.-A. Meyer, & S. W. Wilson (Eds.), From animals to animats 3: Proceedings of the Third International Conference on Simulation of Adaptive Behavior (pp. 12-21 ). Cambridge, MA: MIT Press.
39. Rosen, R. (1985). Anticipatory systems. Oxford: Pergamon Press.
40. Rosen, R. (1991). Life itself. New York: Columbia University Press.
41. Schubotz, R. I., & Cramon, D. Y. von (2001). Functional organization of the lateral premotor cortex. fMRI reveals different regions activated by anticipation of object properties, location and speed. Cognitive Brain Research, 11, 97-112.
42. Stickgold, R. (1998). Sleep: Off-line memory reprocessing. Trends in Cognitive Sciences, 2(12), 484-492.
43. Stock, A., & Hoffmann, J. (2002). Intentional fixation of behavioral learning or how R-E learning blocks S-R learning. European Journal of Cognitive Psychology. 14(1), 127-153.
44. Stolzmann, W. (1997). Antizipative Classifier Systems [Anticipatory classifier systems]. Aachen, Germany: Shaker.
45. Stolzmann, W. (2000). An introduction to anticipatory classifier systems. In P. L. Lanzi, W. Stolzmann, & S. W. Wilson (Eds.), Learning classifier systems: From foundations to applications (pp. 175-194). Berlin: Springer.
46. Stolzmann, W., Butz, M. V., Hoffmann, J., & Goldberg, D. E. (2000). First cognitive capabilities in the anticipatory classifier system. In J.-A. Meyer, A. Berthoz, D. Floreano, H. Roitblat, & S. W. Wilson (Eds.), From animals to animats 6: Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior (pp. 287-296). Cambridge, MA: MIT Press.
47. Sutton, R., Precup, D., & Singh, S. (1999). Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning. Artificial Intelligence, 112, 181-211.
48. Sutton, R. S. (1991a). Dyna, an integrated architecture for learning, planning, and reacting. In Working Notes of the 1991 AAAI Spring Symposium on Integrated Intelligent Architectures (pp. 151-155). Palo Alto, CA: Stanford University.
49. Sutton, R. S. (1991b). Reinforcement learning architectures for animals. In J.-A. Meyer & S. W. Wilson (Eds.), From animals to animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior (pp. 288-296). Cambridge, MA: MIT Press.
50. Sutton, R. S., & Barto, A. G. (1998). Reinforcement learning: An introduction, Cambridge, MA: MIT Press.
51. Tani, J. (1996). Model-based learning for mobile robot navigation from the dynamical systems perspective. IEEE Transactions System, Man and Cybernetics B [Special Issue on Learning Autonomous Systems], 26(3), 421-436.
52. Thislethwaite, D. (1951). A critical review of latent learning and related experiments. Psychological Bulletin, 48(2), 97-129.
53. Thorndike, E. L. (1911). Animal intelligence: Experimental studies. New York: Macmillan.
54. Tolman, E. C. (1932). Purposive behavior in animals and men. New York: Appleton.
55. Tolman, E. C. (1949). There is more than one kind of learning. Psychological Review, 5b, 144-155.
56. Tolman, E. C., & Honzik, C. (1930). Introduction and removal of reward, and maze performance in rats. University of California, Publications in Psychology, 4, 257-275.
57. Watkins, C. J. C. H. (1989). Learning from delayed rewards. Doctoral dissertation, King's College, Cambridge, UK.
58. Whitehead, S. D., & Ballard, D. H. (1991). Learning to perceive and act. Machine Learning, 7(1), 45-83.
59. Wilson, S. W. (1991). The animat path to Al. In J.-A. Meyer & S. W. Wilson (Eds.), From animals to animats: Proceedings of the First International Conference on Simulation of Adaptive Behavior (pp. 15-21 ). Cambridge, MA: MIT Press.
60. Wilson, S. W. (1995). Classifier fitness based on accuracy, Evolutionary Computation, 3(2), 149-175.
61. Wilson, S. W. (1998). Generalization in the XCS classifier system. In J. R. Koza, W. Banzhaf, K. Chellapilla, K. Deb, M. Dorigo, D. B. Fogel, M. H. Grazon, D. E. Goldberg, H. Iba, & R. L. Riolo (Eds.), Genetic programming 1998: Proceedings of the Third Annual Conference (pp. 665-674). San Francisco: Morgan Kaufmann.
62. Witkowski, C. M. (1997). Schemes for learning and behaviour: A new expectancy model. Doctoral dissertation, Department of Computer Science, Queen Mary Westfield College, University of London.
63. Witkowski, C. M. (2000). The role of behavioral extinction in animat action selection. In J.-A. Meyer, A. Berthoz, D. Floreano, H. Roitblat, & S. W. Wilson (Eds.), From animals to animats 6: Proceedings of the Sixth International Conference on Simulation of Adaptive Behavior (pp. 177-186), Cambridge, MA: MIT Press.
64. Ziessler, M. (1998). Response-effect learning as a major component of implicit serial learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 962-978.
65. Ziessler, M., & Nattkemper, D. (2001). Learning of event sequences is based on response-effect learning: Further evidence from serial reaction task. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27, 595-613.