Discovery of high-level behavior from observation of human performance in a strategic game

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

Volumn 38, Issue 3, 2008, Pages 855-874

  Stensrud, Brian S ^a   Gonzalez, Avelino J ^b  

^a SOAR TECHNOLOGY INC   (United States)

^b University of Central Florida   (United States)

Author keywords

Context Based Reasoning (CxBR); Fuzzy ARTMAP (FAM); Learning from observation; Neural network; Poker; Template based interpretation (TBI)

Indexed keywords

CYBERNETICS; GAME THEORY;

CONTEXT-BASED REASONING; HUMAN BEHAVIOR; LEARNING TASK; STRATEGIC GAME SIMULATION; TEMPLATE-BASED INTERPRETATION;

FUZZY NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; DECISION MAKING; GAME; HUMAN; METHODOLOGY; TASK PERFORMANCE;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; DECISION MAKING; GAME THEORY; HUMANS; PATTERN RECOGNITION, AUTOMATED; TASK PERFORMANCE AND ANALYSIS;

EID: 44849115010     PISSN: 10834419     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2008.922062     Document Type: Article

References (70)

1. Webster's New World Dictionary of the American Language, 1960, Cleveland, OH: World.
2. A. J. Gonzalez and R. Ahlers, "Context-bassed representation of intelligent behavior in training simulations," Trans. Soc. Comput. Simul. Int., vol. 15, no. 4, pp. 151-166, Mar. 1998.
3. A. J. Gonzalez, J. Castro, and W. J. Gerber, "Automating the acquisition of tactical knowledge for military missions," J. Defense Model. Simul., vol. 3, no. 1, pp. 145-160, Spring 2006.
4. T. A. Sidani and A. J. Gonzalez, "A framework for learning implicit expert knowledge through observation," Trans. Soc. Comput. Simul. Int., vol. 17, pp. 54-72, 2000.
5. H. Fernlund, A. J. Gonzalez, M. Georgiopoulos, and R. F. DeMara, "Learning tactical human behavior through observation of human performance," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 36, no. 1, pp. 128-140, Feb. 2006.
6. R. M. Jones, J. E. Laird, P. E. Nielsen, K. J. Coulter, P. G. Kenny, and F. Koss, "Automated intelligent pilots for combat flight simulation," AI Mag., vol. 20, no. 1, pp. 27-41, 1999.
7. R. M. Jones and J. E. Laird, "Constraints on the design of a high-level model of cognition," Univ. Michigan, Ann Arbor, MI, 1996. Res. Rep.
8. D. J. Patterson, L. Liao, D. Fox, and H. Kautz, "Inferring high-level behavior froili low-level sensors," Univ. Washington, Seattle, WA, 2003. Res. Rep.
9. R. Khardon, "Learning to take actions," Dept. Comput. Sci., Univ. Edinburgh, Edinburgh, U.K., Jun. 12, 1998. Tech. Rep.
10. M. van Lent and J. E. Laird, "Learning hierarchical performance knowledge by observation,", in Proc. Int. Conf. Mach. Learn., 1999, pp. 229-238.
11. C. E. Shannon, "Programming a computer for playing chess," Philos. Mag., vol. 41, no. 314, pp. 256-275, 1950.
12. A. M. Turing, "Digital computers applied to games," in Faster Than Thought: A Symposium on Digital Computing Machines, B. V. Bowden, Ed. London, U.K.: Pitman, 1953, ch. 25, pp. 286-310.
13. A. Newell, J. C. Shaw, and H. A. Simon, "Chess-playing programs and the problem of complexity," IBM J. Res. Develop., vol. 2, no. 4, pp. 320-335, 1958.
14. N. Charness, "Components of skill in bridge," Can. J. Psychol., vol. 33, no. 1, pp. 1-16, Mar. 1979.
15. N. Charness, "Component processes in bridge bidding and novel problem-solving tasks," Can. J. Psychol., vol. 41, no. 2, pp. 223-243, Jun. 1987.
16. N. Charness, "The impact of chess research on cognitive science," Psychol. Res., vol. 54, no. 1, pp. 4-9, Mar. 1992.
17. T. Jastrzembski, N. Charness, and C. Vasyukova, "Expertise and age effects oil knowledge activation in chess," Psychol. Aging, vol. 21, no. 2, pp. 401-405, Jun. 2006.
18. F.-H. Hsu, Behind Deep Blue: Building the Computer that Defeated the World Chess Champion. Princeton, NJ: Princeton Univ. Press, 2002.
19. M. Newborn and D. Kopec, "The twentieth annual ACM North America computer chess championship," Commun. ACM, vol. 33, no. 7, pp. 92-105, Jul. 1990.
20. D. Coursey, "Deep Thought is a chess-playing computer," MIS Week vol. 11, no. 14, p. 33, Apr. 2, 1990.
21. A. L. Zobrist, "Feature extraction and representation for pattern recognition and the game of GO," Ph.D. dissertation, Univ. Wisconsin, Madison, WI, 1970.
22. J. L. Ryder, "Heuristic analysis of large trees as generated in the game of Go," Ph.D. dissertation, Stanford Univ., Stanford, CA, 1971.
23. A. Kierulf and J. Nievergelt, "Computer GO: A smart board and its applications," in Go Worlds, vol. 42. Tokyo, Japan: Ishi Press, Winter 1986.
24. A. Kierulf, K. Chen, and J. Nievergelt, "Smart game board and go explorer," Commun. ACM, vol. 33, no. 2, pp. 152-167, Feb. 1990.
25. B. Wilcox, "Reflections on building two GO programs," SIGART Newsl. vol. 94, pp. 29-43, Oct. 1985.
26. A. L. Samuel, "Some studies in machine learning using the game of checkers," in Computers and Thought, E. A. Feigenbaum and J. Feldman, Eds. New York: McGraw-Hill, pp. 71-105.
27. D. A. Waterman, "Generalization learning techniques for automating the learning of heuristics," Artif. Intell., vol. 1, no. 1/2, pp. 121-170, 1970.
28. R. S. Michalski, J. G. Carbonel, and T. M. Mitchell, Machine Learning: An Artificial Intelligence Approach. Los Altos, CA: Morgan Kaufmann, 1986.
29. M. X. Zhou and S. Ma, "Toward applying machine learning to design rule acquisition for automated graphics generation," IBM T. J. Watson Res. Center, Hawthorne, NY, 2000. Res. Rep.
30. C. Sammut, S. Hurst, D. Kedzier, and D. Michie, "Learning to fly," in Proc. 9th Int. Conf. Mach. Learn., Aberdeen, U.K., 1992, pp. 335-339.
31. R. Camacho, "Inducing models of human control skills," in Proc. 10th ECML, Chemnitz, Germany, 1998, pp. 107-118.
32. A. Isaac and C. Sammut, "Goal-directed learning to fly," in Proc. 20th Int. Conf. Mach. Learn., Washington, DC, 2003, pp. 258-265.
33. A. E. Henninger, "Neural network based movement models to improve the predictive utility of entity state synchronization methods for distributed simulations," Ph.D. dissertation, School Electr. Eng. Comput. Sci., Univ. Central Florida, Orlando, FL, 2001.
34. W. J. Gerber, "Real-time synchronization of behavioral models with human performance in a simulation,", Ph.D. dissertation, School Electr. Eng. Comput. Sci., Univ. Central Florida, Orlando, FL, 2001.
35. M. J. Johnson, M. McGinnis, M. Mollaghasemi, and T. Damarly, "Methodology for human decision making using fuzzy ARTMAP neural networks," in Proc. Int. Joint Conf. Neural Netw., Honolulu, HI, 2002, pp. 2668-2673.
36. G. A. Carpenter, S. Grossberg, N. Markuzon, J. H. Reynolds, and D. B. Rosen, "Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps," IEEE Trans. Neural Netw., vol. 3, no. 5, pp. 698-713, Sep. 1992.
37. M. van Lent and J. E. Laird, "Learning procedural knowledge through observation," in Proc. 1st Int. Conf. K-CAP, Victoria, BC, Canada, 2001, pp. 179-186.
38. T. Konik and J. E. Laird, "Learning goal hierarchies from structured observations and expert annotations," in Proc. 14th Int. Conf. Inductive Logic Program., 2004, pp. 198-215.
39. S. Schaal, "Is imitation learning the route to humanoid robots?," Trends Cogn. Sci., vol. 3, no. 6, pp. 233-242, Jun. 1999.
40. S. Walczak and P. Fishwick, "A quantitative analysis of pattern production and its relationship to expert performance," J. Exp. Theor. Artif. Intell., vol. 9, no. 1, pp. 883-101, Jan. 1997.
41. J. E. Laird, P. Rosenbloom, and A. Newell, Universal Subgoaling and Chunking. Norwell, MA: Kluwer, 1986.
42. D. Pomerlau, C. Thorpe, D. Longer, J. K. Rosenblatt, and R. Sukthankar, "AVCS research at Carnegie Mellon University," in Proc. Intell. Vehicle Highway Syst., Amer. Amin. Meeting, 1994, pp. 257-262.
43. D. Bentivegna and C. Atkeson, "Learning from observation using primitives," in Proc. IEEE Int. Conf. Robot. Autom., Seoul, Korea, May 21-26, 2001, pp. 1988-1993.
44. A. Moukas and G. Hayes, "Synthetic robotic language acquisition by observation," in Proc. 4th Int. Conf. Simul. Adapt. Behav., 1996, pp. 568-579.
45. B. Yang and H. Asada, "Hybrid linguistic/numeric control of deburring robots based on human skills," in Proc. IEEE Int. Conf. Robot. Autom., 1992, vol. 2, pp. 1467-1474.
46. D. Floreano and F. Mondada, "Automatic creation of an autonomous agent: Genetic evolution of a neural-network driver robot," in Proc. Conf. Simul. Adapt. Behav., 1994, pp. 421-430.
47. A. Pentland and A. Liu, "Toward augmented control systems," in Proc. Intell. Vehicles, 1995, vol. 1, pp. 350-355.
48. L. J. Fogel, W Porto, and M. Owen, "An intelligently interactive non-rule-based computer generated force," in Proc. 6th Conf. Comput. Generated Forces Behavior Representation, Orlando, FL, Jul. 23-25, 1996, pp. 265-271.
49. J. D. Morrison, "Real time learning of doctrine and tactics using neural networks and combat simulations," Mil. Oper. Res., vol. 2, no. 3, pp. 45-60, 1996.
50. M. Crowe, "The application of artificial neural systems to the training of air combat decision-making skills," in Proc. 12th Inter-Service/Industry Training Syst. Conf., Orlando, FL, 1990, pp. 302-312.
51. H. Friedrich, M. Kaiser, and R. Dillman, "What can robots learn from humans?," Annu. Rev. Control, vol. 20, pp. 167-172, 1996.
52. M. Kaiser and R. Dillman, "Building element tary robot skills from human demonstration," in Proc. IEEE Int. Conf. Robot. Autom., Minneapolis, MN, 1996, pp. 2700-2705.
53. S. Rajput, C. R. Karr, J. E. Cisneros, and R. J. Parsons, "Learning the selection of reactive behaviors," in Proc. 6th Conf. Comput. Generated Forces Behavioral Representation, Orlando, FL, Jul. 23-25, 1996, pp. 3-8.
54. M. R. Hieb, G. Tecuci, and M. Pullen, "Training ModSAF command agent through demonstration," in Proc. 6th Conf. Comput. Generated Forces Behavioral Representation, Orlando, FL, Jun. 1996.
55. C. G. Gingrich, D. R. Kuespert, and T. J. McAvoy, "Modeling human operators using neural networks," ISA Trans., vol 31, no. 3, pp. 81-90, 1992.
56. G. E. Hovland, P. Sikka, and B. J. McCarragher, "Skill acquisition from human demonstration using a hidden Markov model," in Proc. IEEE Int. Conf. Robot. Autom., 1996, pp. 2706-2711.
57. K. Kosuge, T. Fukuda, and H. Asada, "Acquisition of human skills for robotic systems," in Proc. IEEE Intell. Symp. Intell. Control, 1991, pp. 469-474.
58. S. Lee and J. Chen, "Skill learning from observations," in Proc. IEEE Conf. Robot. Autom., 1994, vol. 4, pp. 3245-3250.
59. S. Lee and H. Chen, "Robot skill discovery based on observed data," in Proc. IEEE Int. Conf. Robot. Autom., Minneapolis, MN, 1996, pp. 2694-2699.
60. A. Modjitahedzadeh and R. A. Hess, "A model of driver steering control behavior for use in assessing vehicle handling qualities," Trans. ASME, J. Dyn. Syst. Meas. Control, vol. 115, no. 3, pp. 456-464, 1993.
61. E. Fix and H. G. Armstrong, "Neural network based human performance modeling," in Proc. IEEE Nat. Aerosp. Electron. Conf., 1990, vol. 3, pp. 1162-1165.
62. M. C. Nechyba and Y. Xu, "Learning and transfer of human real-time control strategies," J. Adv. Comput. Intell., vol. 1, no. 2, pp. 137-154, 1997.
63. M. C. Nechyba and Y. Xu, "Cascade neural networks with node-decoupled extended Kalman filtering," in Proc. IEEE Int. Symp. Comput. Intell. Robot. Autom., 1997, vol. 1, pp. 214-219.
64. P. Drewes, "Automated student performance monitoring in training simulation," Ph.D., dissertation. Dept. Electr. Comput. Eng., Univ. Central Florida, Orlando, FL, 1997.
65. M. Georgiopoulos and C. Christodoulou, Applications of Neural Networks in Electromagnetics. Norwood, MA: Artech House, 2001.
66. B. S. Stensrud, "FAMTILE: An algorithm for learning high-level, tactical behavior from observation," Ph.D. dissertation, School Electr. Eng. Comput. Sci., Univ. Central Florida, Orlando, FL, 2005.
67. D. Brunson, Doyle Brunson's Super System, 3rd ed. New York: Cardoza, 1979.
68. D. Sklansky, The Theory of Poker, 3rd ed. Henderson, NV: Two Plus Two Publications, 1989.
69. D. Sklansky and M. Malmouth, Hold'em Poker for Advanced Players. Las Vegas, NV: Creel, 2003. 21st Century Edition.
70. D. Billings, D. Papp, J. Schaeffer and D. Szafron, "Poker as a testbed for AI research," in Proc. 12th Can. Conf. Artif. Intell., 1998, pp. 228-238.