An Adaptive Visual Attentive Tracker for human communicational behaviors using HMM-based TD learning with new state distinction capability

IEEE Transactions on Robotics

Volumn 21, Issue 3, 2005, Pages 497-504

  Ho, Minh Anh T ^a   Yamada, Yoji ^b   Umetani, Yoji ^c  

^a Shibuya Cross Tower 28F   (Japan)

^b NATIONAL INSTITUTE OF ADVANCED INDUSTRIAL SCIENCE AND TECHNOLOGY AIST   (Japan)

^c Secom Intelligent Systems Laboratories   (Japan)

Author keywords

Hidden Markov models (HMMs); Intended gestures; Model update capability; New state distinction; Reinforcement learning; Visual tracking

Indexed keywords

COMMUNICATION CHANNELS (INFORMATION THEORY); GESTURE RECOGNITION; LEARNING ALGORITHMS; MARKOV PROCESSES; MATHEMATICAL MODELS; PROBABILITY; USER INTERFACES; WAVELET TRANSFORMS;

ADAPTIVE VISUAL ATTENTIVE TRACKER; HIDDEN MARKOV MODELS; INTENDED GESTURES; NEW STATE DISTINCTION; REINFORCEMENT LEARNING; TEMPORAL DIFFERENCE LEARNING; VISUAL TRACKING;

MAN MACHINE SYSTEMS;

EID: 20844437386     PISSN: 15523098     EISSN: None     Source Type: Journal    
DOI: 10.1109/TRO.2004.840912     Document Type: Article

References (23)

1. A. Pentland, "Looking at people: Sensing for ubiquitous and wearable computing," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 1, pp. 107-119, Jan. 2000.
2. V. Pavlovic, R. Sharma, and T. S. Huang, "Visual interpretation of hand gestures for human-computer interaction: a review," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 677-695, Jul. 1997.
3. J. K. Aggarwal and Q. Cai, "Human motion analysis: a review," Computer Vis. Image Understanding, vol. 73, no. 3, pp. 428-440, Mar. 1999.
4. S. Baron-Cohen, "Mindblindness: an essay on autism and theory of mind," in Mindreading: Nature's Choice. Cambridge, MA: MIT Press, 1996, ch. 3.
5. T. Sato, Y. Nishida, J. Ichikawa, Y. Hatamura, and H. Mizoguchi, "Active understanding of human intention by a robot through monitoring of human behavior," in Proc. IEEE IROS, 1994, pp. 405-414.
6. M. Mitsuishi, H. Watanabe, H. Nakanishi, H. Kubota, and Y. Iizuka, "Dexterity enhancement for a tele-micro co-located operation point manipulators and understanding of the operator's intention," in CVRMed-MRCAS'97, Grenoble, France, 1997, pp. 822-830.
7. J. Yamato, J. Ohya, and K. Ishii, "Recognizing human action in time-sequential images using hidden Markov models," in Proc. Int. Conf. Pattern Recognition, 1992, pp. 379-385.
8. T. Startner and A. Pentland, "Visual recognition of American sign language using hidden Markov models," in Proc. Int. Conf. Automatic Face and Gesture Recognition, 1995, pp. 189-194.
9. N. M. Oliver, B. Rosario, and A. P. Pentland, "A Bayesian computer vision system for modeling human interactions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 831-843, Aug. 2000.
10. M. A. T. Ho, Y. Yamada, T. Sakai, T. Morizono, and Y. Umetani, "Proposal of an adaptive vision-based interactional intention inference system in human/robot coexistence," IEICE Trans. Inform. Syst., vol. E84-D, no. 12, pp. 1596-1602, Dec. 2001.
11. R. D. Rimey and C. M. Brown, "Selective attention as sequential behavior: modeling eye movements with an augmented hidden markov model," in Proc. DARPA, Image Understanding Workshop, 1990, pp. 840-849.
12. L. Chrisman, "Reinforcement learning with perceptual aliasing: the perceptual distinctions approach," in Proc AAAI, 1992, pp. 183-188.
13. A. K. McCallum, "Reinforcement learning with selective perception and hidden state," Ph.D. dissertation, Univ. Rochester, Rochester, NY, 1996.
14. T. G. Dietterich, "Hierarchical reinforcement learning with the MAXQ value function decomposition," J. Artif. Intell. Res., vol. 13, pp. 227-303, 2000.
15. C. Gaskett, D. Wettergreen, and A. Zelinsky, "Q-learning in continuous state and action spaces," in Proc. 12th Australian Joint Conf. Artif. Intell., Sydney, Australia, Dec. 1999, pp. 417-428.
16. D. Chapman and L. P. Kaelbling, "Input generalizaiton in delayed reinforcement learning: an algorithm and performance comparisons," in Proc. Int. Joint Conf. Artif. Intell., 1991, pp. 726-731.
17. R. Munos and A. Moore, "Variable resolution discretization in optimal control," Machine Learning, vol. 49, no. 2/3, pp. 291-323, 2002.
18. M. A. T. Ho, Y. Yamada, and Y. Umetani, "An HMM-based temporal difference learning with state transition update capability for visual tracking of human communicational behaviors," J. Robot. Mechatron., vol. 15, no. 3, pp. 271-277, 2003.
19. J.-C. Terrillon, M. David, and S. Akamatsu, "Automatic detection of human faces in natural scene images by use of a skin color model and of invariant moments," in Proc. IEEE Conf. Automatic Face and Gesture Recognition, 1998, pp. 112-117.
20. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, no. 2, pp. 257-285, Feb. 1989.
21. S. Mallat, A Wavelet Tour of Signal Processing, 2nd ed. San Diego, CA: Academic, 1998.
22. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
23. V. Tadic, "On the convergence of temporal-difference learning with linear function approximation," Machine Learning, vol. 42, pp. 241-267, 2001.