Real-time multimodal humanavatar interaction

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 18, Issue 4, 2008, Pages 467-477

  Fu, Yun ^a   Li, Renxiang ^b   Huang, Thomas S ^a   Danielsen, Mike ^b  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b MOTOROLA INC   (United States)

Author keywords

Avatar; DAZ3D; DAZ3D,; Head tracking; Human computer interaction; M3G; Mobile 3 D (M3G); Multimodal system; Real time multi modal human avatar interaction (RTM HAI); RTM HAI; TTS; Visual communication

Indexed keywords

AVATAR; DAZ3D; HEAD TRACKING; M3G; MOBILE 3-D (M3G); MULTIMODAL SYSTEM; REAL-TIME MULTI MODAL HUMAN-AVATAR INTERACTION (RTM-HAI); RTM-HAI; TTS;

ANIMATION; ATMOSPHERIC RADIATION; COMMUNICATION; COMPUTERS; FACE RECOGNITION; IMAGE COMMUNICATION SYSTEMS; KNOWLEDGE MANAGEMENT; MOBILE DEVICES; MOBILE TELECOMMUNICATION SYSTEMS; THREE DIMENSIONAL; VISUAL COMMUNICATION; WIRELESS NETWORKS;

HUMAN COMPUTER INTERACTION;

EID: 54949109437     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2008.918441     Document Type: Article

References (32)

1. R. Li, "A technical report on motorola avatar framework," Applications Research Center, Motorola Labs, Schaumburg, IL, TR-2006, 2006.
2. Y. Fu and N. Zheng, "M-face: An appearance-based photorealistic model for multiple facial attributes rendering," IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 7, pp. 830-842, Jul. 2006.
3. Y. Fu and T. S. Huang, "hMouse: Head tracking driven virtual computer mouse," in Proc. IEEE Workshop WACV, Feb. 2007, p. 30.
4. Y. Fu, R. Li, T. S. Huang, and M. Danielsen, "Real-time humanoid avatar for multimodal human-machine interaction," in Proc. IEEE Conf. ICME, 2007, pp. 991-994.
5. T. Yang, S. Li, Q. Pan, J. Li, and C. H. Zhao, "Reliable and fast tracking of faces under varying pose," in Proc. IEEE Conf. FGR, 2006, pp. 421-428.
6. H. Tao and T. S. Huang, "Explanation-based facial motion tracking using a piecewise Bézier volume deformation model," in Proc. IEEE Conf. CVPR, 1999, vol. 1, pp. 611-617.
7. P. Hong, Z. Wen, and T. S. Huang, "Real-time speech-driven face animation with expressions using neural networks," IEEE Trans. Neural Netw., vol. 13, no. 4, pp. 916-927, Jul. 2002.
8. Z.Wen and T. S. Huang, 3D Face Processing: Modeling, Analysis and Synthesis, 1st ed. Berlin, Germany: Springer-Verlag, 2004.
9. M. Mancini, R. Bresin, and C. Pelachaud, "A virtual head driven by music expressivity," IEEE Trans. Audio, Speech, Lang. Process., vol. 15, no. 6, pp. 1833-1841, Aug. 2007.
10. M. La Cascia, S. Sclaroff, and V. Athitsos, "Fast, reliable head tracking under varying illumination: An approach based on registration of texture-mapped 3-D models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 4, pp. 322-336, Apr. 2000.
11. H. Nanda and K. Fujimura, "A robust elliptical head tracker," in Proc. IEEE Conf. FGR, 2004, pp. 469-474.
12. P. A. Viola and M. J. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. IEEE Conf. CVPR, 2001, pp. 511-518.
13. G. R. Bradski, "Real time face and object tracking as a component of a perceptual user interface," in Proc. IEEE Workshop WACV, 1998, pp. 214-219.
14. M. T. Wenzel and W. H. Schiffmann, "Head pose estimation of partially occluded faces," in Proc. 2nd Can. Conf. Comput. Robot Vision, 2005, pp. 353-360.
15. Y. Wang, VisGenie [Online]. Available: http://www.ee.columbia.edu/ ̃ywang/Research/VisGenie/
16. Open Source Computer Vision Library (OpenCV). Intel Corporation. [Online]. Available: http://www.intel.com/technology/computing/opencv/index.htm
17. Java Mobile 3D [Online]. Available: http://www.jcp.org/en/jsr/detail?id= 184
18. X. L. Yang, D. C. Petriu, T. E. Whalen, and E. M. Petriu, "Hierarchical animation control of avatars in 3-D virtual environments," IEEE Trans. Instrum. Meas., vol. 54, no. 3, pp. 1333-1341, Jun. 2005.
19. V. Blanz and T. Vetter, "A morphable model for the synthesis of 3-D faces," in Proc. SIGGRAPH, 1999, pp. 187-194.
20. F. I. Parke, "Computer generated animation of faces," in Proc. ACM Nat. Conf., 1972, pp. 451-457.
21. K. Perlin and A. Goldberg, "Improv: A system for scripting interactive actors in virtual worlds," in Proc. SIGGRAPH, 1996, pp. 205-216.
22. C. S. Pinhanez, K. Mase, and A. F. Bobick, "Interval scripts: A design paradigm for story-based interactive systems," in Proc. ACM SIGCHI, 1997, pp. 287-294.
23. N. Badler, C. Phillips, and B. Webber, "Simulating humans," in Computer Graphics Animation and Control. New York: Oxford Univ. Press, 1993.
24. M. Betke, J. Gips, and P. Fleming, "The camera mouse: Visual tracking of body features to provide computer access for people with severe disabilities," IEEE Trans. Neural Syst. Rehab. Eng., vol. 10, no. 1, pp. 1-10, Mar. 2002.
25. S. Rusdorf, G. Brunnett, M. Lorenz, and T. Winkler, "Real-time interaction with a humanoid avatar in an immersive table tennis simulation," IEEE Trans. Visualiz. Comput. Graphics, vol. 13, no. 1, pp. 15-25, Jan. 2007.
26. H. Ishii, C.Wisneski, J. Orbanes, B. Chun, and J. Paradiso, "PingPong-Plus: Design of an athletic-tangible interface for computer-supported cooperative play," in Proc. ACM SIGCHI, 1999, pp. 394-401.
27. V. Wulf, E. F. Moritz, C. Henneke, K. Al-Zubaidi, and G. Stevens, "Computer supported collaborative sports: Creating social spaces filled with sports activities," in Proc. Int. Conf. Educational Computing, 2004, pp. 80-89.
28. I. Pandzic and R. Forchheimer, Eds., MPEG-4 Facial Animation. Chichester, U.K.: Wiley, 2002.
29. DAZ3D [Online]. Available: http://www.daz3d.com/
30. K.-H. Choi and J.-N. Hwang, "Constrained optimization for audio-tovisual conversion," IEEE Trans. Signal Process., vol. 52, no. 6, pp. 1783-1790, Jun. 2004.
31. J. Ma, R. Cole, B. Pellom, W. Ward, and B. Wise, "Accurate visible speech synthesis based on concatenating variable length motion capture data," IEEE Trans. Visualiz. Comput. Graphics, vol. 12, no. 2, pp. 266-276, Feb. 2006.
32. J. W. Davis and A. F. Bobick, "The representation and recognition of human movement using temporal templates," in Proc. IEEE Conf. CVPR, 1997, pp. 928-934.