A multi-gesture interaction system using a 3-D iris disk model for gaze estimation and an active appearance model for 3-D hand pointing

IEEE Transactions on Multimedia

Volumn 13, Issue 3, 2011, Pages 474-486

  Reale, Michael J ^a   Canavan, Shaun ^a   Yin, Lijun ^a   Hu, Kaoning ^a   Hung, Terry ^b  

^a State University of New York at Binghamton   (United States)

^b Corning Corp ^*  (Taiwan)

Author keywords

Gaze estimation; hand tracking; humancomputer interaction (HCI)

Indexed keywords

ACTIVE APPEARANCE MODELS; COMPLEX HARDWARE; CONTOUR POINTS; CONTROL COMPONENTS; DISK MODEL; ENTIRE SYSTEM; ESTIMATION ALGORITHM; EYE-GAZE; GAZE DIRECTION; GAZE ESTIMATION; HAND FINGERS; HAND GESTURE; HAND TRACKING; HEAD POSE; HUMAN-COMPUTER INTERACTION SYSTEM; HUMANCOMPUTER INTERACTION (HCI); INTERACTION SYSTEMS; INTERACTIVE INFORMATION VISUALIZATION; NON-INTRUSIVE; OPTICAL AXIS; PAN-TILT-ZOOM CAMERA; ROUGH LOCATION; TOPOGRAPHIC FEATURES; VISION BASED; WIDE-ANGLE CAMERA;

ALGORITHMS; CAMERAS; ESTIMATION; EYE MOVEMENTS; FACE RECOGNITION; INFORMATION SYSTEMS; SOIL STRUCTURE INTERACTIONS; SPHERES; THREE DIMENSIONAL; USER INTERFACES; VISUALIZATION;

GESTURE RECOGNITION;

EID: 79957456055     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2011.2120600     Document Type: Conference Paper

References (49)

1. Z. Zeng, M. Pantic, G. Roisman, and T. Huang, "A survey of affect recognition methods: Audio, visual, and spontaneous expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 1, pp. 39-58, Jan. 2009.
2. S. Zhai, C. Morimoto, and S. Ihde, "Manual and gaze input cascaded (magic) pointing," in Proc. CHI'99, 1999, pp. 246-253.
3. D. W. Hansen and Q. Ji, "In the eye of the beholder:A survey of models for eyes and gaze," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 3, pp. 478-500, Mar. 2010.
4. S.Wang, X. Xiong, Y. Xu, C.Wang, W. Zhang, X. Dai, and D. Zhang, "Face-tracking as an augmented input in video games: Enhancing presence role-playing and control," in Proc. CHI'06, 2006, pp. 1097-1106. (Pubitemid 44032204)
5. G. G. Mateos and S. F. Munoz, "Tierra inhospita: Exploring a virtual world with your face," in Proc. ACM SIGCHI Int. Conf. Advances in Computer Entertainment Technology (ACE'05), 2005, pp. 383-384.
6. C. Harrison and A. K. Dey, "Lean and zoom: Proximity-aware user interface and content magnification," in Proc. CHI'08, 2008, pp. 507-510.
7. T. B. Moeslund and E. Granum, "A survey of computer vision-based human motion capture," Comput. Vis. Image Understand., vol. 81, pp. 231-268, Mar. 2001. (Pubitemid 32415976)
8. K. Tollmar, D. Demirdjian, and T. Darrell, "Navigating in virtual environments using a vision-based interface," in Proc. Nordic Conf. Human-Computer Interaction (NordiCHI'04), 2004, pp. 113-120.
9. M.-Y. Chen, L. Mummert, P. Pillai, A. Hauptmann, and R. Sukthankar, "Controlling your tv with gestures," in Proc. Int. Conf. Multimedia Information Retrieval (MIR'10), 2010, pp. 405-408.
10. J. P. Wachs, M. Kölsch, H. Stern, and Y. Edan, "Vision-based handgesture applications," Commun. ACM, vol. 54, pp. 60-71, Feb. 2011.
11. D. Beymer and M. Flickner, "Eye gaze tracking using an active stereo head," in Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR'03), Jun. 2003, vol. 2, pp. 451-458.
12. B. Noureddin, P. D. Lawrence, and C. F. Man, "A non-contact device for tracking gaze in a human computer interface," Comput. Vis. Image Understand., vol. 98, pp. 52-82, Apr. 2005.
13. T. Ohno and N. Mukawa, "A free-head simple calibration, gaze tracking system that enables gaze-based interaction," in Proc. Symp. Eye Tracking Research & Applications (ETRA'04), 2004, pp. 115-122.
14. K. Nguyen, C. Wagner, D. Koons, and M. Flickner, "Differences in the infrared bright pupil response of human eyes," in Proc. Symp. Eye Tracking Research & Applications (ETRA'02), 2002, pp. 133-138.
15. K. Bernardin, H. K. Ekenel, and R. Stiefelhagen, "Multimodal identity tracking in a smart room," Personal Ubiq. Comput., vol. 13, pp. 25-31, Jan. 2009.
16. J.-G. Wang and E. Sung, "Gaze determination via images of irises," Image Vision Comput., vol. 19, no. 12, pp. 891-911, 2001. (Pubitemid 33663916)
17. D. Xia and Z. Ruan, "IR image based eye gaze estimation," in Proc. ACIS Int. Conf. Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing (SNPD'07), 2007, vol. 1, pp. 220-224.
18. C. Colombo, D. Comanducci, and A. D. Bimbo, "Robust tracking and remapping of eye appearance with passive computer vision," ACM Trans. Multimedia Comput. Commun. Appl., vol. 3, pp. 2:1-2:20, Dec. 2007.
19. J. Wang, L. Yin, and J. Moore, "Using geometric properties of topographic manifold to detect and track eyes for human-computer interaction," ACM Trans. Multimedia Comput. Commun. Appl., vol. 3, pp. 3:1-3:20, Dec. 2007.
20. D. Schnieders, X. Fu, and K.-Y.Wong, "Reconstruction of display and eyes from a single image," in Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR'10), Jun. 2010, pp. 1442-1449.
21. E. Pogalin, A. Redert, I. Patras, and E. A. Hendriks, "Gaze tracking by using factorized likelihoods particle filtering and stereo vision," in Proc. Int. Symp. 3D Data Processing, Visualization, and Transmission (3DPVT'06), 2006, pp. 57-64.
22. J. Xie and X. Lin, "Gaze direction estimation based on natural head movements," in Proc. Int. Conf. Image and Graphics (ICIG'07), 2007, pp. 672-677.
23. H. Wu, Y. Kitagawa, T. Wada, T. Kato, and Q. Chen, "Tracking iris contour with a 3D eye-model for gaze estimation," in Proc. Asian Conf. Comput. Vision-Volume Part I (ACCV'07), 2007, vol. Part I, pp. 688-697.
24. H. Yamazoe, A. Utsumi, T. Yonezawa, and S. Abe, "Remote and head-motion-free gaze tracking for real environments with automated head-eye model calibrations," in Proc. IEEE CVPR Workshop Human Communicative Behavior Analysis (CVPR4HB), Jun. 2008, pp. 1-6.
25. V. I. Pavlovic, R. Sharma, and T. S. Huang, "Visual interpretation of hand gestures for human-computer interaction: A review," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 7, pp. 677-695, Jul. 1997. (Pubitemid 127762261)
26. Y.Wu and T. S. Huang, "Vision-based gesture recognition: A review," in Proc. Int. Gesture Workshop Gesture-Based Communication in Human-Computer Interaction (GW'99), 1999, pp. 103-115.
27. R. Y. Wang and J. Popović, "Real-time hand-tracking with a color glove," ACM Trans. Graphics, vol. 28, pp. 63:1-63:8, Jul. 2009.
28. C. Manders, F. Farbiz, J. Chong, K. Tang, G. Chua, M. Loke, and M. Yuan, "Robust hand tracking using a skin tone and depth joint probability model," in Proc. IEEE Int. Conf. Automatic Face & Gesture Recognition (FG'08), Sep. 2008, pp. 1-6.
29. A. Utsumi, N. Tetsutani, and S. Igi, "Hand detection and tracking using pixel value distribution model for multiple-camera-based gesture interactions," in Proc. IEEE Workshop Knowledge Media Networking (KMN'02), 2002, pp. 31-36.
30. M. Lee, D. Weinshall, E. Cohen-Solal, A. Colmenarez, and D. Lyons, "A computer vision system for on-screen item selection by finger pointing," in Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR'01), 2001, vol. 1, pp. 1026-1033.
31. C. Colombo, A. D. Bimbo, and A. Valli, "Visual capture and understanding of hand pointing actions in a 3-D environment," IEEE Trans. Syst., Man, Cybern. B, vol. 33, no. 4, pp. 677-686, Aug. 2003.
32. N. Jojic, B. Brumitt, B. Meyers, S. Harris, and T. Huang, "Detection and estimation of pointing gestures in real-time stereo sequences," in Proc. IEEE Int. Conf. Automatic Face & Gesture Recognition (FG'00), 2000, pp. 468-475.
33. Y. Yamamoto, I. Yoda, and K. Sakaue, "Arm-pointing gesture interface using surrounded stereo cameras system," in Proc. Int. Conf. Pattern Recognition (ICPR'04), 2004, vol. 4, pp. 965-970.
34. M. Kolsch and M. Turk, "Analysis of rotational robustness of hand detection with a Viola-Jones detector," in Proc. Int. Conf. Pattern Recognition (ICPR'04), 2004, vol. 3, pp. 107-110.
35. T. T. Nguyen, N. D. Binh, and H. Bischof, "An active boosting-based learning framework for real-time hand detection," in Proc. IEEE Int. Conf. Automatic Face & Gesture Recognition (FG'08), Sep. 2008, pp. 1-6.
36. L. Cinque, M. Cupelli, and E. Sangineto, "Fast viewpoint-invariant articulated hand detection combining curve and graph matching," in Proc. IEEE Int. Conf. Automatic Face & Gesture Recognition (FG'08), Sep. 2008, pp. 1-6.
37. M. Kolsch and M. Turk, "Robust hand detection," in Proc. IEEE Int. Conf. Automatic Face & Gesture Recognition (FG'04), May 2004, pp. 614-619.
38. K. Oka, Y. Sato, and H. Koike, "Real-time fingertip tracking and gesture recognition," IEEE Comput. Graphics Appl., vol. 22, no. 6, pp. 64-71, Nov./Dec. 2002. (Pubitemid 35421720)
39. C.-B. Park, M.-C. Roh, and S.-W. Lee, "Real-time 3D pointing gesture recognition in mobile space," in Proc. IEEE Int. Conf. Automatic Face & Gesture Recognition (FG'08), Sep. 2008, pp. 1-6.
40. P. Viola and M. J. Jones, "Robust real-time face detection," Int. J. Comput. Vis., vol. 57, pp. 137-154, May 2004.
41. T. F. Cootes, G. J. Edwards, and C. J. Taylor, "Active appearance models," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 6, pp. 681-685, Jun. 2001. (Pubitemid 32585279)
42. T. Hung, "Real-time face detection and applications," M.S. thesis, Binghamton Univ., Binghamton, NY, 2008.
43. Seeing machines faceapi. [Online]. Available: http://www.seeingmachines. com/product/faceapi.
44. D. Li, D.Winfield, and D. Parkhurst, "Starburst: A hybrid algorithm for video-based eye tracking combining feature-based and model-based approaches," in Proc. IEEE CVPR Workshop Vision for Human-Computer Interaction (V4HCI), Jun. 2005, p. 79.
45. V. Vezhnevets and A. Degtiareva, "Robust and accurate eye contour extraction," in Proc. GraphiCon'03, 2003, pp. 81-84.
46. F. Coutinho and C. Morimoto, "Free head motion eye gaze tracking using a single camera and multiple light sources," in Proc. Brazilian Symp. Computer Graphics and Image Processing (SIBGRAPI'06), Oct. 2006, pp. 171-178. (Pubitemid 47515406)
47. R. Bailey, A. McNamara, N. Sudarsanam, and C. Grimm, "Subtle gaze direction," in Proc. ACM SIGGRAPH 2007 posters, 2007.
48. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C: The Art of Scientific Computing, 2nd ed. New York: Cambridge Univ. Press, 1992.
49. J. Orozco, F. X. Roca, and J. Gonzàlez, "Real-time gaze tracking with appearance-based models," Mach. Vision Appl., vol. 7, pp. 1-12, Oct. 2009.