Continuous body and hand gesture recognition for natural human-computer interaction

ACM Transactions on Interactive Intelligent Systems

Volumn 2, Issue 1, 2012, Pages 1-28

  Song, Yale ^a   Demirdjian, David ^a   Davis, Randall ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Conditional random fields; Gesture recognition; Human computer interaction; Multilayered filtering; Online sequence labeling and segmentation; Pose tracking

Indexed keywords

FORECASTING; HUMAN COMPUTER INTERACTION; IMAGE SEGMENTATION; MOTION TRACKING; PALMPRINT RECOGNITION; RANDOM PROCESSES; STEREO IMAGE PROCESSING; STEREO VISION; SUPPORT VECTOR MACHINES; THREE DIMENSIONAL COMPUTER GRAPHICS;

APPEARANCE BASED APPROACH; CONDITIONAL RANDOM FIELD; GESTURE RECOGNITION SYSTEM; MULTI-CLASS SUPPORT VECTOR MACHINES; MULTI-LAYERED; NATURAL HUMAN COMPUTER INTERACTIONS; POSE TRACKING; SEQUENCE LABELING;

GESTURE RECOGNITION;

EID: 84983483288     PISSN: 21606455     EISSN: 21606463     Source Type: Journal    
DOI: 10.1145/2133366.2133371     Document Type: Article

References (43)

1. BARR,A. 1981. Superquadrics andangle-preservingtransformations. IEEEComput. Graph. Appl. 1, 1, 11-23..
2. BOBICK, A. F. AND DAVIS, J. W. 2001. The recognition of human movement using temporal templates. IEEE Trans. Pattern Anal. Mach. Intell. 23, 3, 257-267..
3. BRADSKI, G. AND KAEHLER, A. 2008. Learning OpenCV: Computer Vision with the OpenCVLibrary. O'Reilly, Cambridge, MA..
4. BRAND, M. 1999. Shadow puppetry. In Proceedings of the IEEE International Conference on Computer Vision. 1237-1244..
5. CHANG, C.-C. AND LIN, C.-J. 2011. Libsvm: A library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 3, 27..
6. DALAL, N. AND TRIGGS, B. 2005. Histograms of oriented gradients for human detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 886-893..
7. DENAVIT, J. AND HARTENBERG, R. S. 1955. A kinematic notation for lower-pair mechanisms based on matrices. ASME J. Appl. Mechan. 23, 215-221..
8. DEUTSCHER, J., BLAKE, A., AND REID, I. D. 2000. Articulated body motion capture by annealed particle filtering. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2126-2133..
9. ENGIN, A. 1980. On the biomechanics of the shoulder complex. J. Biomechan. 13, 7, 575-581, 583-590..
10. EROL, A., BEBIS, G., NICOLESCU, M., BOYLE, R. D., AND TWOMBLY, X. 2007. Vision-based hand pose estimation: A review. Comput. Vis. Image Understand. 108, 1-2, 52-73..
11. FENG, X., YANG, J., AND ABDEL-MALEK, K. 2008. Survey of biomechanical models for the human shoulder complex. Tech. rep., SAE International..
12. FOFI, D., SLIWA, T., AND VOISIN, Y. 2004. A comparative survey on invisible structured light. In Proceedings of SPIE Machine Vision Applications in Industrial Inspection XII..
13. FREEMAN, W. T., ANDERSON, D. B., BEARDSLEY, P. A., DODGE, C., ROTH, M., WEISSMAN, C. D., YERAZUNIS, W. S., KAGE, H., KYUMA, K., MIYAKE, Y., AND ICHI TANAKA, K. 1998. Computer vision for interactive computer graphics. IEEE Comput. Graph. Appl. 18, 3, 42-53..
14. GOKTURK, S., YALCIN, H., AND BAMJI, C. 2004. A time-of-flight depth sensor\-System description, issues and solutions. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshop..
15. GUNAWARDANA, A., MAHAJAN, M., ACERO, A., AND PLATT, J. C. 2005. Hidden conditional random fields for phone classification. In Proceedings of the 9th European Conference on Speech Communication and Technology. 1117-1120..
16. HARRIS, F 1978. On the use of windows for harmonic analysis with the discrete fourier transform. Proc. IEEE 66, 1, 51-83..
17. HSU, C.-W. AND LIN, C.-J. 2002. A comparison ofmethods for multiclass support vector machines. IEEE Trans. Neural Netw. 13, 2, 415-425..
18. HUANG, L., MORENCY, L.-P., AND GRATCH, J. 2011. Virtual rapport 2.0. In Proceedings of the 11th International Conference on Intelligent VirtualAgents. Lecture Notes in Computer Science Series, vol. 6895, Springer, 68-79..
19. ISARD, M. AND BLAKE, A. 1998. CONDENSATION\-Conditional density propagation for visual tracking. Int. J. Comput. Vis. 29, 1, 5-28..
20. KIM, K., CHALIDABHONGSE, T. H., HARWOOD, D., AND DAVIS, L. S. 2005. Real-Time foreground-background segmentation using codebook model. Real-Time Imag. 11, 3, 172-185..
21. KNERR, S., PERSONNAZ, L., AND DREYFUS, G. 1990. Single-Layer learning revisited: A stepwise procedure for building and training a neural network. In Neurocomputing: Algorithms, Architectures and Applications, J. Fogelman, Ed., Springer-Verlag..
22. LAFFERTY, J. D., MCCALLUM, A., AND PEREIRA, F C. N. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the 18th International Conference on Machine Learning. Morgan Kaufmann, 282-289..
23. LEE, M. W. AND COHEN, I. 2006. A model-based approach for estimating human 3d poses in static images. IEEE Trans. Pattern Anal. Mach. Intell. 28, 905-916..
24. MITRA, S. AND ACHARYA, T. 2007. Gesture recognition: A survey. IEEE Trans. Syst. Man, Cybernet. C: Appl. Rev. 37, 3, 311-324..
25. MORENCY, L.-P., QUATTONI, A., AND DARRELL, T. 2007. Latent-Dynamic discriminative models for continuous gesture recognition. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition..
26. MORI, G. AND MALIK, J. 2006. Recovering 3d human body configurations using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. 28, 7, 1052-1062..
27. MURPHY, K. 2002. Dynamic bayesian networks: Representation, inference and learning. Ph.D. thesis Computer Science Division, UC, Berkeley
28. NASA. 1995. Man-Systems Integration Standards: Vol. 1. Section 3. Anthropometry and Biomechanics. http://msis.jsc.hasa.gov/sections/section03.htm. .
29. NOCEDAL, J. AND WRIGHT, S. J. 1999. Numerical Optimization. Springer-Verlag..
30. PEARL, J. 1988. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann..
31. POPPE, R. 2007. Vision-Basedhuman motion analysis: An overview. Comput. Vis. Image Understand. 108, 1-2, 4-18..
32. QUATTONI, A., WANG, S. B., MORENCY, L.-P., COLLINS, M., AND DARRELL, T. 2007. Hidden conditional random fields. IEEE Trans. Pattern Anal. Mach. Intell. 29, 10, 1848-1852..
33. SCHWARZ, L. A., MKHITARYAN, A., MATEUS, D., ANDNAVAB, N. 2011. Estimatinghuman 3dpose fromtime-of-flight images based on geodesic distances and optical flow. In Proceedings of the IEEE International Conference onAutomatic Face and Gesture Recognition. 700-706..
34. SHAKHNAROVICH, G., VIOLA, P. A., AND DARRELL, T. 2003. Fast pose estimation withparameter-sensitive hashing. In Proceedings of the IEEE International Conference on Computer Vision. 750-759..
35. SHOTTON, J., FITZGIBBON, A., COOK, M., SHARP, T., FINOCCHIO, M., MOORE, R., KIPMAN, A., AND BLAKE, A. 2011. Real-time human pose recognition in parts from single depth images. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition..
36. SMINCHISESCU, C. AND TRIGGS, B. 2003. Kinematicjump processes for monocular 3d human tracking. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 69-76..
37. SONG, Y., DEMIRDJIAN, D., AND DAVIS, R. 2011a. Multi-Signal gesture recognition using temporal smoothing hidden conditional random fields. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 388-393..
38. SONG, Y., DEMIRDJIAN, D., AND DAVIS, R. 2011b. Tracking body and hands for gesture recognition: Natops aircraft handling signals database. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition. 500-506..
39. SUTTON, C. A., ROHANIMANESH, K., AND MCCALLUM, A. 2004. Dynamic conditional random fields: factorized probabilistic models for labeling and segmenting sequence data. In Proceedings of the 18th International Conference on Machine Learning. Morgan Kaufmann..
40. VAPNIK, V. N. 1995. The Nature of Statistical Learning Theory. Springer, New York..
41. WANG, Y. AND MORI, G. 2009. Max-Margin hidden conditional random fields for human action recognition. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 872-879..
42. WEINLAND, D., RONFARD, R., AND BOYER, E. 2011. A survey ofvision-based methods for action representation, segmentation and recognition. Comput. Vis. Image Understand. 115, 2, 224-241..
43. YIN, Y. AND DAVIS, R. 2010. Toward natural interaction in the real world: Real-Time gesture recognition. In Proceedings of the 12th International Conference on Multimodal Interfaces/International Workshop on Machine Learningfor Multimodal Interaction. 15..