Real-time continuous pose recovery of human hands using convolutional networks

ACM Transactions on Graphics

Volumn 33, Issue 5, 2014, Pages

  Tompson, Jonathan ^a   Stein, Murphy ^a   Lecun, Yann ^a   Perlin, Ken ^a  

^a NEW YORK UNIVERSITY   (United States)

Author keywords

Analysis by synthesis; Hand tracking; Markerless motion capture; Neural networks

Indexed keywords

CLASSIFICATION (OF INFORMATION); COMPUTER SYSTEM RECOVERY; CONVOLUTION; IMAGE SEGMENTATION; INVERSE KINEMATICS; MOTION TRACKING; NEURAL NETWORKS; RECOVERY;

ANALYSIS BY SYNTHESIS; CONVOLUTIONAL NETWORKS; HAND TRACKING; LABELED DATASET; MARKERLESS MOTION CAPTURE; RANDOMIZED DECISIONS; ROBUST METHODS; STATE OF THE ART;

CONVOLUTIONAL NEURAL NETWORKS;

EID: 84907552337     PISSN: 07300301     EISSN: 15577368     Source Type: Journal    
DOI: 10.1145/2629500     Document Type: Article

References (35)

1. GEAR. 2014. 3gear sytems hand-tracking development platform. http:// www.threegear.com/.
2. B. Allen, B. Curless, and Z. Popovic. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Trans. Graph. 22, 3, 587-594.
3. thEuropean Conference on Computer Vision. 640-653.
4. Y. Boykov, O. Veksler, and R. Zabih. 2001. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23, 11, 1222-1239.
5. D. A. Butler, S. Izadi, O. Hilliges, D. Molyneaux, S. Hodges, and D. Kim. 2012. Shake'n'sense: Reducing interference for overlapping structured light depth cameras. In Proceedings of the ACM Annual Conference on Human Factors in Computing Systems. 1933-1936.
6. R. Collobert, K. Kavukcuoglu, and C. Farabet. 2011. Torch7: A matlab-like environment for machine learning, http://ronan.collobert.com/pub/matos/2011torch7nipsw.pdf.
7. C. Couprie, C. Farabet, L. Najman, and Y. Lecun. 2013. Indoor semantic segmentation using depth information. In Proceedings of the International Conference on Learning Representations.
8. A. Erol, G. Bebis, M. Nicolescu, R. D. Boyle, and X. Twombly. 2007. Visionbased hand pose estimation: A review. Comput. Vis. Image Understand. 108, 1-2, 52-73.
9. C. Farabet, C. Couprie, L. Najman, and Y. Lecun. 2013. Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35, 8, 1915-1929.
10. B. K. P. Horn. 1987. Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Amer. 4, 4, 629-642.
11. K. Jarrett, K. Kavukcuoglu, M. Ranzato, and Y. Lecun. 2009. What is the best multi-stage architecture for object recognition? In Proceedings of the 12th IEEE International Conference on Computer Vision. 2146-2153.
12. M. Jiu, C. Wolf, G. W. Taylor, and A. Baskurt. 2013. Human body part estimation from depth images via spatially-constrained deep learning. Pattern Recogn. Lett. (to appear).
13. C. Keskin, F. Kirac, Y. Kara, and L. Akarun. 2011. Real time hand pose estimation using depth sensors. In Proceedings of the IEEE International Computer Vision Workshops. 1228-1234.
14. thEuropean Conference on Computer Vision. Vol. 6. Springer, 852-863.
15. A. Krizhevsky, I. Sutskever, and G. Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Proceedings of the Neural Information Processing Systems Conference. P. Bartlett, F. Pereira, C. Burges, L. Bottou, and K. Weinberger, Eds., 1106-1114.
16. Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11, 2278-2324.
17. Y. Lecun, F. J. Huang, and L. Bottou. 2004. Learning methods for generic object recognition with invariance to pose and lighting. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Vol. 2. 97-104.
18. H. Li, R. W. Sumner, and M. Pauly. 2008. Global correspondence optimization for non-rigid registration of depth scans. Comput. Graph. Forum 27, 5, 1421-1430.
19. H. Li, J. Yu, Y. Ye, and C. Bregler. 2013. Realtime facial animation with on-the-fly correctives. ACM Trans. Graph. 32, 4.
20. S. Melax, L. Keselman, and S. Orsten. 2013. Dynamics based 3D skeletal hand tracking. In Proceedings of the ACM Symposium on Interactive 3D Graphics and Games.
21. J. Nagi, F. Ducatelle, G. Di Caro, D. Ciresan, U. Meier, A. Giusti, F. Nagi, J. Schmidhuber, and L. Gambardella. 2011. Max-pooling convolutional neural networks for vision-based hand gesture recognition. In Proceedings of the IEEE International Conference on Signal and Image Processing Applications. 342-347.
22. S. J. Nowlan and J. C. Platt. 1995. A convolutional neural network hand tracker. In Proceedings of the Neural Information Processing Systems Conference. 901-908.
23. I. Oikonomidis, N. Kyriazis, and A. Argyros. 2011. Efficient model-based 3D tracking of hand articulations using kinect. In Proceedings of the British Machine Vision Conference.
24. M. Osadchy, Y. Lecun, M. L. Miller, and P. Perona. 2005. Synergistic face detection and pose estimation with energy-based model. In Proceedings of the Neural Information Processing Systems Conference. 1017-1024.
25. J. M. Rehg and T. Kanade. 1994. Visual tracking of high dof articulated structures: An application to human hand tracking. In Proceedings of the 3rd European Conference on Computer Vision. 35-46.
26. M. Saric. 2011. Libhand: A library for hand articulation. http://www.libhand.org/.
27. J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, and A. Blake. 2011. Real-time human pose recognition in parts from single depth images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1297-1304.
28. M. Stein, J. Tompson, X. Xiao, C. Hendeee, H. Ishii, and K. Perlin. 2012. Arcade: A system for augmenting gesture-based computer graphic presentations. In Proceedings of the ACM SIGGRAPH Computer Animation Festival. 77-77.
29. G. W. Taylor, I. Spiro, C. Bregler, and R. Fergus. 2011. Learning invariance through imitation. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 2729-2736.
30. P. Tseng. 1995. Fortified-descent simplicial search method: A general approach. SIAM J. Optim. 10, 1, 269-288.
31. R. Wang, S. Paris, and J. Popovic. 2011. 6d hands: Markerless tracking for computer aided design. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology. 549-558.
32. R. Y. Wang, and J. Popovic. 2009. Real-time hand-tracking with a color glove. ACM Trans. Graph. 28, 3.
33. T. Weise, H. Li, L. Van Gool, and M. Pauly. 2009. Face/off: Live facial puppetry. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation.
34. T. Yasuda, K. Ohkura, and Y. Matsumura. 2010. Extended PSO with partial randomization for large scale multimodal problems. In Proceedings of the World Automation Congress. 1-6.
35. W. Zhao, J. Chai, and Y.-Q. Xu. 2012. Combining marker-based MOCAP and RGB-d camera for acquiring high-fidelity hand motion data. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Eurographics Association, 33-42.