EgoCAp: Egocentric marker-less motion capture with two fisheye cameras

ACM Transactions on Graphics

Volumn 35, Issue 6, 2016, Pages

  Rhodin, Helge ^a   Richardt, Christian ^a,b,c   Casas, Dan ^a   Insafutdinov, Eldar ^a   Shafiei, Mohammad ^a   Seidel, Hans Peter ^a   Schiele, Bernt ^a   Theobalt, Christian ^a  

^a MAX PLANCK INSTITUTE FOR INFORMATICS   (Germany)

^b Intel Visual Computing Institute   (Germany)

^c UNIVERSITY OF BATH   (United Kingdom)

Author keywords

Crowded scenes; First person vision; Inside in; Large scale; Markerless; Motion capture; Optical

Indexed keywords

EXOSKELETON (ROBOTICS); VIRTUAL REALITY;

CROWDED SCENES; FIRST-PERSON VISIONS; LARGE-SCALE; MARKERLESS; MOTION CAPTURE; OPTICAL;

CAMERAS;

EID: 85044267747     PISSN: 07300301     EISSN: 15577368     Source Type: Journal    
DOI: 10.1145/2980179.2980235     Document Type: Article

References (72)

1. AMIN, S., ANDRILUKA, M., ROHRBACH, M., AND SCHIELE, B. 2009. Multi-view pictorial structures for 3D human pose estimation. In BMVC.
2. ANDRILUKA, M., PISHCHULIN, L., GEHLER, P., AND SCHIELE, B. 2014. 2D human pose estimation: New benchmark and state of the art analysis. In CVPR.
3. BAAK, A., MÜLLER, M., BHARAJ, G., SEIDEL, H.-P., AND THEOBALT, C. 2011. A data-driven approach for real-time full body pose reconstruction from a depth camera. In ICCV.
4. BELAGIANNIS, V., AMIN, S., ANDRILUKA, M., SCHIELE, B., NAVAB, N., AND ILIC, S. 2014. 3D pictorial structures for multiple human pose estimation. In CVPR.
5. BLANZ, V., AND VETTER, T. 1999. A morphable model for the synthesis of 3D faces. In SIGGRAPH.
6. BREGLER, C., AND MALIK, J. 1998. Tracking people with twists and exponential maps. In CVPR.
7. BURENIUS, M., SULLIVAN, J., AND CARLSSON, S. 2013. 3D pictorial structures for multiple view articulated pose estimation. In CVPR.
8. CEREZO, E., PÉREZ, F., PUEYO, X., SERON, F. J., AND SILLION, F. X. 2005. A survey on participating media rendering techniques. The Visual Computer 21, 5, 303-328.
9. CHAI, J., AND HODGINS, J. K. 2005. Performance animation from low-dimensional control signals. ACM Transactions on Graphics 24, 3, 686-696.
10. CHEN, X., AND YUILLE, A. L. 2014. Articulated pose estimation by a graphical model with image dependent pairwise relations. In NIPS.
11. EGOCAP, 2016. EgoCap dataset. http://gvv.mpi-inf.mpg.de/projects/EgoCap/.
12. ELHAYEK, A., DE AGUIAR, E., JAIN, A., TOMPSON, J., PISHCHULIN, L., ANDRILUKA, M., BREGLER, C., SCHIELE, B., AND THEOBALT, C. 2015. Efficient ConvNet-based markerless motion capture in general scenes with a low number of cameras. In CVPR.
13. FATHI, A., FARHADI, A., AND REHG, J. M. 2011. Understanding egocentric activities. In ICCV.
14. GALL, J., ROSENHAHN, B., BROX, T., AND SEIDEL, H.-P. 2010. Optimization and filtering for human motion capture. International Journal of Computer Vision 87, 1-2, 75-92.
15. HA, S., BAI, Y., AND LIU, C. K. 2011. Human motion reconstruction from force sensors. In SCA.
16. HE, K., ZHANG, X., REN, S., AND SUN, J. 2016. Deep residual learning for image recognition. In CVPR.
17. HOLTE, M. B., TRAN, C., TRIVEDI, M. M., AND MOESLUND, T. B. 2012. Human pose estimation and activity recognition from multi-view videos: Comparative explorations of recent developments. IEEE Journal of Selected Topics in Signal Processing 6, 5, 538-552.
18. INSAFUTDINOV, E., PISHCHULIN, L., ANDRES, B., ANDRILUKA, M., AND SCHIELE, B. 2016. DeeperCut: A deeper, stronger, and faster multi-person pose estimation model. In ECCV.
19. JAIN, A., TOMPSON, J., ANDRILUKA, M., TAYLOR, G. W., AND BREGLER, C. 2014. Learning human pose estimation features with convolutional networks. In ICLR.
20. JAIN, A., TOMPSON, J., LECUN, Y., AND BREGLER, C. 2015. MoDeep: A deep learning framework using motion features for human pose estimation. In ACCV.
21. JIANG, H., AND GRAUMAN, K. 2016. Seeing invisible poses: Estimating 3D body pose from egocentric video. arXiv:1603.07763.
22. JOHNSON, S., AND EVERINGHAM, M. 2011. Learning effective human pose estimation from inaccurate annotation. In CVPR.
23. JONES, A., FYFFE, G., YU, X., MA, W.-C., BUSCH, J., ICHIKARI, R., BOLAS, M., AND DEBEVEC, P. 2011. Head-mounted photometric stereo for performance capture. In CVMP.
24. JOO, H., LIU, H., TAN, L., GUI, L., NABBE, B., MATTHEWS, I., KANADE, T., NOBUHARA, S., AND SHEIKH, Y. 2015. Panoptic studio: A massively multiview system for social motion capture. In ICCV.
25. KIM, D., HILLIGES, O., IZADI, S., BUTLER, A. D., CHEN, J., OIKONOMIDIS, I., AND OLIVIER, P. 2012. Digits: Freehand 3D interactions anywhere using a wrist-worn gloveless sensor. In UIST.
26. KITANI, K. M., OKABE, T., SATO, Y., AND SUGIMOTO, A. 2011. Fast unsupervised ego-action learning for first-person sports videos. In CVPR.
27. LOPER, M., MAHMOOD, N., AND BLACK, M. J. 2014. MoSh: Motion and shape capture from sparse markers. ACM Transactions on Graphics 33, 6, 220:1-13.
28. MA, M., FAN, H., AND KITANI, K. M. 2016. Going deeper into first-person activity recognition. In CVPR.
29. MEKA, A., ZOLLHÖFER, M., RICHARDT, C., AND THEOBALT, C. 2016. Live intrinsic video. ACM Transactions on Graphics 35, 4, 109:1-14.
30. MENACHE, A. 2010. Understanding Motion Capture for Computer Animation, 2nd ed. Morgan Kaufmann.
31. MOESLUND, T. B., HILTON, A., KRÜGER, V., AND SIGAL, L., Eds. 2011. Visual Analysis of Humans: Looking at People. Springer.
32. MOULON, P., MONASSE, P., AND MARLET, R. 2013. Global fusion of relative motions for robust, accurate and scalable structure from motion. In ICCV.
33. MURRAY, R. M., SASTRY, S. S., AND ZEXIANG, L. 1994. A Mathematical Introduction to Robotic Manipulation. CRC Press.
34. NEWELL, A., YANG, K., AND DENG, J. 2016. Stacked hourglass networks for human pose estimation. arXiv:1603.06937.
35. OHNISHI, K., KANEHIRA, A., KANEZAKI, A., AND HARADA, T. 2016. Recognizing activities of daily living with a wrist-mounted camera. In CVPR.
36. PARK, S. I., AND HODGINS, J. K. 2008. Data-driven modeling of skin and muscle deformation. ACM Transactions on Graphics 27, 3, 96:1-6.
37. PARK, H. S., JAIN, E., AND SHEIKH, Y. 2012. 3D social saliency from head-mounted cameras. In NIPS.
38. PFISTER, T., CHARLES, J., AND ZISSERMAN, A. 2015. Flowing ConvNets for human pose estimation in videos. In ICCV.
39. PISHCHULIN, L., INSAFUTDINOV, E., TANG, S., ANDRES, B., ANDRILUKA, M., GEHLER, P., AND SCHIELE, B. 2016. Deep-Cut: Joint subset partition and labeling for multi person pose estimation. In CVPR.
40. PONS-MOLL, G., BAAK, A., HELTEN, T., MÜLLER, M., SEIDEL, H.-P., AND ROSENHAHN, B. 2010. Multisensor-fusion for 3D full-body human motion capture. In CVPR.
41. PONS-MOLL, G., BAAK, A., GALL, J., LEAL-TAIXÉ, L., MÜLLER, M., SEIDEL, H.-P., AND ROSENHAHN, B. 2011. Outdoor human motion capture using inverse kinematics and von Mises-Fisher sampling. In ICCV.
42. PONS-MOLL, G., FLEET, D. J., AND ROSENHAHN, B. 2014. Posebits for monocular human pose estimation. In CVPR.
43. RHINEHART, N., AND KITANI, K. M. 2016. Learning action maps of large environments via first-person vision. In CVPR.
44. RHODIN, H., ROBERTINI, N., RICHARDT, C., SEIDEL, H.-P., AND THEOBALT, C. 2015. A versatile scene model with differentiable visibility applied to generative pose estimation. In ICCV.
45. RHODIN, H., ROBERTINI, N., CASAS, D., RICHARDT, C., SEIDEL, H.-P., AND THEOBALT, C. 2016. General automatic human shape and motion capture using volumetric contour cues. In ECCV.
46. ROGEZ, G., KHADEMI, M., SUPANCIC, III, J. S., MONTIEL, J. M. M., AND RAMANAN, D. 2014. 3D hand pose detection in egocentric RGB-D images. In ECCV Workshops.
47. SAPP, B., AND TASKAR, B. 2013. MODEC: Multimodal decomposable models for human pose estimation. In CVPR.
48. SCARAMUZZA, D., MARTINELLI, A., AND SIEGWART, R. 2006. A toolbox for easily calibrating omnidirectional cameras. In IROS.
49. SHIRATORI, T., PARK, H. S., SIGAL, L., SHEIKH, Y., AND HODGINS, J. K. 2011. Motion capture from body-mounted cameras. ACM Transactions on Graphics 30, 4, 31:1-10.
50. SHOTTON, J., FITZGIBBON, A., COOK, M., SHARP, T., FINOC-CHIO, M., MOORE, R., KIPMAN, A., AND BLAKE, A. 2011. Real-time human pose recognition in parts from single depth images. In CVPR.
51. SIGAL, L., BALAN , A. O., AND BLACK, M. J. 2010. HumanEva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion. International Journal of Computer Vision, 87, 4-27.
52. SIGAL, L., ISARD, M., HAUSSECKER, H., AND BLACK, M. J. 2012. Loose-limbed people: Estimating 3D human pose and motion using non-parametric belief propagation. International Journal of Computer Vision 98, 1, 15-48.
53. SRIDHAR, S., MUELLER, F., OULASVIRTA, A., AND THEOBALT, C. 2015. Fast and robust hand tracking using detection-guided optimization. In CVPR.
54. STOLL, C., HASLER, N., GALL, J., SEIDEL, H.-P., AND THEOBALT, C. 2011. Fast articulated motion tracking using a sums of Gaussians body model. In ICCV.
55. SU, Y.-C., AND GRAUMAN, K. 2016. Detecting engagement in egocentric video. In ECCV.
56. SUGANO, Y., AND BULLING, A. 2015. Self-calibrating head-mounted eye trackers using egocentric visual saliency. In UIST.
57. TAUTGES, J., ZINKE, A., KRÜGER, B., BAUMANN, J., WEBER, A., HELTEN, T., MÜLLER, M., SEIDEL, H.-P., AND EBER-HARDT, B. 2011. Motion reconstruction using sparse accelerometer data. ACM Transactions on Graphics 30, 3, 18:1-12.
58. TEKIN, B., ROZANTSEV, A., LEPETIT, V., AND FUA, P. 2016. Direct prediction of 3D body poses from motion compensated sequences. In CVPR.
59. THEOBALT, C., DE AGUIAR, E., STOLL, C., SEIDEL, H.-P., AND THRUN, S. 2010. Performance capture from multi-view video. In Image and Geometry Processing for 3-D Cinematography, R. Ronfard and G. Taubin, Eds. Springer, 127-149.
60. TOMPSON, J. J., JAIN, A., LECUN, Y., AND BREGLER, C. 2014. Joint training of a convolutional network and a graphical model for human pose estimation. In NIPS.
61. TOSHEV, A., AND SZEGEDY, C. 2014. DeepPose: Human pose estimation via deep neural networks. In CVPR.
62. URTASUN, R., FLEET, D. J., AND FUA, P. 2006. Temporal motion models for monocular and multiview 3D human body tracking. Computer Vision and Image Understanding 104, 2, 157-177.
63. VLASIC, D., ADELSBERGER, R., VANNUCCI, G., BARNWELL, J., GROSS, M., MATUSIK, W., AND POPOVIĆ, J. 2007. Practical motion capture in everyday surroundings. ACM Transactions on Graphics 26, 3, 35.
64. WANG, R. Y., AND POPOVIĆ, J. 2009. Real-time hand-tracking with a color glove. ACM Transactions on Graphics 28, 3, 63.
65. WANG, J., CHENG, Y., AND FERIS, R. S. 2016. Walk and learn: Facial attribute representation learning from egocentric video and contextual data. In CVPR.
66. WEI, X., ZHANG, P., AND CHAI, J. 2012. Accurate realtime full-body motion capture using a single depth camera. ACM Transactions on Graphics 31, 6, 188:1-12.
67. WEI, S.-E., RAMAKRISHNA, V., KANADE, T., AND SHEIKH, Y. 2016. Convolutional pose machines. In CVPR.
68. YANG, Y., AND RAMANAN, D. 2013. Articulated human detection with flexible mixtures of parts. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 12, 2878-2890.
69. YASIN, H., IQBAL, U., KRÜGER, B., WEBER, A., AND GALL, J. 2016. A dual-source approach for 3D pose estimation from a single image. In CVPR.
70. YIN, K., AND PAI, D. K. 2003. Footsee: an interactive animation system. In SCA.
71. YONEMOTO, H., MURASAKI, K., OSAWA, T., SUDO, K., SHIMA-MURA, J., AND TANIGUCHI, Y. 2015. Egocentric articulated pose tracking for action recognition. In International Conference on Machine Vision Applications (MVA).
72. ZHANG, P., SIU, K., ZHANG, J., LIU, C. K., AND CHAI, J. 2014. Leveraging depth cameras and wearable pressure sensors for full-body kinematics and dynamics capture. ACM Transactions on Graphics 33, 6, 221:1-14.