Facial performance sensing head-mounted display

ACM Transactions on Graphics

Volumn 34, Issue 4, 2015, Pages

  Li, Hao ^a   Trutoiu, Laura ^b   Olszewski, Kyle ^a   Wei, Lingyu ^a   Trutna, Tristan ^b   Hsieh, Pei Lun ^a   Nicholls, Aaron ^b   Ma, Chongyang ^a  

^a UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^b FACEBOOK   (United States)

Author keywords

Depth camera; Head mounted display; Real time facial performance capture; Strain gauge; Virtual reality; Wearable sensors

Indexed keywords

CAMERAS; INTERACTIVE COMPUTER GRAPHICS; SENSORY PERCEPTION; STRAIN GAGES; STREET TRAFFIC CONTROL; VIRTUAL REALITY; WEARABLE SENSORS;

COMMUNICATION PLATFORMS; DEPTH CAMERA; ELECTRONIC MATERIALS; FACE-TO-FACE INTERACTION; GAUSSIAN MIXTURE DISTRIBUTION; HEAD MOUNTED DISPLAYS; PERFORMANCE CAPTURE; TRADITIONAL TECHNIQUES;

HELMET MOUNTED DISPLAYS;

EID: 84945212482     PISSN: 07300301     EISSN: 15577368     Source Type: Journal    
DOI: 10.1145/2766939     Document Type: Conference Paper

References (43)

1. ABRASH, M., 2012. Latency - the sine qua non of AR and VR. http://blogs.valvesoftware.com/abrash/latency-the-sine-qua-non-of-ar-and-vr/.
2. BHAT, K. S., GOLDENTHAL, R., YE, Y., MALLET, R., AND KOPERWAS, M. 2013. High fidelity facial animation capture and retargeting with contours. In SCA '13, 7-14.
3. BICKEL, B., BOTSCH, M., ANGST, R., MATUSIK, W., OTADUY, M., PFISTER, H., AND GROSS, M. 2007. Multi-scale capture of facial geometry and motion. ACM Trans. Graph. 26, 3.
4. BLANZ, V., AND VETTER, T. 1999. A morphable model for the synthesis of 3d faces. In SIGGRAPH '99, 187-194.
5. BOUAZIZ, S.,WANG, Y., AND PAULY, M. 2013. Online modeling for realtime facial animation. ACM Trans. Graph. 32, 4, 40:1-40:10.
6. BRAND, M. 1999. Voice puppetry. In SIGGRAPH'99, 21-28.
7. BREGLER, C., COVELL, M., AND SLANEY, M. 1997. Video rewrite: Driving visual speech with audio. In SIGGRAPH '97, 353-360.
8. CAO, C., WENG, Y., LIN, S., AND ZHOU, K. 2013. 3d shape regression for real-time facial animation. ACM Trans. Graph. 32, 4, 41:1-41:10.
9. CAO, C., HOU, Q., AND ZHOU, K. 2014. Displaced dynamic expression regression for real-time facial tracking and animation. ACM Trans. Graph. 33, 4, 43:1-43:10.
10. CHAI, J.-X., XIAO, J., AND HODGINS, J. 2003. Vision-based control of 3d facial animation. In SCA '03, 193-206.
11. CHEN, Y.-L., WU, H.-T., SHI, F., TONG, X., AND CHAI, J. 2013. Accurate and robust 3d facial capture using a single rgbd camera. In ICCV, IEEE, 3615-3622.
12. COOTES, T. F., EDWARDS, G. J., AND TAYLOR, C. J. 2001. Active appearance models. IEEE TPAMI 23, 6, 681-685.
13. CRISTINACCE, D., AND COOTES, T. 2008. Automatic feature localisation with constrained local models. Pattern Recogn. 41, 10, 3054-3067.
14. EKMAN, P., AND FRIESEN, W. 1978. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press, Palo Alto.
15. GALES, M. 1998. Maximum likelihood linear transformations for hmm-based speech recognition. Computer Speech and Language 12, 75-98.
16. GARRIDO, P., VALGAERT, L., WU, C., AND THEOBALT, C. 2013. Reconstructing detailed dynamic face geometry from monocular video. ACM Trans. Graph. 32, 6, 158:1-158:10.
17. GRUEBLER, A., AND SUZUKI, K. 2014. Design of a wearable device for reading positive expressions from facial emg signals. Affective Computing, IEEE Transactions on 5, 3, 227-237.
18. HIBBELER, R. 2005. Mechanics of Materials. Prentice Hall, Upper Saddle River.
19. HSIEH, P.-L., MA, C., YU, J., AND LI, H. 2015. Unconstrained realtime facial performance capture. In CVPR, to appear.
20. INTEL, 2014. Intel realsense ivcam. https://software.intel.com/en-us/intel-realsense-sdk.
21. KRAMER, J., AND LEIFER, L. 1988. The talking glove. SIGCAPH Comput. Phys. Handicap., 39, 12-16.
22. LI, H.,WEISE, T., AND PAULY, M. 2010. Example-based facial rigging. ACM Trans. Graph. 29, 4, 32:1-32:6.
23. LI, H., YU, J., YE, Y., AND BREGLER, C. 2013. Realtime facial animation with on-the-fly correctives. ACM Trans. Graph. 32, 4, 42:1-42:10.
24. LUCERO, J. C., AND MUNHALL, K. G. 1999. A model of facial biomechanics for speech production. The Journal of the Acoustical Society of America 106, 5, 2834-2842.
25. MCFARLAND, D. J., AND WOLPAW, J. R. 2011. Brain-computer interfaces for communication and control. Commun. ACM 54, 5, 60-66.
26. NI, 2014. National instruments. http://www.ni.com/whitepaper/3642/en/.
27. OCULUS VR, 2014. Oculus rift dk2. https://www.oculus.com/dk2/.
28. OMEGA, 2014. Omega strain gages. http://www.omega.com/Pressure/pdf/KFH.pdf.
29. PARKE, F. I., AND WATERS, K. 1996. Computer Facial Animation. A. K. Peters.
30. PIGHIN, F., AND LEWIS, J. P. 2006. Performance-driven facial animation. In ACM SIGGRAPH 2006 Courses, SIGGRAPH '06.
31. RASMUSSEN, C. E., AND WILLIAMS, C. K. I. 2005. Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press.
32. RUSINKIEWICZ, S., AND LEVOY, M. 2001. Efficient variants of the icp algorithm. In International Conference on 3-D Digital Imaging and Modeling, IEEE, 145-152.
33. SARAGIH, J. M., LUCEY, S., AND COHN, J. F. 2011. Deformable model fitting by regularized landmark mean-shift. International Journal of Computer Vision 91, 2, 200-215.
34. SCHEIRER, J., FERNANDEZ, R., AND PICARD, R. W. 1999. Expression glasses: A wearable device for facial expression recognition. In CHI EA '99, 262-263.
35. SEKITANI, T., KALTENBRUNNER, M., YOKOTA, T., AND SOMEYA, T. 2014. Imperceptible electronic skin. SID Symposium Digest of Technical Papers 45, 1, 122-125.
36. SHI, F., WU, H.-T., TONG, X., AND CHAI, J. 2014. Automatic acquisition of high-fidelity facial performances using monocular videos. ACM Trans. Graph. 33, 6, 222:1-222:13.
37. SIFAKIS, E., NEVEROV, I., AND FEDKIW, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. 24, 3, 417-425.
38. SMI, 2014. Sensomotoric instruments. http://www.smivision.com/.
39. SUGIMOTO, T., FUKUSHIMA, M., AND IBARAKI, T. 1995. A parallel relaxation method for quadratic programming problems with interval constraints. Journal of Computational and Applied Mathematics 60, 12, 219-236.
40. SUMNER, R.W., AND POPOVIĆ , J. 2004. Deformation transfer for triangle meshes. ACM Trans. Graph. 23, 3, 399-405.
41. TERZOPOULOS, D., AND WATERS, K. 1990. Physically-based facial modelling, analysis, and animation. The Journal of Visualization and Computer Animation 1, 2, 73-80.
42. WEISE, T., BOUAZIZ, S., LI, H., AND PAULY, M. 2011. Realtime performance-based facial animation. ACM Trans. Graph. 30, 4, 77:1-77:10.
43. XIONG, X., AND DE LA TORRE, F. 2013. Supervised descent method and its application to face alignment. In CVPR, IEEE, 532-539.