Real-time 3D human capture system for mixed-reality art and entertainment

IEEE Transactions on Visualization and Computer Graphics

Volumn 11, Issue 6, 2005, Pages 706-720

  Nguyen, Ta Huynh Duy ^a,b   Qui, Tran Cong Thien ^a,b   Xu, Ke ^a,b   Cheok, Adrian David ^a,b,d,e   Teo, Sze Lee ^b   Zhou, Zhiying ^b,e   Mallawaarachchi, Asitha ^b   Lee, Shang Ping ^b   Liu, Wei ^b   Teo, Hui Siang ^b   Thang, Le Nam ^b   Li, Yu ^b   Kato, Hirokazu ^c,f  

^a IEEE   (Singapore)

^b NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^c HIROSHIMA CITY UNIVERSITY   (Japan)

^d NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^e ACM SIGCHI

^f OSAKA UNIVERSITY   (Japan)

Author keywords

3D viewpoint; Art; Entertainment; Mixed reality; Tangible interaction

Indexed keywords

3D HUMAN CAPTURE SYSTEMS; 3D VIEWPOINT; ENTERTAINMENT; MIXED REALITY ART; TANGIBLE INTERACTION;

ALGORITHMS; ANIMATION; ARTS COMPUTING; HUMAN COMPUTER INTERACTION; PERSONAL COMPUTERS; PROGRAM PROCESSORS; VIRTUAL REALITY;

REAL TIME SYSTEMS;

ART; ARTICLE; BIOLOGICAL MODEL; COMPUTER ASSISTED DIAGNOSIS; COMPUTER INTERFACE; COMPUTER SIMULATION; COMPUTER SYSTEM; ENVIRONMENT; EQUIPMENT; EQUIPMENT DESIGN; HUMAN; INFORMATION PROCESSING; INSTRUMENTATION; LEISURE; MAN MACHINE INTERACTION; METHODOLOGY; MULTIMEDIA; ONLINE SYSTEM; SIGNAL PROCESSING; THREE DIMENSIONAL IMAGING;

ART; COMPUTER SIMULATION; COMPUTER SYSTEMS; DATA DISPLAY; ENVIRONMENT; EQUIPMENT DESIGN; EQUIPMENT FAILURE ANALYSIS; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGING, THREE-DIMENSIONAL; LEISURE ACTIVITIES; MAN-MACHINE SYSTEMS; MODELS, BIOLOGICAL; MULTIMEDIA; ONLINE SYSTEMS; SIGNAL PROCESSING, COMPUTER-ASSISTED; USER-COMPUTER INTERFACE;

EID: 27644575944     PISSN: 10772626     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVCG.2005.105     Document Type: Article

References (33)

1. ARToolKit, http://www.hitl.washington.edu/artoolkit/, 2005.
2. CAVE Quake II, http://brighton.ncsa.uiuc.edu/prajlich/caveQuake/, 2005.
3. MXRToolkit, http://sourceforge.net/projects/mxrtoolkit/, 2005.
4. OpenCV, http://sourceforge.net/projects/opencvlibrary/, 2005.
5. Point Grey Research Inc., http://www.ptgrey.com, 2005.
6. RGB "Bayer" Color and MicroLenses, http://www.siliconimaging. com/RGB Bayer.htm, 2005.
7. A. Amory, K. Naicker, J. Vincent, and C. Adams, "The Use of Computer Games as an Educational Tool: Identification of Appropriate Game Types and Elements," pp. 311-321, British J. Educational Technology, 1999.
8. R. Azuma, "A Survey of Augmented Reality," Presence, 1997.
9. R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. MacIntyre, "Recent Advances in Augmented Reality," IEEE Computer Graphics and Applications, Nov./Dec. 2001.
10. S. Bjork, J. Falk, R. Hansson, and P. Ljungstrand, "Pirates! - Using the Physical World as a Game Board," Proc. Interact 2001, IFIP TC. 13 Conf. Human-Computer Interaction, 2001.
11. A.D. Cheok, K.H. Goh, W. Liu, F. Farzbiz, S.W. Fong, S.Z. Teo, Y. Li, and X. Yang, "Human Pacman: A Mobile Wide-Area Entertainment System Based on Physical, Social, and Ubiquitous Computing," Personal and Ubiquitous Computing, vol. 8, no. 2, pp. 71-81, May 2004.
12. A.D. Cheok, X. Yang, Z. Zhou, M. Billinghurst, and H. Kato, "Touch-Space: Mixed Reality Game Space Based on Ubiquitous, Tangible, and Social Computing," J. Personal and Ubiquitous Computing, pp. 430-442, 2002.
13. J.M. Hasenfratz, M. Lapierre, and F. Sillion, "A Real-Time System for Full Body Interaction," Virtual Environments, pp. 147-156, 2004.
14. T. Horprasert et al., "A Statistical Approach for Robust Background Subtraction and Shadow Detection," Proc. IEEE Int'l Conf. Computer Vision (ICCV '99) Frame Rate Workshop, 1999.
15. H. Kato, K. Tachibana, M. Tanabe, T. Nakajima, and Y. Fukuda, "Magiccup: A Tangible Interface for Virtual Objects Manipulation in Table-Top Augmented Reality," Proc. Augmented Reality Toolkit Workshop (ART03), pp. 85-86, 2003.
16. K.N. Kutulakos and S.M. Seitz, "A Theory of Shape by Space Carving," Int'l J. Computer Vision, vol. 38, no. 3, 2000.
17. S.P. Lee, F. Farbiz, and A.D. Cheok, "Touchy Internet: A Cybernetic System for Human-Pet Interaction through the Internet," SIGGRAPH 2003, Sketches and Application, 2003.
18. T.W. Malone, "Toward a Theory of Intrinsically Motivating Instruction," vol. 5, pp. 333-369, 1981.
19. R.L. Mandryk and K.M. Inkpen, "Supporting Free Play in Ubiquitous Computer Games," Proc. Workshop Designing Ubiquitous Computer Games (UbiComp), 2001.
20. G. Markus, W. Stephan, N. Martin, L. Edouard, S. Christian, K. Andreas, K.M. Esther, S. Tomas, V.G. Luc, L. Silke, S. Kai, V.M. Andrew, and S. Oliver, "blue-c: A Spatially Immersive Display and 3D Video Portal for Telepresence," Proc. ACM SIGGRAPH, 2003.
21. W. Matusik, C. Buehler, and L. McMillan, "Polyhedral Visual Hulls for Real-Time Rendering," Proc. 12th Eurographics Workshop Rendering Techniques, 2001.
22. R. Mester, T. Aach, and L. Dmbgen, "Illumination-Invariant Change Detection Using Statistical Colinearity Criterion," Proc. DAGM Symp. Pattern Recognition, pp. 170-177, 2001.
23. T. Oshima, K. Satoh, H. Yamamoto, and H. Tamura, "Ar2 Hockey System: A Collaboration Mixed Reality System," vol. 3, no. 2, pp. 55-60, Trans. VRSJ, 1998.
24. S.J.D. Prince, A.D. Cheok, F. Farbiz, T. Williamson, N. Johnson, M. Billinghurst, and H. Kato, "Live 3-Dimensional Content for Augmented Reality," IEEE Trans. Multimedia, submitted.
25. S.J.D. Prince, A.D. Cheok, F. Farbiz, T. Williamson, N. Johnson, M. Billinghurst, and H. Kato, "3D Live: Real Time Captured Content for Mixed Reality," Proc. Int'l Symp. Mixed and Augmented Reality (ISMAR), 2002.
26. H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "Rtp: A Transport Protocol for Real-Time Applications," Internet Eng. Task Force, Audio-Video Transport Working Group, 1996.
27. S. Seitz and C. Dyer, "Photorealistic Scene Reconstruction by Voxel Coloring," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 1997.
28. M. Seki, H. Fujiwara, and K. Sumi, "A Robust Background Subtraction Method for Changing Background," Proc. Fifth IEEE Int'l Workshop Applications of Computer Vision, 2000.
29. G. Slabaugh, W.B. Culbertson, T. Malzbender, and R. Schafer, "A Survey of Volumetric Scene Reconstruction Methods from Photographs," Proc. Int'l Workshop Volume Graphics, 2001.
30. G. Slabaugh, R. Schafer, and M. Hans, "Image-Based Photo Hulls," Proc. First Int'l Symp. 3D Data Processing, Visualization, and Transmission, 2002.
31. G.G. Slabaugh, W.B. Culbertson, T. Malzbender, M.R. Stevens, and R.W. Schafer, "Methods for Volumetric Reconstruction of Visual Scenes," Int'l J. Computer Vision, 2004.
32. R. Szeliski, "Video Mosaics for Virtual Environments," IEEE Computer Graphics and Applications, Mar. 1996.
33. H. Tamura, H. Yamamoto, and A. Katayama, "Mixed Reality: Future Dreams Seen at the Border between Real and Virtual Worlds," IEEE Computer Graphics and Applications, vol. 21, no. 6, pp. 64-70, 2001.