Enhanced personal autostereoscopic telepresence system using commodity depth cameras

Computers and Graphics (Pergamon)

Volumn 36, Issue 7, 2012, Pages 791-807

  Maimone, Andrew ^a   Bidwell, Jonathan ^a   Peng, Kun ^a   Fuchs, Henry ^a  

^a Brooks Computer Science Building   (United States)

Author keywords

Camera calibration; Color calibration; Filtering; Sensor fusion; Teleconferencing; Tracking

Indexed keywords

CALIBRATION; CAMERAS; COLOR; FILTRATION; GEOMETRICAL OPTICS; IMAGE ENHANCEMENT; MERGING; PIPELINE PROCESSING SYSTEMS; RENDERING (COMPUTER GRAPHICS); SURFACE DISCHARGES; TELECONFERENCING; THREE DIMENSIONAL COMPUTER GRAPHICS; THREE DIMENSIONAL DISPLAYS; VISUAL COMMUNICATION;

AUTO-STEREOSCOPIC DISPLAY; AUTOMATIC COLOR CALIBRATION; CAMERA CALIBRATION; COLOR CALIBRATION; MARKERLESS TRACKING SYSTEMS; MULTIPLE DEPTH CAMERAS; SENSOR FUSION; SOFTWARE AND HARDWARES;

STEREO IMAGE PROCESSING;

EID: 84865477781     PISSN: 00978493     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cag.2012.04.011     Document Type: Conference Paper

References (30)

1. Raskar R, Welch G, Cutts M, Lake A, Stesin L, Fuchs H. The office of the future: a unified approach to image-based modeling and spatially immersive displays. In: Proceedings of the 25th annual conference on computer graphics and interactive techniques. SIGGRAPH 98. New York, NY, USA: ACM. ISBN 0-89791-999-8; 1998. p. 179-88. http://doi.acm.org/10.1145/280814.280861.
2. S.J. Gibbs, C. Arapis, and C.J. Breiteneder Teleport - towards immersive copresence Multimedia Syst 7 1999 214 221 http://dx.doi.org/10.1007/ s005300050123
3. W. Matusik, and H. Pfister 3d tv: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes ACM Trans Graph 23 2004 814 824 http://doi.acm.org/10.1145/1015706.1015805
4. Schreer O, Feldmann I, Atzpadin N, Eisert P, Kauff P, Belt H. 3D presence - a system concept for multi-user and multi-party immersive 3d videoconferencing. In: Fifth European conference on visual media production (CVMP 2008); 2008. p. 1-8.
5. A. Jones, M. Lang, G. Fyffe, X. Yu, J. Busch, and I. McDowall Achieving eye contact in a one-to-many 3d video teleconferencing system ACM Trans Graph 28 2009 http://doi.acm.org/10.1145/1531326.1531370
6. Balogh T, Kovács PT. Real-time 3d light field transmission. In: Real-time image and video processing 2010, SPIE, vol. 7724; 2010.
7. Kuster C, Popa T, Zach C, Gotsman C, Gross M. Freecam: a hybrid camera system for interactive free-viewpoint video. In: Proceedings of vision, modeling, and visualization (VMV); 2011.
8. M. Gross, S. Würmlin, M. Naef, E. Lamboray, C. Spagno, and A. Kunz blue-c: a spatially immersive display and 3d video portal for telepresence ACM Trans Graph 22 2003 819 827 http://doi.acm.org/10.1145/882262.882350
9. Kurillo G, Bajcsy R, Nahrsted K, Kreylos O. Immersive 3d environment for remote collaboration and training of physical activities. In: Virtual reality conference, VR 08, IEEE; 2008. p. 269-270. http://dx.doi.org.10.1109/VR.2008. 4480795.
10. H.H. Baker, N. Bhatti, D. Tanguay, I. Sobel, D. Gelb, and M.E. Goss Understanding performance in coliseum, an immersive videoconferencing system ACM Trans Multimedia Comput Commun Appl 1 2005 190 210 http://doi.acm.org/10.1145/ 1062253.1062258
11. B. Petit, J.D. Lesage, C. Menier, J. Allard, J.S. Franco, and B. Raffin Multicamera real-time 3d modeling for telepresence and remote collaboration Int J Digital Multimedia Broadcast 2010 2009
12. Maimone A, Fuchs H. Encumbrance-free telepresence system with real-time 3d capture and display using commodity depth cameras. In: Tenth IEEE international symposium on mixed and augmented reality (ISMAR); 2011. p. 137-46. http://dx.doi.org.10.1109/ISMAR.2011.6092379.
13. L. Muhlbach, M. Bocker, and A. Prussog Telepresence in videocommunications: a study on stereoscopy and individual eye contact Hum Factors 37 1995 16
14. Freedman B, Shpunt A, Machline M, Arieli Y. Depth mapping using 1032 projected patterns. Patent Application US 2010/0118123 A1; 2010.
15. Meehan M. Physiological reaction as an objective measure of presence in virtual environments. Ph.D. thesis; University of North Carolina at Chapel Hill; 2001.
16. Zhang Z. Flexible camera calibration by viewing a plane from unknown orientations. In: The proceedings of the seventh IEEE international conference on computer vision, 1999; vol. 1; 1999. http://dx.doi.org.10.1109/ICCV.1999. 791289.
17. Maimone A, Fuchs H. A first look at a telepresence system with room-sized real-time 3d capture and large tracked display. In: The 21st international conference on artificial reality and telexistence (ICAT); 2011.
18. Berger K, Ruhl K, Brümmer C, Schröder Y, Scholz A, Magnor M. Markerless motion capture using multiple color-depth sensors. In: Proceedings of vision, modeling and visualization (VMV); 2011.
19. Merrell P, Akbarzadeh A, Wang L, Mordohai P, Frahm JM, Yang R, et al. Real-time visibility-based fusion of depth maps. In: IEEE 11th international conference on computer vision, 2007. ICCV 2007; 2007. p. 1-8. http://dx.doi.org.10.1109/ICCV.2007.4408984.
20. McGuire M. A fast, small-radius GPU median filter. In: ShaderX6. 2008, 〈 http://graphics.cs.williams.edu/papers/MedianShaderX6/ 〉.
21. Tomasi C, Manduchi R. Bilateral filtering for gray and color images. In: Sixth international conference on computer vision; 1998, p. 839-46. http://dx.doi.org.10.1109/ICCV.1998.710815.
22. Kawata H, Kanai T. Image-based point rendering for multiple range images. In: Proceedings of the 2nd international conference on information technology for application (ICITA 2004). Sydney: Macquarie Scientific Publishing; 2004, p. 478-83.
23. Ilie A, Welch G. Ensuring color consistency across multiple cameras. In: Proceedings of the tenth IEEE international conference on computer vision - volume 2. ICCV 05; Washington, DC, USA: IEEE Computer Society; 2005, p. 1268-75. ISBN 0-7695-2334-X-02. http://dx.doi.org/10.1109/ICCV.2005.88.
24. M.A. Fischler, and R.C. Bolles Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography Commun ACM 24 1981 381 395 http://doi.acm.org/10.1145/358669.358692
25. Cai Q, Gallup D, Zhang C, Zhang Z. 3d deformable face tracking with a commodity depth camera. In: Proceedings of the 11th European conference on computer vision conference on Computer vision: Part III. ECCV10; Berlin, Heidelberg: Springer-Verlag; 2010, p. 229-42. ISBN 3-642-15557-X, 978-3-642-15557-4 〈 http://portal.acm.org/citation.cfm?id=1927006.1927026 〉.
26. Viola P, Jones M. Rapid object detection using a boosted cascade of simple features. In: CVPR 2001. Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition, vol. 1; 2001. http://dx.doi.org.10.1109/CVPR.2001.990517.
27. Lienhart R, Maydt J. An extended set of haar-like features for rapid object detection. In: Proceedings of the international conference on image processing, vol. 1; 2002. http://dx.doi.org.10.1109/ICIP.2002.1038171.
28. M. Castrillón Santana, O. Déniz Suárez, M. Hernández Tejera, and C. Guerra Artal Encara2: Real-time detection of multiple faces at different resolutions in video streams J Visual Commun Image Representation 2007 130 140
29. Ye G, State A, Fuchs H. A practical multi-viewer tabletop autostereoscopic display. In: Ninth IEEE international symposium on mixed and augmented reality (ISMAR); 2010. p. 147-56. http://dx.doi.org.10.1109/ISMAR. 2010.5643563.
30. Peterka T, Kooima R, Girado J, Ge J, Sandin D, Johnson A, et al., Dynallax: solid state dynamic parallax barrier autostereoscopic VR display. In: Virtual reality conference, 2007, VR 07, IEEE: 2007, p. 155-62. http://dx.doi.org.10.1109/VR.2007.352476.