Virtual mirror rendering with stationary RGB-D cameras and stored 3-D background

IEEE Transactions on Image Processing

Volumn 22, Issue 9, 2013, Pages 3433-3448

  Shen, Ju ^a   Su, Po Chang ^a   Cheung, Sen Ching Samson ^a   Zhao, Jian ^a  

^a UNIVERSITY OF KENTUCKY   (United States)

Author keywords

3 D scene scanning; client server systems; depth image denoising; image denoising; image reconstruction; Markov random field; Mirrors; RGB D system

Indexed keywords

CLIENT-SERVER SYSTEMS; COMPLETION ALGORITHMS; FOREGROUND/BACKGROUND; MARKOV RANDOM FIELDS; MEDICAL INTERVENTION; PROBABILISTIC GRAPHICAL MODELS; REAL TIME PERFORMANCE; RGB-D SYSTEM;

CAMERAS; IMAGE DENOISING; IMAGE RECONSTRUCTION; MARKOV PROCESSES; MIRRORS; MODELS; THREE DIMENSIONAL COMPUTER GRAPHICS; VISUAL COMMUNICATION;

THREE DIMENSIONAL;

EID: 84880546020     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2013.2268941     Document Type: Article

References (45)

1. A. Taguchi, T. Aoki, and H. Yasuda, "A study on real-time virtual clothing system based on two-dimensional plane model," in Proc. 6th APSITT, Nov. 2005, pp. 126-130
2. J. Ehara and H. Saito, "Texture overlay for virtual clothing based on PCA of silhouettes," in Proc. 5th IEEE/ACM Int. Symp. Mixed Augmented Reality, Oct. 2006, pp. 139-142
3. P. Eisert, P. Fechteler, and J. Rurainsky, "3-D tracking of shoes for virtual mirror applications," in Proc. Int. Conf. Comput. Vis. Pattern Recognit., Jun. 2008, pp. 1-6
4. V. S. Ramachandran, D. C. Rogers-Ramachandran, and S. Cobb, "Touching the phantom," Nature, vol. 377, no. 6549, pp. 489-490, Oct. 1995
5. V. S. Ramachandran and E. L. Altschuler, "The use of visual feedback, in particular mirror visual feedback, in restoring brain functions," Brain, vol. 132, no. 7, pp. 1693-1710, Jun. 2009
6. P. W. Dowrick, "Self-modeling," in Using Video: Psychological and Social Applications. New York, NY, USA: Wiley, 1983
7. T. Buggey, Seeing Is Believing: Video Self Modeling for People with Autism and Other Developmental Disabilities, Bethesda, MD, USA: Wodbine House, 2009
8. C. H. Hitchcock, P. W. Dowrick, and M. A. Prater, "Video self-modeling intervention in school-based settings: A review," Remed. Special Educ., vol. 24, no. 1, pp. 36-45, Jan./Feb. 2003
9. K. Khoshelham, "Accuracy analysis of kinect depth data," in Proc. ISPRS Workshop Laser Scanning, 2011, p. 1
10. R. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohli, J. Shotton, S. Hodges, and A. Fitzgibbon, "KinectFusion: Real-time dense surface mapping and tracking," in Proc. 10th IEEE Int. Symp. Mixed Augmented Reality, Oct. 2011, pp. 127-136
11. J. Shen, S-C S. Cheung, and J. Zhao, "Virtual mirror by fusing multiple RGB-D cameras," in Proc. APSIPA Ann. Summit Conf., Dec. 2012, pp. 1-9
12. M. Biehl, A. Ghosh, and B. Hammer, "Learning vector quantization: The dynamics of winner-takes-all algorithms," Neurocomputing, vol. 69, nos. 7-9, pp. 660-670, Mar. 2006
13. V. Kitanovski and E. Izquierdo, "3-D tracking of facial features for augmented reality applications," in Proc. Int. Workshop Image Anal. Multimedia Interact. Services, vol. 2. Apr. 2011
14. C. Cullinan and S. Agamanolis, "Reflexion: A responsive virtual mirror for interpersonal communication," in Proc. Eur. Conf. Comput. Supported Cooperat. Work, 2003
15. V. Kitanovski and E. Izquierdo, "Augmented reality mirror for virtual facial alterations," in Proc. ICIP, Sep. 2011, pp. 1093-1096
16. A. R. J. Francois and E.-Y. E. Kang, "A handheld mirror simulation," in Proc. Int. Conf. Multimedia Expo, vol. 2. Jul. 2003, pp. 745-748
17. A. Hilsmann and P. Eisert, "Realistic cloth augmentation in single view video," in Proc. Vis., Modell., Visualizat. Workshop, 2009, pp. 55-62
18. L. Wang, R. Villamil, S. Samarasekera, and R. Kumar, "Magic mirror: A virtual handbag shopping system," in Proc. IEEE Comput. Soc. Conf. CVPRW, Jun. 2012, pp. pp. 19-24
19. M. Straka, S. Hauswiesner, M. Ruther, and H. Bischof, "A freeviewpoint virtual mirror with marker-less user interaction," in Proc. 17th SCIA, May 2011, pp. 635-645
20. D. Porquet, J. M. Dischler, and D. Ghazanfarpour, "Real-time highquality view-dependent texture mapping using per-pixel visibility," in Proc. 3rd Int. Conf. Comput. Graph. Interact. Tech. Australasia South East Asia, 2005, pp. 213-220
21. C. Eisenacher, Q. Meyer, and C. Loop, "Real-time view-dependent rendering of parametric surfaces," in Proc. Symp. Interact. 3-D Graph. Games, 2009, pp. 137-143
22. P. Debevec, Y. Yu, and G. Boshokov, "Efficient view-dependent imagebased rendering with projective texture-mapping," EECS Dept., Univ. California at Berkeley, Berkeley, CA, USA, Tech. Rep. UCB/CSD-98- 1003, Jun. 1998
23. V. Garro, C. dal Mutto, P. Zanuttigh, and G. M. Cortelazzo, "A novel interpolation scheme for range data with side information," in Proc. IEEE Vis. Media Prod. Conf., Nov. 2009, pp. 52-60
24. J. Park, H. Kim, Y. W. Tai, M. S. Brown, and I. Kweon, "High quality depth map upsampling for 3-D-TOF cameras," in Proc. IEEE ICCV, Nov. 2011, pp. 1623-1630
25. L. Wang, H. Jin, R. Yang, and M. Gong, "Stereoscopic inpainting: Joint color and depth completion from stereo images," in Proc. IEEE CVPR, Jun. 2008, pp. 1-8
26. Q. Zhang, M. Ye, R. Yang, Y. Matsushita, B. Wilburn, and H. Yu, "Edgepreserving photometric stereo via depth fusion," in Proc. IEEE CVPR, Jun. 2012, pp. 2472-2479
27. M. Camplani and L. Salgado, "Efficient spatio-temporal hole filling strategy for kinect depth maps," Proc. SPIE, pp. 82900E-1-82900E-10, 2012, doi:10.1117/12.911909
28. J. Kopf, M. F. Cohen, D. Lischinski, and M. Uyttendaele, "Joint bilateral upsampling," ACM Trans. Graph., vol. 26, no. 3, p. 26, 2007
29. B. Freedman, A. Shpunt, M. Machline, and Y. Arieli, "Depth mapping using projected patterns," U.S. Patent 20 100 118 123, Nov. 30, 2010
30. J. H. Cho, S. Y. Kim, Y. S. Ho, and K. Lee, "Dynamic 3-D human actor generation method using a time-of-flight depth camera," IEEE Trans. Consum. Electron., vol. 54, no. 4, pp. 1514-1521, Nov. 2008
31. R. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohli, J. Shotton, S. Hodges, and A. Fitzgibbon, "KinectFusion: Real-time dense surface mapping and tracking," in Proc. IEEE ISMAR, Oct. 2011, pp. 127-136
32. A. Maimone and H. Fuchs, "Reducing interference between multiple structured light depth sensors using motion," in Proc. VRW, Mar. 2012, pp. 51-54
33. W. T. Freeman, E. C. Pasztor, and O. T. Carmichael, "Learning low-level vision," Int. J. Comput. Vis., vol. 40, no. 1, pp. 25-47, 2000
34. M. Harville, "A framework for high-level feedback to adaptive, per-pixel, mixture-of-gaussian background models," in Proc. ECCV, May 2002, pp. pp. 37-49
35. B. Curless and M. Levoy, "A volumetric method for building complex models from range images," in Proc. 23rd Ann. Conf. Comput. Graph. Interact. Tech., 1996, pp. 303-312
36. J. Besl and N. McKay, "A method for registration of 3-D shapes," IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, no. 2, pp. 239-256, Feb. 1992
37. D. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, Nov. 2004
38. M. Fischler and R. Bolles, "Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography," Commun. ACM, vol. 24, no. 6, pp. 381-395, Jun. 1981
39. J. Amanatides and A. Woo, "A fast voxel traversal algorithm for ray tracing," in Proc. Eurographics, 1987, pp. 3-10
40. C. Yan, Y. Wang, and Z. Zhang, "A novel real-time eye detection in human-computer interaction," in Proc. Int. Conf. Innovative Comput. Informat., Sep. 2011, pp. 530-538
41. M. Biehl, A. Ghosh, and B. Hammer, "Learning vector quantization: The dynamics of winner-takes-all algorithms," Neurocomputing, vol. 69, nos. 7-9, pp. 660-670, Mar. 2006
42. S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. Davison, and A. Fitzgibbon, "KinectFusion: Real-time 3-D reconstruction and interaction using a moving depth camera," in Proc. 24th Ann. ACM Symp. User Inter. Softw. Technol., 2011, pp. 559-568
43. J. Tong, J. Zhou, L. Liu, Z. Pan, and H. Yan, "Scanning 3-D full human bodies using kinects," IEEE Trans. Visualizat. Comput. Graph., vol. 18, no. 4, pp. 643-650, Apr. 2012
44. T. Shao, W. Xu, K. Zhou, J. Wang, D. Li, and B. Guo, "An interactive approach to semantic modeling of indoor scenes with an rgbd camera," ACM Trans. Graph., vol. 31, no. 6, pp. 136-147, Nov. 2012
45. E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, "Color transfer between images," IEEE Comput. Graph. Appl., vol. 21, no. 5, pp. 34-41, Oct. 2001