PixMix: A real-time approach to high-quality Diminished Reality

ISMAR 2012 - 11th IEEE International Symposium on Mixed and Augmented Reality 2012, Science and Technology Papers

Volumn , Issue , 2012, Pages 141-150

  Herling, Jan ^a   Broll, Wolfgang ^a  

^a ILMENAU UNIVERSITY OF TECHNOLOGY   (Germany)

Author keywords

H.5.1 Information Interfaces and Presentation : Multimedia Information Systems Artificial, augmented, and virtual realities; I.3.6 Computing Methodologies : Computer Graphics Methodology and Techniques; I.4.9 Computing Methodologies : Image Processing and Computer Vision Applications

Indexed keywords

BETTER PERFORMANCE; CAMERA MOVEMENT; FRAME-TO-FRAME COHERENCE; H.5.1 [INFORMATION INTERFACES AND PRESENTATION]: MULTIMEDIA INFORMATION SYSTEMS - ARTIFICIAL , AUGMENTED , AND VIRTUAL REALITIES; HIGH QUALITY; HIGH QUALITY IMAGES; HOMOGRAPHY-BASED OBJECT TRACKING; IMAGE PROCESSING AND COMPUTER VISION; INTERACTIVE RATES; LINEAR CAMERAS; METHODOLOGY AND TECHNIQUES; NON-TRIVIAL; STATIC CAMERAS; VIDEO STREAMS;

AUGMENTED REALITY; IMAGE PROCESSING; VIDEO STREAMING; VIRTUAL REALITY;

CAMERAS;

EID: 84873539130     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ISMAR.2012.6402551     Document Type: Conference Paper

References (38)

1. P. Arbeláez, M. Maire, C. Fowlkes, and J. Malik. Contour detection and hierarchical image segmentation. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 33(5):898-916, may 2011.
2. M. Ashikhmin. Synthesizing natural textures. In Proceedings of the 2001 symposium on Interactive 3D graphics, I3D '01, pages 217-226, New York, NY, USA, 2001. ACM.
3. X. Bai, J. Wang, D. Simons, and G. Sapiro. Video snapcut: robust video object cutout using localized classifiers. In ACM SIGGRAPH, pages 1-11, New York, 2009. ACM.
4. C. Barnes, E. Shechtman, A. Finkelstein, and D. B. Goldman. Patch-match: a randomized correspondence algorithm for structural image editing. In ACM SIGGRAPH, pages 1-11, New York, 2009. ACM.
5. C. Barnes, E. Shechtman, D. B. Goldman, and A. Finkelstein. The generalized patchmatch correspondence algorithm. In Proceedings of the 11th ECCV, pages 29-43. Springer-Verlag, 2010.
6. A. Bugeau, M. Bertalmío, V. Caselles, and G. Sapiro. A comprehensive framework for image inpainting. Trans. Img. Proc., 19(10):2634-2645, 2010.
7. A. Criminisi, P. Perez, and K. Toyama. Region filling and object removal by exemplar-based image inpainting. Image Processing, IEEE Transactions on, 13(9):1200-1212, sept. 2004.
8. L. Demanet, B. Song, and T. Chan. Image inpainting by correspondence maps: a deterministic approach. Computer, 1100(03-40):21750, 2003.
9. I. Drori, D. Cohen-Or, and H. Yeshurun. Fragment-based image completion. In ACM SIGGRAPH 2003 Papers, SIGGRAPH '03, pages 303-312, New York, NY, USA, 2003. ACM.
10. A. A. Efros and T. Leung. Texture synthesis by non-parametric sampling. In The Proceedings of the Seventh IEEE ICCV, volume 2, pages 1033 -1038, 1999.
11. A. Enomoto and H. Saito. Diminished reality using multiple handheld cameras. In ACCV'07 Workshop on Multidimensional and Multi-view Image Processing, 2007.
12. J. Herling and W. Broll. Advanced self-contained object removal for realizing real-time diminished reality in unconstrained environments. In 9th IEEE ISMAR, pages 207 -212, oct. 2010.
13. J. Jia, Y.-W. Tai, T.-P.Wu, and C.-K. Tang. Video repairing under variable illumination using cyclic motions. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 28(5):832-839, may 2006.
14. J. Jia, T.-P.Wu, Y.-W. Tai, and C.-K. Tang. Video repairing: inference of foreground and background under severe occlusion. In In Proceedings of CVPR' 2004, volume 1, pages 364-371, June 2004.
15. M. Kass, A. Witkin, and D. Terzopoulos. Snakes: Active contour models. Int. Journal of Computer Vision, 1(4):321-331, 1988.
16. O. Korkalo, M. Aittala, and S. Siltanen. Light-weight marker hiding for augmented reality. In 9th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2010, Seoul, Korea, 13-16 October 2010, pages 247-248. IEEE, 2010.
17. T.-H. Kwok, H. Sheung, and C. C. L. Wang. Fast query for exemplar-based image completion. Trans. Img. Proc., 19:3106-3115, December 2010.
18. A. Levin, D. Lischinski, and Y. Weiss. A closed-form solution to natural image matting. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 30(2):228-242, feb. 2008.
19. M. Liu, S. Chen, J. Liu, and X. Tang. Video completion via motion guided spatial-temporal global optimization. In Proceedings of the 17th ACM international conference on Multimedia, MM '09, pages 537-540, New York, NY, USA, 2009. ACM.
20. J. Ning, L. Zhang, D. Zhang, and C. Wu. Interactive image segmentation by maximal similarity based region merging. Pattern Recogn., 43(2):445-456, Feb. 2010.
21. K. A. Patwardhan, G. Sapiro, and M. Bertalmio. Video inpainting under constrained camera motion. IEEE Transactions On Image Processing, 16(2):545-553, Feb 2007.
22. P. Pérez, M. Gangnet, and A. Blake. Poisson image editing. In ACM SIGGRAPH 2003 Papers, SIGGRAPH '03, pages 313-318, New York, NY, USA, 2003. ACM.
23. Y. Pritch, E. Kav-Venaki, and S. Peleg. Shift-map image editing. In Computer Vision, 2009 IEEE 12th International Conference on, pages 151 -158, 29 2009-oct. 2 2009.
24. C. Rother, V. Kolmogorov, and A. Blake. "grabcut": interactive foreground extraction using iterated graph cuts. In ACM SIGGRAPH 2004 Papers, SIGGRAPH '04, pages 309-314, New York, NY, USA, 2004. ACM.
25. J. Shen, X. Jin, C. Zhou, and C. C. L.Wang. Technical section: Gradient based image completion by solving the poisson equation. Comput. Graph., 31:119-126, January 2007.
26. Y. Shen, F. Lu, X. Cao, and H. Foroosh. Video completion for perspective camera under constrained motion. In Pattern Recognition, 2006. ICPR 2006. 18th International Conference on, volume 3, pages 63 -66, 0-0 2006.
27. T. Shiratori, Y. Matsushita, X. Tang, and S. B. Kang. Video completion by motion field transfer. In IEEE Conference on CVPR' 2006, volume 1, pages 411 - 418, june 2006.
28. S. Siltanen. Texture generation over the marker area. In Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality, ISMAR '06, pages 253-254, Washington, DC, USA, 2006. IEEE Computer Society.
29. D. Simakov, Y. Caspi, E. Shechtman, and M. Irani. Summarizing visual data using bidirectional similarity. In CVPR. IEEE Computer Society, 2008.
30. J. Sun, L. Yuan, J. Jia, and H.-Y. Shum. Image completion with structure propagation. In ACM SIGGRAPH 2005 Papers, SIGGRAPH '05, pages 861-868, New York, NY, USA, 2005. ACM.
31. R. Szeliski. Computer Vision : Algorithms and Applications. Springer, 2010.
32. K. Takeda and R. Sakamoto. Diminished reality for landscape video sequences with homographies. In Proceedings of the 14th international conference on Knowledge-based and intelligent information and engineering systems: Part III, KES'10, pages 501-508, Berlin, Heidelberg, 2010. Springer-Verlag.
33. R. Tong, Y. Zhang, and M. Ding. Video brush: A novel interface for efficient video cutout. Comput. Graph. Forum, 30(7):2049-2057, 2011.
34. M. V. Venkatesh, S.-C. S. Cheung, and J. Zhao. Efficient object-based video inpainting. Pattern Recogn. Lett., 30(2):168-179, Jan. 2009.
35. Y. Wexler, E. Shechtman, and M. Irani. Space-time completion of video. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 29(3):463-476, march 2007.
36. Z. Xu and J. Sun. Image inpainting by patch propagation using patch sparsity. Trans. Img. Proc., 19:1153-1165, May 2010.
37. Y. Zhang, J. Xiao, and M. Shah. Motion layer based object removal in videos. In Application of Computer Vision, 2005. WACV/MOTIONS '05 Volume 1. Seventh IEEEWorkshops on, volume 1, pages 516 -521, jan. 2005.
38. S. Zokai, J. Esteve, Y. Genc, and N. Navab. Multiview paraperspective projection model for diminished reality. In Proceedings of the 2nd IEEE/ACM International Symposium on Mixed and Augmented Reality, ISMAR '03, pages 217-, Washington, DC, USA, 2003. IEEE Computer Society.