Spatiotemporal inpainting for recovering texture maps of occluded building facades

IEEE Transactions on Image Processing

Volumn 16, Issue 9, 2007, Pages 2262-2271

  Korah, Thommen ^a   Rasmussen, Christopher ^a  

^a University of Delaware   (United States)

Author keywords

Foreground removal; Layered appearance models; Spatiotemporal inpainting; Texture maps

Indexed keywords

ALGORITHMS; CAMERAS; IMAGE RECOGNITION; IMAGE SEGMENTATION; PIXELS;

FOREGROUND REMOVAL; SPATIOTEMPORAL INPAINTING; TEMPORAL MEDIAN FILTERING;

IMAGE RECONSTRUCTION;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; COMPUTER GRAPHICS; CONSTRUCTION WORK AND ARCHITECTURAL PHENOMENA; IMAGE ENHANCEMENT; MATHEMATICAL COMPUTING; METHODOLOGY; PAINTING; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; THREE DIMENSIONAL IMAGING;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER GRAPHICS; FACILITY DESIGN AND CONSTRUCTION; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; IMAGING, THREE-DIMENSIONAL; NUMERICAL ANALYSIS, COMPUTER-ASSISTED; PAINTINGS; PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY;

EID: 34548279916     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2007.903263     Document Type: Article

References (49)

1. S. Teller, M. Antone, Z. Bodnar, M. Bosse, S. Coorg, M. Jethwa, and N. Master, "Calibrated, registered images of an extended urban area," Int. J. Comput. Vis., vol. 53, pp. 93-107, 2003.
2. F. van den Heuvel, "Automation in architectural photogrammetry; line-photogrammetry for the reconstruction from single and multiple images," Ph.D. dissertation, Delft Univ. Technol., Delft, The Netherlands, 2003.
3. D. Farin, P. de With, and W. Effelsberg, "Robust background estimation for complex video sequences," presented at the IEEE Int. Conf. Image Processing, 1997.
4. F. Odone, A. Fusiello, and E. Trucco, "Layered representation of a video shot with mosaicing," Pattern Anal. Appl., vol. 5, pp. 296-305, 2002.
5. C. Rother, V. Kolmogorov, and A. Blake, "Grabcut - interactive foreground extraction using iterated graph cuts," presented at the SIGGRAPH, 2004.
6. J. Wang and E. Adelson, "Representing moving images with layers," IEEE Trans. Image Process., vol. 3, no. 5, pp. 625-638, Sep. 1994.
7. N. Jojic and B. Frey, "Learning flexible sprites in video layers," presented at the IEEE Conf. Computer Vision and Pattern Recognition, 2001.
8. J. Xiao and M. Shah, "Motion layer extraction in the presence of occlusion using graph cut," presented at the IEEE Conf. Computer Vision and Pattern Recognition, 2004.
9. Y. Wexler, A. Fitzgibbon, and A. Zisserman, "Bayesian estimation of layers from multiple images," presented at the Eur. Conf. Computer Vision, 2002.
10. A. Kokaram, B. Collis, and S. Robinson, "A bayesian framework for recursive object removal in movie post-production," presented at the IEEE Int. Conf. Image Processing, 2003.
11. A. Criminisi, G. Cross, A. Blake, and V. Kolmogorov, "Bilayer segmentation of live video," presented at the IEEE Conf. Computer Vision and Pattern Recognition, 2006.
12. J. Sun, W. Zhang, X. Tang, and H. Shum, "Background cut," presented at the Eur. Conf. Computer Vision, 2006.
13. M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, "Image inpainting," in Proc. SIGGRAPH, 2000, pp. 417-424.
14. T. Chan and J. Shen, Image Processing And Analysis: Variational, PDE, Wavelet, And Stochastic Methods. Philadelphia, PA: SIAM, 2005.
15. A. Efros and W. Freeman, "Image quilting for texture synthesis and transfer," presented at the SIGGRAPH, 2001.
16. A. Criminisi, P. Pérez, and K. Toyama, "Region filling and object removal by exemplar-based image inpainting," IEEE Trans. Image Process., vol. 13, no. 9, pp. 1200-1212, Sep. 2004.
17. Y. Wexler, E. Shechtman, and M. Irani, "Space-time video completion," presented at the IEEE Conf. Computer Vision and Pattern Recognition, 2004.
18. J. Jia, T. Wu, Y. Tai, and C. Tang, "Video repairing: Inference of foreground and background under severe occlusion," presented at the IEEE Conf. Computer Vision and Pattern Recognition, 2004.
19. S. Kumar, M. Biswas, and T. Nguyen, "Spatio-temporal texture synthesis and image inpainting for video applications," presented at the IEEE Int. Conf. Image Processing, 2005.
20. I. Drori, D. Cohen-Or, and H. Yeshurun, "Fragment-based image completion," ACM Trans. Graph., vol. 22, pp. 303-312, 2003.
21. V. Kwatra, I. Essa, A. Bobick, and N. Kwatra, "Texture optimization for example-based synthesis," ACM Trans. Graph., pp. 795-802, Aug. 2005.
22. L. Liang, C. Liu, Y.-Q. Xu, B. Guo, and H.-Y. Shum, "Real-time texture synthesis by patch-based sampling," ACM Trans. Graph., vol. 20, pp. 127-150, 2001.
23. M. Ashikhmin, "Synthesizing natural textures," presented at the ACM Sym. Interactive 3D Graphics, 2001.
24. J. Sun, L. Yuan, J. Jia, and H.-Y. Shum, "Image completion with structure propagation," ACM Trans. Graph., vol. 24, pp. 861-868, 2005.
25. N. Komodakis and G. Tziritas, "Image completion using global optimization," presented at the IEEE Computer Society Conf. Computer Vision and Pattern Recognition, 2006.
26. P. Perez, M. Gangnet, and A. Blake, "Patchworks: Example-based region tiling for image editing," Microsoft Res. Rep. TR-2004-04, Tech. Rep., 2004.
27. C. Rasmussen and T. Korah, "Spatiotemporal inpainting for recovering texture maps of partially occluded building facades," presented at the IEEE Int. Conf. Image Processing, 2005.
28. T. Korah and C. Rasmussen, "PCA-based recognition for efficient inpainting," presented at the Proc. Asian Conf. Computer Vision, 2006.
29. T. Korah and C. Rasmussen, "Improving spatiotemporal inpainting with layer appearance models," presented at the Int. Symp. Visual Computing, 2006.
30. L. Yatziv, G. Sapiro, and M. Levoy, "Lightfield completion," in Proc. Int. Conf. Image Processing, 2004, pp. 1787-1790.
31. K. Patwardhan, G. Sapiro, and M. Bertalmío, "Video inpainting of occluding and occluded objects," presented at the IEEE Int. Conf. Image Processing, 2005.
32. J. Davis, "Mosaics of scenes with moving objects," presented at the IEEE Conf. Computer Vision and Pattern Recognition, 1998.
33. M. Hansen, P. Anandan, K. Dana, G. van den Wal, and P. Burt, "Real-time scene stabilization and mosaic construction," presented at the DARPA Image Understanding Workshop, 1994.
34. R. Szeliski, "Video mosaics for virtual environments," in Proc. IEEE Computer Graphics and Applications, 1996, vol. 16, no. 2, pp. 22-30.
35. R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge, U.K.: Cambridge Univ. Press, 2000.
36. J. Shi and C. Tomasi, "Good features to track," presented at the IEEE Conf. Computer Vision and Pattern Recognition, 1994.
37. T. Tommasini, A. Fusiello, E. Trucco, and V. Roberto, "Making good features to track better," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1998, pp. 178-183.
38. A. Fusiello, E. Trucco, T. Tommasini, and V. Roberto, "Improving feature tracking with robust statistics," Pattern Anal. Appl., vol. 2, no. 4, pp. 312-320, 1999.
39. K. Toyama, J. Krumm, B. Brumitt, and B. Meyers, "Principles and practice of background maintenance," presented at the Int. Conf. Computer Vision, 1999.
40. M. Varma, and A. Zisserman, "A statistical approach to texture classification from single images," Int. J. Comput. Vis., vol. 62, no. 1-2, pp. 61-81, Apr. 2005.
41. J. Winn, A. Criminisi, and T. Minka, "Object categorization by learned universal visual dictionary," presented at the 10th IEEE Int. Conf. Computer Vision, 2005.
42. L. Lu, K. Toyama, and G. D. Hager, "A two level approach for scene recognition," presented at the IEEE Conf. Computer Vision and Pattern Recognition, 2005.
43. M. Turk and A. Pentland, "Face recognition using eigenfaces," presented at the IEEE Conf. Computer Vision and Pattern Recognition, 1991.
44. T. K. Leung and J. Malik, "Recognizing surfaces using three-dimensional textons," presented at the Int. Conf. Computer Vision, 1999.
45. R. Duda, P. Hart, and D. Stork, Pattern Classification, 2nd ed. New York: Wiley, 2001.
46. T. Joachims, "Making large-scale SVM learning practical," in Advances in Kernel Methods: Support Vector Learning, B. Schölkopf, C. Burges, and A. Smola, Eds. Cambridge, MA: MIT Press, 1999.
47. P. Perez, M. Gangnet, and A. Blake, "Poisson image editing," in Proc. ACM SIGGRAPH, 2003, pp. 313-318.
48. S. J. Kim and M. Pollefeys, "Radiometric alignment of image sequences," presented at the IEEE Conf. Computer Vision and Pattern Recognition, 2004.
49. D. Capel and A. Zisserman, "Computer vision applied to super resolution," IEEE Signal Process. Mag., vol. 20, no. 3, pp. 75-86, May 2003.