Video inpainting on digitized vintage films via maintaining spatiotemporal continuity

IEEE Transactions on Multimedia

Volumn 13, Issue 4, 2011, Pages 602-614

  Tang, Nick C ^a   Hsu, Chiou Ting ^b   Su, Chih Wen ^c   Shih, Timothy K ^d   Liao, Hong Yuan Mark ^a,e  

^a INSTITUTE OF INFORMATION SCIENCE   (Taiwan)

^b NATIONAL TSING HUA UNIVERSITY   (Taiwan)

^c CHUNG YUAN CHRISTIAN UNIVERSITY   (Taiwan)

^d NATIONAL CENTRAL UNIVERSITY   (Taiwan)

^e NATIONAL CHIAO TUNG UNIVERSITY   (Taiwan)

Author keywords

Frame completion; motion completion; motion estimation; video inpainting

Indexed keywords

CULTURAL ARTIFACTS; DAMAGED AREA; DIGITAL FORMAT; DIGITAL VIDEOS; FRAME COMPLETION; KEY TECHNIQUES; MOTION COMPLETION; MOTION INFORMATION; SPATIAL CONTINUITY; TEMPORAL CONTINUITY; VIDEO CLIPS; VIDEO ENHANCEMENT; VIDEO INPAINTING;

ALGORITHMS; MOTION ESTIMATION; MULTIMEDIA SYSTEMS; REPAIR;

COMPUTER GRAPHICS;

EID: 79960668667     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2011.2112642     Document Type: Article

References (38)

1. J. Bergen, P. Anandan, K. Hanna, and R. Hingorani, "Hierarchical model-based motion estimation," in Proc. 2nd Eur. Conf. Computer Vision, 1992, pp. 237-252.
2. J. Bouguet, "Pyramidal implementation of the Lucas Kanade feature tracker: Description of the algorithm," OpenCV Document, Intel, Microprocessor Research Labs, 2000.
3. M. Brown and D. Lowe, "Recognizing panoramas," in Proc. Int. Conf. Computer Vision, 2003, pp. 1218-1225.
4. P. J. Burt and A. H. Adelson, "A multiresolution spline with application to image mosaics," ACM Trans. Graph., vol. 2, pp. 217-236, 1983.
5. D. Capel and A. Zisserman, "Automated mosaicing with super-resolution zoom," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1998, pp. 885-891.
6. A. Cenci, A. Fusiello, and V. Roberto, "Image stabilization by feature tracking," in Proc. 10th Int. Conf. Image Analysis and Processing, 1999, pp. 665-670.
7. T.-J. Cham and R. Cipolla, "A statistical framework for longrange feature matching in uncalibrated image mosaicing," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1998, pp. 442-447.
8. C. H. Cheung and L. M. Po, "Novel cross-diamond-hexagonal search algorithms for fast block motion estimation," IEEE Trans. Multimedia, vol. 7, no. 1, pp. 16-22, Feb. 2005.
9. M. V. Venkatesh, S.-C. S. Cheung, and J. Zhao, "Efficient object based video inpainting," Pattern Recognit. Lett., vol. 30, no. 2, pp. 168-179, Jan. 2009.
10. D. Comaniciu and P. Meer, "Mean shift: A robust approach toward feature space analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 5, pp. 603-619, May 2002. (Pubitemid 34550429)
11. A. Criminisi, P. Perez, and K. Toyama, "Region filling and object removal by exemplar-based image inpainting," IEEE Trans. Image Process., vol. 13, no. 9, pp. 1200-1212, Sep. 2004.
12. E. Cuevas, D. Zaldivar, and R. Rojas, Kalman Filter for Vision Tracking, Freie Univ. Berlin, 2005, Tech. Rep. B 05-12.
13. E. Cuevas, D. Zaldivar, and R. Rojas, Particle Filter in Vision Tracking, Instituet fur Informatik, Freie Univ. Berlin, Tech. Rep. B-05-13.
14. I. Drori, D. Cohen-Or, and H. Yeshurun, "Fragment-based image completion," ACM Trans. Graph., vol. 22, pp. 303-312, 2003.
15. K. M. Gullu, O. Urhan, and S. Erturk, "Scratch detection via temporal coherency analysis," in Proc. 2006 IEEE Int. Symp. Circuits and Systems, 2006.
16. A. M. Huang and T. Nguyen, "Correlation-based motion vector processing with adaptive interpolation scheme for motion-compensated frame interpolation," IEEE Trans. Image Process., vol. 18, no. 4, pp. 740-752, Apr. 2009.
17. J. Jia, T.-P. Wu, Y.-W. Tai, and C.-K. Tang, "Video repairing: Inference of foreground and background under severe occlusion," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Jun.-Jul. 2004, pp. I: 364-371.
18. J. Jia, Y.-W. Tai, T.-P. Wu, and C.-K. Tang, "Video repairing under variable illumination using cyclic motions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 5, pp. 832-839, May 2006.
19. L. Joyeux, O. Buisson, B. Besserer, and S. Boukir, "Detection and removal of line scratches in motion picture films," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Jun. 1999.
20. A. C. Kokaram and S. J. Godsill, "Joint detection, interpolation, motion and parameter estimation for image sequences with missing data," in Proc. Int. Conf. Image Processing, Oct. 26-29, 1997, vol. 2, pp. 191-194.
21. B. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," in Proc. Int. Joint Conf. Artificial Intelligence, 1981, pp. 674-679. (Pubitemid 14478744)
22. A. Machi and F. Collura, "Accurate spatio-temporal restoration of compact single frame defects in aged motion pictures," in Proc. 12th Int. Conf. Image Analysis and Processing, 2003, pp. 454-459.
23. Y. Matsushita, E. Ofek, X. Tang, and H. Y. Shum, "Full-frame video stabilization," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, Jun. 2005, vol. 1, pp. 50-57.
24. K. A. Patwardhan, G. Sapiro, and M. Bertalmio, "Video inpainting under constrained camera motion," IEEE Trans. Image Process., vol. 12, no. 2, pp. 545-553, Feb. 2007. (Pubitemid 46421694)
25. P. Perez, M. Gangnet, and A. Blake, "Poisson image editing," in Proc. ACM SIGGRAPH 2003, 2003, pp. 313-318.
26. L. M. Po and W. C. Ma, "A novel four-step search algorithm for fast block motion estimation," IEEE Trans. Circuits Syst.Video Technol., vol. 6, no. 3, pp. 313-317, Jun. 1996.
27. I. W. Selensnick and K. Y. Li, "Video denoising using 2D and 3D dual-tree complex wavelet transforms," in Proc. SPIE Wavelets X, San Diego, CA, Aug. 2003.
28. Y. Shen, F. Lu, X.-C. Cao, and H. Foroosh, "Video completion for perspective camera under constrained motion," in Proc. Int. Conf. Pattern Recognition, 2006, pp. 63-66.
29. J. Shi and C. Tomasi, "Good features to track," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1994, pp. 593-600.
30. T. K. Shih, N. C. Tang, and J.-N. Hwang, "Exemplar-based video inpainting without ghost shadow artifacts by maintaining temporal continuity," IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 2, pp. 347-360, Mar. 2009.
31. T. Shiratori, Y. Matsushita, S.-B. Kang, and X. Tang, "Video completion by motion field transfer," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006, pp. 411-418. (Pubitemid 44931358)
32. J. Sun, L. Yuan, J. Jia, and H. Y. Shum, "Image completion with structure propagation," in Proc. ACM SIGGRAPH 2005, 2005, vol. 24, pp. 861-868.
33. N. C. Tang, H. Y. M. Liao, C. W. Su, F. Huang, and T. K. Shih, "Video inpainting on digitized old films," in Proc. Int. Conf. Knowledge-Based and Intelligent Information and Engineering Systems: Part II, 2009, pp. 421-430.
34. D. Turaga, M. Alkanhal, and T. Chen, Correlation Based Search Algorithms for Motion Estimation, Carnegie Mellon Univ., 1999, Tech. Rep.
35. Y. Wexler, E. Shechtman, and M. Irani, "Space-time completion of video," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 3, pp. 463-476, Mar. 2007. (Pubitemid 46336421)
36. Y. Zhang, J. Xiao, and M. Shah, "Motion layer based object removal in videos," in Proc. 7th IEEE Workshop Application of Computer Vision, 2005, pp. 516-521.
37. Z. Zhu, G. Xu, Y. Yang, and J. Jin, "Camera stabilization based on 2.5D motion estimation and inertial motion filtering," in Proc. IEEE Int. Conf. Intelligent Vehicles, 1998, vol. 2, pp. 329-334.
38. I. Zoghlami, O. Faugeras, and R. Deriche, "Using geometric corners to build a 2D mosaic from a set of images," in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997, pp. 420-425.