Automatic inpainting scheme for video text detection and removal

IEEE Transactions on Image Processing

Volumn 22, Issue 11, 2013, Pages 4460-4472

  Mosleh, Ali ^a   Bouguila, Nizar ^b   Hamza, Abdessamad Ben ^b  

^a Dept of Electrical and Computer Engineering   (Canada)

^b NONE   (Canada)

Author keywords

Bandlets; edge detection; inpainting; regularization; stroke width transform; text detection

Indexed keywords

BANDLETS; INPAINTING; REGULARIZATION; STROKE WIDTHS; TEXT DETECTION;

EDGE DETECTION; MATHEMATICAL TRANSFORMATIONS; VIDEOTEX;

CHARACTER RECOGNITION;

ALGORITHM; ARTICLE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; IMAGE ENHANCEMENT; IMAGE SUBTRACTION; INFORMATION RETRIEVAL; METHODOLOGY; PHOTOGRAPHY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; VIDEORECORDING; WRITING; PROCEDURES;

ALGORITHMS; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INFORMATION STORAGE AND RETRIEVAL; PATTERN RECOGNITION, AUTOMATED; PHOTOGRAPHY; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SUBTRACTION TECHNIQUE; VIDEO RECORDING; WRITING;

EID: 84884537452     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2013.2273672     Document Type: Article

References (72)

1. T.-H. Tsai and C.-L. Fang, "Text-video completion using structure repair and texture propagation," IEEE Trans. Multimedia, vol. 13, no. 1, pp. 29-39, Feb. 2011.
2. M. Favorskaya, A. Zotin, and M. Damov, "Intelligent inpainting system for texture reconstruction in videos with text removal," in Proc. ICUMT, Oct. 2010, pp. 867-874.
3. C.-L. Fang and T.-H. Tsai, "Advertisement video completion using hierarchical model," in Proc. IEEE ICME, Apr. 2008, pp. 1557-1560.
4. W.-Q. Yan and M. Kankanhalli, "Erasing video logos based on image inpainting," in Proc. ICME, vol. 2. Jan. 2002, pp. 521-524.
5. C. W. Lee, K. Jung, and H. J. Kim, "Automatic text detection and removal in video sequences," Pattern Recognit. Lett., vol. 24, no. 15, pp. 2607-2623, 2003. (Pubitemid 37027782)
6. M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, "Image inpainting," in Proc. 27th Annu. Conf. Comput. Graph. Interact. Tech. SIGGRAPH, 2000, pp. 417-424. (Pubitemid 32394006)
7. J. Kim, H. Kim, and S. Wachenfeld, "Restoration of regions occluded by a caption in TV scene," in Proc. Conf. Converg. Technol. Asia-Pacific Region, vol. 2. Oct. 2003, pp. 817-820.
8. D. Chen, J.-M. Odobez, and H. Bourlard, "Text detection and recognition in images and video frames," Pattern Recognit., vol. 37, no. 3, pp. 595-608, 2004. (Pubitemid 38003096)
9. M. Lyu, J. Song, and M. Cai, "A comprehensive method for multilingual video text detection, localization, and extraction," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 2, pp. 243-255, Feb. 2005. (Pubitemid 40265948)
10. L. Agnihotri and N. Dimitrova, "Text detection for video analysis," in Proc. IEEE Workshop CBAIVL, Jan. 1999, pp. 109-113.
11. W. Kim and C. Kim, "A new approach for overlay text detection and extraction from complex video scene," IEEE Trans. Image Process., vol. 18, no. 2, pp. 401-411, Feb. 2009.
12. A. Mosleh, N. Bouguila, and A. Ben Hamza, "Image text detection using a bandlet-based edge detector and stroke width transform," in Proc. Brit. Mach. Vis. Conf., Sep. 2012, pp. 63.1-63.12.
13. H. Li, D. Doermann, and O. Kia, "Automatic text detection and tracking in digital video," IEEE Trans. Image Process., vol. 9, no. 1, pp. 147-156, Jan. 2000. (Pubitemid 30562093)
14. R. Lienhart and W. Effelsberg, "Automatic text segmentation and text recognition for video indexing," Multimedia Syst., vol. 8, no. 1, pp. 69-81, 2000.
15. K. Jung, K. I. Kim, and A. K. Jain, "Text information extraction in images and video: A survey," Pattern Recognit., vol. 37, no. 5, pp. 977-997, 2004. (Pubitemid 38473840)
16. J. Liang, D. Doermann, and H. Li, "Camera-based analysis of text and documents: A survey," Int. J. Document Anal. Recognit., vol. 7, no. 2, pp. 84-104, Jul. 2005. (Pubitemid 41051508)
17. M. Cai, J. Song, and M. Lyu, "A new approach for video text detection," in Proc. IEEE ICIP, Feb. 2002, pp. 117-120.
18. C. Liu, C. Wang, and R. Dai, "Text detection in images based on unsupervised classification of edge-based features," in Proc. 8th Int. Conf. Document Anal. Recognit., 2005, pp. 610-614. (Pubitemid 46442397)
19. E. K. Wong and M. Chen, "A new robust algorithm for video text extraction," Pattern Recognit., vol. 36, no. 6, pp. 1397-1406, 2003. (Pubitemid 36225874)
20. K. Kim, K. Jung, and J. H. Kim, "Texture-based approach for text detection in images using support vector machines and continuously adaptive mean shift algorithm," IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 12, pp. 1631-1639, Dec. 2003.
21. M. Lyu, J. Song, and M. Cai, "A comprehensive method for multilingual video text detection, localization, and extraction," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 2, pp. 243-255, Feb. 2005. (Pubitemid 40265948)
22. J. Gllavata, R. Ewerth, and B. Freisleben, "Text detection in images based on unsupervised classification of high-frequency wavelet coefficients," in Proc. ICPR, vol. 1. Aug. 2004, pp. 425-428.
23. H. Li, D. Doermann, and O. Kia, "Automatic text detection and tracking in digital video," IEEE Trans. Image Process., vol. 9, no. 1, pp. 147-156, Jan. 2000. (Pubitemid 30562093)
24. Y. Zhong, H. Zhang, and A. Jain, "Automatic caption localization in compressed video," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 4, pp. 385-392, Apr. 2000.
25. H. Takahashi and M. Nakajima, "Region graph based text extraction from outdoor images," in Proc. 3rd ICITA, vol. 1. Jul. 2005, pp. 680-685. (Pubitemid 43763131)
26. X. Chen and A. Yuille, "Detecting and reading text in natural scenes," in Proc. IEEE CVPR, Jul. 2004, pp. 366-373.
27. B. Epshtein, E. Ofek, and Y. Wexler, "Detecting text in natural scenes with stroke width transform," in Proc. CVPR, Jun. 2010, pp. 2963-2970.
28. P. Shivakumara, T. Q. Phan, and C. L. Tan, "A laplacian approach to multi-oriented text detection in video," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 2, pp. 412-419, Feb. 2011.
29. H. Chen, S. T. G. Schroth, D. Chen, R. Grzeszczuk, and B. Girod, "Robust text detection in natural images with edge-enhanced maximally stable extremal regions," in Proc. IEEE ICIP, Sep. 2011, pp. 2609-2612.
30. Y.-F. Pan, X. Hou, and C.-L. Liu, "A hybrid approach to detect and localize texts in natural scene images," IEEE Trans. Image Process., vol. 20, no. 3, pp. 800-813, Mar. 2011.
31. M. Zhao, S. Li, and J. Kwok, "Text detection in images using sparse representation with discriminative dictionaries," Image Vis. Comput., vol. 28, no. 12, pp. 1590-1599, 2010.
32. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Discriminative learned dictionaries for local image analysis," in Proc. IEEE CVPR, Jun. 2008, pp. 1-8.
33. T. F. Chan and J. Shen, "Mathematical models for local nontexture inpaintings," SIAM J. Appl. Math., vol. 62, pp. 1019-1043, May 2001.
34. M. Bertalmio, "Strong-continuation, contrast-invariant inpainting with a third-order optimal PDE," IEEE Trans. Image Process., vol. 16, no. 7, pp. 1934-1938, Jul. 2006. (Pubitemid 43985071)
35. M. Bertalmio, A. L. Bertozzi, and G. Sapiro, "Navier-stokes, fluid dynamics, and image and video inpainting," in Proc. IEEE Comput. Soc. Conf. CVPR), Jan. 2001, pp. 355-362.
36. M. Ghoniem, Y. Chahir, and A. Elmoataz, "Geometric and texture inpainting based on discrete regularization on graphs," in Proc. IEEE ICIP, Nov. 2009, pp. 1349-1352.
37. H. Grossauer and O. Scherzer, "Using the complex Ginzburg-Landau equation for digital inpainting in 2D and 3D," in Proc. 4th Int. Conf. Scale Space Methods Comput. Vis., 2003, pp. 225-236.
38. A. Criminisi, P. Perez, and K. Toyama, "Region filling and object removal by exemplar-based image inpainting," IEEE Trans. Image Process., vol. 13, no. 9, pp. 1200-1212, Sep. 2004.
39. K. Patwardhan, G. Sapiro, and M. Bertalmio, "Video inpainting of occluding and occluded objects," in Proc. IEEE ICIP, Sep. 2005, pp. 69-72. (Pubitemid 44538879)
40. K. A. Patwardhan, G. Sapiro, and M. Bertalmio, "Video inpainting under constrained camera motion," IEEE Trans. Image Process., vol. 16, no. 2, pp. 4545-553, Feb. 2007.
41. T. K. Shih, N. C. Tang, W.-S. Yeh, T.-J. Chen, and W. Lee, "Video inpainting and implant via diversified temporal continuations," in Proc. ACM Int. Conf. Multimedia, 2006, pp. 133-136. (Pubitemid 47113521)
42. T. K. Shih, N. C. Tang, and J.-N. Hwang, "Exemplar-based video inpainting without ghost shadow artifacts by maintaining temporal continuity," IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 3, pp. 347-360, Mar. 2009.
43. Y. Wexler, E. Shechtman, and M. Irani, "Space-time completion of video," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 3, pp. 463-476, Mar. 2007. (Pubitemid 46336421)
44. J. Jia, Y.-W. Tai, T.-P. Wu, and C. Tang, "Video repairing under variable illumination using cyclic motions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 5, pp. 832-839, May 2006.
45. J. Jia and C.-K. Tang, "Image repairing: Robust image synthesis by adaptive ND tensor voting," in Proc. IEEE Comput. Soc. Conf. CVPR, Jun. 2003, pp. 643-650.
46. Y. Zhang, J. Xiao, and M. Shah, "Motion layer based object removal in videos," in Proc. 7th IEEE WACV/MOTIONS, Jan. 2005, pp. 516-521.
47. T. Shiratori, Y. Matsushita, X. Tang, and S. B. Kang, "Video completion by motion field transfer," in Proc. IEEE Comput. Soc. Conf. CVPR, Jun. 2006, pp. 411-418. (Pubitemid 44931358)
48. N. Tang, C.-T. Hsu, C.-W. Su, T. Shih, and H.-Y. Liao, "Video inpainting on digitized vintage films via maintaining spatiotemporal continuity," IEEE Trans. Multimedia, vol. 13, no. 4, pp. 602-614, Aug. 2011.
49. X. Li and Y. Zheng, "Patch-based video processing: A variational Bayesian approach," IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 1, pp. 27-40, Jan. 2009.
50. V. Cheung, B. J. Frey, and N. Jojic, "Video epitomes," in Proc. IEEE Comput. Soc. Conf. CVPR, Jun. 2005, pp. 42-49.
51. O. Guleryuz, "Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-Part I: Theory," IEEE Trans. Image Process., vol. 15, no. 3, pp. 539-554, Mar. 2006. (Pubitemid 43257554)
52. O. G. Guleryuz, "Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-Part II: Adaptive algorithms," IEEE Trans. Image Process., vol. 15, no. 3, pp. 555-571, Mar. 2006. (Pubitemid 43257555)
53. Z. Xu and J. Sun, "Image inpainting by patch propagation using patch sparsity," IEEE Trans. Image Process., vol. 19, no. 5, pp. 1153-1165, May 2010.
54. A. Mosleh, N. Bouguila, and A. Ben Hamza, "A video completion method based on bandlet transform," in Proc. IEEE ICME, Jul. 2011, pp. 1-6.
55. A. Mosleh, N. Bouguila, and A. Ben Hamza, "Video completion using bandlet transform," IEEE Trans. Multimedia, vol. 14, no. 6, pp. 1591-1601, Dec. 2012.
56. S. Mallat and G. Peyre, "A review of bandlet methods for geometrical image representation," Numer. Algorithms, vol. 44, pp. 205-234, Mar. 2007.
57. E. L. Pennec and S. Mallat, "Sparse geometric image representations with bandelets," IEEE Trans. Image Process., vol. 14, no. 4, pp. 423-438, Apr. 2005. (Pubitemid 40552380)
58. E. L. Pennec and S. Mallat, "Bandelet image approximation and compression," SIAM Multiscale Model. Simul., vol. 4, no. 3, pp. 992-1039, 2005. (Pubitemid 44340296)
59. S. Mallat and G. Peyre, "Surface compression with geometric bandelets," ACM Trans. Graph., vol. 24, pp. 601-608, Jul. 2005.
60. S. Mallat and G. Peyre, "Orthogonal bandlets bases for geometric image approximation," Commun. Pure Appl. Math., vol. 61, pp. 1173-1212, Jul. 2008.
61. S. Mallat and S. Zhong, "Characterization of signals from multiscale edges," IEEE Trans. Pattern Anal. Mach. Intell., vol. 14, no. 7, pp. 710-732, Jul. 1992.
62. W. Zhang, Q. Zhang, and C. Yang, "Edge detection with multiscale products for SAR image despeckling," Electron. Lett., vol. 48, no. 4, pp. 211-212, Feb. 2012.
63. J. Tang, J. Kim, and E. Peli, "Image enhancement in the JPEG domain for people with vision impairment," IEEE Trans. Bio-Med. Eng., vol. 51, no. 11, pp. 2013-2023, Nov. 2004. (Pubitemid 39391340)
64. E. Peli, "Contrast sensitivity function and image discrimination," J. Opt. Soc. Amer., vol. 18, no. 2, pp. 283-293, Nov. 2001.
65. E. Reinhard, P. Shirley, M. Ashikhmin, and T. Troscianko, "Second order image statistics in computer graphics," in Proc. 1st Symp. Appl. Perception Graph. Visualizat., 2004, pp. 99-106. (Pubitemid 40290747)
66. G. Bradski, "Real time face and object tracking as a component of a perceptual user interface," in Proc. 4th IEEE WACV, Oct. 1998, pp. 214-219.
67. B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," in Proc. Int. Joint Conf. Artif. Intell., 1981, pp. 674-679. (Pubitemid 14478744)
68. D. L. Donoho and J. M. Johnstone, "Ideal spatial adaptation by wavelet shrinkage," Biometrika, vol. 81, no. 3, pp. 425-455, 1994.
69. J. Starck, M. Elad, and D. Donoho, "Redundant multiscale transforms and their application for morphological component separation," Adv. Imag. Electron Phys., vol. 132, pp. 287-348, Mar. 2004.
70. S. Lucas, "ICDAR 2005 text locating competition results," in Proc. 8th ICDAR, Aug./Sep. 2005, pp. 80-84. (Pubitemid 46442295)
71. S. Lucas, A. Panaretos, L. Sosa, A. Tang, S. Wong, and R. Young, "ICDAR 2003 robust reading competitions," in Proc. 7th ICDAR, Aug. 2003, pp. 682-687.
72. M. Do and M. Vetterli, "The contourlet transform: An efficient directional multiresolution image representation," IEEE Trans. Image Process., vol. 14, no. 12, pp. 2091-2106, Dec. 2005. (Pubitemid 41781516)