Maintaining temporal coherence in video retargeting using mosaic-guided scaling

IEEE Transactions on Image Processing

Volumn 20, Issue 8, 2011, Pages 2339-2351

  Yen, Tzu Chieh ^a   Tsai, Chia Ming ^b   Lin, Chia Wen ^c  

^a ASUSTEK COMPUTER INC   (Taiwan)

^b NATIONAL CHUNG CHENG UNIVERSITY   (Taiwan)

^c NATIONAL TSING HUA UNIVERSITY   (Taiwan)

Author keywords

Spatio temporal coherence; video adaptation; video retargeting; video scaling

Indexed keywords

CAMERA MOTIONS; CONTENT-AWARE; GLOBAL MAP; GLOBAL SCALING; INFORMATION LOSS; LOCAL SCALING; LOWER RESOLUTION; OBJECT MOTION; SPATIAL COHERENCE; SPATIO-TEMPORAL; SPATIO-TEMPORAL COHERENCE; TEMPORAL COHERENCE; VIDEO ADAPTATION; VIDEO FRAME; VIDEO RETARGETING; VIDEO SCALING; VIDEO SHOTS; VISUAL INFORMATION; VISUAL QUALITIES;

CAMERAS;

EID: 79960527817     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2011.2114357     Document Type: Article

References (28)

1. A. Shamir and O. Sorkine, "Visual media retargeting," in Proc. ACM SIGGRAPH ASIA Courses (SIGGRAPH ASIA'09), 2009, pp. 1-13.
2. S. Avidan and A. Shamir, "Seam carving for content-aware image resizing," ACM Trans. Graph., vol. 26, no. 3, pp. 10:1-10:10, Aug 2007.
3. M. Rubinstein, A. Shamir, and S. Avidan, "Improved seam carving for video retargeting," ACM Trans. Graph., vol. 27, no. 3, pp. 16:1-16:9, Aug. 2008.
4. L.Wolf, M. Guttmann, and D. Cohen-Or, "Non-homogeneous contentdriven video-retargeting," in Proc. IEEE Int. Conf. Comput. Vis., Rio de Janeiro, Brazil, 2007, pp. 1-6.
5. P. Krähenbühl, M. Lang, A. Hornung, and M. Gross, "A system for retargeting of streaming video," ACM Trans. Graph., vol. 28, no. 5, pp. 126:1-126:10, Dec. 2009.
6. J.-S. Kim, J.-H. Kim, and C.-S. Kim, "Adaptive image and video retargeting technique based on fourier analysis," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., Kyoto, Japan, Sep. 2009, pp. 1730-1737.
7. L. Shi, J. Wang, L. Y. Duan, and H. Lu, "Consumer video retargeting: Context assisted spatial-temporal grid optimization," in Proc. ACMInt. Conf. Multimedia, Beijing, China, Oct. 2009, pp. 301-310.
8. S. Kopf, T. Haenselmann, D. Farin, and W. Effelsberg, "Automatic generation of summaries for the web," in Proc. IS&T/SPIE Conf. Storage Retrieval For Media Databases, San Jose, CA, Jan. 2004, pp. 417-428.
9. F. Liu and M. Gleicher, "Video retargeting: Automating pan and scan," in Proc. ACMInt. Conf. Multimedia, Santa Barbara, CA, Oct. 2006, pp. 241-250. (Pubitemid 47113539)
10. Y.-S. Wang, H. Fu, O. Sorkine, T.-Y. Lee, and H.-P. Seidel, "Motionaware temporal coherence for video resizing," ACMTrans. Graph., vol. 28, no. 5, pp. 127:1-127:10, Dec. 2009.
11. W.-H. Cheng, C.-W. Wang, and J.-L. Wu, "Video adaptation for small display based on content recomposition," IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 1, pp. 43-58, Jan. 2007. (Pubitemid 46113449)
12. V. Setlur, T. Lechner,M. Nienhaus, and B. Gooch, "Retargeting images and video for preserving information saliency," IEEE Comput. Grap. Appl., vol. 27, no. 5, pp. 80-88, Sep./Oct. 2007. (Pubitemid 47434072)
13. Y. Guo, F. Liu, J. Shi, Z.-H. Zhou, and M. Gleicher, "Image retargeting using mesh parametrization," IEEE Trans. Multimedia, vol. 11, no. 5, pp. 856-867, Aug. 2009.
14. T. Ren, Y. Liu, and G. Wu, "Image retargeting based on global energy optimization," in Proc. IEEE Int. Conf. Multimedia Expo, New York, Jun. 2009, pp. 406-409.
15. Y.-S. Wang, C.-L. Tai, O. Sorkine, and T.-Y. Lee, "Optimized scaleand- stretch for image resizing," ACM Trans. Graph., vol. 27, no. 5, pp. 118:1-118:8, Dec. 2008.
16. C.-K. Chiang, S.-F.Wang, Y.-L. Chen, and S.-H. Lai, "Fast JND-based video carving with GPU acceleration for real-time video retargeting," IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 11, pp. 1588-1597, Nov. 2009.
17. D. Han, X. Wu, and M. Sonka, "Optimal multiple surfaces searching for video/image resizing-a graph-theoretic approach," in Proc. IEEE Int. Conf. Comput. Vis., Kyoto, Japan, 2009, pp. 1026-1033.
18. S. Kopf, J. Kiess, H. Lemelson, and W. Effelsberg, "FSCAV-fast seam carving for size adaptation of videos," in Proc. ACM Int. Conf. Multimedia, Beijing, China, Oct. 2009, pp. 321-330.
19. Y.-F. Zhang, S.-M. Hu, and R. R. Martin, "Shrinkability maps for content- aware video resizing," Comput. Graph. Forum, vol. 27, no. 7, pp. 1797-1804, 2008.
20. Z. Lu, W. Lin, X. Yang, E. Ong, and S. Yao, "Modeling visual attention's modulatory aftereffects on visual sensitivity and quality evaluation," IEEE Trans. Image Process., vol. 14, no. 11, pp. 1928-1942, Nov. 2005. (Pubitemid 41655695)
21. L. Itti, C.Koch, and E. Niebur, "A model of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998. (Pubitemid 128742001)
22. D. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, Nov. 2004.
23. R. Szeliski, "Image alignment and stitching: A tutorial," FNT Comput. Graph. Vis., vol. 2, no. 1, pp. 1-104, 2006.
24. M. A. Fischler and R. C. Bolles, "Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography," ACM Commun., vol. 24, no. 6, pp. 381-395, Jun. 1981.
25. R. H. Byrd, M. E. Hribar, and J. Nocedal, "An interior point algorithm for large-scale nonlinear programming," SIAM J Optim, vol. 9, no. 4, pp. 877-900, 1999. (Pubitemid 129489310)
26. NTHU, Video Scaling Project. [Online]. Available: http://www.ee. nthu.edu.tw/cwlin/scaling/scaling.htm
27. Y. Li, J. Sun, and H.-Y. Shum, "Video object cut and paste," ACM Trans. Graph., vol. 24, pp. 595-600, 2005.
28. X. Bai, J. Wang, D. Simons, and G. Sapiro, "Video snapcut: Robust video object cutout using localized classifiers," ACM Trans. Graph., vol. 28, pp. 70:1-70:12, Jul. 2009.