Spatiotemporal grid flow for video retargeting

IEEE Transactions on Image Processing

Volumn 23, Issue 4, 2014, Pages 1615-1628

  Li, Bing ^a,b   Duan, Ling Yu ^b   Wang, Jinqiao ^c   Ji, Rongrong ^b   Lin, Chia Wen ^d   Gao, Wen ^b  

^a INSTITUTE OF COMPUTING TECHNOLOGY   (China)

^b PEKING UNIVERSITY   (China)

^c INSTITUTE OF AUTOMATION   (China)

^d NATIONAL TSING HUA UNIVERSITY   (Taiwan)

Author keywords

dynamic programming; quadratic programming; Video retargeting; video warping

Indexed keywords

CONTENT CONSISTENCY; DISPLAY RESOLUTIONS; GRID INTERPOLATION; QUADRATIC PROGRAMMING PROBLEMS; TEMPORAL COHERENCE; TEMPORAL REDUNDANCY; VIDEO RETARGETING; VIDEO WARPING;

DYNAMIC PROGRAMMING; INTERPOLATION; VIDEO CAMERAS;

QUADRATIC PROGRAMMING;

EID: 84900645737     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2014.2305843     Document Type: Article

References (41)

1. L.-Q. Chen, X. Xie, X. Fan, W.-Y. Ma, H. Zhang, and H.-Q. Zhou, "A visual attention model for adapting images on small displays," Multimedia Syst., vol. 9, no. 4, pp. 353-364, Oct. 2003.
2. H. Liu, X. Xie, W.-Y. Ma, and H. Zhang, "Automatic browsing of large pictures on mobile devices," in Proc. 11th ACM Int. Conf. Multimedia, Nov. 2003, pp. 148-155.
3. B. Suh, H. Ling, B. B. Bederson, and D. W. Jacobs, "Automatic thumbnail cropping and its effectiveness," in Proc. 16th Annu. ACM Symp. User Inter. Softw. Technol., May 2003, pp. 95-104. (Pubitemid 40730678)
4. A. Santella, M. Agrawala, D. DeCarlo, D. Salesin, and M. Cohen, "Gaze-based interaction for semi-automatic photo cropping," in Proc. SIGCHI Conf. Human Factors Comput. Syst., Apr. 2006, pp. 771-780. (Pubitemid 44032166)
5. S. Avidan and A. Shamir, "Seam carving for content-aware image resizing," ACM Trans. Graph., vol. 26, no. 3, pp. 1-10, Jul. 2007.
6. A. Mansfield, P. Gehler, L. Van Gool, and C. Rother, "Scene carving: Scene consistent image retargeting," in Proc. ECCV, Sep. 2010, pp. 143-156.
7. Y.-S. Wang, C.-L. Tai, O. Sorkine, and T.-Y. Lee, "Optimized scale-andstretch for image resizing," ACM Trans. Graph., vol. 27, no. 5, p. 118, Dec. 2008.
8. Y. Guo, F. Liu, J. Shi, Z. Zhou, and M. Gleicher, "Image retargeting using mesh parametrization," IEEE Trans. Multimedia, vol. 11, no. 5, pp. 856-867, Aug. 2009.
9. B. Li, L.-Y. Duan, J. Wang, J. Chen, R. Ji, and W. Gao, "Grid-based retargeting with transformation consistency smoothing," in Proc. 17th Int. Multimedia Model. Conf., Jan. 2011, pp. 12-24.
10. B. Li, Y. Chen, J. Wang, L.-Y. Duan, and W. Gao, "Fast retargeting with adaptive grid optimization," in Proc. IEEE ICME, Jul. 2011, pp. 1-4.
11. D. Panozzo, O. Weber, and O. Sorkine, "Robust image retargeting via axis-aligned deformation," Comput. Graph. Forum, vol. 31, no. 2, pp. 229-236, May 2012.
12. P.-Y. Laffont, J. Y. Jun, C. Wolf, Y.-W. Tai, K. Idrissi, G. Drettakis, et al., "Interactive content-aware zooming," in Proc. Graph. Inter., 2010, pp. 79-87.
13. C.-H. Chang, "A line-structure-preserving approach to image resizing," in Proc. IEEE Int. Conf. CVPR, Jun. 2012, pp. 1075-1082.
14. M. Rubinstein, A. Shamir, and S. Avidan, "Multi-operator media retargeting," ACM Trans. Graph., vol. 28, no. 3, p. 23, Aug. 2009.
15. W. Dong, N. Zhou, J.-C. Paul, and X. Zhang, "Optimized image resizing using seam carving and scaling," ACM Trans. Graph., vol. 29, no. 5, pp. 1-10, Dec. 2009.
16. A. Shamir and O. Sorkine, "Visual media retargeting," in Proc. ACM SIGGRAPH ASIA Courses, 2009, pp. 1-13.
17. L. Wolf, M. Guttmann, and D. Cohen-Or, "Non-homogeneous contentdriven video-retargeting," in Proc. IEEE 11th ICCV, Oct. 2007, pp. 1-6.
18. P. Krähenbühl, M. Lang, A. Hornung, and M. Gross, "A system for retargeting of streaming video," ACM Trans. Graph., vol. 28, no. 5, pp. 1-10, Dec. 2009.
19. P. Greisen, M. Lang, S. Heinzle, and A. Smolic, "Algorithm and VLSI architecture for real-time 1080p60 video retargeting," in Proc. 4th ACM SIGGRAPH/Eurograph. Conf. High Perform. Graph., 2012, pp. 57-66.
20. Y.-F. Zhang, S.-M. Hu, and R. R. Martin, "Shrinkability maps for content-aware video resizing," Comput. Graph. Forum, vol. 27, no. 7, pp. 1797-1804, Oct. 2008.
21. Y. Niu, F. Liu, X. Li, and M. Gleicher, "Warp propagation for video resizing," in Proc. IEEE Int. Conf. CVPR, Jun. 2010, pp. 537-544.
22. B. Yan, K. Sun, and L. Liu, "Matching area based seam carving for video retargeting," IEEE Trans. Circuits Syst. Video Technol., vol. 23, no. 2, pp. 302-310, Feb. 2013.
23. W.-L. Chao, H.-H. Su, S.-Y. Chien, W. Hsu, and J.-J. Ding, "Coarse-tofine temporal optimization for video retargeting based on seam carving," in Proc. IEEE ICME, Jul. 2011, pp. 1-6.
24. L. Shi, J. Wang, L. Duan, and H. Lu, "Consumer video retargeting: Context assisted spatial-temporal grid optimization," in Proc. 17th ACM Int. Conf. Multimedia, Oct. 2009, pp. 301-310.
25. M. Rubinstein, A. Shamir, and S. Avidan, "Improved seam carving for video retargeting," ACM Trans. Graph., vol. 27, no. 3, p. 16, Aug. 2008.
26. T.-C. Yen, C.-M. Tsai, and C.-W. Lin, "Maintaining temporal coherence in video retargeting using mosaic-guided scaling," IEEE Trans. Image Process., vol. 20, no. 8, pp. 2339-2351, Aug. 2011.
27. Y.-S. Wang, H. Fu, O. Sorkine, T.-Y. Lee, and H.-P. Seidel, "Motionaware temporal coherence for video resizing," ACM Trans. Graph., vol. 28, no. 5, p. 127, Dec. 2009.
28. Y.-S. Wang, H.-C. Lin, O. Sorkine, and T.-Y. Lee, "Motion-based video retargeting with optimized crop-and-warp," ACM Trans. Graph., vol. 29, no. 4, p. 90, Jul. 2010.
29. Y.-S. Wang, J.-H. Hsiao, O. Sorkine, and T.-Y. Lee, "Scalable and coherent video resizing with per-frame optimization," ACM Trans. Graph., vol. 30, no. 4, p. 88, Aug. 2011.
30. Y. Hu and D. Rajan, "Hybrid shift map for video retargeting," in Proc. IEEE Int. Conf. CVPR, Jun. 2010, pp. 577-584.
31. J. Sun and H. Ling, "Scale and object aware image retargeting for thumbnail browsing," in Proc. IEEE ICCV, Nov. 2011, pp. 1511-1518.
32. T. Deselaers, P. Dreuw, and H. Ney, "Pan, zoom, scan-Time-coherent, trained automatic video cropping," in Proc. IEEE Int. Conf. CVPR, Jun. 2008, pp. 1-8.
33. G. Hua, C. Zhang, Z. Liu, Z. Zhang, and Y. Shan, "Efficient scale-space spatiotemporal saliency tracking for distortion-free video retargeting," in Proc. 9th ACCV, 2010, pp. 182-192.
34. F. Liu and M. Gleicher, "Video retargeting: Automating pan and scan," in Proc. 14th Annu. ACM Int. Conf. Multimedia, Oct. 2006, pp. 241-250. (Pubitemid 47113539)
35. X. Fan, X. Xie, H. Q. Zhou, and W. Y. Ma, "Looking into video frames on small displays," in Proc. 11th ACM Int. Conf. Multimedia, Nov. 2003, pp. 247-250.
36. Y. Fang, Z. Chen, W. Lin, and C.-W. Lin, "Saliency detection in the compressed domain for adaptive image retargeting," IEEE Trans. Image Process., vol. 21, no. 9, pp. 3888-3901, Sep. 2012.
37. P. Viola and M. J. Jones, "Robust real-time face detection," Int. J. Comput. Vis., vol. 57, no. 2, pp. 137-154, May 2004.
38. J. Nocedal and S. Wright, Numerical Optimization, 2nd ed. New York, NY, USA: Springer-Verlag, 2006.
39. M. Kozlov, S. Tarasov, and L. Khachiyan, "Polynomial solvability of convex quadratic programming," USSR Comput. Math. Math. Phys., vol. 20, no. 5, pp. 223-228, 1980.
40. - (2013). Video Retargeing Project [Online]. Available: http://vipl. ict.ac.cn/members/bli
41. H. A. David, The Method of Paired Comparisons. London, U.K.: Griffin, 1963.