Patch-based video processing: A variational Bayesian approach

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 19, Issue 1, 2009, Pages 27-40

  Li, Xin ^a   Zheng, Yunfei ^a  

^a West Virginia University   (United States)

Author keywords

Patch based models; Sparsity based priors; Variational bayesian; Variational EM; Video processing; Weighted averaging

Indexed keywords

CLUSTERING ALGORITHMS; INFERENCE ENGINES; MOTION ESTIMATION; VIDEO RECORDING; VIDEO SIGNAL PROCESSING;

PATCH-BASED MODELS; SPARSITY-BASED PRIORS; VARIATIONAL BAYESIAN; VARIATIONAL EM; VIDEO PROCESSING; WEIGHTED AVERAGING;

BAYESIAN NETWORKS;

EID: 58449083065     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSVT.2008.2005805     Document Type: Article

References (85)

1. B. Horn, Robot Vision. Cambridge, MA: MIT Press, 1986.
2. A. M. Tekalp, Digital Video Processing. Upper Saddle River, NJ: Prentice-Hall, 1995.
3. E. Simoncelli, "Distributed Representation and Analysis of Visual Mo-tion," Ph.D. dissertation, MIT, Cambridge, Jan. 1993.
4. M. Szummer and R. Picard, "Temporal texture modeling," in Proc. Int. Conf. Image Process., 1996, pp. 65-70.
5. X. Li, "Video processing via implicit and mixture motion models," IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 8, pp. 953-963, Aug. 2007.
6. A. Buades, B. Coll, and J.-M. Morel, "A non-local algorithm for image denoising," Proc. CVPR, vol. 2, pp. 60-65, 2005.
7. A. Buades, B. Coll, and J. M. Morel, "Denoising image sequences does not require motion estimation," in Proc. IEEE Conf. Advanced Video Signal Based Surveillance, Sep. 2005, pp. 70-74.
8. M. Mahmoudi and G. Sapiro, "Fast image and video denoising via non-local means of similar neighborhoods," IEEE Signal Proc. Lett., vol. 12, no. 12, pp. 839-842, Dec. 2005.
9. V. Cheung, B. J. Frey, and N. Jojic, "Video epitomes," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2005, vol. 1, pp. 42-49.
10. J. Boulanger, C. Kervrann, and P. Bouthemy, "Space-time adaptation for patch-based image sequence restoration," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, pp. 1096-1102, Jun. 2007.
11. K. Dabov, A. Foi, and K. Egiazarian, "Video denoising by sparse 3d transform-domain collaborative filtering," in Proc. 15th Eur. Signal Processing Conf., Sep. 2007, pp. 145-149.
12. M. Protter and M. Elad, "Image sequence denoising via sparse and redundant representations," IEEE Trans. Image Process., 2007, to be published.
13. S. Chen, D. Donoho, and M. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Scient. Comput., vol. 20, pp. 33-61, 1998.
14. M. Beal, "Variational algorithms for approximate Bayesian inference," Ph.D. dissertation, Gatsby Computational .Neuroscience Unit, Univ. College London, U.K., 2003.
15. W. Freeman, E. Pasztor, and O. Carmichael, "Learning low-level vi-sion," Int. J. Comput. Vis., vol. 40, no. 1, pp. 25-47, Oct. 2000.
16. R. Duda, P. Hart, and D. Stork, Pattern Classification, 2nd ed. New York: Wiley, 2001.
17. M. Orchard and G. Sullivan, "Overlapped block motion compensation: An estimation-theoretic approach," IEEE Trans. Image Process., vol. 3, no. 5, pp. 693-699, Sep. 1994.
18. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, "Image denoising by sparse 3-d transform-domain collaborative filtering," IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080-2095, Aug. 2007.
19. S. Mallat, A Wavelet Tour of Signal Processing, 2nd ed. New York: Academic, 1999.
20. O. G. Guleryuz, "Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part I: Theory," IEEE Trans. Image Process., vol. 15, no. 3, pp. 539-554, mar. 2006.
21. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries," IEEE Trans. Image Process., vol. 15, no. 12, pp. 3736-3745, Dec. 2006.
22. W. Hong, J. Wright, K. Huang, and Y. Ma, "Multi-scale hybrid linear models for lossy image representation," IEEE Trans. Image Process., vol. 15, no. 12, pp. 3655-3671, Dec. 2006.
23. J. B. Tenenbaum, V. de Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science, vol. 290, no. 5500, pp. 2319-2323, 2000.
24. S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol. 290, no. 5500, pp. 2323-2326, 2000.
25. A. Srivastava, A. Lee, E. Simoncelli, and S. Zhu, "On advances in statistical modeling of natural images," J. Math. Imaging Vis., vol. 18, no. 1, pp. 17-33, Jan. 2003.
26. M. D. Carmo, Differential Geometry of Curves and Surfaces. Upper Saddle River, NJ: Prentice-Hall, 1976.
27. A. K. Jain, M. N. Murty, and P. J. Flynn, "Data clustering: A review," ACM Comput. Surv., vol. 31, no. 3, pp. 264-323, 1999.
28. N. Kambhatla and T. K. Leen, "Dimension reduction by local principal component analysis," Neural Comput., vol. 9, no. 7, pp. 1493-1516, 1997.
29. A. Jain, Fundamentals of Digital Image Processing. Upper Saddle River, NJ: Prentice-Hall, 1989.
30. R. Vidal, Y. Ma, and S. Sastry, "Generalized principal component analysis (gpca)," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 12, pp. 1945-1959, Dec. 2005.
31. J. Portilla, V. Strela, M. Wainwright, and E. Simoncelli, "Image denoising using scale mixtures of gaussians in the wavelet domain," IEEE Trans. Image Process., vol. 12, no. 11, pp. 1338-1351, Nov. 2003.
32. E. P. Simoncelli and E. H. Adelson, "Noise removal via bayesian wavelet coring," in Proc. Int. Conf. Image Process., 1996, vol. 1, pp. 379-386.
33. X. Li and M. Orchard, "Spatially adaptive image denoising under overcomplete expansion," in IEEE Int. Conf. Image Process., 2000, pp. 300-303.
34. D. Donoho and I. Johnstone, "Ideal spatial adaptation by wavelet shrinkage," Biometrika, vol. 8.1, pp. 425-455, 1994.
35. O. G. Guleryuz, "Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part ii : Adaptive algorithms," IEEE Trans. Image Process., vol. 15, no. 3, pp. 555-571, Mar. 2006.
36. K. Hirakawa and T. Parks, "Image denoising using total least squares," IEEE Trans. Image Process., vol. 15, no. 9, pp. 2730-2742, Sep. 2006.
37. J. A. Hoeting, D. Madigan, A. E. Raftery, and C. T. Volinsky, "Bayesian model averaging: A tutorial," Stat. Sci., vol. 14, pp. 382-401, 1999.
38. R. A. Jacobs, M. I. Jordan, S. J. Nowlan, and G. E. Hinton, "Adaptive mixtures of local experts," Neural Computation, vol. 3, no. 1, pp. 79-87, 1991.
39. O. G. Guleryuz, "Weighted averaging for denoising over overcomplete dictionaries," IEEE Trans. Image Process., vol. 16, no. 12, pp. 3020-3034, Dec. 2007.
40. R. M. Neal and G. E. Hinton, A View of the EM Algorithm That Justifies Incremental, Sparse, and Other variants. Cambridge: MIT Press, 1999, pp. 355-368.
41. T. S. Lee and D. Mumford, "Hierarchical Bayesian inference in the visual cortex," J. Opt. Soc. Amer. A, vol. 20, pp. 1434-1448, Jul. 2003.
42. C. Kervrann and J. Boulanger, "Unsupervised patch-based image regularization and representation," in Proc. ECCV06, 2006, pp. TV: 555-IV: 567.
43. C. Kervrann, J. Boulanger, and P. Coupe, "Bayesian non-local means filter, image redundancy and adaptive dictionaries for noise removal," in Proc. Scale Space Variational Methods Comput. Vis., 2007, pp. 520-532.
44. S. Roth and M. J. Black, "Fields of experts: A framework for learning image priors," in Proc. CVPR, 2005, vol. 2, pp. 860-867.
45. S. P. Awate and R. T. Whitaker, "Unsupervised, information- theoretic, adaptive image filtering for image restoration," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 3, pp. 364-376, Mar. 2006.
46. J. Mairal, G. Sapiro, and M. Elad, "Learning multiscale sparse representations for image and video restoration," SIAM Multiscale Modeling and Simulation, vol. 7, no. 1, pp. 214-241, 2008.
47. R. Dugad and N. Ahuja, "Video denoising by combining Kalman and Wiener estimates," in Proc. Int. Conf. Image Process., 1999, pp. 152-156.
48. I. W. Selesnick and K. Y. Li, "Video denoising using 2-D and 3-D dual-tree complex wavelet transforms," in Proc. Wavelets: Appl. Signal Image Processing X., Nov. 2003, vol. 5207, pp. 607-618.
49. D. Rusanovskyy and K. Egiazarian, "Video denoising algorithm in sliding 3d dct domain," in Proc. 7th Advanced Concepts Intelligent Vis. Syst., Sep. 2005, pp. 618-625.
50. E. Baister, Y. Zheng, and R. Ewing, "Combined spatial and temporal domain wavelet shrinkage algorithm for video denoising," IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 2, pp. 220-230, Feb. 2006.
51. V. Zlokolica, A. Pizurica, and W. Philips, "Wavelet-domain video denoising based on reliability measures," IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 8, pp. 993-1007, Aug. 2006.
52. T. Moldovan, S. Roth, and M. Black, "Denoising archival films using a learned bayesian model," in Proc. ICIP, 2006, pp. 2641-2644.
53. L. Guo, O. Au, M. Ma, and Z. Liang, "Temporal video denoising based on multihypothesis motion compensation," IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 10, pp. 1423-1429, Oct. 2007.
54. M. Bertalmio, V. Caselles, and A. Pardo, "Movie denoising by average of warped lines," IEEE Trans. Image Process., vol. 16, no. 9, pp. 2333-2347, Sep. 2007.
55. S. G. Chang, B. Yu, and M. Vetterli, "Adaptive wavelet thresholding for image denoising and compression," IEEE Trans. Image Process., vol. 9, no. 9, pp. 1532-1546, Sep. 2000.
56. R. E. Blahut, Theory of Remote Image Formation. Cambridge, U.K.: Cambridge Univ. Press, Jan. 2005.
57. H. Robbins and S. Monro, "A stochastic approximation method," Ann. Math. Statist., vol. 22, no. 3, pp. 400-407, 1951.
58. V. Zlokolica and W. Philips, "Motion- and detail-adaptive denoising of video," in Proc. Image Process.: Algorithms . Systems III., May 2004, vol. 5298, pp. 403-412.
59. G. S. M. Bertalmio and A. Bertozzi, "Navier-stokes, fluid dynamics, and image and video inpainting," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2001, pp. 355-362.
60. Y. Wexler, E. Shechtman, and M. Irani, "Space-time video completion," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2004, pp. 120-127.
61. P. Salama, N. B. Shroff, E. J. Coyle, and E. J. Delp, "Error concealment techniques for encoded video streams," in Proc. ICIP, 1995, pp. 9-12.
62. Y. Wang and Q.-F. Zhu, "Error control and concealment for video communication: A review," Proc. IEEE, vol. 86, no. 5, pp. 974-997, May 1998.
63. W. Zhao, "Motion-based spatial-temporal image repairing," in Proc. ICIP, 2004, pp. I: 291-1: 294.
64. F. Lauze and M. Nielsen, "A variational algorithm for motion compensated inpainting," in Proc. BMVC, 2004, pp. 777-778.
65. C. Rasmussen and T. Korah, "Spatiotemporal inpainting for recovering texture maps of partially occluded building facades," in Proc. ICIP, 2005, pp. III: 125-III: 128.
66. M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, "Image inpainting," in Proc. SIGGRAPH, New Orleans, LA, 2000, pp. 417-424.
67. A. Criminisi, P. Perez, and K. Toyama, "Region filling and object removal by exemplar-based image inpainting," IEEE Trans. Image Process., vol. 13, no. 9, pp. 1200-1212, Sep. 2004.
68. G. Peyre, "Non-negative sparse modeling of textures," in Proc. Scale Space Variational Methods Comput. Vis., 2007, pp. 628-639.
69. S. Aign and K. Fazel, "Temporal and spatial error concealment techniques for hierarchical mpeg-2 video codec," in Proc. Int. Conf. Commun., 1995, vol. 3, pp. 1778-1783.
70. P. KA, S. G., and B. M., "Video inpainting under constrained camera motion," IEEE Trans. Image Process., vol. 16, no. 2, pp. 545-553, Feb. 2007.
71. F. Wang, D. Anastassiou, and A. Netravali, "Time-recursive deinterlacing for idtv and pyramid coding," Signal Process.: IC, vol. 2, no. 3, pp. 365-374, Oct. 1990.
72. K. Jensen and D. Anastassiou, "Digitally assisted deinterlacing for edtv," IEEE Trans. Circuits Syst. Video Technol., vol. 3, no. 2, pp. 99-106, Apr. 1993.
73. J. Kovacevic, R. Safranek, and E. Yeh, "Deinterlacing by successive approximation," IEEE Trans. Image Process., vol. 6, no. 2, pp. 339-344, Feb. 1997.
74. G. D. Haan and E. Bellers, "Deinterlacing-an overview," Proc. IEEE, vol. 86, no. 9, pp. 1839-1857, Sep. 1998.
75. R. Li, B. Zeng, and M. Liou, "Reliable motion detection/compensation for interlaced sequences and its applications to deinterlacing," IEEE Trans. Circuits Syst. Video Technol., vol. 10, no. 1, pp. 23-30, Feb. 2000.
76. S. Yang, Y. Jung, Y. Lee, and R. Park, "Motion compensation assisted motion adaptive interlaced-to-progressive conversion," IEEE Trans. Circuits Syst. Video Technol., vol. 14, no. 9, pp. 1138-1148, Sep. 2004.
77. Y. Chang, S. Lin, C. Chen, and L. Chen, "Video de-interlacing by adaptive 4-field global/local motion compensated approach," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 12, pp. 1569-1582, Dec. 2005.
78. C. e. a. Ballester, "An inpainting-Based deinterlacing method," IEEE Trans. Image Process., vol. 16, no. 10, pp. 2476-2491, Oct. 2007.
79. O. Kwon, K. Sohn, and C. Lee, "Deinterlacing using directional interpolation and motion compensation," Image Video Commun. Processing, vol. 5022, pp. 604-611, May 2003.
80. D. Wang, A. Vincent, and P. Blanchfield, "Hybrid de-interlacing algorithm based on motion vector reliability," IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 8, pp. 1019-1025, Aug. 2005.
81. P. List, A. Joch, J. Lainema, G. Bjntegaard, and M. Karczewicz, "Adaptive deblocking filter," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 614-619, Jul. 2003.
82. B. F. N. Jojic and A. Kannan, "Epitomic analysis of appearance and shape," in Proc. Int. Conf. Comput. Vis., 2003, vol. 1, pp. 34-41.
83. A. Foi, V. Katkovnik, and K. Egiazarian, "Pointwise shape-adaptive dct for high-quality denoising and deblocking of grayscale and color images," IEEE Trans. Image Process., vol. 16, no. 5, pp. 1395-1411, May 2007.
84. S. Arya, D. M. Mount, N. S. Netanyahu, R. Silverman, and A. Wu, "An optimal algorithm for approximate nearest neighbor searching," in Proc. 5th Annu. ACM-SIAM Symp. Discrete Algorithms, 1994, pp. 573-582.
85. S. A. Nene and S. K. Nayar, "A simple algorithm for nearest neighbor search in high dimensions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 19, no. 9, pp. 989-1003, Sep. 1997.