On missing data treatment for degraded video and film archives: A survey and a new Bayesian approach

IEEE Transactions on Image Processing

Volumn 13, Issue 3, 2004, Pages 397-415

  Kokaram, Anil C ^a  

^a TRINITY COLLEGE DUBLIN   (Ireland)

Author keywords

Autoregressive models; Bayesian inference; Composition sampling; Factored sampling; Film and video post production; Gibbs sampling; Image processing; Marginalization; Markov chain Monte Carlo; Missing data reconstruction; Motion estimation; Noise reduction; Video processing; Video restoration

Indexed keywords

IMAGE ANALYSIS; IMAGE SEGMENTATION; MARKOV PROCESSES; MONTE CARLO METHODS; MOTION ESTIMATION; PHOTOGRAPHIC FILMS; SPURIOUS SIGNAL NOISE; VIDEO RECORDING;

AUTOREGRESSIVE MODELS; BAYESIAN INFERENCE; FILM POST PRODUCTION; GIBBS SAMPLING; MISSING DATA TREATMENT; VIDEO POST PRODUCTION; VIDEO RESTORATION; VISUAL DIGITAL MEDIA;

IMAGE RECONSTRUCTION;

ALGORITHM; AUTOMATED PATTERN RECOGNITION; BAYES THEOREM; COMPARATIVE STUDY; COMPUTER ASSISTED DIAGNOSIS; CULTURAL ANTHROPOLOGY; EVALUATION; IMAGE ENHANCEMENT; IMAGE SUBTRACTION; INFORMATION CENTER; INFORMATION RETRIEVAL; METHODOLOGY; PHOTOGRAPHY; QUALITY CONTROL; REPRODUCIBILITY; REVIEW; SENSITIVITY AND SPECIFICITY; SIGNAL PROCESSING; STATISTICAL MODEL; VALIDATION STUDY; VIDEORECORDING;

ALGORITHMS; ANTHROPOLOGY, CULTURAL; ARCHIVES; BAYES THEOREM; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; INFORMATION STORAGE AND RETRIEVAL; MODELS, STATISTICAL; PATTERN RECOGNITION, AUTOMATED; PHOTOGRAPHY; QUALITY CONTROL; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; SIGNAL PROCESSING, COMPUTER-ASSISTED; SUBTRACTION TECHNIQUE; VIDEO RECORDING;

EID: 1942421690     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2004.823815     Document Type: Article

References (81)

1. P. V. M. Roosmalen, A. Kokaram, and J. Biemond, "Noise reduction of image sequences as preprocessing for mpeg2 encoding," in Eur. Conf. Signal Processing (EUSIPCO'98), vol. 4, Sept. 1998, pp. 2253-2256.
2. F. Chen and D. Suter, "Motion estimation for noise reduction in historical films: Mpeg encoding effects," in Proc. 6th Digital Image Computing: Techniques and Applications (DICTA2002) Conf., 2002, pp. 207-212.
3. D. Suter and S. Richardson, "Historical film restoration and video coding," in Proc. Picture Coding Symp., 1996, pp. 389-394.
4. P. Schallauer, G. Thallinger, and M. J. Addis, "Diamant - Digital film manipulation platform," in Proc. IEE Seminar on Digital Restoration of Film and Video Archives (Ref. no. 2001/049), Jan. 2001, pp. 3/1-3/3.
5. S. M. Sommerville, "Archangel - Automated real-time archive restoration," in Proc. IEE Seminar on Digital Restoration of Film and Video Archives (Ref. no. 2001/049), Jan. 2001, pp. 5/1-5/30.
6. A. C. Kokaram, Motion Picture Restoration: Digital Algorithms for Artefact Suppression in Degraded Motion Picture Film and Video. Berlin, Germany: Springer-Verlag, 1998.
7. ____, "Reconstruction of severely degraded image sequence," in Image Analysis and Processing. Berlin, Germany: Springer-Verlag, Sept. 1997, vol. 2, pp. 773-780.
8. D. Sidorov and A. Kokaram, "Suppression of moiré patterns via spectral analysis," Proc. SPIE, vol. 4671, pp. 895-906, Jan. 2002.
9. ____, "Removing moire from degraded video archives," in Proc. 11th Eur. Conf. Signal Processing (EUSIPCO 2002), Sept. 2002.
10. P. J. W. Rayner, S. Armstrong, and A. C. Kokaram, "Restoring video images taken from scratched 2-inch tape," in Workshop on Non-Linear Model Based Image Analysis, NMBIA'98, S. Marshall, N. Harvey, and D. Shah, Eds. Berlin, Germany: Springer-Verlag, Sept. 1998, pp. 83-88.
11. S. Manhall and N. R. Harvey, "Film and Video Archive Restoration Using Mathematical Morphology,", Jan. 2001.
12. S.-J. Ko, S.-H. Lee, S.-W. Jeon, and E.-S. Kang, "Fast digital image stabilizer based on gray-coded bit-plane matching," IEEE Trans. Consumer Electron., vol. 45, pp. 598-603, Aug. 1999.
13. K. Uomori, A. Morimura, H. Ishii, T. Sakaguchi, and Y. Kitamura, "Automatic image stabilizing system by full-digital signal processing," IEEE Trans. Consumer Electron., vol. 36, pp. 510-519, Aug. 1990.
14. J. Tucker and A. de Sam Lazaro, "Image stabilization for a camera on a moving platform," in Proc. IEEE Pacific Rim Conf. Communications, Computers, Signal Processing, vol. 2, May 1993, pp. 734-737.
15. S.-J. Ko, S.-H. Lee, and K.-H. Lee, "Digital image stabilizing algorithms based on bit-plane matching," IEEE Trans. Consumer Electron., vol. 44, pp. 617-622, Aug. 1998.
16. K. Ratakonda, "Real-time digital video stabilization for multimedia applications," in Proc. IEEE Int. Symp. Circuits and Systems, vol. 4, Monterey, CA, May 1998, pp. 69-72.
17. T. Vlachos, "Simple method for estimation of global motion parameters using sparse translational motion vector fields," Electron. Lett., vol. 34, no. 1, pp. 60-62, Jan. 1998.
18. J.-M. Odobez and P. Bouthémy, "Robust multiresolution estimation of parametric motion models," J. Vis. Commun. Image Representat., vol. 6, pp. 348-365, 1995.
19. L. Hill and T. Vlachos, "On the estimation of of global motion using phase correlation for broadcasting applications," in Proc. 7th Int. Conf. Image Processing Applications, vol. 2, July 1999, pp. 721-725.
20. ____, "Global and local motion estimation using higher-order search," in Proc. 5th Meeting on Image Recognition and Understanding (MIRU 2000), vol. 1, July 2000, pp. 18-21.
21. F. Dufaux and J. Konrad, "Efficient, robust and fast global motion estimation for video coding," IEEE Trans. Image Processing, vol. 9, pp. 497-501, 2000.
22. A. Smolic and J.-R. Ohm, "Robust global motion estimation using a simplified m-estimator approach," in Proc. IEEE Int. Conf. Image Processing, Vancouver, BC, Canada, Sept. 2000.
23. W. Qi and Y. Zhong, "New robust global motion estimation approach used in MPEG-4," J. Tsinghua Univ. Sci. Technol., 2001.
24. P. M. B. Van Roosmalen, R. L. Lagendijk, and J. Biemond, "Flicker reduction in old film sequences," in Time-Varying Image Processing and Moving Object Recognition. Amsterdam, The Netherlands: Elsevier, 1997, vol. 4, pp. 9-17.
25. ____, "Correction of intensity flicker in old film sequences," IEEE Trans. Circuits Syst. Video Technol., submitted for publication.
26. T. Vlachos and G. A. Thomas, "Motion estimation for the correction of twin-lens telecine flicker," in IEEE Int. Conf. Image Processing, vol. 1, Sept. 1996, pp. 109-112.
27. P. Read and M.-P. Meyer, Restoration of Motion Picture Film. New York: Butterworth Heinemann, 2000.
28. V. Bruni and D. Vitulano, "Scratch detection via underdamped harmonic motion," in Proc. Int. Conf. Pattern Recognition, vol. III, 2002, pp. 887-890.
29. L. Joyeux, S. Boukir, and B. Besserer, "Film line scratch removal using kalman filtering and bayesian restoration," in Proc. 5th IEEE Workshop on Applications of Computer Vision, 2000, pp. 8-13.
30. L. Joyeux, S. Boukir, B. Besserer, and O. Buisson, "Detection and removal of line scratches in motion picture films," in IEEE Int. Conf. Computer Vision and Pattern Recognition, June 1999, pp. 548-553.
31. ____, "Reconstruction of degraded image sequences. Application to film restoration," Image Vis. Comput., no. 19, pp. 503-516, 2001.
32. A. Kokaram, "Detection and removal of line scratches in degraded motion picture sequences," Signal Process. VIII, vol. I, pp. 5-8, Sept. 1996.
33. A. Kokaram, R. Morris, W. Fitzgerald, and P. Rayner, "Detection of missing data in image sequences," IEEE Trans. Image Processing, pp. 1496-1508, Nov. 1995.
34. "Interpolation of missing data in image sequences," IEEE Trans. Image Processing, pp. 1509-1519, Nov. 1995.
35. P. M. B. van Roosmalen, "Restoration of Archived Film and Video," Ph.D. dissertation, Tech. Univ. Delft, Delft, The Netherlands, Sept. 1999.
36. P. V. M. Roosmalen, A. Kokaram, and J. Biemond, "Fast high quality interpolation of missing data in image sequences using a controlled pasting scheme," in Proc. IEEE Conf. Acoustics Speech and Signal Processing (ICASSP'99), vol. IMDSP 1.2, Mar. 1999, pp. 3105-3108.
37. G. R. Arce, "Multistage order statistic filters for image sequence processing," IEEE Trans. Signal Processing, vol. 39, pp. 1146-1161, May 1991.
38. R. Storey, "Electronic detection and concealment of film dirt," SMPTE J., pp. 642-647, June 1985.
39. M. J. Nadenau and S. K. Mitra, "Blotch and scratch detection in image sequences based on rank ordered differences," in Proc. 5th Int. Workshop on Time-Varying Image Processing and Moving Object Recognition, Sept. 1996.
40. A. C. Kokaram, "Advances in the detection and reconstruction of blotches in archived film and video," in IEE Seminar on Digital Restoration of Film and Video Archives (Ref. no. 2001/049), Jan. 2001, pp. 7/1-7/6.
41. A. Kokaram and S. J. Godsill, "MCMC for joint noise reduction and missing data treatment in degraded video," IEEE Trans. Signal Processing, vol. 50, pp. 189-205, Feb. 2002.
42. R. Storey, "Electronic Detection and Concealment of Film Dirt," 1984.
43. A. Kokaram, R. Bornard, A. Rares, D. Sidorov, J.-H. Chenot, L. Laborelli, and J. Biemond, "Robust and automatic digital restoration systems: Coping with reality," in Proc. Int. Broadcasting Convention (IBC), 2002.
44. A. Kokaram, R. Bornard, A. Rares, D. Sidorov, J.-H. Chenot, L. Laborelli, and J. Biemond, "Automatic digital restoration systems: Coping with reality," J. Soc. Motion Picture Television Eng., to be published.
45. F. Paisan and A. Crise, "Restoration of signals degraded by impulsive noise by means of a low distortion, nonlinear filter," Signal Process, vol. 6, pp. 67-76, 1984.
46. E. D. Ferrandière, Motion Picture Restoration Using Morphological Tools. Norwell, MA: Kluwer, May 1999, pp. 361-368.
47. E. D. Ferrandìere and J. Serra, "Detection of local defects in old motion pictures," in Proc. VII Nat. Symp. Pattern Recognition Image Analysis, Apr. 1997, pp. 145-150.
48. E. Ferrandière, Mathematical Morphology and Motion Picture Restoration. New York: Wiley, 2001.
49. O. Buisson, "Analyse Deséquences d'images Haute résolution, Application à la Restauration Numérique de Films Cinématographiques," Ph.D. dissertation, Univ. de La Rochelle, Rochelle, France, Dec. 1997.
50. E. D. Ferrandière, "Restauration Automatique de Films Anciens," Ph.D. dissertation, Ecole des Mines de Paris, Paris, France, Dec. 1997.
51. O. Buisson, B. Besserer, S. Boukir, and F. Helt, Deterioration Detection for Digital Film Restoration. New York: IEEE Press, June 1997, vol. 1, pp. 78-84.
52. L. Tenze, G. Ramponi, and S. Carrato, "Blotches correction and contrast enhancement for old film pictures," in IEEE Int. Conf. Image Processing, 2000, p. TP06.05.
53. ____, "Robust detection and correction of blotches in old films using spatio-temporal information," Proc. SPIE, Jan. 2002.
54. T. Saito, T. Komatsu, T. Ohuchi, and T. Seto, "Image processing for restoration of heavily-corrupted old film sequences," in Proc. Int. Conf. Pattern Recognition 2000, vol. 3, 2000, pp. 17-20.
55. G. W. Boon, M. N. Chong, S. Kalra, and D. Krishnan, "Bidirectional 3D autoregressive model approach to motion picture restoration," in Proc. IEEE Int. Conf. Acoustics and Signal Processing, Apr. 1996, pp. 2275-2278.
56. S. Kalra, M. N. Chong, and D. Krishnan, "A new autoregressive (AR) model based algorithm for motion picture restoration," in Proc. IEEE Int. Conf. Acoustics and Signal Processing, Apr. 1997, pp. 2557-2560.
57. M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, "Image inpainting," in Proc. SIGGRAPH, 2000.
58. M. Bertalmio et al., "Filling-in by joint interpolation of vector fields and gray levels," IEEE Trans. Image Processing, vol. 10, Aug. 2001.
59. A. Kokaram, "Parametric texture synthesis for filling holes in pictures," in Proc. IEEE Int. Conf. Image Processing 2002, Sept. 2002.
60. ____, "A statistical framework for picture reconstruction using ar models," in Proc. Eur. Conf. Computer Vision Workshop on Statistical Methods in Video Processing (ISBN 0-9 581 044-0-9), Australia, June 2002, pp. 73-78.
61. A. A. Efros and T. K. Leung, "Texture synthesis by nonparametric sampling," in Proc. IEEE Int. Conf. Computer Vision (ICCV), vol. 2, Sept. 1999, pp. 1033-1038.
62. A. A. Efros and W. T. Freeman, "Image quilting for texture synthesis and transfer," in Proc. SIGGRAPH, 2001, pp. 341-346.
63. R. Bornard, "Approaches Probabilistes Appliquèes À La Restauration Numérique D'Archives Télévisées," Ph.D. dissertation, École Centrale Des Arts Et Manufactures, École, Centrale Paris, Paris, France, 2002.
64. R. Bornard, E. Lecan, L. Laborelli, and J.-H. Chenot, "Missing data correction in still images and image sequences," ACM Multimedia, Dec. 2002.
65. R. D. Morris, "Image Sequence Restoration Using Gibbs Distributions," Ph.D. dissertation, Cambridge Univ., Cambridge, U.K., 1995.
66. A. Kokaram and S. Godsill, "Joint detection, interpolation, motion and parameter estimation for image sequences with missing data," in Proc. IEEE Int. Conf. Image Processing, Oct. 1997, pp. 191-194.
67. A. Kokaram, "Practical MCMC for missing data treatment in degraded video," in Proc. Eur. Conf. Computer Vision Workshop Statistical Methods Video Processing, Australia, June 2002, pp. 85-90.
68. A. Kokaram and P. Rayner, "A system for the removal of impulsive noise in image sequences," Proc. SPIE, pp. 322-331, Nov. 1992.
69. R. D. Morris and W. J. Fitzgerald, "Detection and correction of speckle degradation in image sequences using a 3D markov random field," in Proc. Int. Conf. Image Processing: Theory and Applications (IPTA'93), June 1993.
70. J. J. O. Ruanaidh and W. J. Fitgerald, Numerical Bayesian Methods Applied to Signal Processing. Berlin, Germany: Springer-Verlag, 1996.
71. J. Konrad and E. Dubois, "Bayesian estimation of motion vector fields," IEEE Trans. Pattern Anal. Machine Intell., vol. 14, Sept. 1992.
72. J. Besag, "On the statistical analysis of dirty pictures," J. R. Statist. Soc. B, vol. 48, pp. 259-302, 1986.
73. A. Kokaram and S. J. Godsill, "Joint noise reduction, motion estimation, missing data reconstruction and model parameter estimation for degraded motion pictures," Proc. SPIE Int., July 1998.
74. A. Katsagellos, J. Driessen, S. Efstratiadis, and R. Lagendijk, "Spatio-temporal motion compensated noise filtering of image sequences," in Proc. SPIE VCIP, 1989, pp. 61-70.
75. E. Dubois and S. Sabri, "Noise reduction in image sequences using motion compensated temporal filtering," IEEE Trans. Commun., vol. COM-32, pp. 826-831, July 1984.
76. M. N. Chong, P. Liu, W. B. Goh, and D. Krishnan, "A new spatio-temporal MRF model for the detection of missing data in image sequences," in IEEE Int. Conf. Acoustics Signal Processing, vol. 4, Apr. 1997, pp. 2977-2980.
77. J. Biemond, A. Rares, and M. J. T. Reinders, "Statistical analysis of pathological motion areas," in Proc. 2001 IEE Seminar on Digital Restoration of Film and Video Archives, London, U.K., Jan. 2001.
78. ____, "Complex event classification in degraded image sequences," in Proc. IEEE Int. Conf. Image Processing, Thessaloniki, Greece, Oct. 2001.
79. ____, "Image sequence restoration in the presence of pathological motion and severe artifacts," in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing, Orlando, FL, May 2002.
80. ____, "A spatiotemporal image sequence restoration algorithm," in Proc. IEEE Int. Conf. Image Processing 2002, New York, Sept. 2002.
81. I. McLean and S. Witt, "Telecine nose reduction," in Proc. IEE Seminar Digital Restoration of Film and Video Archives, Jan. 2001, pp. 2/1-2/6.