Model-aided coding: a new approach to incorporate facial animation into motion-compensated video coding

IEEE Transactions on Circuits and Systems for Video Technology

Volumn 10, Issue 3, 2000, Pages 344-358

  Eisert, Peter ^b   Wiegand, Thomas ^a,b   Girod, Bernd ^a,b  

^a IEEE   (Germany)

^b UNIVERSITY OF ERLANGEN NUREMBERG   (Germany)

Author keywords

[No Author keywords available]

Indexed keywords

BLOCK BASED VIDEO CODER; FACIAL ANIMATION PARAMETERS; MODEL AIDED CODING; MOTION COMPENSATED PREDICTION; MOVING PICTURES EXPERT GROUP; WAVEFORM CODING;

ANIMATION; COMPUTER SIMULATION; DECODING; IMAGE COMPRESSION; LAGRANGE MULTIPLIERS; MOTION COMPENSATION; SIGNAL TO NOISE RATIO; STANDARDS; THREE DIMENSIONAL; VIDEO CONFERENCING; VIDEO TELEPHONE EQUIPMENT;

IMAGE CODING;

EID: 0033904352     PISSN: 10518215     EISSN: None     Source Type: Journal    
DOI: 10.1109/76.836279     Document Type: Article

References (40)

1. Video coding for low bitrate communication, ITU-T Recommendation H.263 Version 2 (H.263+), Jan. 1998.
2. Generic Coding of Audio-Visual Objects: (MPEG-4 video), Final Draft Int. Standard, Document N2502, 1999.
3. W. J. Welsh, S. Searsby, and J. B. Waite, "Model-based image coding," British Telecom Technol. J., vol. 8, no. 3, pp. 94-106, July 1990.
4. D. E. Pearson, "Developments in model-based video coding," Proc. IEEE, vol. 83, pp. 892-906, June 1995.
5. P. Eisert and B. Girod, "Analyzing facial expressions for virtual conferencing," IEEE Comput. Graph. Applicat., vol. 18, pp. 70-78, Sept. 1998.
6. E. Reusens, R. Castagno, C. le Buhan, L. Piron, T. Ebrahimi, and M. Kunt, "Dynamic video coding - An overview," in Proc. Int. Conf. Image Processing, vol. 2, Lausanne, Switzerland, Sept. 1996, pp. 377-380.
7. M. F. Chowdhury, A. F. Clark, A. C. Downton, E. Morimatsu, and D. E. Pearson, "A switched model-based coder for video signals," IEEE Trans. Circuits Syst. Video Technol. , vol. 4, pp. 216-227, June 1994.
8. H. G. Musmann, "A layered coding system for very low bit rate video coding," Signal Processing: Image Commun., vol. 7, no. 4-6, pp. 267-278, Nov. 1995.
9. M. Kampmann and J. Ostermann, "Automatic adaptation of a face model in a layered coder with an object-based analysis-synthesis layer and a knowledge-based layer," Signal Processing: Image Communication, vol. 9, no. 3, pp. 201-220, Mar. 1997.
10. T. Wiegand, X. Zhang, and B. Girod, "Long-term memory motion-compensated prediction," IEEE Trans. Circuits Syst. Video Technol., vol. 9, pp. 70-84, Feb. 1999.
11. T. Wiegand, E. Steinbach, A. Stensrud, and B. Girod, "Multiple reference picture coding using polynomial motion models," in Proc. SPIE Conf. Visual Communications and Image Processing, San Jose, CA, Feb. 1998, pp. 134-145.
12. E. Steinbach, T. Wiegand, and B. Girod, "Using multiple global motion models for improved block-based video coding," in Proc. IEEE Int. Conf. Image Processing, Kobe, Japan, Oct. 1999.
13. H. Everett III, "Generalized Lagrange multiplier method for solving problems of optimum allocation of resources," Oper. Res., vol. 11, pp. 399-417, 1963.
14. Y. Shoham and A. Gersho, "Efficient bit allocation for an arbitrary set of quantizers, "IEEE Trans. Acoust., Speech, Signal Processing, vol. 36, pp. 1445-1453, Sept. 1988.
15. P. A. Chou, T. Lookabaugh, and R. M. Gray, "Entropy-constrained vector quantization," IEEE Trans. Acoust., Speech, Signal Processing, vol. 37, pp. 31-42, Jan. 1989.
16. M. Rydfalk, "CANDIDE: A parameterized face," Ph.D. dissertation, Linköping University, Linköping , Sweden, 1978.
17. G. Greiner and H. P. Seidel, "Splines in computer graphics: Polar forms and triangular B-spline surfaces," Eurographics, 1993.
18. M. Hoch, G. Fleischmann, and B. Girod, "Modeling and animation of facial expressions based on B-splines," Vis. Comput., vol. 11, pp. 87-95, 1994.
19. J. D. Foley, A. van Dam, S. K. Feiner, and J. F. Hughes, Computer Graphics, Principles and Practice, 2nd ed. Reading, MA: Addison-Wesley, 1990.
20. R. Y. Tsai, "A versatile camera calibration technique for high accuracy 3-D machine vision metrology using off-the-shelf cameras and lenses," IEEE J. Robot. Automat., vol. RA-3, pp. 323-344, Aug. 1987.
21. P. Eisert and B. Girod, "Model-based estimation of facial expression parameters from image sequences," in Proc. Int. Conf. Image Processing, vol. 2, Santa Barbara, Oct. 1997, pp. 418-421.
22. H. Li, P. Roivainen, and R. Forchheimer, "3-D motion estimation in model-based facial image coding," IEEE Trans. Pattern Anal. Machine Intell., vol. 15, pp. 545-555, June 1993.
23. P. Eisert and B. Girod, "Model-based coding of facial image sequences at varying illumination conditions," in Proc. 10th Image and Multidimensional Digital Signal Processing Workshop IMDSP '98, Alpbach, Austria, July 1998, pp. 119-122.
24. J. Ostermann, "Object-based analysis-synthesis coding (OBASC) based on the source model of moving flexible 3-D-objects," IEEE Trans. Image Processing, vol. 3, pp. 705-711, Sept. 1994.
25. D. DeCarlo and D. Metaxas, "The integration of optical flow and deformable models with applications to human face shape and motion estimation," Comput. Vis. Pattern Recognit., pp. 231-238, 1996.
26. C. L. Lawson and R. J. Hanson, Solving Least Squares Problems. Englewood Cliffs, NJ: Prentice-Hall, 1974.
27. J. Stauder, "Estimation of point light source parameters for object-based coding," Signal Processing: Image Commun., vol. 7, no. 4/6, pp. 355-379, Nov. 1995.
28. G. Bozdagi, A. M. Tekalp, and L. Onural, "3-D motion estimation and wireframe adaption including photometric effects for model-based coding of facial image sequences," IEEE Trans. Circuits Systems Video Technol., vol. 4, pp. 246-256, June 1994.
29. C. A. Poynton, "Gamma and its disguises: The nonlinear mappings of intensity in perception, crts, film and video," SMPTE J., pp. 1099-1108, Dec. 1993.
30. Video Codec Test Model, Near Term, Version 10 (TMN-10), Draft 1, Download via anonymous ftp to: standard.pictel.com/video-site/9804_Tam/q15d65d1.doc, Apr. 1998.
31. An Improved H.263-Codec using Rate-Distortion Optimization, Available FTP: standard.pictel.com/video-site/9804_Tam/q15d13.doc, Apr. 1998.
32. G. J. Sullivan and R. L. Baker, "Motion compensation for video compression using control grid interpolation," in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, vol. 4, May 1991, pp. 2713-2716.
33. B. Girod, "Rate-constrained motion estimation," in Proc. SPIE Conf. Visual Communications and Image Processing, Chicago, IL, Sept. 1994, pp. 1026-1034.
34. T. Wiegand, M. Lightstone, T. G. Campbell, and S. K. Mitra, "Efficient mode selection for block-based motion compensated video coding," in Proc. IEEE Int. Conf. Image Processing, Washington, DC, Oct. 1995, pp. 559-562.
35. G. M. Schuster and A. K. Katsaggelos, "Fast and efficient mode and quantizer selection in the rate distortion sense for H.263," in Proc. SPIE Conf. Visual Communications and Image Processing, Orlando, FL, Mar. 1996, pp. 784-795.
36. T. Wiegand, M. Lightstone, D. Mukherjee, T. G. Campbell, and S. K. Mitra, "Rate-distortion optimized mode selection for very low bit rate video coding and the emerging H.263 standard," IEEE Trans. Circuits Syst. Video Technol., vol. 6, pp. 182-190, Apr. 1996.
37. G. J. Sullivan and T. Wiegand, "Rate-distortion optimization for video compression," IEEE Signal Processing Mag., vol. 15, pp. 74-90, Nov. 1998.
38. B. Girod, "Image sequence coding using 3-D scene models," in Proc. Visual Computation and Image Processing (VCIP), vol. 3, Sept. 1994, pp. 1576-1591.
39. M. Magnor and B. Girod, "Fully embedded coding of triangle meshes," in Proc. Vision, Modeling, and Visualization (VMV'99), Erlangen, Germany, Nov 1999, pp. 253-259.
40. T. Vetter and V. Blanz, "Estimating colored 3-D face models from single images: An example based approach," in Proc. Euro. Conf. Computer Vision (ECCV), 1998.