Video compression-from concepts to the H.264/AVC standard

Proceedings of the IEEE

Volumn 93, Issue 1, 2005, Pages 18-31

  Sullivan, Gary J ^a   Wiegand, Thomas ^b  

^a MICROSOFT   (United States)

^b FRAUNHOFER HEINRICH HERTZ INSTITUTE   (Germany)

Author keywords

Advanced Video Coding (AVC); Compression; H.264; H.26x; Joint Video Team (JVT); Moving Picture Experts Group (MPEG); MPEG 4; Standards; Video; Video coding; Video Coding Experts Group (VCEG); Video compression

Indexed keywords

DECODING; IMAGE CODING; IMAGE COMMUNICATION SYSTEMS; MOTION PICTURES; NETWORK PROTOCOLS; ROBUSTNESS (CONTROL SYSTEMS); SIGNAL TO NOISE RATIO; STANDARDS; VIDEODISKS;

ADVANCED VIDEO CODING (AVC); H.264; H.26X; JOINT VIDEO TEAM (JVT); MOVING PICTURE EXPERTS GROUP (MPEG); MPEG-4; VIDEO CODING EXPERTS GROUP (VCEG);

IMAGE COMPRESSION;

EID: 20744456892     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2004.839617     Document Type: Article

References (78)

1. "Generic coding of moving pictures and associated audio information-Part 1: Systems," Int. Telecommun. Union-Telecommun. (ITU-T) and Int. Standards Org./Int. Electrotech. Comm. (ISO/IEC) JTC 1, Recommendation H.222.0 and ISO/IEC 13 818-1 (MPEG-2 Systems), Nov. 1994.
2. "Narrow-band visual telephone systems and terminal equipment," Int. Telecommun. Union-Telecommun. (ITU-T), Recommendation H.320, 1999.
3. "Packet-based multimedia communications systems," Int. Telecommun. Union-Telecommun. (ITU-T), Recommendation H.323, 1998.
4. "Terminal for low bit rate multimedia communication," Int. Telecommun. Union-Telecommun. (ITU-T), Recommendation H.324, 1996.
5. J. Rosenberg, H. Schulzrinne, G. Camarillo, A. Johnston, J. Peterson, R. Sparks, M. Handley, and E. Schooler, "SIP: Session Initiation Protocol," Internet Eng. Task Force (IETF), Request for Comments (RFC) 3261, 2002.
6. H. Schulzrinne, S. Casner, R. Frederick, and V. Jacobson, "RTP: A transport protocol for real-time applications," Internet Eng. Task Force (IETF), Request for Comments (RFC) 1889, 1996.
7. C. E. Shannon, "Coding theorems for a discrete source with a fidelity criterion," IRE Nat. Conv. Rec., pt. 4, pp. 142-163, 1959.
8. "Studio encoding parameters of digital television for standard 4:3 and wide-screen 16:9 aspect ratios," Int. Telecommun. Union-Radiocommun. (ITU-R), Recommendation BT.601-5, 1995.
9. H. Enomoto and K. Shibata, "Features of Hadamard transformed television signal," presented at the Nat. Conf. IECE Jpn., 1965, Paper 881.
10. H. C. Andrews and W. K. Pratt, "Fourier transform coding of images," in Proc. Hawaii Int. Conf. System Sciences, 1968, pp. 677-679.
11. "Digital Compression and Coding of Continuous-Tone Still Images," Int. Telecommun. Union-Telecommun. (ITU-T) and Int. Standards Org./Int. Electrotech. Comm. (ISO/IEC) Joint Tech. Comm. (JTC) 1, Recommendation T.81 and ISO/IEC 10918-1 (JPEG), Sep. 1992.
12. N. Ahmed, T. Natarajan, and K. R. Rao, "On image processing and a discrete cosine transform," IEEE Trans. Comput., vol. C-23, no. 1, pp. 90-93, Jan. 1974.
13. R. D. Kell, "Improvements relating to electric picture transmission systems," British Patent 341 811, 1929.
14. C. Reader, "History of MPEG video compression-Ver. 4.0," Joint Video Team (JVT), JVT-E066, Oct. 2002.
15. F. W. Mounts, "A video encoding system with conditional picture element replenishment," Bell Syst. Tech. J., vol. 48, no. 7, pp. 2545-2554, Sep. 1969.
16. "Codec for videoconferencing using primary digital group transmission," Int. Telecommun. Union-Telecommun. (ITU-T), Recommendation H.120, version 1, 1984; version 2, 1988; version 3, 1993.
17. A. Habibi, "Hybrid coding of pictorial data," IEEE Trans. Commun., vol. COM-22, no. 5, pp. 614-624, May 1974.
18. "Video codec for audiovisual services at p × 64 kbit/s," Int. Telecommun. Union-Telecommun. (ITU-T), Recommendation H.261, version 1, 1990; version 2, 1993.
19. "Coding of moving pictures and associated audio for digital storage media at up to about 1.5 Mbit/s-Part 2: Video," Int. Standards Org./Int. Electrotech. Comm. (ISO/IEC) JTC 1, ISO/IEC 11 172-2 (MPEG-1), Mar. 1993.
20. "Generic coding of moving pictures and associated audio information-Part 2: Video," Int. Telecommun. Union-Telecommun. (ITU-T) and Int. Standards Org./Int. Electrotech. Comm. (ISO/IEC) JTC 1, Recommendation H.262 and ISO/IEC 13 818-2 (MPEG-2 Video), Nov. 1994.
21. "Video coding for low bit rate communication," Int. Telecommun. Union-Telecommun. (ITU-T), Recommendation H.263, version 1, 1995; version 2, 1998; version 3, 2000.
22. "Coding of audio-visual objects-Part 2: Visual," Int. Standards Org./Int. Electrotech. Comm. (ISO/IEC) JTC 1, ISO/IEC 14 496-2 (MPEG-4 visual version 1), 1999-2003.
23. "Advanced video coding for generic audiovisual services," Int. Telecommun. Union-Telecommun. (ITU-T) and Int. Standards Org./Int. Electrotech. Comm. (ISO/IEC) JTC 1, Recommendation H.264 and ISO/IEC 14 496-10 (MPEG-4) AVC, 2003.
24. Y. Taki, M. Hatori, and S. Tanaka, "Interframe coding that follows the motion," in Proc, Institute of Electronics and Communication Engineers Jpn. Annu. Conv. (IECEJ), 1974, p. 1263.
25. J. R. Jain and A. K. Jain, "Displacement measurement and its application in interframe image coding," IEEE Trans. Commun., vol. COM-29, no. 12, pp. 1799-1808, Dec. 1981.
26. S. Brofferio and F. Rocca, "Interframe redundancy reduction of video signals generated by translating objects," IEEE Trans. Commun., vol. COM-25, pp. 448-455, Apr. 1977.
27. B. Girod, "The efficiency of motion-compensating prediction for hybrid coding of video sequences," IEEE J. Sel. Areas Commun., vol. 5, no. 7, pp. 1140-1154, Aug. 1987.
28. _, "Motion-compensating prediction with fractional-pel accuracy," IEEE Trans. Commun., vol. 41, no. 4, pp. 604-612, Apr. 1993.
29. G. J. Sullivan and R. L. Baker, "Motion compensation for video compression using control grid interpolation," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), 1991, pp. 2713-2716.
30. T. Hidaka, "Description of the proposing algorithm and its score for moving image (A part of the proposal package)," Int. Standards Org./Int. Electrotech. Comm. (ISO/IEC) JTC 1, ISO/IEC JTC 1/SC 2/WG 8 MPEG 89/188, Oct. 1989.
31. F. Giorda and A. Racciu, "Bandwidth reduction of video signals via shift vector transmission," IEEE Trans. Commun., vol. 23, no. 9, pp. 1002-1004, Sep. 1975.
32. G. J. Sullivan, "Multi-hypothesis motion compensation for low bitrate video coding," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), 1993, pp. 437-440.
33. M. T. Orchard and G. J. Sullivan, "Overlapped block motion compensation: An estimation-theoretic approach," IEEE Trans. Image Process., vol. 3, no. 5, pp. 693-699, Sep. 1994.
34. M. Flierl, T. Wiegand, and B. Girod, "A locally optimal design algorithm for block-based multi-hypothesis motion-compensated prediction," in Proc. IEEE Data Compression Conf. (DCC), 1998, pp. 239-248.
35. B. Girod, "Efficiency analysis of multi-hypothesis motion-compensated prediction," IEEE Trans. Image Process., vol. 9, no. 2, pp. 173-183, Feb. 2000.
36. T. Wiegand, X. Zhang, and B. Girod, "Long-term memory motioncompensated prediction," IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 1, pp. 70-84, Feb. 1999.
37. T. Wiegand and B. Girod, Multi-Frame Motion-Compensated Prediction for Video Transmission. Norwell, MA: Kluwer, 2001.
38. H. Everett, "Generalized Lagrange multiplier method for solving problems of optimum allocation of resources," Oper. Res., vol. 11, pp. 399-417, 1963.
39. Y. Shoham and A. Gersho, "Efficient bit allocation for an arbitrary set of quantizers," IEEE Trans. Acoust., Speech, Signal Process., vol. 36, pp. 1445-1453, Sep. 1988.
40. P. A. Chou, T. Lookabaugh, and R. M. Gray, "Entropy-constrained vector quantization," IEEE Trans. Acoust., Speech, Signal Process., vol. 37, no. 1, pp. 31-42, Jan. 1989.
41. G. J. Sullivan and R. L. Baker, "Rate-distortion optimized motion compensation for video compression using fixed or variable size blocks," in Proc. IEEE Global Telecommunications Conf. (GLOBECOM), 1991, pp. 85-90.
42. S.-W. Wu and A. Gersho, "Enhanced video compression with standardized bitstream syntax," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), vol. 1, 1993, pp. 103-106.
43. K. Ramchandran, A. Ortega, and M. Vetterli, "Bit allocation for dependent quantization with applications to multiresolution and MPEG video coders," IEEE Trans. Image Process., vol. 3, no. 5, pp. 533-545, Sep. 1994.
44. A. Ortega, K. Ramchandran, and M. Vetterli, "Optimal trellis-based buffered compression and fast approximations," IEEE Trans. Image Process., vol. 3, no. 1, pp. 26-40, Jan. 1994.
45. T. Wiegand, M. Lightstone, D. Mukherjee, T. G. Campbell, and S. K. Mitra, "Rate-distortion optimized mode selection for very low bit rate video coding and the emerging H.263 standard," IEEE Trans. Circuits Syst. Video Technol., vol. 6, no. 2, pp. 182-190, Apr. 1996.
46. M. C. Chen and A. N. Willson, "Design and optimization of a differentially coded variable block size motion compensation system," in Proc. IEEE Int. Conf. Image Processing, vol. 3, 1996, pp. 259-262.
47. G. M. Schuster and A. K. Katsaggelos, "A video compression scheme with optimal bit allocation among segmentation, motion, and residual error," IEEE Trans. Image Process., vol. 6, pp. 1487-1502, Nov. 1997.
48. M. C. Chen and A. N. Willson, "Rate-distortion optimal motion estimation algorithms for motion-compensated transform video coding," IEEE Trans. Circuits Syst. Video Technol., vol. 8, no. 2, pp. 147-158, Apr. 1998.
49. A. Ortega and K. Ramchandran, "Rate-distortion methods for image and video compression: An overview," IEEE Signal Process. Mag., pp. 23-50, Nov. 1998.
50. G. J. Sullivan and T. Wiegand, "Rate-distortion optimization for video compression," IEEE Signal Process. Mag., vol. 15, pp. 74-90, Nov. 1998.
51. T. Wiegand, H. Schwarz, A. Joch, F. Kossentini, and G. J. Sullivan, "Rate-constrained coder control and comparison of video coding standards," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 688-703, Jul. 2003.
52. B. Girod, A. Aaron, S. Rane, and D. Rebollo-Monedero, "Distributed video coding," Proc. IEEE, vol. 93, no. 1, pp. 71-83, Jan. 2005.
53. E. Steinbach, N. Färber, and B. Girod, "Standard compatible extension of H.263 for robust video transmission in mobile environments," IEEE Trans. Circuits Syst. Video Technol., vol. 7, no. 6, pp. 872-881, Dec. 1997.
54. B. Girod and N. Färber, "Feedback-based error control for mobile video transmission," Proc. IEEE, vol. 97, no. 10, pp. 1707-1723, Oct. 1999.
55. Q. F. Zhu and L. Kerofsky, "Joint source coding, transport processing, and error concealment for H.323-based packet video," Proc. SPIE, Visual Commun. Image Process., vol. 3653, pp. 52-62, Jan. 1999.
56. R. O. Hinds, T. N. Pappas, and J. S. Lira, "Joint block-based video source/channel coding for packet-switched networks," in Proc. SPIE, Visual Commun. Image Process., vol. 3309, Jan. 1998, pp. 124-133.
57. R. Zhang, S. L. Regunathan, and K. Rose, "Video coding with optimal Inter/Intra mode switching for packet loss resilience," IEEE J. Sel. Areas Commun., vol. 18, no. 6, pp. 966-976, Jun. 2000.
58. G. Bjøntegaard, "An error resilience method based on back channel signaling and FEC," Telenor R&D, ITU-T, SG15/WP15/1, LBC-95-033, Jan. 1996. Also submitted to ISO/IEC JTC 1/SC 29/WG 11 as MPEG96/M0616.
59. S. Fukunaga, T. Nakai, and H. Inoue, "Error-resilient video coding by dynamic replacing of reference pictures," in IEEE Global Telecommunications Conf. (GLOBECOM), vol. 3, 1996, pp. 1503-1508.
60. Y. Tomita, T. Kimura, and T. Ichikawa, "Error resilient modified inter-frame coding system for limited reference picture memories," in Proc. Picture Coding Symp., 1997, pp. 743-748.
61. M. Budagavi and J. D. Gibson, "Multiframe block motion compensated video coding for wireless channels," in Proc. Asilomar Conf. Signals, Systems, and Computers, vol. 2, 1996, pp. 953-957.
62. _, "Error propagation in motion compensated video over wireless channels," in Proc. IEEE Int. Conf. Image Processing (ICIP), vol. 2, 1997. pp. 89-92.
63. _, "Random lag selection in multi-frame motion compensation," presented at the IEEE Int. Symp. Information Theory, Boston, MA, 1998.
64. T. Wiegand, N. Färber, K. Stuhlmüller, and B. Girod, "Error-resilient video transmission using long-term memory motion-compensated prediction," IEEE J. Sel. Areas Commun., vol. 18, no. 6, pp. 1050-1062, Jun. 2000.
65. P. Salama, N. B. Shroff, and E. J. Delp, "Error concealment in encoded video," in Image Recovery Techniques for Image Compression Applications. Norwell, MA: Kluwer, 1998.
66. W. M. Lam, A. R. Reibman, and B. Liu, "Recovery of lost or erroneously received motion vectors," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), vol. 5, 1993, pp. 417-420.
67. V. Varsa, M. M. Hannuksela, and Y.-K. Wang, "Non-normative error concealment algorithms," Int. Telecommun. Union-Telecommun. (ITU-T), VCEG-N62, Sep. 2001.
68. Y.-K. Wang, M. M. Hannuksela, V. Varsa, A. Hourunranta, and M. Gabbouj, "The error concealment feature in the H.26L test model," in Proc. IEEE Int. Conf. Image Processing (ICIP), vol. 2, 2002, pp. 729-732.
69. T. Wiegand, G. J. Sullivan, G. Bjøntegaard, and A. Luthra, "Overview of the H.264/AVC video coding standard," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 560-576, Jul. 2003.
70. S. Wenger, "H.264/AVC over IP," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 645-656, Jul. 2003.
71. T. Stockhammer, M. M. Hannuksela, and T. Wiegand, "H.264/AVC in wireless environments," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 657-673, Jul. 2003.
72. M. Karczewicz and R. Kurçeren, "The SP and SI frames design for H.264/AVC," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 637-644, Jul. 2003.
73. T. Wedi and H. G. Musmann, "Motion- and aliasing-compensated prediction for hybrid video coding," IEEE Trans. Circuits Syst. Video Technol, vol. 13, no. 7, pp. 577-586, Jul. 2003.
74. M. Flierl and B. Girod, "Generalized B pictures and the draft JVT/H.264 video compression standard," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 587-597, Jul. 2003.
75. H. Malvar, A. Hallapuro, M. Karczewicz, and L. Kerofsky, "Lowcomplexity transform and quantization in H.264/AVC," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 598-603, Jul. 2003.
76. D. Marpe, H. Schwarz, and T. Wiegand, "Context-adaptive binary arithmetic coding for H.264/AVC," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 620-636, Jul. 2003.
77. P. List, A. Joch, J. Lainema, G. Bjøntegaard, and M. Karczewicz, "Adaptive deblocking filter," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 614-619, Jul. 2003.
78. J. Ribas-Corbera, P. A. Chou, and S. L. Regunathan, "A generalized hypothetical reference decoder for H.264/AVC," IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 674-687, Jul. 2003.