On the minimization of glass-to-glass and glass-to-algorithm delay in video communication

IEEE Transactions on Multimedia

Volumn 20, Issue 1, 2018, Pages 238-252

  Bachhuber, Christoph ^a   Steinbach, Eckehard ^a   Freundl, Martin ^a   Reisslein, Martin ^b  

^a TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^b Arizona State University   (United States)

Author keywords

Delay analysis; Delay measurement; Frame skipping; Preemption; Prototype system

Indexed keywords

AUTOMATION; CAMERAS; GLASS; IMAGE COMMUNICATION SYSTEMS; MEDIA STREAMING; REAL TIME SYSTEMS;

DELAY ANALYSIS; DELAY MEASUREMENTS; DELAYS; FRAME SKIPPING; MECHANICAL SENSORS; PREEMPTION; PROTOTYPE SYSTEM; PROTOTYPES; ROBOT VISION SYSTEMS; STREAMING MEDIA;

VIDEO CAMERAS;

EID: 85023645352     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2017.2726189     Document Type: Article

References (70)

1. C. Bachhuber and E. Steinbach, "A system for high precision glass-toglass delay measurements in video communication, " in Proc. IEEE Int. Conf. Image Process. 2016, pp. 2132-2136.
2. A.Geiger, P. Lenz, and R.Urtasun, "Are we ready for autonomous driving? The kitti vision benchmark suite, " in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2012, pp. 3354-3361.
3. K. Okumura, H. Oku, and M. Ishikawa, "High-speed gaze controller for millisecond-order pan/tilt camera, " in Proc. IEEE Int. Conf. Robot. Autom., 2011, pp. 6186-6191.
4. N. Alt, C. Claus, and W. Stechele, "Hardware/software architecture of an algorithm for vision-based real-time vehicle detection in dark environments, " in Proc. ACM Conf. Des., Autom. Test Eur., 2008, pp. 176-181.
5. N. Alt and E. Steinbach, "Visuo-haptic sensor for force measurement and contact shape estimation, " in Proc. IEEE Int. Symp. Haptic Audio Vis. Environ. Games, 2013, pp. 24-28.
6. W. Zhang, M. S. Branicky, and S. M. Phillips, "Stability of networked control systems, " IEEE Control Syst., vol. 21, no. 1, pp. 84-99, Feb. 2001.
7. Y. Shi and B. Yu, "Output feedback stabilization of networked control systems with random delays modeled by Markov chains, " IEEE Trans. Automat. Control, vol. 54, no. 7, pp. 1668-1674, Jul. 2009.
8. J. P. Hespanha, P. Naghshtabrizi, and Y. Xu, "A survey of recent results in networked control systems, " Proc. IEEE, vol. 95, no. 1, pp. 138-162, Jan. 2007.
9. G. C. Walsh, H. Ye, and L. G. Bushnell, "Stability analysis of networked control systems, " IEEE Trans. Control Syst. Technol., vol. 10, no. 3, pp. 438-446, May 2002.
10. D. Yue, Q.-L. Han, and C. Peng, "State feedback controller design of networked control systems, " in Proc. IEEE Int. Conf. Control Appl., 2004, vol. 1, pp. 242-247.
11. Y. Tipsuwan and M.-Y. Chow, "Control methodologies in networked control systems, " Control Eng. Pract., vol. 11, no. 10, pp. 1099-1111, Oct. 2003.
12. L. Zhang, H. Gao, and O. Kaynak, "Network-induced constraints in networked control systems-A survey, " IEEE Trans. Ind. Informat., vol. 9, no. 1, pp. 403-416, Feb. 2013.
13. R. Yang, G.-P. Liu, P. Shi, C. Thomas, and M. V. Basin, "Predictive output feedback control for networked control systems, " IEEE Trans. Ind. Electron., vol. 61, no. 1, pp. 512-520, Jan. 2014.
14. M. B. Cloosterman, N.Van deWouw, W.Heemels, and H. Nijmeijer, "Stability of networked control systems with uncertain time-varying delays, " IEEE Trans. Automat. Control, vol. 54, no. 7, pp. 1575-1580, Jul. 2009.
15. G. P. Fettweis, "The tactile internet: Applications and challenges, " IEEE Veh. Technol. Mag., vol. 9, no. 1, pp. 64-70, Mar. 2014.
16. G. Baltas and G. Xylomenos, "Evaluating the impact of network I/O on ultra-low delay packet switching, " in Proc. IEEE Symp. Comput. Commun., 2015, pp. 397-402.
17. O. Bondarenko, K. De Schepper, I.-J. Tsang, B. Briscoe, A. Petlund, and C. Griwodz, "Ultra-low delay for all: Live experience, live analysis, " in Proc. ACM Int. Conf. Multimedia Syst., 2016, pp. 33:1-33:4.
18. B. Briscoe et al., "Reducing internet latency: A survey of techniques and their merits, " IEEE Commun. Surveys Tuts., vol. 18, no. 3, pp. 2149-2196, Third Quarter 2016.
19. C. Greco, M. Cagnazzo, and B. Pesquet-Popescu, "Low-latency video streaming with congestion control in mobile ad-hoc networks, " IEEE Trans. Multimedia, vol. 14, no. 4, pp. 1337-1350, Aug. 2012.
20. U. Jennehag, S. Forsstrom, and F. V. Fiordigigli, "Low delay video streaming on the internet of things using Raspberry Pi, " Electronics, vol. 5, no. 3, pp. 1-11 2016.
21. Y. Liu, D. Niu, and B. Li, "Delay-optimized video traffic routing in software-defined interdatacenter networks, " IEEE Trans. Multimedia, vol. 18, no. 5, pp. 865-878, May 2016.
22. J. Pilz, M. Mehlhose, T. Wirth, D. Wieruch, B. Holfeld, and T. Haustein, "A tactile internet demonstration: 1 ms ultra low delay for wireless communications towards 5G, " in Proc. IEEE Conf. Comput. Commun. Workshops, 2016, pp. 862-863.
23. A. M. Sheikh, A. Fiandrotti, and E. Magli, "Distributed scheduling for low-delay and loss-resilient media streaming with network coding, " IEEE Trans. Multimedia, vol. 16, no. 8, pp. 2294-2306, Dec. 2014.
24. T. H. Szymanski, "An ultra-low-latency guaranteed-rate internet for cloud services, " IEEE/ACMTrans. Netw., vol. 24, no. 1, pp. 123-136, Feb. 2016.
25. J. Wu, C. Yuen, N.-M. Cheung, and J. Chen, "Delay-constrained high definition video transmission in heterogeneous wireless networks with multi-homed terminals, " IEEE Trans. Mobile Comput., vol. 15, no. 3, pp. 641-655, Mar. 2016.
26. J. Wu, B. Cheng, C. Yuen, N.-M. Cheung, and J. Chen, "Trading delay for distortion in one-way video communication over the internet, " IEEE Trans. Circuits Syst. Video Technol., vol. 26, no. 4, pp. 711-723, Apr. 2016.
27. M. Annett, A. Ng, P. Dietz, W. F. Bischof, and A. Gupta, "How low should we go?: Understanding the perception of latency while inking, " in Proc. Graph. Interface Conf., 2014, pp. 167-174.
28. R. Jota, A. Ng, P. Dietz, and D. Wigdor, "How fast is fast enough?: A study of the effects of latency in direct-touch pointing tasks, " in Proc. ACM SIGCHI Conf. Human Factors Comput. Syst., 2013, pp. 2291-2300.
29. K. Mania, B. D. Adelstein, S. R. Ellis, and M. I. Hill, "Perceptual sensitivity to head tracking latency in virtual environments with varying degrees of scene complexity, " in Proc. ACM Symp. Appl. Perception Graph. Vis., 2004, pp. 39-47.
30. S. Kawamura and R. Kijima, "Effect of head mounted display latency on human stability during quiescent standing on one foot, " in Proc. IEEE Virtual Reality, 2016, pp. 199-200.
31. C. Bachhuber and E. Steinbach, "Are todays video communication solutions ready for the tactile internet, " in Proc. IEEE Wireless Commun. Netw. Conf. Workshops, 2017, pp. 1-6.
32. M. Baldi and Y. Ofek, "End-to-end delay analysis of videoconferencing over packet-switched networks, " IEEE/ACM Trans. Netw., vol. 8, no. 4, pp. 479-492, Aug. 2000.
33. A. Vinel, E. Belyaev, K. Egiazarian, and Y. Koucheryavy, "An overtaking assistance system based on joint beaconing and real-time video transmission, " IEEE Trans. Veh. Technol., vol. 61, no. 5, pp. 2319-2329, Jun. 2012.
34. R. Song, Y.-L. Wang, Y. Han, and Y.-S. Li, "Statistically uniform intrablock refresh algorithm for very low delay video communication, " J. Zhejiang Univ. Sci. C, vol. 14, no. 5, pp. 374-382, May 2013.
35. R. M. Schreier, A.M. T. I. Rahman, G. Krishnamurthy, and A. Rothermel, "Architecture analysis for low-delay video coding, " in Proc. IEEE Int. Conf. Multimedia Expo., 2006, pp. 2053-2056.
36. R. M. Schreier and A. Rothermel, "A latency analysis on H.264 video transmission systems, " in Proc. IEEE Int. Conf. Consum. Electron., 2008, pp. 1-2.
37. P. Holub, J. Matela, M. Pulec, and M. ?Srom, "Ultragrid: Low-latency high-quality video transmissions on commodity hardware, " in Proc. ACM Int. Conf. Multimedia, 2012, pp. 1457-1460.
38. T. Inatsuki, M. Matsuura, K. Morinaga, H. Tsutsui, and Y. Miyanaga, "An FPGA implementation of low-latency video transmission system using lossless and near-lossless line-based compression, " in Proc. IEEE Int. Conf. Digit. Signal Process., 2015, pp. 1062-1066.
39. M. U. K. Khan, J. M. Borrmann, L. Bauer, M. Shafique, and J. Henkel, "An H.264 Quad-Full HD low-latency intra video encoder, " in Proc. EDA Conf. Des., Autom. Test Europe, 2013, pp. 115-120.
40. T. Liu and C. Choudary, "Real-time content analysis and adaptive transmission of lecture videos formobile applications, " in Proc. ACMInt. Conf. Multimedia, 2004, pp. 400-403.
41. A. Doulamis and G. Tziritas, "Content-based video adaptation in low/variable bandwidth communication networks using adaptable neural network structures, " in Proc. IEEE Int. Joint Conf. Neural Netw., 2006, pp. 4037-4044.
42. P. Usach-Molina, J. Sastre, V. Naranjo, L. Vergara, and J. M. L. Muñoz, "Content-based dynamic threshold method for real-time keyframe selecting, " IEEE Trans.Circuits Syst. Video Technol., vol. 20, no. 7, pp. 982-993, Jul. 2010.
43. H. Lee and S. Kim, "Rate-constrained key frame selection using iteration, " in Proc. IEEE Int. Conf. Image Process., 2002, vol. 1, pp. I-928-I-931.
44. P. Navakitkanok and S. Aramvith, "Improved rate control for advanced video coding (AVC) standard under low delay constraint, " in Proc. IEEE Int. Conf. Inf. Technol., Coding Comput., 2004, vol. 2, pp. 664-668.
45. C.-Y. Chang, C.-F. Chou, D.-Y. Chan, T. Lin, and M.-H. Chen, "Accurate bitrate model and greedy-based rate controller for low delay video transmission, " IEEE Syst. J., vol. 6, no. 3, pp. 414-425, Sep. 2012.
46. H. Lin, X. He, Q.-Z. Teng, W. Fu, and S. Xiong, "Adaptive bit allocation scheme for extremely low-delay intraframe rate control in high efficiency video coding, " SPIE J. Electron. Imag., vol. 25, no. 4, pp. 043008-1-043008-13, Jul. 2016.
47. S. Sanz-Rodriguez, T. Mayer, M. Alvarez-Mesa, and T. Schierl, "A lowcomplexity parallel-friendly rate control algorithm for ultra-low delay high definition video coding, " in Proc. IEEE Int. Conf. Multimedia Expo. Workshops, 2013, pp. 1-4.
48. J. Ribas-Corbera and S. Lei, "Rate control in DCT video coding for lowdelay communications, " IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 1, pp. 172-185, Feb. 1999.
49. F. Zhang and E. Steinbach, "Improved ρ-domain rate control with accurate header size estimation, " in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2011, pp. 813-816.
50. S. Y. Chien, K. H. Lok, and Y. C. Lu, "Low-decoding-latency buffer compression for graphics processing units, " IEEE Trans. Multimedia, vol. 14, no. 2, pp. 250-263, Apr. 2012.
51. T. Wiegand, G. J. Sullivan, G. Bjøntegaard, and A. Luthra, "Overview of the H.264/AVC video coding standard, " IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp. 560-576, Jul. 2003.
52. D. Grois, D. Marpe, T. Nguyen, and O. Hadar, "Comparative assessment of H. 265/MPEG-HEVC, VP9, and H.264/MPEG-AVC encoders for lowdelay video applications, " in Proc. SPIE, 2014, pp. 92170Q-1-92170Q-10.
53. T. Mallikarachchi, D. S. Talagala, H. K. Arachchi, and A. Fernando, "Content-adaptive feature-based CU size prediction for fast low-delay video encoding in HEVC, " IEEE Trans. Circuits Syst. Video Technol., to be published.
54. G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, "Overview of the high efficiency video coding (HEVC) standard, " IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 12, pp. 1649-1668, Dec. 2012.
55. "x264-A free open-source H.264 encoder, " Jun. 2007. [Online]. Available: http://www.videolan.org/developers/x264.html. Accessed on: Jul. 20, 2017.
56. "x265-A free open-source H.265 encoder, " Apr. 2014. [Online]. Available: http://www.videolan.org/developers/x265.html. Accessed on: Jul. 20, 2017.
57. K. L. Kaiser, Transmission Lines, Matching, and Crosstalk. Boca Raton, FL, USA: CRC Press, 2005.
58. Z.Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity, " IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.
59. Y.-F. Ou, T. Liu, Z. Zhao, Z. Ma, and Y. Wang, "Modeling the impact of frame rate on perceptual quality of video, " in Proc. IEEE Int. Conf. Image Process., Oct. 2008, pp. 689-692.
60. Y.-F. Ou, Z. Ma, T. Liu, and Y. Wang, "Perceptual quality assessment of video considering both frame rate and quantization artifacts, " IEEE Trans. Circuits Syst. Video Technol., vol. 21, no. 3, pp. 286-298, Mar. 2011.
61. Z. Lu et al., "Measuring the negative impact of frame dropping on perceptual visual quality, " Proc. SPIE, vol. 5666, pp. 554-562 2005.
62. R. Hill, C. Madden, A. van den Hengel, H. Detmold, and A. Dick, "Measuring latency for video surveillance systems, " in Proc. IEEE Digit. Image Comput., Technol. Appl., 2009, pp. 89-95.
63. J. Jansen and D. C. Bulterman, "User-centric video delay measurements, " in Proc. ACM Workshop Netw. Oper. Syst. Support Digit. Audio Video, 2013, pp. 37-42.
64. J. MacCormick, "Video chat with multiple cameras, " in Proc. ACM Conf. Comput. Supported Cooperative Work Companion, 2013, pp. 195-198.
65. T. Sielhorst, W. Sa, A. Khamene, F. Sauer, and N. Navab, "Measurement of absolute latency for video see through augmented reality, " in Proc. IEEE/ACM Int. Symp. Mixed Augmented Reality, 2007, pp. 1-4.
66. O. Boyaci, A. Forte, S. A. Baset, and H. Schulzrinne, "vDelay: A tool to measure capture-to-display latency and frame rate, " in Proc. IEEE Int. Symp. Multimedia, 2009, pp. 194-200.
67. M. C. Jacobs et al., "Managing latency in complex augmented reality systems, " in Proc. ACM Symp. Interact. 3D Graph., 1997, pp. 49-54.
68. P. Massart, "The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality, " Ann. Probability, vol. 18, no. 3, pp. 1269-1283, Jul. 1990.
69. A. Dvoretzky, J. Kiefer, and J. Wolfowitz, "Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator, " Ann. Math. Statist., vol. 27, no. 3, pp. 642-669, Sep. 1956.
70. E. Kohler, R.Morris, B. Chen, J. Jannotti, and M. F. Kaashoek, "The click modular router, " ACM Trans. Comput. Syst., vol. 18, no. 3, pp. 263-297, Aug. 2000.