Encoding, fast and slow: Low-latency video processing using thousands of tiny threads

Proceedings of the 14th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2017

Volumn , Issue , 2017, Pages 363-376

  Fouladi, Sadjad ^a   Wahby, Riad S ^a   Shacklett, Brennan ^a   Balasubramaniam, Karthikeyan Vasuki ^b   Zeng, William ^a   Bhalerao, Rahul ^b   Sivaraman, Anirudh ^c   Porter, George ^b   Winstein, Keith ^a  

^a STANFORD UNIVERSITY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ENCODING (SYMBOLS); FUNCTIONAL PROGRAMMING; SIGNAL ENCODING; SYSTEMS ANALYSIS;

COMPRESSION EFFICIENCY; FINE-GRAINED PARALLELISM; LOW LATENCY; PARALLEL COMPUTATION; TINY THREADS; VIDEO ENCODER; VIDEO ENCODINGS; VIDEO PROCESSING;

VIDEO SIGNAL PROCESSING;

EID: 85076889929     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. AARON, A., AND RONCA, D. High quality video encoding at scale. In Netflix Tech Blog (December 9, 2015). http://techblog.netflix.com/2015/12/ high-quality-video-encoding-at-scale.html.
2. AGARWAL, R., KHANDELWAL, A., AND STOICA, I. Succinct: Enabling queries on compressed data. In NSDI (May 2015).
3. AKRAMULLAH, S., AHMAD, I., AND LIOU, M. A data-parallel approach for real-time MPEG-2 video encoding. Journal of Parallel and Distributed Computing 30, 2 (1995), 129–146.
4. Azure Functions—Serverless Architecture. https://azure.microsoft.com/en-us/services/functions/.
5. CHACON, S., AND STRAUB, B. Pro Git: Git Branching - Rebasing, 2nd ed. Apress, Berkeley, CA, USA, 2014. https://git-scm.com/book/en/v2/ Git-Branching-Rebasing.
6. Cisco VNI forecast and methodology, 2015–2020. http://www.cisco.com/c/en/us/solutions/collateral/service-provider/visual-networking-index-vni/ complete-white-paper-c11-481360.html.
7. Xiph Daala repository. https://github.com/xiph/ daala.
8. DEAN, J., AND GHEMAWAT, S. MapReduce: Simplified data processing on large clusters. Commun. ACM 51, 1 (Jan. 2008), 107–113.
9. ELDER. http://web.archive.org/web/20080105113156/http://www.funknmary.de/ bergdichter/projekte/index.php?page=ELDER.
10. Cloud Functions—Serverless Microservices. https://cloud.google.com/functions/.
11. Advanced video coding for generic audiovisual services, May 2003. Rec. ITU-T H.264 and ISO/IEC 14496-10 (http://www.itu.int/rec/T-REC-H.264-201602-I/en).
12. Apache Hadoop. http://hadoop.apache.org.
13. ISARD, M., BUDIU, M., YU, Y., BIRRELL, A., AND FETTERLY, D. Dryad: Distributed data-parallel programs from sequential building blocks. In EuroSys (Mar. 2007).
14. JONAS, E. Microservices and teraflops. http://ericjonas.com/pywren.html (Oct. 2016).
15. JONAS, E., VENKATARAMAN, S., STOICA, I., AND RECHT, B. Occupy the Cloud: Distributed computing for the 99%. https://arxiv.org/abs/1702. 04024 (Feb. 2017).
16. LINPACK_BENCH: The LINPACK benchmark. https://people.sc.fsu.edu/~jburkardt/cpp_src/linpack_bench/linpack_bench.html.
17. Generic coding of moving pictures and associated audio information [MPEG-2] – Part 2: Video, May 1996. Rec. ITU-T H.262 and ISO/IEC 13818-2 (http://www.iso.org/iso/iso_catalogue/catalogue_ics/catalogue_detail_ics.htm?csnumber=61152).
18. Dynamic adaptive streaming over HTTP (DASH) — Part 1: Media presentation description and segment formats, April 2012. ISO/IEC 23009-1 (http://standards.iso.org/ittf/PubliclyAvailableStandards).
19. IBM Bluemix OpenWhisk. http://www.ibm.com/cloud-computing/bluemix/openwhisk/.
20. ROOSENDAAL, T. Sintel. In SIGGRAPH (Aug. 2011).
21. SAMBE, Y., WATANABE, S., YU, D., NAKAMURA, T., AND WAKAMIYA, N. High-speed distributed video transcoding for multiple rates and formats. IEICE- Trans. Inf. Syst. E88-D, 8 (Aug. 2005), 1923–1931.
22. SHEN, K., ROWE, L. A., AND DELP III, E. J. Parallel implementation of an MPEG-1 encoder: faster than real time. In SPIE (July 1995).
23. Tears of Steel: Mango open movie project. https://mango.blender.org.
24. Apache Tez. https://tez.apache.org/.
25. Examples of how to use AWS Lambda. http://docs.aws.amazon.com/lambda/latest/dg/use-cases.html.
26. VP9 Bitstream & Decoding Process Specification Version 0.6, March 2016. http://www.webmproject.org/vp9/.
27. VP8 Encode Parameter Guide. http://www.webmproject.org/docs/encoder-parameters/.
28. WANG, Z., BOVIK, A. C., SHEIKH, H. R., AND SIMONCELLI, E. P. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13, 4 (2004), 600–612.
29. WILKINS, P., XU, Y., QUILLIO, L., BANKOSKI, J., SALONEN, J., AND KOLESZAR, J. VP8 Data Format and Decoding Guide. RFC 6386, Nov. 2011.
30. x264 Settings. http://www.chaneru.com/Roku/HLS/X264_Settings.htm#threads.
31. YU, Y., AND ANASTASSIOU, D. Software implementation of MPEG-II video encoding using socket programming in LAN. In SPIE (Feb. 1994).
32. ZAHARIA, M., CHOWDHURY, M., DAS, T., DAVE, A., MA, J., MCCAULY, M., FRANKLIN, M. J., SHENKER, S., AND STOICA, I. Resilient Distributed Datasets: A fault-tolerant abstraction for in-memory cluster computing. In NSDI (Apr. 2012).