Darkroom: Compiling high-level image processing code into hardware pipelines

ACM Transactions on Graphics

Volumn 33, Issue 4, 2014, Pages

  Hegarty, James ^a   Brunhaver, John ^a   DeVito, Zachary ^a   Ragan Kelley, Jonathan ^b   Cohen, Noy ^a   Bell, Steven ^a   Vasilyev, Artem ^a   Horowitz, Mark ^a   Hanrahan, Pat ^a  

^a STANFORD UNIVERSITY   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Domain specific languages; FPGAs; Hardware synthesis; Image processing; Video processing.

Indexed keywords

ENERGY EFFICIENCY; FIELD PROGRAMMABLE GATE ARRAYS (FPGA); HARDWARE; IMAGE PROCESSING; INTEGER PROGRAMMING; INTERACTIVE COMPUTER GRAPHICS; MULTICORE PROGRAMMING; PIPELINES; PROBLEM ORIENTED LANGUAGES; PROGRAM COMPILERS; SCHEDULING; SEMANTICS; VIDEO SIGNAL PROCESSING; BUFFER STORAGE; DYNAMIC RANDOM ACCESS STORAGE; PIPELINE PROCESSING SYSTEMS;

BATTERY POWERED DEVICES; DOMAIN SPECIFIC LANGUAGES; HARDWARE SYNTHESIS; HIGH ENERGY EFFICIENCY; IMAGE PROCESSING PIPELINE; INTEGER LINEAR PROGRAMS; LOW-LEVEL FEATURE DETECTIONS; VIDEO PROCESSING; ARCHITECTURAL PATTERN; DOMAINS SPECIFIC LANGUAGES; IMAGE SIGNAL; IMAGES PROCESSING; MEMORY BANDWIDTHS; ON-CHIP BUFFERS; SIGNAL PROCESSOR;

PIPELINE CODES; FIELD PROGRAMMABLE GATE ARRAYS (FPGA);

EID: 84905730031     PISSN: 07342071     EISSN: 15577333     Source Type: Journal    
DOI: 10.1145/2601097.2601174     Document Type: Conference Paper

References (33)

1. ADAMS, A., TALVALA, E.-V., PARK, S. H., JACOBS, D. E., AJDIN, B., GELFAND, N., DOLSON, J., VAQUERO, D., BAEK, J., TICO, M., LENSCH, H. P. A., MATUSIK, W., PULLI, K., HOROWITZ, M., AND LEVOY, M. 2010. The Frankencamera: An experimental platform for computational photography. ACM Transactions on Graphics 29, 4 (July), 29:1-29:12.
2. APTINA. Aptina MT9P111. http://www.aptina.com/products/soc/mt9p111/.
3. BERKELAAR, M., EIKLAND, K., NOTEBAERT, P., ET AL. 2004. lpsolve: Open source (mixed-integer) linear programming system. Eindhoven U. of Technology.
4. BILSEN, G., ENGELS, M., LAUWEREINS, R., AND PEPER-STRAETE, J. 1995. Cyclo-static data flow. In 1995 International Conference on Acoustics, Speech, and Signal Processing, vol. 5, 3255-3258.
5. BOUGUET, J.-Y. 2001. Pyramidal implementation of the affine Lucas Kanade feature tracker description of the algorithm. Tech. rep., Intel Corporation.
6. CANNY, J. 1986. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 679-698.
7. DATTA, K., MURPHY, M., VOLKOV, V., WILLIAMS, S., CARTER, J., OLIKER, L., PATTERSON, D., SHALF, J., AND YELICK, K. 2008. Stencil computation optimization and auto-tuning on state-of-the-art multicore architectures. In Proceedings of the 2008 ACM/IEEE conference on Supercomputing, IEEE Press, 4.
8. DEVITO, Z., HEGARTY, J., AIKEN, A., HANRAHAN, P., AND VITEK, J. 2013. Terra: A multi-stage language for highperformance computing. In Proceedings of the 34th ACM SIG-PLAN Conference on Programming Language Design and Implementation, 105-116.
9. ELLIOTT, C. 2001. Functional image synthesis. In Proceedings of Bridges.
10. FRIGO, M., AND STRUMPEN, V. 2005. Cache oblivious stencil computations. In Proceedings of the 19th annual international conference on Supercomputing, ACM, 361-366.
11. GUMMARAJU, J., AND ROSENBLUM, M. 2005. Stream programming on general-purpose processors. In Proceedings of the 38th Annual IEEE/ACM International Symposium on Microarchitecture, IEEE, 343-354.
12. HAMEED, R., QADEER, W., WACHS, M., AZIZI, O., SOLOMAT-NIKOV, A., LEE, B. C., RICHARDSON, S., KOZYRAKIS, C., AND HOROWITZ, M. 2010. Understanding sources of inefficiency in general-purpose chips. In Proceedings of the 37th Annual International Symposium on Computer Architecture, ACM, 37-47.
13. HARRIS, C., AND STEPHENS, M. 1988. A combined corner and edge detector. In Proceedings of the 4th Alvey Vision Conference, 147-151.
14. HOLZMANN, G. 1988. Beyond Photography: The Digital Darkroom. Prentice Hall.
15. KUNG, H. T. 1979. Let's design algorithms for VLSI systems. In Proceedings of the Caltech Conference on Very Large Scale Integration.
16. LATTNER, C., AND ADVE, V. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In Proceedings of the 2004 International Symposium on Code Generation and Optimization (CGO'04).
17. LEE, E. A., AND MESSERSCHMITT, D. G. 1987. Static scheduling of synchronous data flow programs for digital signal processing. IEEE Transactions on Computers 100, 1, 24-35.
18. LEISERSON, C. E., AND SAXE, J. B. 1991. Retiming synchronous circuitry. Algorithmica 6, 1-6, 5-35.
19. LUCAS, B. D., KANADE, T., ET AL. 1981. An iterative image registration technique with an application to stereo vision. In IJCAI, vol. 81, 674-679.
20. MALLADI, K., NOTHAFT, F., PERIYATHAMBI, K., LEE, B., KOZYRAKIS, C., AND HOROWITZ, M. 2012. Towards energy-proportional datacenter memory with mobile dram. In 2012 39th Annual International Symposium on Computer Architecture (ISCA), 37-48.
21. MURALIMANOHAR, N., AND BALASUBRAMONIAN, R. 2009. Cacti 6.0: A tool to understand large caches. Tech. rep., HP Labs.
22. MURTHY, P., BHATTACHARYYA, S., AND LEE, E. 1997. Joint minimization of code and data for synchronous dataflow programs. Formal Methods in System Design 11, 1, 41-70.
23. NGUYEN, A., SATISH, N., CHHUGANI, J., KIM, C., AND DUBEY, P. 2010. 3.5-d blocking optimization for stencil computations on modern cpus and gpus. In in Proc. of the 2010 ACM/IEEE Intl Conf. for High Performance Computing, Networking, Storage and Analysis, 2010, 1-13.
24. OPENCV. OpenCV. http://opencv.org/.
25. QUALCOMM. Qualcomm hexagon SDK. https://developer.qualcomm.com/mobile- development/maximize-hardware/mobile-multimedia-optimization-hexagon-sdk.
26. RAGAN-KELLEY, J., ADAMS, A., PARIS, S., LEVOY, M., AMA-RASINGHE, S., AND DURAND, F. 2012. Decoupling algorithms from schedules for easy optimization of image processing pipelines. ACM Transactions on Graphics (TOG) 31, 4, 32.
27. RAGAN-KELLEY, J., BARNES, C., ADAMS, A., PARIS, S., DURAND, F., AND AMARASINGHE, S. 2013. Halide: A language and compiler for optimizing parallelism, locality, and recompu-tation in image processing pipelines. In Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation, ACM, 519-530.
28. RICHARDSON, W. H. 1972. Bayesian-based iterative method of image restoration. JOSA 62, 1, 55-59.
29. SHACHAM, O., GALAL, S., SANKARANARAYANAN, S., WACHS, M., BRUNHAVER, J., VASSILIEV, A., HOROWITZ, M., DANOWITZ, A., QADEER, W., AND RICHARDSON, S. 2012. Avoiding game over: Bringing design to the next level. In Proceedings of the 49th Annual Design Automation Conference (DAC), 623-629.
30. SHANTZIS, M. A. 1994. A model for efficient and flexible image computing. In Proceedings of the 21st annual conference on Computer graphics and interactive techniques, ACM, 147-154.
31. SUGERMAN, J., FATAHALIAN, K., BOULOS, S., AKELEY, K., AND HANRAHAN, P. 2009. Gramps: A programming model for graphics pipelines. ACM Transactions on Graphics (TOG) 28, 1 (Feb.), 4:1-4:11.
32. TANG, Y., CHOWDHURY, R. A., KUSZMAUL, B. C., LUK, C.-K., AND LEISERSON, C. E. 2011. The Pochoir stencil compiler. In Proceedings of the 23rd ACM Symposium on Parallelism in Algorithms and Architectures, ACM, 117-128.
33. VIVADO. vivado. http://www.xilinx.com/products/design-tools/vivado/ integration/esl-design/.