SPIRAL: Code generation for DSP transforms

Proceedings of the IEEE

Volumn 93, Issue 2, 2005, Pages 232-273

  Püschel, Markus ^a   Moura, José M F ^a   Johnson, Jeremy R ^b   Padua, David ^c   Veloso, Manuela M ^d   Singer, Bryan W ^e   Xiong, Jianxin ^f   Franchetti, Franz ^a   Gačić, Aca ^a   Voronenko, Yevgen ^a   Chen, Kang ^g   Johnson, Robert W ^e   Rizzolo, Nicholas ^h  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b Drexel University   (United States)

^c University of Illinois at Urbana Champaign   (United States)

^d Carnegie Mellon University   (United States)

^e STMicroelectronics Inc ^*  (United States)

^f 3315 Digital Computer Laboratory   (United States)

^g STMICROELECTRONICS   (Italy)

^h UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

Author keywords

Adaptation; Automatic performance tuning; Code optimization; Discrete cosine transform (DCT); Discrete Fourier transform (DFT); Fast Fourier transform (FFT); Filter; Genetic and evolutionary algorithm; High performance computing; Learning; Library generation

Indexed keywords

ADAPTIVE ALGORITHMS; ALGORITHMS; CODES (STANDARDS); COMPUTER ARCHITECTURE; COSINE TRANSFORMS; DISCRETE FOURIER TRANSFORMS; EVOLUTIONARY ALGORITHMS; FAST FOURIER TRANSFORMS; GENETIC ALGORITHMS; MARKOV PROCESSES; SIGNAL THEORY; WAVE FILTERS;

ADAPTATION; AUTOMATIC PERFORMANCE TUNING; CODE OPTIMIZATION; DISCRETE COSINE TRANSFORM (DCT); LINEAR SIGNAL TRANSFORMS; MARKOV DECISION PROCESSES; SEARCH; WAVELET;

DIGITAL SIGNAL PROCESSING;

EID: 19344368072     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2004.840306     Document Type: Conference Paper

References (92)

1. K. Yotov, X. Li, G. Ren, M. Cibulskis, G. DeJong, M. Garzaran, D. Padua, K. Pingali, P. Stodghill, and P. Wu, "A comparison of empirical and model-driven optimization," in Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation (PLDI), 2003, pp. 63-76.
2. T. Kisuki, P. Knijnenburg, and M. O'Boyle, "Combined selection of tile sizes and unroll factors using iterative compilation," in Proc. Parallel Architectures and Compilation Techniques (PACT), 2000, pp. 237-246.
3. W. Abu-Sufah, D. J. Kuck, and D. H. Lawrie, "Automatic program transformations for virtual memory computers," in Proc. Nat. Computer Conf., 1979, pp. 969-974.
4. D. J. Kuck, R. H. Kuhn, D. A. Padua, B. Leasure, and M. Wolfe, "Dependence graphs and compiler optimizations," in Proc. 8th ACM SIGPLAN-SIGACT Symp, Principles of Programming Languages, 1981, pp. 207-218.
5. F. Allen and J. Cocke, "A catalogue of optimizing transformations," in Design and Optimization of Compilers, R. Rustin, Ed. Englewood Cliffs, NJ: Prentice-Hall, 1972, pp. 1-30.
6. M. E. Wolf and M. S. Lam, "A data locality optimizing algorithm," in Proc. ACM SIGPLAN Conf Programming Language Design and Implementation (PLDI), 1991, pp. 30-44.
7. I. Kodukula, N. Ahmed, and K. Pingali, "Data-centric multi-level blocking," in Proc. ACM SIGPLAN Conf Programming Language Design and Implementation (PLDI), 1997, pp. 346-357.
8. K. Kennedy and R. Allen, Optimizing Compilers for Modern Architectures: A Dependence-Based Approach. San Francisco, CA: Morgan Kaufmann, 2001.
9. R. Metzger and Z. Wen, Automatic Algorithm Recognition and Replacement: A New Approach to Program Optimization. Cambridge, MA: MIT Press, 2000.
10. D. Barthou, P. Feautrier, and X. Redon, "On the equivalence of two systems of affine recurrence equations," in Lecture Notes in Computer Science, Euro-Par 2002. Heidelberg, Germany: Springer-Verlag, 2002, vol. 2400, pp. 309-313.
11. R. C. Whaley, A. Petitet, and J. J. Dongarra, "Automated empirical optimization of software and the ATLAS project," Parallel Comput., vol. 27, no. 1-2, pp. 3-35, 2001.
12. J. Demmel, J. Dongarra, V. Eijkhout, E. Fuentes, A. Petitet, R. Vuduc, C. Whaley, and K. Yelick, "Self-adapting linear algebra algorithms and software," Proc. IEEE, vol. 93, no. 2, pp. 293-312, Feb. 2005.
13. E. Anderson, Z. Bai, C. Bischof, S. Blackford, J. Demmel, J. Dongarra, J. Du Croz, A. Greenbaum, S. Hammarling, A. McKenney, and D. Sorensen, LAPACK Users ' Guide, 3rd ed. Philadelphia, PA: Society for Industrial and Applied Mathematics, 1999.
14. K. Yotov, X. Li, G. Ren, M. Garzaran, D. Padua, K. Pingali, and P. Stodghill, "A comparison of empirical and model-driven optimization," Proc. IEEE, vol. 93, no. 2, pp. 358-386, Feb. 2005.
15. E.-I. Im, K. Yelick, and R. Vuduc, "Sparsity: Optimization framework for sparse matrix kernels," Int. J. High Perform. Comput. Appl., vol. 18, no. 1, pp. 135-158, 2004.
16. G. Baumgartner, A. Auer, D. E. Bernholdt, A. Bibireata, V. Choppella, D. Cociorva, X. Gao, R. J. Harrison, S. Hirata, S. Krishanmoorthy, S. Krishnan, C.-C. Lam, Q. Lu, M. Nooijen, R. M. Pitzer, J. Ramanujam, P. Sadayappan, and A. Sibiryakov, "Synthesis of high-performance parallel programs for a class of ab initio quantum chemistry models," Proc. IEEE, vol. 93, no. 2, pp. 276-292, Feb. 2005.
17. G. Baumgartner, D. Bernholdt, D. Cociovora, R. Harrison, M. Nooijen, J. Ramanujan, and P. Sadayappan, "A performance optimization framework for compilation of tensor contraction expressions into parallel programs," in Proc. Int. Workshop High-Level Parallel Programming Models and Supportive Environments [Held in Conjunction With IEEE Int. Parallel and Distributed Processing Symp. (IPDPS)], 2002, pp. 106-114.
18. M. Frigo and S. G. Johnson, "The design and implementation of FFTW3," Proc. IEEE, vol. 93, no. 2, pp. 216-231, Feb. 2005.
19. _, "FFTW: An adaptive software architecture for the FFT," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), vol. 3, 1998, pp. 1381-1384. [Online]. Available: http://www.fftw.org.
20. M. Frigo, "A fast Fourier transform compiler," in Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation (PLDI), 1999, pp. 169-180.
21. D. Mirković and S. L. Johnsson, "Automatic performance tuning in the UHFFT library," in Lecture Notes in Computer Science, Computational Science - ICCS 2001. Heidelberg, Germany: Springer-Verlag, 2001, vol. 2073, pp. 71-80.
22. S. Egner, "Zur Algorithmischen Zerlegungstheorie Linearer Transformationen Mit Symmetrie (On the algorithmic decomposition theory of linear transforms with symmetry)," Ph.D. dissertation, Institut für Informatik, Univ. Karlsruhe, Karlsruhe, Germany, 1997.
23. K. S. Gatlin and L. Carter, "Faster FFT's via architecture cognizance," in Proc. Parallel Architectures and Compilation Techniques (PACT), 2000, pp. 249-260.
24. _, "Architecture-cognizant divide and conquer algorithms," presented at the Conf. Supercomputing, Portland, OR, 1999.
25. D. H. Bailey, "Unfavorable strides in cache memory systems," Sci. Program., vol. 4, pp. 53-58, 1995.
26. N. Park, D. Kang, K. Bondalapati, and V. K. Prasanna, "Dynamic data layouts for cache-conscious factorization of DFT," in Proc. IEEE Int. Parallel and Distributed Processing Symp. (IPDPS), 2000, pp. 693-701.
27. J. Johnson and M. Püschel, "In search for the optimal Walsh-Hadamard transform," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), vol. 4, 2000, pp. 3347-3350.
28. J. Lebak, J. Kepner, H. Hoffmann, and E. Rutledge, "Parallel VSIPL++: An open standard software library for high-performance parallel signal processing," Proc. IEEE, vol. 93, no. 2, pp. 313-330, Feb. 2005.
29. M. Arnold, S. Fink, V. Sarkar, and P. F. Sweeney, "A comparative study of static and profile-based heuristics for inlining," in Proc. ACM SIGPLAN Workshop Dynamic and Adaptive Compilation and Optimization, 2000, pp. 52-64.
30. P. P. Chang, S. A. Mahlke, W. Y. Chen, and W. M. W. Hwu, "Profileguided automatic inline expansion for C programs," Softw. Pract. Exper., vol. 22, no. 5, pp. 349-369, 1992.
31. A. V. Oppenheim and R. W. Schafer, Discrete-Time Signal Processing, 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 1999.
32. "Information Technology - JPEG 2000 Image Coding System - Part 1: Core Coding System," Int. Org. Standardization/Int. Electrotech. Comm., ISO/IEC 15444-1:2000.
33. J. R. Johnson, R. W. Johnson, D. Rodriguez, and R. Tolimieri, "A methodology for designing, modifying, and implementing Fourier transform algorithms on various architectures," Circuits, Syst., Signal Process., vol. 9, no. 4, pp. 449-500, 1990.
34. C. Van Loan, Computational Framework of the Fast Fourier Transform. Philadelphia, PA: SIAM, 1992.
35. G. E. Révész, Introduction to Formal Languages. New York: McGraw-Hill, 1983.
36. R. Tolimieri, M. An, and C. Lu, Algorithms for Discrete Fourier Transforms and Convolution, 2nd ed. New York: Springer-Verlag, 1997.
37. M. Püschel, "Cooley-Tukey FFT like algorithms for the DCT," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), vol. 2, 2003, pp. 501-504.
38. P. P. Vaidyanathan, Multirate Systems and Filter Banks. Englewood Cliffs, NJ: Prentice-Hall, 1993.
39. G. Strang and T. Nguyen, Wavelets and Filter Banks. Reading, MA: Addison-Wesley, 1998.
40. I. Daubechies and W. Sweldens, "Factoring wavelet transforms into lifting steps," J. Fourier Anal. Appl., vol. 4, no. 3, pp. 247-269, 1998.
41. A. Graham, Kronecker Products and Matrix Calculus With Applications. New York: Wiley, 1981.
42. N. Dershowitz and D. A. Plaisted, "Rewriting," in Handbook of Automated Reasoning, A. Robinson and A. Voronkov, Eds. New York: Elsevier, 2001, vol. 1, ch. 9, pp. 535-610.
43. M. Püschel, B. Singer, M. Veloso, and J. M. F. Moura, "Fast automatic generation of DSP algorithms," in Lecture Notes in Computer Science, Computational Science - ICCS 2001, Heidelberg, Germany: Springer - Verlag, 2001, vol. 2073, pp. 97-106.
44. M. Püschel, B. Singer, J. Xiong, J. M. F. Moura, J. Johnson, D. Padua, M. Veloso, and R. W. Johnson, "SPIRAL: A generator for platform-adapted libraries of signal processing algorithms," Int. J. High Perform. Comput. Appl., vol. 18, no. 1, pp. 21-45, 2004.
45. (1997) GAP - Groups, algorithms, and programming. GAP Team, Univ. St. Andrews, St. Andrews, U.K. [Online]. Available: http://www-gap.dcs.st-and.ac.uk/ ~gap/
46. N. D. Jones, C. K. Gomard, and P. Sestoft, Partial Evaluation and Automatic Program Generation. Englewood Cliffs, NJ: Prentice-Hall, 1993.
47. J. Xiong, J. Johnson, R. Johnson, and D. Padua, "SPL: A language and compiler for DSP algorithms," in Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation (PLDI), 2001, pp. 298-308.
48. N. Rizzolo and D. Padua, "Hilo: High level optimization of FFTs," presented at the Workshop Languages and Compilers for Parallel Computing (LCPC), West Lafayette, IN, 2004.
49. Y. Voronenko and M. Püschel, "Automatic generation of implementations for DSP transforms on fused multiply-add architectures," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), vol. 5, 2004, pp. V-101-V-104.
50. C. W. Fraser, D. R. Hanson, and T. A. Proebsting, "Engineering a simple, efficient code-generator generator," ACM Lett. Program. Lang. Syst., vol. 1, no. 3, pp. 213-226, 1992.
51. E. Linzer and E. Feig, "Implementation of efficient FFT algorithms on fused multiply-add architectures," IEEE Trans. Signal Process., vol. 41, no. 1, p. 93, Jan. 1993.
52. C. Lu, "Implementation of multiply-add FFT algorithms for complex and real data sequences," in Proc. Int. Symp. Circuits and Systems (ISCAS), vol. 1, 1991, pp. 480-483.
53. F. Franchetti and M. Püschel, "A SIMD vectorizing compiler for digital signal processing algorithms," in Proc. IEEE Int. Parallel and Distributed Processing Symp. (IPDPS), 2002, pp. 20-26.
54. _, "Short vector code generation for the discrete Fourier transform," in Proc. IEEE Int. Parallel and Distributed Processing Symp. (IPDPS), 2003, pp. 58-67.
55. F. Franchetti, "Performance portable short vector transforms," Ph.D. dissertation, Inst. Appl. Math. Numer.Anal., Vienna Univ. Technol., 2003.
56. F. Franchetti, S. Kral, J. Lorenz, and C. Ueberhuber, "Efficient utilization of SIMD extensions," Proc. IEEE, vol. 93, no. 2, pp. 409-425, Feb. 2005.
57. R. E. J. Hoeflinger, Z. Li, and D. Padua, "Experience in the automatic parallelization of four perfect benchmark programs," in Lecture Notes in Computer Science, Languages and Compilers for Parallel Computing, vol. 589. Heidelberg, Germany, 1992, pp. 65-83.
58. R. E. J. Hoeflinger and D. Padua, "On the automatic parallelization of the perfect benchmarks," IEEE Trans. Parallel Distrib. Syst., vol. 9, no. 1, pp. 5-23, Jan. 1998.
59. K. Chen and J. R. Johnson, "A prototypical self-optimizing package for parallel implementation of fast signal transforms," in Proc. IEEE Int. Parallel and Distributed Processing Symp. (IPDPS), 2002, pp. 58-63.
60. _, "A self-adapting distributed memory package for fast signal transforms," in Proc. IEEE Int. Parallel and Distributed Processing Symp. (IPDPS), 2004, p. 44.
61. (1998) OpenMP C and C++ Application Program Interface, Version 1.0. OpenMP. [Online], Available: http://www.openmp.org
62. W. Gropp, S. Huss-Lederman, A. Lumsdaine, E. Lusk, B. Nitzberg, W. Saphir, and M. Snir, MPI: The Complete Reference, 2nd ed. Cambridge, MA: MIT Press, 1998.
63. P. Kumhom, "Design, optimization, and implementation of a universal FFT processor," Ph.D. dissertation, Dept. Elect. Comput. Eng., Drexel Univ., Philadelphia, PA, 2001.
64. F. Ergün, "Testing multivariate linear functions: Overcoming the generator bottleneck," in Proc. ACM Symp. Theory of Computing (STOC), vol. 2, 1995, pp. 407-416.
65. J. Johnson, M. Püschel, and Y. Voronenko, "Verification of linear programs," presented at the Int. Symp. Symbolic and Algebraic Computation (ISSAC), London, ON, Canada, 2001.
66. S. Winograd, Arithmetic Complexity of Computations, ser. CBMS-NSF Regional Conf. Ser. Appl. Math. Philadelphia, PA: SIAM, 1980.
67. J. R. Johnson and A. F. Breitzman, "Automatic derivation and implementation of fast convolution algorithms," J. Symbol. Comput., vol. 37, no. 2, pp. 261-293, 2004.
68. E. Linzer and E. Feig, "New scaled DCT algorithms for fused multiply/add architectures," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), vol. 3, 1991, pp. 2201-2204.
69. N. Higham, Accuracy and Stability of Numerical Algorithms, 2nd ed. Philadelphia, PA: SIAM, 2002.
70. P. R. Cappello and K. Steiglitz, "Some complexity issues in digital signal processing," IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-32, no. 5, pp. 1037-1041, Oct. 1984.
71. O. Gustafsson, A. Dempster, and L. Wanhammar, "Extended results for minimum-adder constant integer multipliers," in IEEE Int. Symp. Circuits and Systems, vol. 1, 2002, pp. I-73-I-76.
72. A. C. Zelinski, M. Püschel, S. Misra, and J. C. Hoe, "Automatic cost minimization for multiplierless implementations of discrete signal transforms," in Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), vol. 5, 2004, pp. V-221-V-224.
73. M. Püschel, A. Zelinski, and J. C. Hoe, "Custom-optimized multiplierless implementations of DSP algorithms," presented at the Int. Conf. Computer Aided Design (ICCAD), San Jose, CA, 2004.
74. "Information technology-coding of moving pictures and associated audio for digital storage media at up to about 1.5 Mbits/s," Int. Org. Standardization/Int. Electrotech. Comm., ISO/IEC 11 172, 1995.
75. H.-J. Huang, "Performance analysis of an adaptive algorithm for the Walsh-Hadamard transform," M.S. thesis, Dept Comput. Sci., Drexel Univ., Philadelphia, PA, 2002.
76. M. Furis, "Cache miss analysis of Walsh-Hadamard transform algorithms," M.S. thesis, Dept. Comput. Sci., Drexel Univ., Philadelphia, PA, 2003.
77. A. Parekh and J. R. Johnson, "Dataflow analysis of the FFT," Dept. Comput. Sci., Drexel Univ., Philadelphia, PA, Tech. Rep. DU-CS-2004-01, 2004.
78. J. Johnson, P. Hitczenko, and H.-J. Huang, "Distribution of a class of divide and conquer recurrences arising from the computation of the Walsh-Hadamard transform," presented at the 3rd Colloq. Mathematics and Computer Science: Algorithms, Trees, Combinatorics and Probabilities, Vienna, Austria, 2004.
79. P. Hitczenko, H.-J. Huang, and J. R. Johnson, "Distribution of a class of divide and conquer recurrences arising from the computation of the Walsh-Hadamard transform," Theor. Comput. Sci., 2003, submitted for publication.
80. D. E. Knuth, The Art of Computer Programming: Fundamental Algorithms, 3rd ed. Reading, MA: Addison-Wesley, 1997, vol. 1.
81. D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley, 1989.
82. B. Singer and M. Veloso, "Stochastic search for signal processing algorithm optimization," Proc. Supercomputing, 2001.
83. L. Torgo, "Inductive learning of tree-based regression models," Ph.D. dissertation, Dept. Comput. Sci., Faculty Sci., Univ. Porto, Porto, Portugal, 1999.
84. B. Singer and M. Veloso, "Learning to construct fast signal processing implementations," J. Mach. Learn. Res., vol. 3, pp. 887-919, 2002.
85. _, "Learning to generate fast signal processing implementations, " in Proc. Int. Conf. Machine Learning, 2001, pp. 529-536.
86. B. Singer and M. M. Veloso, "Automating the modeling and optimization of the performance of signal transforms," IEEE Trans. Signal Process., vol. 50, no. 8, pp. 2003-2014, Aug. 2002.
87. W. H. Press, B. P. Flannery, S. A. Teukolsky, and W. T. Vetterling, Numerical Recipes in C: The Art of Scientific Computing, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, 1992.
88. Ph.D. dissertation, Dept. Elect. Comput. Eng., Carnegie Mellon Univ., Pittsburgh, PA.
89. S. R. Ladd. (2004) ACOVEA: Analysis of Compiler Options via Evolutionary Algorithm. [Online]. Available: http://www.coyotegulch.com/acovea/
90. RS/6000 Enterprise Server Model S80, Technology and Architecture. IBM. [Online]. Available: http://www.rs6000.ibm.com/resource/technology/s80techarch. html
91. T. C. Mowry, M. S. Lam, and A. Gupta, "Design and evaluation of a compiler algorithm for prefetching," in Proc. Int. Conf. Architectural Support for Programming Languages and Operating Systems, 1992, pp. 62-73.
92. L. J. Chang, I. Hong, Y. Voronenko, and M. Püschel, "Adaptive mapping of linear DSP algorithms to fixed-point arithmetic," presented at the Workshop High Performance Embedded Computing (HPEC), Lexington, MA, 2004.