Using programmable graphics hardware for acoustics and audio rendering

AES: Journal of the Audio Engineering Society

Volumn 59, Issue 9, 2011, Pages 628-646

  Tsingos, Nicolas ^a   Jiang, Wenyu ^a   Williams, Ian ^b  

^a DOLBY LABORATORIES   (United States)

^b NVIDIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

3D AUDIO; AUDIO RENDERING; AURALIZATIONS; GENERAL PURPOSE PROCESSORS; GRAPHICS ACCELERATORS; MANY-CORE; MASSIVE PARALLELISM; NUMERICAL ACOUSTICS; PERFORMANCE IMPROVEMENTS; PROCESSING CAPABILITY; PROGRAMMABILITY; PROGRAMMABLE GRAPHICS HARDWARES; SOUND SCATTERING; SOUND SYNTHESIS;

AUDIO SIGNAL PROCESSING; COMPUTER ARCHITECTURE; PROGRAM PROCESSORS; SOUND REPRODUCTION; VIRTUAL REALITY;

AUDIO ACOUSTICS;

EID: 80054026755     PISSN: 15494950     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Review

References (86)

1. M. Ekman, F. Warg, and J. Nilsson, "An In-Depth Look at Computer Performance Growth," SIGARCH Comput. Archit. News, vol. 33, pp. 144-147 (2005).
2. D. Geer, "Taking the Graphics Processor beyond Graphics," Computer, vol. 38, pp. 14-16 (2005 Sept.). (Pubitemid 41387795)
3. A. Bleiweiss, "GPU Accelerated Pathfinding," in Proc. 23rd ACM SIGGRAPH/EUROGRAPHICS Symp. on Graphics Hardware (Aire-la-Ville, Switzerland, 2008), pp. 65-74 (Eurographics Assoc.).
4. GPGPU\-General-Purpose Computation on Graphics Hardware," http://www.gpgpu.org; accessed 2009 May 15.
5. GPGPU\-Tutorial and Courses," http://gpgpu.org/ tag/tutorials- courses; accessed 2009 May 15.
6. J. D. Owens, D. Luebke, N. Govindaraju, M. Harris, J. Krüger, A. E. Lefohn, and T. J. Purcell, "A Survey of General-Purpose Computation on Graphics Hardware," Comput. Graph. Forum, vol. 26, no. 1, pp. 80-113 (2007). (Pubitemid 46481097)
7. M. Pharr, Ed., GPU Gems 2: Programming Techniques, Tips and Tricks for Real-Time Graphics, Part IV: General-Purpose Computation on GPUS: A Primer (Addison-Wesley Professional, 2005), http://http. developer.nvidia.com/GPUGems2/ gpugems2-part04. html.
8. P. Lapsley, J. Bier, A. Shoham, and E. A. Lee, DSP Processor Fundamentals (IEEE Press, New York, 1997).
9. J. Eyre and J. Bier, "The Evolution of DSP Processors," IEEE Signal Process. Mag. (2000); see also http://www.bdti.com/.
10. D. R. Begault, 3D Sound for Virtual Reality and Multimedia (Academic Press Professional, 1994).
11. L. Savioja, J. Huopaniemi, T. Lokki, and R. Väänänen, "Creating Interactive Virtual Acoustic environments," J. Audio Eng. Soc., vol. 47, pp. 675-705 (1999 Sept.).
12. D. Shreiner, M. Woo, J. Neider, and T. Davis, OpenGL Programming Guide: The Official Guide to Learning OpenGL, version 2.1, 6th ed. (Addison-Wesley, Reading, MA, 2007).
13. D. Luebke and G Humphreys, "How GPUs Work," Computer, vol. 40, pp. 96-100 (2007 Feb.).
14. R. Rost, OpenGL® Shading Language, 2nd ed. (Addison-Wesley, Reading, MA, 2006).
15. W. R. Mark, R. Steven Glanville, K. Akeley, and M. J. Kilgard, "Cg: A System for Programming Graphics Hardware in a C-like Language," in Proc. ACM SIG-GRAPH 2003 (New York, NY, 2003), pp. 896-907 (ACM).
16. R. Fernando and M. J. Kilgard, The Cg Tutorial: The Definitive Guide to Programmable Real-Time Graphics (Addison-Wesley Professional, 2003).
17. nVidia, "NVIDIA CUDA C Programming Guide," version 3.1 (2010 May), http://www.nvidia.com/cuda.
18. I. Buck and T. Purcell, "A Toolkit for computation on GPUs," in GPU Gems: Programming Techniques, Tips and Tricks for Real-Time Graphics (Addison-Wesley Professional, 2004), chap. 37, http://http.developer.nvidia. com/GPUGems/gpugems-ch37.html.
19. I. Buck, T. Foley, D. Horn, J. Sugerman, K. Fatahalian, M. Houston, and P. Hanrahan, "Brook for GPUs: Stream Computing on Graphics Hardware," ACM Trans. Graph., vol. 23, pp. 777-786 (2004).
20. J. Owens, "Streaming Architectures and technology Trends," in GPU Gems 2: Programming Techniques, Tips and Tricks for Real-Time Graphics (Addison-Wesley Professional, 2005), chap. 29, http://developer.nvidia. com/object/gpu-gems-2-home.html.
21. K. Moreland and E. Angel, "The FFT on a GPU," in Proc. ACM SIGGRAPH/EUROGRAPHICS Conf. on Graphics Hardware (Aire-la-Ville, Switzerland, 2003), pp. 112-119 (Eurographics Assoc.).
22. J. Krüger and R. Westermann, "Linear Algebra Operators for GPU Implementation of Numerical Algo-rithms," in Proc. ACM SIGGRAPH 2003 (New York, NY, 2003), pp. 908-916 (ACM).
23. GPUmat: A GPU Toolbox for MATLAB," http:// gp-you.org/; accessed 2009 May 15.
24. L. Savioja, V. Välimäki, and J. O. Smith, "Audio Signal Processing Using Graphics Processing Units," J. Audio Eng. Soc., vol. 59, pp. 3-19 (2011 Jan./Feb.).
25. S. Whalen, "Audio and the Graphics Processing Unit," http://www.node99.org/papers/gpuaudio.pdf; ac-cessed 2009 May 15.
26. E. Gallo and N. Tsingos, "Efficient 3D Audio Processing with the GPU," in Proc. ACM Workshop on General Purpose Computing on Graphics Processors (Los Angeles, CA, 2004 Aug.), http://www-sop.inria.fr/ reves/projects/GPUAudio/.
27. LiquidSonics, "Reverberate LE GPU Edition," http://www.liquidsonics.com/software.htm, accessed 2009 May 15.
28. N. Schneider, "VST Plugin: Convolution Reverb on nVidia GPUs," http://www.nilsschneider.de; accessed 2009 May 15.
29. Acustica Audio, "Nebula 3 VST Plugin," http:// www.acusticaudio.net/; accessed 2009 May 15.
30. Adobe PixelBender API," http://labs.adobe.com/ technologies/ pixelbender; accessed 2009 May 15.
31. J. Blauert, Spatial Hearing: The Psychophysics of Human Sound Localization (MIT Press, Cambridge, MA, 1997).
32. V. Larcher, J. M. Jot, G. Guyard, and O. Warusfel, "Study and Comparison of Efficient Methods for 3D Audio Spatialization Based on Linear Decomposition of HRTF Data," presented at the 108th Convention of the Audio Engineering Society, J. Audio Eng. Soc. (Ab-stracts), vol. 48, p. 351 (2000 Apr.), preprint 5097.
33. GPUFFTW: High Performance Power-of-Two FFT Library Using Graphics Processors," http://gamma. cs.unc.edu/GPUFFTW/; accessed 2009 May 15.
34. A. Smirnov and T. Chiueh, "An Implementation of a FIR Filter on a GPU," http://research.alexeysmirnov. name/index.php?area=gr&proj=fog; accessed 2009 May 15.
35. B. Cowan and B. Kapralos, "Real-Time GPU-Base Convolution: A Follow-Up," in Proc. Canada Int. Conf. on the Future of Game Design and Technology (2009 May 12-13).
36. B. Cowan and B. Kapralos, "Spatial Sound for Video Games and Virtual Environments Utilizing Real-Time GPU-Based Convolution," in Proc. 2008 Conf. on Future Play (New York, NY, 2008), pp. 166-172 (ACM).
37. F. Trebien and M. M. Oliveira, "Realistic Real-Time Sound Re-synthesis and Processing for Interactive Virtual Worlds," Visual Comput., vol. 25, pp. 469-477 (2009 May).
38. L. Savioja, V. Välimäki, and J. O. Smith, "Real-Time Additive Synthesis with One Million Sinusoids Using a GPU," presented at the 128th Convention of the Audio Engineering Society, (Abstracts) www.aes.org/ events/128/128thWrapUp.pdf, (May 2010), convention paper 7962.
39. F. Trebien and M. M. Oliveira, Real-Time Audio Processing on the GPU (Charles River Media, 2008), pp. 583-604.
40. Q. Zhang, L. Ye, and Z. Pan, "Physically-Based Sound Synthesis on GPUs," in Proc. 4th Int. Conf. on Entertainment Computing (2007 Sept. 19-21).
41. C. von Tycowicz and J. Loviscach, "A Malleable Drum," inProc. ACMSIGGRAPH2008 (New York, NY, 2008), pp. 1-1 (ACM).
42. P. Boustead, F. Safaei, and M. Dowlatshahi, "DICE: Internet Delivery of Immersive Voice Commu-nication for Crowded Virtual Spaces," in Proc. Virtual Reality, 2005. (2005 Mar.), pp. 35-41 (IEEE). (Pubitemid 41731707)
43. N. Tsingos, "Pre-Computing Geometry-Based Reverberation Effects for Games," in Proc. AES 35th Int. Conf. on Audio for Games (London, UK, 2009 Feb. 11-13).
44. L. Savioja, "Real-Time 3D Finite-Difference Time-Domain Simulation of Low- and Mid-Frequency Room Acoustics," in Proc. Int. Conf. on Digital Audio Effects (DAFx-10), (Graz, Austria, 2010 Sept.).
45. N. Röber, M. Spindler, and M. Masuch, "Wave-guide-Based Room Acoustics through Graphics Hard-ware," in Proc. Int. Computer Music Conf. (ICMC) (2006 Nov. 6-11).
46. N. Raghuvanshi, B. Lloyd, and M.C. Lin, "Efficient Numerical Acoustic Simulation on Graphics Processors Using Adaptive Rectangular Decomposition," inProc. EAA Symp. on Auralization (2009 June 15-17).
47. N. Raghuvanshi, R. Narain, and M. C. Lin, "Efficient and Accurate Sound Propagation Using Adap-tive Rectangular Decomposition," IEEE Trans. Visuali-zation Comput. Graph., vol. 15, no. 5, (2009 Sept./Oct.) pp. 789-801.
48. P. Micikevicius, "3D Finite Difference computation on GPUs Using CUDA," in Proc. 2nd Workshop on General Purpose Processing on Graphics Processing Units (New York, NY, 2009), pp. 79-84 (ACM).
49. S. Siltanen, T. Lokki, and L. Savioja, "Frequency Domain Acoustic Radiance Transfer for Real-Time Auralization," Acta Acustica/Acustica, vol. 95, pp. 106- 117 (2009 Jan./Feb.).
50. N. A. Carr, J. D. Hall, and J. C. Hart, "GPU Algorithms for Radiosity and Subsurface Scattering," in Proc. ACM SIGGRAPH/EUROGRAPHICS Conf. on Graphics Hardware (Aire-la-Ville, Switzerland, 2003), pp. 51-59 (Eurographics Assoc.).
51. C. Dachsbacher, M. Stamminger, G. Drettakis, and F. Durand, "Implicit Visibility and Antiradiance for Interactive Global Illumination," in Proc. ACM SIG-GRAPH2007 (New York, NY, 2007), p. 61 (ACM).
52. J. Lehtinen, M. Zwicker, E. Turquin, J. Kontka-nen, F. Durand, F. Sillion, and T. Aila, "A Meshless Hierarchical Representation for Light Transport," ACM Trans. Graph., vol. 27, no. 3 (2008).
53. T. J. Purcell, I. Buck, W. R. Mark, and P. Hanrahan, "Ray Tracing on Programmable Graphics Hardware," ACM Trans. Graph., vol. 21, pp. 703-712 (2002 July), (Proc. ACMSIGGRAPH2002).
54. N. Röber, U. Kaminski, and M. Masuch, "Ray-Acoustics Using Computer Graphics Technology," in Proc. 10th Int. Conf. on Digital Audio Effects (DAFx) (2007 Sept. 10-15).
55. M. Jedrzejewski and K. Marasek, "Computation of Room Acoustics Using Programmable Video Hardware," in Proc. Computer Vision and Graphics Int. Conf. (ICCVG 2004), (Warsaw, Poland, 2004 Sept.).
56. S. G. Parker, J. Bigler, A. Dietrich, H. Friedrich, J. Hoberock, D. Luebke, D. McAllister, M. McGuire, K. Morley, A. Robison, and M. Stich, "Optix: A General Purpose Ray Tracing Engine," ACM Trans. Graph. (2010 Aug.).
57. A. Chandak, C. Lauterbach, M. Taylor, Z. Ren, and D. Manocha, "AD-Frustum: Adaptive Frustum Tracing for Interactive Sound Propagation," IEEE Trans. Visualization Comput. Graph., vol. 14, pp. 1707-1722 (2008 Nov.-Dec.).
58. N. Tsingos and J.-D. Gascuel, "Soundtracks for Computer Animation: Sound Rendering in Dynamic Environments with Occlusions," in Proc. Graphics Interface'97 (1997 May), pp. 9-16.
59. B. Cowan and B. Kapralos, "Real-Time Acous-tical Diffraction Modeling Using the GPU," in Proc. 10th Western Pacific Acoustics Conf. (2009 Sept. 21-23).
60. N. Tsingos and J.-D. Gascuel, "Fast Rendering of Sound Occlusion and Diffraction Effects for Virtual Acoustic Environments," presented at the 104th Conven-tion of the Audio Engineering Society, J. Audio Eng. Soc. (Abstracts), vol. 46, p. 572 (1998 June), preprint 4699.
61. N. Tsingos, C. Dachsbacher, S. Lefebvre, and M. Dellepiane, "Instant Sound Scattering," in Proc. Euro-graphics Symp. on Rendering (2007).
62. C. Dachsbacher and M. Stamminger, "Reflective Shadow Map," in Proc. I3D'05 (2005).
63. S. Siltanen, T. Lokki, L. Savioja, and C. Christensen, "Geometry Reduction in Room Acoustics Modeling," Acta Acustica/Acustica, pp. 410-418 (2008).
64. L. M. Wang, J. Rathsam, and S. R. Ryherd, "Interactions of Model Detail Level and Scattering Coefficients in Room Acoustic Computer Simulation," in Proc. Intl. Symp. on Room Acoustics (satelite symposium of ICA, Kyoto, Japan, 2004).
65. C. JoslinandN. Magnenat-Thalmann, "Significant Facet Retrieval for Real-Time 3D Sound Rendering in Complex Virtual Environments," in Proc. Virtual Reality Software and Technology (VRST) 2003 (2003 Oct.).
66. R. L. Cook, L. Carpenter, and E. Catmull, "The Reyes Image Rendering Architecture," SIGGRAPH Comput. Graph., vol. 21, pp. 95-102 (1987).
67. J. Cohen, M. Olano, and D. Manocha, "Appear-ance-Preserving Simplification," in Proc. 25th Ann. Conf. on Computer Graphics and Interactive Techniques (New York, NY, 1998), pp. 115-122 (ACM Press).
68. N. Tsingos, C. Dachsbacher, S. Lefebvre, and M. Dellepiane, "Extending Geometrical Acoustics to Highly Detailed Architectural Environments," in Proc. 19th Int. Congr. on Acoustics (2007 Sept.).
69. J. Hirche, A. Ehlert, S. Guthe, and M. Doggett, "Hardware Accelerated Per-Pixel Displacement Map-ping," in Proc. Graphics Interface'04 (Canadian Human-Computer Communications Society, 2004), pp. 153-158.
70. L. Baboud and X. Décoret, "Rendering Geometry with Relief Textures," in Proc. Graphics Interface '06 (2006).
71. U. P. Svensson, R. I. Fred, and J. Vanderkooy, "An Analytic Secondary Source Model of Edge Diffrac-tion Impulse Responses," J. Acoust. Soc. Am., vol. 106, pp. 2331-2344 (1999).
72. U. P. Svensson, P. Calamia, and S. Nakanishi, "Frequency-Domain Edge Diffraction for Finite and Infinite Edges," Acta Acustica/Acustica, vol. 95, no. 3 (2009).
73. E. Wenzel, M. Arruda, D. Kistler, and F. Wightman, "Localization Using Non-individualized Head-Related Transfer Functions," J. Acoust. Soc. Am., vol. 94, pp. 111-123 (1993 July). (Pubitemid 23206530)
74. D. R. Begault, E. M. Wenzel, and M. R. Anderson, "Direct Comparison of the Impact of Head-Tracking, Reverberation, and Individualized Head-Related Transfer Functions on the Spatial Perception of a Virtual Speech Source," J. Audio Eng. Soc., vol. 49, pp. 904-916 (2001 Oct.). (Pubitemid 33090695)
75. J. C. Middlebrooks, E. A. Macpherson, and Z. A. Onsan, "Psychophysical Customization of Directional Transfer Functions for Virtual Sound Localization," J. Acoust. Soc. Am., vol. 108, pp. 3088-3091 (2000).
76. Y. Kahana and P. A. Nelson, "Numerical Modelling of the Spatial Acoustic Response of the Human Pinna," J. Sound and Vib., vol. 292, pp. 148- 178 (2006 Apr.). (Pubitemid 43253684)
77. B. Katz, "Boundary Element Method Calculation of Individual Head-Related Transfer Function. Part I: Rigid Model Calculation," J. Acoust. Soc. Am., vol. 110, pp. 2440-2448 (2001).
78. M. Dellepiane, N. Pietroni, N. Tsingos, M. Asselot, and R. Scopigno, "Reconstructing Head Models from Photographs for Individualized 3D-Audio Process-ing," in Computer Graphics Forum, Special Issue\-Proc. Pacific Graphics, vol. 27, no. 7 (2008).
79. J. Loviscach, "GPU-Based Audio via the VGA Port," in Proc. ACM SIGGRAPH 2008 (New York, NY, 2008), pp. 1-1 (ACM).
80. M. Pharr, A. Lefohn, C. Kolb, P. Lalonde, T. Foley, and G. Berry, "Programmable Graphics\-The Future of Interactive Rendering," http://www. cerlsoundgroup.org/RealTimeMorph/; accessed 2006 Mar. 08.
81. A. J. Berkhout, D. de Vries, and P. Vogel, "Acoustic Control by Wave Field Synthesis," J. Acoust. Soc. Am., vol. 93, pp. 2764-2778 (1993 May). (Pubitemid 23143504)
82. A. O'Donovan, R. Duraiswami, and N. A. Gumerov, "Real Time Capture of Audio Images and Their Use with Video," in Proc. IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (2007 Oct.), pp. 10-13.
83. M. Monks, B. M. Oh, and J. Dorsey, "Audio-ptimization: Goal Based Acoustic Design," IEEE Comput. Graph. Appl., pp. 76-91 (2000 May).
84. P. Cardinal, P. Dumouchel, G. Boulianne, and M. Comeau, "GPU Accelerated Acoustic Likelihood Computations," in Proc. INTERSPEECH (2008).
85. P. R. Dixon, T. Oonishia, and S. Furuia, "Harnessing Graphics Processors for the Fast computation of Acoustic Likelihoods in Speech Recognition," Comput. Speech Lang., vol. 23, pp. 510-526 (2009 Oct.).
86. J. Chong, Y. Yi, A. Faria, N. R. Satish, and K. Keutzer, "Data-Parallel Large Vocabulary Continuous Speech Recognition on Graphics Processors," Tech. Rep. UCB/EECS-2008-69, EECS Dept., University of Cal-ifornia, Berkeley (2008 May).