Adaptive Digital Audio Effects (A-DAFx): A new class of sound transformations

IEEE Transactions on Audio, Speech and Language Processing

Volumn 14, Issue 5, 2006, Pages 1817-1831

  Verfaille, Vincent ^a,b   Zölzer, Udo ^a,c   Arfib, Daniel ^d  

^a IEEE   (Canada)

^b MCGILL UNIVERSITY   (Canada)

^c HELMUT SCHMIDT UNIVERSITY   (Germany)

^d LABORATOIRE DE MÉCANIQUE ET D ACOUSTIQUE   (France)

Author keywords

Adaptive control; Feature extraction; Information retrieval; Music; Psychoacoustic models; Signal processing

Indexed keywords

ADAPTIVE DIGITAL AUDIO EFFECTS; PSYCHOACOUSTIC MODELS; SOUND PERCEPTION; SOUND TRANSFORMATIONS;

COMPUTER MUSIC; FEATURE EXTRACTION; INFORMATION RETRIEVAL; SIGNAL PROCESSING;

ADAPTIVE CONTROL SYSTEMS;

EID: 33745105930     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSA.2005.858531     Document Type: Article

References (92)

1. S. Orfanidis, Introduction to Signal Processing. Englewood Cliffs, NJ: Prentice-Hall, 1996.
2. U. Zölzer, Ed., DAFX - Digital Audio Effects. New York: Wiley, 2002.
3. E. Métois, "Musical gestures and audio effects processing," in Proc. COST-G6 Workshop on Digital Audio Effects, Barcelona, Spain, 1998, pp. 249-253.
4. P. Schaeffer, Le Traité des Objets Musicaux. Paris, France: Seuil, 1966.
5. X. Amatriain, J. Bonada, A. Loscos, J. L. Arcos, and V. Verfaille, "Content-based transformations," J. New Music Res., vol. 32, no. 1, pp. 95-114, 2003.
6. E. Zwicker and B. Scharf, "A model of loudness summation," Psychol. Rev., vol. 72, pp. 3-26, 1965.
7. E. Zwicker, "Procedure for calculating loudness of temporally variable sounds," J. Acoust. Soc. Amer., vol. 62, no. 3, pp. 675-682, 1977.
8. P. Desain and H. Honing, Music, Mind and Machine: Studies in Computer Music, Music Cognition, and Artificial Intelligence. Amsterdam, The Netherlands: Thesis, 1992.
9. J. Laroche, "Estimating tempo, swing and beat locations in audio recordings," in Proc. IEEE Workshop Applications of Digital Signal Processing to Audio and Acoustics, 2001, pp. 135-138.
10. R. Shepard, "Geometrical approximations to the structure of musical pitch," Psychol. Rev., vol. 89, no. 4, pp. 305-333, 1982.
11. A. de Cheveigné, Pitch, C. Plack and A. Oxenham, Eds. Berlin, Germany: Springer-Verlag, 2004, ch. Pitch Perception Models.
12. USA Standard Acoustic Terminology, ANSI, 1960.
13. J.-C. Risset and D. L. Wessel, Exploration of Timbre by Analysis and Synthesis, D. Deutsch, Ed. New York: Academic, 1999, pp. 113-169.
14. P. Masri and A. Bateman, "Improved modeling of attack transients in music analysis-resynthesis," in Proc. Int. Computer Music Conf., Hong Kong, 1996, pp. 100-103.
15. S. McAdams, S. Winsberg, G. de Soete, and J. Krimphoff, "Perceptual scaling of synthesized musical timbres: Common dimensions, specificities, and latent subject classes," Psychol. Res., vol. 58, pp. 177-192, 1995.
16. D. Cabrera, "'PsySound': A computer program for the psychoacoustical analysis of music," presented at the Australasian Computer Music Conf., MikroPolyphonie, vol. 5, Wellington, New Zealand, 1999.
17. J. C. Brown and M. S. Puckette, "Calculation of a narrowed autocorrelation function," J. Acoust. Soc. Amer., vol. 85, pp. 1595-1601, 1989.
18. S. Dubnov and N. Tishby, "Testing for gaussianity and non linearity in the sustained portion of musical sounds," in Proc. Journées Informatique Musicale, 1996, pp. 288-295.
19. D. Arfib, F. Keiler, and U. Zölzer, DAFX - Digital Audio Effects, U. Zoelzer, Ed. New York: Wiley, 2002, ch. Source-Filter Processing, pp. 299-372.
20. J. Sundberg, The Science of the Singing Voice. Dekalb, IL: Northern Illinois Univ. Press, 1987.
21. W. Slawson, Sound Color. Berkeley, CA: Univ. California Press, 1985.
22. S. Rossignol, P. Depalle, J. Soumagne, X. Rodet, and J.-L. Collette, "Vibrato: Detection, estimation, extraction, modification," in Proc. COST-G6 Workshop on Digital Audio Effects, Trondheim, The Netherlands, 1999, pp. 175-179.
23. J. Blauert, Spatial Hearing: The Psychophysics of Human Sound Localization. Cambridge, MA: MIT Press, 1983.
24. J. Chowning, "The simulation of moving sound sources," J. Audio Eng. Soc., vol. 19, no. 1, pp. 1-6, 1971.
25. M. Portnoff, "Implementation of the digital phase vocoder using the fast fourier transform," IEEE Trans. Acoustics, Speech, Signal Process., vol. ASSP-24, no. 3, pp. 243-248, Jun. 1976.
26. M. Dolson, "The phase vocoder: A tutorial," Comput. Music J., vol. 10, no. 4, pp. 14-27, 1986.
27. D. Arfib, F. Keiler, and U. Zölzer, DAFX-Digital Audio Effects, U. Zoelzer, Ed. New York: Wiley, 2002, ch. Time-Frequency Processing, pp. 237-297.
28. E. Moulines and F. Charpentier, "Pitch synchronous waveform processing techniques for text-to-speech synthesis using diphones," Speech Commun., vol. 9, no. 5/6, pp. 453-467, 1990.
29. J. Laroche, Applications of Digital Signal Processing to Audio & Acoustics, M. Kahrs and K. Brandenburg, Eds. Norwell, MA: Kluwer, 1998, ch. Time and Pitch Scale Modification of Audio Signals, pp. 279-309.
30. R. J. McAulay and T. F. Quatieri, "Speech analysis/synthesis based on a sinusoidal representation," IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-34, no. 4, pp. 744-754, Aug. 1986.
31. X. Serra and J. O. Smith, "A sound decomposition system based on a deterministic plus residual model," J. Acoust. Soc. Amer., vol. 89, no. 1, pp. 425-134, 1990.
32. T. Verma, S. Levine, and T. Meng, "Transient modeling synthesis: A flexible analysis/synthesis tool for transient signals," in Proc. Int. Computer Music Conf., Thessaloniki, Greece, 1997, pp. 164-167.
33. J. B. Allen and L. R. Rabiner, "A unified approach to short-time fourier analysis and synthesis," Proc. IEEE, vol. 65, no. 10, pp. 1558-1564, Oct. 1977.
34. J. B. Allen, "Short term spectral analysis, synthesis and modification by discrete fourier transform," IEEE Trans. Acoust., Speech, Signal Process., vol. ASSP-25, no. 3, pp. 235-238, Jun. 1977.
35. M. S. Puckette, "Phase-locked vocoder," presented at the IEEE ASSP Conf., Mohonk, NY, 1995.
36. J. Laroche and M. Dolson, "About this phasiness business," in Proc. Int. Computer Music Conf., Thessaloniki, Greece, 1997, pp. 55-58.
37. D. Arfib and N. Delprat, "Selective transformations of sound using time-frequency representations: An application to the vibrato modification," presented at the 104th Conv. Audio Eng. Soc., Amsterdam, The Netherlands, 1998.
38. R. Bristow-Johnson, "A detailed analysis of time-domain formant-corrected pitch-shifting algorithm," J. Audio Eng. Soc., vol. 43, no. 5, pp. 340-352, 1995.
39. E. Moulines and J. Laroche, "Non-parametric technique for pitch-scale and time-scale modification," Speech Commun., vol. 16, pp. 175-205, 1995.
40. S. Abrams, D. V. Oppenheim, D. Pazel, and J. Wright, "Higher-level composition control in music sketcher: Modifiers and smart harmony," in Proc. Int. Computer Music Conf., Beijing, China, 1999, pp. 13-16.
41. (2002) Voice One, Voice Prism. TC-Helicon. [Online]. Available: http://www.tc-helicon.tc/
42. (2003) Autotune. Antares. [Online]. Available: http://www.antarestech. com/
43. R. C. Maher and J. Beauchamp, "An investigation of vocal vibrato for synthesis," Appl. Acoust., vol. 30, pp. 219-245, 1990.
44. C. E. Seashore, "Psychology of the vibrato in voice and speech," Studies Psychol. Music, vol. 3, 1936.
45. V. Verfaille, C. Guastavino, and P. Depalle. Perceptual evaluation of vibrato models, presented at Colloq. Interdisciplinary Musicology. [Online]. Available: http://www.oicm.umontreal.ca/cim05/cim05.articles/VERFAILLE_V_CIM05. pdf
46. H. Honing, "The vibrato problem, comparing two solutions," Comput. Music J., vol. 19, no. 3, pp. 32-49, 1995.
47. M. Mathews and J. Kohut, "Electronic simulation of violin resonances," J. Acoust. Soc. Amer., vol. 53, no. 6, pp. 1620-1626, 1973.
48. J. Dattoro, "Effect design, part 2: Delay-line modulation and chorus," J. Audio Eng. Soc., pp. 764-788, 1997.
49. B. Bartlett. "A scientific explanation of phasing (flanging)," J. Audio Eng. Soc., vol. 18, no. 6, pp. 674-675, 1970.
50. W. M. Hartmann, "Flanging and phasers," J. Audio Eng. Soc., vol. 26, pp. 439-443, 1978.
51. J. O. Smith, "An allpass approach to digital phasing and flanging," in Proc. Int. Computer Music Conf., Paris, France, 1984, pp. 103-108.
52. E. Favreau, "Phase vocoder applications in GRM tools environment," in Proc. COST-G6 Workshop on Digital Audio Effects, Limerick, Ireland, 2001, pp. 134-137.
53. S. Haykin, Adaptive Filter Theory, 3rd ed. Englewood Cliffs, NJ: Prentice-Hall, 1996.
54. V. Pulkki, "Virtual sound source positioning using vector base amplitude panning," J. Audio Eng. Soc., vol. 45, no. 6, pp. 456-466, 1997.
55. M. A. Gerzon, "Ambisonics in mutichannel broadcasting and video," J. Audio Eng. Soc., vol. 33, no. 11, pp. 859-871, 1985.
56. J. O. Smith, S. Serafin, J. Abel, and D. Berners, "Doppler simulation and the Leslie," in Proc. Int. Conf. Digital Audio Effects, Hamburg, Germany, 2002. pp. 13-20.
57. T. I. Laakso, V. Välimäki, M. Karjalainen, and U. K. Laine, "Splitting the unit delay," IEEE Signal Process. Mag., no. 1, pp. 30-60, Jan. 1996.
58. M. R. Schroeder and B. Logan, "'Colorless' artificial reyerberation," J. Audio Eng. Soc., vol. 9, pp. 192-197, 1961.
59. J. A. Moorer, "About this reverberation business," Comput. Music J., vol. 3, no. 2, pp. 13-8, 1979.
60. O. Warusfel and N. Misdariis, "Directivity synthesis with a 3D array of loudspeakers - Application for stage performance," in Proc. COST-G6 Workshop Digital Audio Effects, Limerick, Ireland, 2001, pp. 232-236.
61. P. Dutilleux, "Vers la Machine à Sculpter le son. Modification en Tempsréel des Caractéristiques Fréquentielles et Temporelles des Sons," Ph.D. dissertation, Univ. Aix-Marseille II, Marseille, France, 1991.
62. V. Verfaille and D. Arfib, "ADAFx: Adaptive digital audio effects," in Proc. COST-G6 Workshop on Digital Audio Effects, Limerick, Ireland, 2001, pp. 10-14.
63. V. Verfaille, "Effets Audionumériques Adaptatifs: Théorie, Mise enoeuvre et Usage en Création Musicale Numérique," Ph.D. dissertation, Univ. Méditerranée Aix-Marseille II, Marseille, France, 2003.
64. D. Arfib, Recherches et Applications en Informatique Musicale. Paris, France: Hermès, 1998, ch. Des Courbes et des Sons, pp. 277-86.
65. R. Moog, "A voltage-controlled low-pass, high-pass filter for audio signal processing," presented at the 17th Annu. AES Meet., 1965.
66. J. W. Beauchamp, "Synthesis by spectral amplitude and "brightness" matching of analyzed musical instrument tones," J. Audio Eng. Soc., vol. 30, no. 6, pp. 396-406, 1982.
67. W. von Aures, "Der sensorische wohlklang als funktion psychoakustischer empfindungsgröfsen," Acustica, vol. 58, pp. 282-290, 1985.
68. E. Zwicker and H. Fastl, Psychoacoustics: Facts and Models. Berlin, Germany: Springer-Verlag, 1999.
69. G. Peeters, S. McAdams, and P. Herrera, "Instrument sound description in the context of MPEG-7," in Proc. Int. Computer Music Conf., Berlin, Germany, 2000, pp. 166-169.
70. M. M. Wanderley, "Mapping strategies in real-time computer music," Org. Sound, vol. 7, no. 2, 2002.
71. M. M. Wanderley and P. Depalle, "Gesturally controlled digital audio effects," in Proc. COST-G6 Workshop on Digital Audio Effects, Verona, Italy, 2000, pp. 165-169.
72. D. Arfib and V. Verfaille, "Driving pitch-shifting and time-scaling algorithms with adaptive and gestural techniques," in Proc. Int. Conf. Digital Audio Effects, London, U.K., 2003, pp. 106-111.
73. D. Arfib, J.-M. Couturier, L. Kessous, and V. Verfaille, "Strategies of mapping between gesture parameters and synthesis model parameters using perceptual spaces," Org. Sound, vol. 7, no. 2, pp. 135-152, 2002.
74. V. Verfaille and D. Arfib, "Implementation strategies for adaptive digital audio effects," in Proc. Int. Conf. Digital Audio Effects, Hamburg, Germany, 2002, pp. 21-26.
75. V. Verfaille, M. M. Wanderley, and Ph. Depalle, "Mapping Strategies for Gestural Control of Adaptive Digital Audio Effects," J. New Music Res., 2005.
76. C. Cadoz, Les Nouveaux Gestes de la Musique, H. Genevois and R. de Vivo, Eds., 1999, ch. Musique, geste, technologie, pp. 47-92.
77. P. Dutilleux and U. Zölzer, DAFX - Digital Audio Effects, U. Zoelzer, Ed. New York: Wiley, 2002, ch. Nonlinear Processing, pp. 93-135.
78. J. Bonada, "Automatic technique in frequency domain for near-lossless time-scale modification of audio," in Proc. Int. Computer Music Conf., Berlin, Germany, 2000, pp. 369-399.
79. G. Pallone, "Dilatation et Transposition Sous Contraintes Perceptives des Signaux Audio: Application au Transfert Cinéma- Vidéo," Ph.D. dissertation, Univ. Aix-Marseille II, Marseille, France, 2003.
80. F. Gouyon, L. Fabig, and J. Bonada, "Rhythmic expressiveness transformations of audio recordings: Swing modifications," in Proc. Int. Conf. Digital Audio Effects, London, U.K., 2003, pp. 94-99.
81. V. Verfaille and P. Depalle, "Adaptive effects based on STFT, using a source-filter model," in Proc. Int. Conf. Digital Audio Effects, Naples, Italy, 2004, pp. 296-301.
82. E. Gómez, G. Peterschmitt, X. Amatriain, and P. Herrera, "Content-based melodic transformations of audio material for a music processing application," in Proc. Int. Conf. Digital Audio Effects, London, U.K., 2003. pp. 333-338.
83. A. D. Cristo, Prolégomènes à L'étude de L'intonation, Ed. CNRS, 1982.
84. P. Depalle, G. Garcia, and X. Rodet, "Reconstruction of a castrato voice: Farinelli's voice," in Proc. IEEE Workshop Applications of Digital Signal Processing to Audio and Acoustics, 1995, pp. 242-245.
85. P. Cano, A. Loscos, J. Bonada, M. de Boer, and X. Serra, "Voice morphing system for impersonating in karaoke applications," in Proc. Int. Computer Music Conf., Berlin, Germany, 2000, pp. 109-12.
86. X. Amatriain, J. Bonada, A. Loscos, and X. Serra, DAFX - Digital Audio Effects, U. Zoelzer, Ed. New York: Wiley, 2002, ch. Spectral Processing, pp. 373-438.
87. A. Bregman, Auditory Scene Analysis. Cambridge, MA: MIT Press, 1990.
88. J. Chowning, "The synthesis of complex audio spectra by means of frequency modulation," J. Audio Eng. Soc., vol. 21, pp. 526-534, 1971.
89. J.-C. Risset, "Computer study of trampet tones," J. Acoust. Soc. Amer., vol. 33, pp. 912-912, 1965.
90. A. Sédès, B. Courribet, J.-B. Thiebaut, and V. Verfaule, "Visualization de l'Espace Sonore, vers la Notion de Transduction: Une Approche Interactive Temps-Réel," Espaces Sonores - Actes de Recherches, pp. 125-43, 2003.
91. X. Amatriain, J. Bonada, A. Loscos, and X. Serra, "Spectral modeling for higher-level sound transformations," presented at the MOSART Workshop Current Research Dir. in Computer Music, Barcelona, Spain, 2001, IUA-UPF.
92. V. Verfaule and D. Lebel, "AUvolution: Implementation of Adaptive Digital Audio Effects Using the AudioUnit Framework," Sound Process. Control Lab., Schulich School of Music, McGill Univ., Montréal, QC, Canada, 2005.