Structured Audio: Creation, Transmission, and Rendering of Parametric Sound Representations

Proceedings of the IEEE

Volumn 86, Issue 5, 1998, Pages 922-939

  Yercoe, Barry L ^a,b   Gardner, William G ^a   Scheirer, Eric D ^a  

^a MIT MEDIA LAB   (United States)

^b Wave Arts Inc   (United States)

Author keywords

Audio coding; Audio systems; Digital audio broadcasting; Multimedia computing; Multimedia data bases; Music

Indexed keywords

COMPUTATIONAL LINGUISTICS; DATA STRUCTURES; DIGITAL SIGNAL PROCESSING; DISTRIBUTED DATABASE SYSTEMS; INFORMATION RETRIEVAL; SIGNAL ENCODING; VIRTUAL REALITY;

AUDIO CODING; DIGITAL AUDIO BROADCASTING; MULTIMEDIA COMPUTING; MULTIMEDIA DATABASE SYSTEMS;

INTERACTIVE COMPUTER SYSTEMS;

EID: 0032070996     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/5.664280     Document Type: Article

References (124)

1. J. B. Allen, "Short term spectral analysis, synthesis, and modification by discrete Fourier transform," IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-25, pp. 235-238, Mar. 1977.
2. J. B. Allen and L. R. Rabiner, "A unified approach to shorttime Fourier analysis and synthesis," Proc. IEEE, vol. 65, pp. 1558-1564, Nov. 1977.
3. S.-I. Amari, A. Cichocki, and H. H. Yang, "A new learning algorithm for blind signal separation," in Advances in Neural Information Processing Systems 8. Cambridge, MA: MIT Press, 1996.
4. B. S. Atal, "Automatic speaker recognition based on pitch contours," J. Acoust. Soc. Amer., vol. 52, pp. 1687-1697, 1972.
5. B. S. Atal and S. L. Hanauer, "Speech analysis and synthesis by linear prediction of the speech wave," J. Acoust. Soc. Amer., vol. 50, pp. 637-655, 1971.
6. B. S. Atal and M. R. Schroeder, "Predictive coding of speech signals and subjective error criteria," IEEE Trans. Acoust., Speech, Signal Proc., vol. ASSP-27, pp. 247-254, 1979.
7. J. Beauchamp, "Brass tone synthesis by spectrum evolution matching with nonlinear functions," Comput. Music J., vol. 3, no. 2, pp. 35-3, Summer 1979.
8. A. J. Bell and T. Sejnowski, "An information maximization approach to blind separation and blind deconvolution," Neural Computation, 1989, pp. 1129-1159.
9. J. Bilmes, 'Timing is of the essence: Perceptual and computational techniques for representing, learning, and reproducing expressive timing in percussive music," S.M. thesis, Massachusetts Institute of Technology Media Lab, Cambridge, 1993.
10. A. Bregman, Auditory Scene Analysis. Cambridge, MA: MIT Press, 1989.
11. J. Brown, "Computer identification of musical instruments using pattern recognition," presented at the Society for Music Perception and Cognition Meeting, Cambridge, MA, 1997.
12. J. Brown and M. Puckette, "Calculation of a 'narrowed' autocorrelation function," J. Acoust. Soc. Amer., vol. 85, pp. 1595-1601, 1989.
13. M. A. Casey, "Understanding musical sound with forward models and physical models," Connection Sei., vol. 6, pp. 355-371, 1994.
14. M. A. Casey and P. Smaragdis, "Netsound," in Proc. 1996 Int. Computer Music Con/, Hong Kong, P.R.C., 1996.
15. M. A. Casey, Auditory group theory with applications to statistical basis methods for structured audio, Ph.D. dissertation, Massachusetts Institute of Technology Media Lab, Cambridge, 1997.
16. J. Cahn, "Generating expression in synthesized speech," M.S. thesis, Massachusetts Institute of Technology Media Lab, Cambridge, 1990.
17. J. Chowning, "The synthesis of complex audio spectra by means of frequency modulation," J. Audio Eng. Soc., July 1977, pp. 526-534.
18. P. R. Cook, "Identification of control parameters in an articulatory vocal tract model, with applications to the synthesis of singing," Ph.D. dissertation, CCRMA, Stanford Univ., Stanford, CA, 1991.
19. _, "TBone: an interactive waveguide brass instrument synthesis workbench for the next machine," in Proc. Int. Computer Music Conf., Montreal, Canada, 1991, pp. 297-299.
20. _, "SPASM, a real-time vocal tract physical model controller; and Singer, the companion software synthesis system," Comput. Music J., Spring 1993, pp. 30-44.
21. P. R. Cook, "A hierarchical system for controlling synthesis by physical modeling," in Proc. Int. Computer Music Conf., 1995, Banff, CA, pp. 108-109.
22. M. Cooke, Modeling Auditory Processing and Organization. Cambridge, U.K.: Cambridge Univ. Press, 1993.
23. G. De Poli and A. Piccialli, "Pitch-synchronous granular synthesis," in Representations of Musical Signals, G. De Poli, A. Piccialli, and C. Roads, Eds. Cambridge, MA: MIT Press, 1991, pp. 187-220.
24. G. De Poli, A. Piccialli, and C. Roads, Eds., Representations of Musical Signals. Cambridge, MA: MIT Press, 1991.
25. G. Doddington, "Speaker recognition-Identifying people from their voices," Proc. IEEE, vol. 83, pp. 1651-1664, Nov. 1985.
26. M. B. Dolson, "The phase vocoder: A tutorial," Compiit. Music J., Winter 1986, pp. 14-27.
27. H. Dudley, "Remaking speech," J. Acoust. Soc. Amer., vol. 11, pp. 169-177, 1939.
28. D. P. W. Ellis, "A computer implementation of psychacoustic grouping rules," in Proc. 12th Int. Conf. Pattern Recognition, Jerusalem, Israel, Oct. 1994.
29. _, "Prediction-driven computational auditory scene analysis," Ph.D. dissertation, Dept. of Electrical Engineering and Computer Science, Massachusetts Institute of Technology, Cambridge, 1996.
30. C. Faloutsos, R. Barber, M. Rickner, J. Hafner, W. Niblack, D. Petkovic, and W. Equitz, "Efficient and effective querying by image content." J, Intell. Inform. Syst., 1994, pp. 231-262.
31. Firefly Network, Inc. (1997). WWW white paper. [Online]. Available: http://www.firefly.net/products/FirenyTools.html.
32. J.-L. Florens and C. Cadoz, 'The physical model: Modeling and simulating the instrumental universe," in Representations of Musical Signals, G. De Poli, A. Piccialli, and C. Roads, Eds. Cambridge, MA: MIT Press, 1991, pp. 227-268.
33. J. Foote, "A similarity measure for audio classification," in Proc. AAAI1997 Spring Symp. Intelligent Integration and Use of Text, Image, Video, and Audio Corpora, Stanford, CA, Mar. 1997.
34. J. T. Foote, M. G. Brown, G. J. F. Jones, K. Spdrck Jones, and S. J. Young, "Video mail retrieval by voice: Toward intelligent retrieval and browsing of multimedia documents," in Proc. IMMI-1, 1st Int. Workshop Intelligence and Miiltimodality in Multimedia Interfaces, Edinburgh, Scotland, July 1995.
35. M. D. Freedman, "Analysis of musical instrument tones," J. Acoust. Soc. Amer., vol. 41, pp. 793-806, 1967.
36. W. G. Gardner, "Reverberation algorithms," in Applications of Signal Processing to Audio and Acoustics, M. Kahrs and K. Brandenberg, Eds. New York: Kluwer, 1998.
37. _, "3-D audio using loudspeakers," Ph.D. dissertation. Massachusetts of Technology Media Lab, Cambridge, 1997.
38. G. E. Garnett, "Music, signals, and representations: A survey," in Representations of Musical Signals, G. De Poli, A. Piccialli, and C. Roads, Eds. Cambridge, MA: MIT Press, 1991, pp. 325-370. .
39. B. Garton, "Virtual performance modeling," in Proc. Int. Computer Music Conf., 1992, pp. 219-222.
40. A. Gersho, "Advances in speech and audio compression," Proc. IEEE, vol. 82, pp. 900-918, June 1994.
41. M. Goto and Y. Muraoka, "Music understanding at the beat level: Real-time beat tracking for audio signals," in Readings in Computational Auditory Scene Analysis, H. Okuno and D. Rosenthal, Eds. Mahweli, NJ: Erlbaum, 1998.
42. J. M. Grey, "Multidimensional perceptual scaling of musical timbres," J. Acoust. Soc. Amer., vol. 61, pp. 1270-1277, 1977.
43. J. M. Grey and J. A. Moorer, "Perceptual evaluations of synthesized musical instrument tones," J. Acoust. Soc. Amer., vol. 62, pp. 454-462, 1977.
44. B. Grill, B. Edler, M. Hahn, K. lijima, N. Iwakami, T. Moriya, J. Ostermann, S. Nakajima, M. Nishiguchi, T. Nomura, W. Oomen, H. Pumhagen, E. Scheirer, N. Tanaka, A. P. Tan, and R. Taori, Eds. "MPEG-4 audio Working Draft 4.0," ISO/IEC JTC1/SC29/WG11 (MPEG) Bristol Document N1631, 1997.
45. A. Horner, B. Beauchamp, and L. Haken, "FM matching synthesis with genetic algorithms," Compiit. Music J., vol. 17, no. 4, pp. 17-29, Winter 1993.
46. N. Jayant, J. Johnston, and R. Safranek, "Signal compression based on models of human perception," Proc. IEEE, vol. 81, pp. 1385-1421, Oct. 1993.
47. J. M. Jot, "Synthesizing three-dimensional sound scenes in audio or multimedia production and interactive human-computer interfaces," presented at the Fifth International Conference on Interfaces to Real and Virtual Worlds, 1996.
48. M. Karjalainen, U. K. Laine, T. Laakso, and V. Valimaki, 'Transmission-line modeling and real-time synthesis of string and wind instruments," in Proc. Int. Computer Music Conf., San Francisco, CA, 1991, pp. 293-296.
49. K. Kashino, K. Nakadai, T. Kinoshita, and H. Tanaka, "Application of Bayesian probability network to music scene analysis," in Readings in Computational Auditory Scene Analysis, H. Okuno and D. Rosenthal, Eds. Mahweh, NJ: Erlbaum, 1998.
50. W. B. Kühn, "A real-time pitch recognition algorithm for music applications," Compitt. Music J., pp. 60-71, Fall 1990.
51. R. G. Laughlin, B. D. Truax, and B. V. Funt, "Synthesis of acoustic timbres using principal component analysis," in Proc. Int. Computer Music Conf., Glasgow, Scotland, 1990, pp 95-99.
52. M. LeBrun, "A derivation of the spectrum of FM with a complex modulating wave," Compitt. Music J., pp. 51-52, Winter 1977.
53. _, "Digital waveshaping synthesis," J. Audio Eng. Soc., pp. I 250-266, Apr. 1979. I
54. M. Li and P. M. B. Vitanyi, An Introduction to Kolmogorov I Complexity and Its Applications. New York: Springer-Verlag, I 1993. I
55. A. Lindsay, "Using contour as a mid-level representation of I melody," M.S. thesis, Massachusetts Institute of Technology Media Lab, Cambridge, 1996.
56. _, "Multimodal interaction in the perception of musical interpolation," presented at the Society for Music Perception and Cognition Meeting, Cambridge, MA, 1997.
57. D. G. Loy, "Musicians make a standard: the MIDI phenomenon," Comput. Music J., pp. 181-198, Winter 1989.
58. R. F. Lyon, "Computational models of neural auditory processing," in Proc. IEEE 1CASSP, 1984.
59. T. Machover. (1997). The brain opera archive. [Online]. Available: http://brainop.media.mit.edu/Archive/index.htmI.
60. R. Mäher, "An approach for the separation of voices in composite musical signals," Ph.D. dissertation, Univ. of Illinois Urbana-Champaign, 1989.
61. J. Makhoul, "Linear prediction: a tutorial review," Proc. IEEE, vol. 63, pp. 561-580, Mar. 1975.
62. K. Martin, "Automatic transcription of simple polyphonic music: Robust front end processing," Massachusetts Institute of Technology Media Lab, Cambridge, Perceptual Computing Tech. Rep. 399, 1996.
63. K. Martin and E. D. Scheirer, "Automatic transcription of , simple polyphonic music: Incorporating high-level knowledge," presented at the Society for Music Perception and Cognition Conference, 1997.
64. D. C. Massie, "Wavetable Sampling Synthesis," in Applications of Signal Processing to Audio and Acoustics, M. Kahrl and K. Brandenberg, Eds. New York: Kluwer, to be published.
65. M. Mathews, The Technology of Computer Music. Cambridge, MA: MIT Press, 1969.
66. R. J. McAulay and T. F. Quatieri, "Speech analysis/synthesis based on a sinusoidal representation," IEEE Trans. Acoust., Speech. Signal Processing, vol. ASSP-34, 1986, pp. 744-754.
67. R. Meddis and M. Hewitt, "Virtual pitch and phase sensitivity of a computer model of the auditory periphery. I: Pitch identification," J. Acoust. Soc. Amer., vol. 89, pp. 2866-2882, 1991.
68. D. Mellinger, "Event formation and separation in musical sound," Ph.D. dissertation, CCRMA Stanford Univ., Stanford, CA, 1991.
69. R. Moog, "MIDI: Musical Instrument Digital Interface," J. Audio Eng. Soc., May 1986, pp. 394-404.
70. F. R. Moore, "The dysfunctions of MIDI," Comput. Music J., pp. 19-28, Spring 1988.
71. J. A. Moorer, "On the segmentation and analysis of continuous musical sound by digital computer," Ph.D. dissertation, CCRMA, Stanford Univ., Stanford, CA, 1975.
72. _, "The use of the phase vocoder in computer music applications," J. Audio Eng. Soc., pp. 42-15, Feb. 1978.
73. T. Nakatani, M. Goto, T. Ito, and H. Okuno, "Multi-agent based binaural sound stream segregation," in Readings in Computational Auditory Scene Analysis, H. Okuno and D. Rosenthal, Eds. Mahweh, NJ: Erlbaum, 1998..
74. S. H. Nawab, C. Espy-Wilson, R. Mani, and N. Bitar, "Knowledge-based analysis of speech mixed with sporadic environmental sounds" in Readings in Computational Auditory Scene Analysis, H. Okuno and D. Rosenthal, Eds. Mahweh, NJ: Erlbaum, 1998.
75. R. Picard, "A society of models for video and image libraries," "IBM Syst. J., vol. 35, pp. 292-312, 1996.
76. S. T. Pope, "Music composition and scoring by computer," in Music Processing, G. Haus, Ed. Los Angeles, CA: Ä-R Editions, 1992.
77. M. Puckette, "Combining event and signal processing in the
78. MAX graphical programming environment," Comput. Music J., pp. 68-77, Fall 1991.
79. L. R. Rabiner, "Speech synthesis by rule: an acoustic domain approach," Bell Sysl. Tech. J., pp. 17-37, 1968.
80. _, "Digital formant synthesizer for speech synthesis studies," J. Acoust. Soc. Amer., vol. 43, pp. 822-828, 1968.
81. L. R. Rabiner and R. W. Schafer, Digital Processing of Speech Signals. Englewood Cliffs, NJ: Prentice-Hall, 1978.
82. J. Rissanen, "Modeling by the shortest data description," Automatica, pp. 465-71, 1978.
83. J.-C. Risset and M. V. Mathews, "Analysis of musical-instrument tones," Phys. Today, pp. 23-30, 1969.
84. J.-C. Risset and D. L. Wessel, "Exploration of Timbre by analysis and synthesis," in D. Deutsch, Ed., Psychology of Music. Orlando, FL: Academic, 1982.
85. C. Roads, "A tutorial on nonlinear distortion or waveshaping synthesis," Comput. Music J., pp. 29-34, Summer 1979.
86. _, "Granular Synthesis of Sound," in Foundations of Computer Music, C. Roads and J. Strawn, Eds. Cambridge, MA: MIT Press, 1985, pp. 145-159.
87. _, "Asynchronous Granular Synthesis," in Representations of Musical Signals, G. De Poli, A. Piccialli, and C. Roads, Eds. Cambridge, MA: MIT Press, 1991, pp. 143-186.
88. _, The Computer Music Tutorial. Cambridge, MA: MIT Press, 1996.
89. X. Rodet, 'Time-domain formant wave-function synthesis," Comput. Music J., pp. 9-14, Fall 1984.
90. D. Rossum, "The SoundFont 2.0 file format," Joint E-Mu/Creative Tech Center white paper, 1995.
91. R. Rowe, Interactive Music Systems. . Cambridge, MA: MIT Press, 1993.
92. N. Saint-Arnaud and K. Popat, "Analysis and synthesis of sound textures," in Readings in Computational Auditory Scene Analysis, H. Okuno and D. Rosenthal, Eds. Mahweh, NJ: Erlbaum, 1998.
93. G. J. Sandell and W. L. Martens, "Perceptual evaluation of principal-component-based synthesis of musical timbres," J. Audio Eng. Soc., pp. 1013-1028, Dec. 1995.
94. C. Scaletti and K. Hebel, "An object-based representation for digital audio signals," in Representations of Musical Signals, G. De Poli, A. Piccialli, and C. Roads, Eds. Cambridge, MA: MIT Press, 1991, pp. 371-390.
95. E. D. Scheirer, "Tempo and beat analysis of acoustic musical signals," J. Acoust. Soc. Amer. vol. 103, pp. 588-601, Jan. 1998.
96. _, "The MPEG-4 structured audio standard," in Proc. IEEE ICASSP, 1998.
97. _, "Using musical knowledge to extract expressive performance information from audio recordings," in Readings in Computational Auditory Scene Analysis, H. Okuno and D. Rosenthal, Eds. Mahweh, NJ: Erlbaum, 1998.
98. _, "Structured audio and effects processing in the MPEG-4 multimedia standard," ACM Multimedia Syst., to be published.
99. E. D. Scheirer and M. Slaney, "Construction and evaluation of a robust multifeature speech-music discriminator," in Proc. IEEE ICASSP, 1997.
100. K. R. Scherer, "Vocal affect expression: A review and a model for future research," Psych. Bul., pp. 143-165, 1986.
101. B. Schottstaedt, "The simulation of natural instrumeht tones using frequency modulation with a complex modulating wave," Comput. Music J., pp. 46-50, Winter 1977.
102. E. Selfridge-Field, Ed., Beyond MIDI: The Handbook of Musical Codes. Cambridge, MA: MIT Press, 1997.
103. X. Serra, "A system for sound analysis/transformation/synthesis based on a deterministic plus stochastic decomposition," Ph.D. dissertation, CCRMA, Stanford Univ., Stanford, CA, 1989.
104. C. Shannon, "A mathematical theory of communications," Bell Syst. Tech. J., pp. 379-23, 623-656, 1948.
105. P. Smaragdis, "Information theoretic approaches to source separation," S.M. thesis, Massachusetts Institute of Technology Media Lab, Cambridge, 1997.
106. R. N. Shepard, "The analysis of proximities: Multidimensional scaling with an unknown distance function," Psychometrika, pp. 125-140, 1962.
107. J. O. Smith, 'Techniques for digital filter design and system identification with application to the violin," Ph.D. dissertation, CCRMA, Stanford Univ., Stanford, CA, 1983.
108. _, "Musical applications of digital waveguides," Music Department, Stanford Univ., Stanford, ÇA, Rep. STAN-M-39, 1987.
109. _, "Physical modeling using digital waveguides," Comput. Music J., pp. 74-9, Winter 19921.
110. J. Stapleton and S. C. Bass, "Synthesis of musical tones based on the Karhunen-Loève transform," IEEE Trans. Acoust., Speech, Signal Processing, Mar. 1988, pp. 305-319.
111. W. Strong and M. Clark, "Synthesis of wind-instrument tones," J. Acoust. Soc. Amer., vol. 41, pp. 39-52, 1967.
112. N. P. Todd, "The auditory 'primal sketch': A multiscale model of rhythmic grouping," J. New Music Res., pp. 25-70, 1994.
113. S. Van Duyne and J. O. Smith, "Physical modeling with the 2D digital waveguide mesh," in Proc. 1993 Int. Computer Music Conf., pp. 40-47.
114. V. Valimaki, M. Karjalainen, and T. I. Laakso, "Modeling of woodwind bores with finger holes," in Proc. 1993 Int. Computer Music Conf., pp. 32-39.
115. B. Vercoe, "The synthetic performer in the context of live performance," in Proc. 1984 Int. Computer Music Conf., pp. 199-200.
116. _, "Perceptually-based music pattern recognition and response," in Proc. 1994 Int. Conf. Music Perception and Cognition., pp. 59-60
117. B. Vercoe, "Csound: A manual for the audio processing system," Massachusetts Institute of Technology Media Lab, Cambridge, 1996.
118. B. Vercoe and M. Puckette, "Synthetic rehearsal: Training the synthetic performer," in Proc. 1985 Int. Computer Music Conf., pp. 275-278.
119. Virtual reality modeling language. (1997). [Online]. Available: http://www.vrml.org.
120. D. Wang, "Stream segregation based on oscillatory correlation," in Readings in Computational Auditory Scene Analysis, H. Okuno and D. Rosenthal, Eds. Mahweh, NJ: Erlbaum, 1998.
121. W. Warren and R. Verbrugge, "Auditory perception of breaking and bouncing objects," in Natural Computation, W. Richards, Ed. Cambridge, MA: MIT Press, 1988.
122. D. Weintraub, "A theory and computational model of auditory monaural sound separation," Ph.D. dissertation, CCRMA, Stanford Univ., Stanford, CA, 1985
123. D. L. Wessel, 'Timbre space as a musical control structure," Comput. Music J., pp. 45-52, Summer 1979.
124. L. Wyse and S. Smoliar, "Toward content-based audio indexing and retrieval," in Readings in Computational Auditory Scene Analysis, H. Okuno and D. Rosenthal, Eds. Mahweh, NJ: Erlbaum, 1998.