A Comparison of Approaches to Timbre Descriptors in Music Information Retrieval and Music Psychology

Journal of New Music Research

Volumn 45, Issue 1, 2016, Pages 27-41

  Siedenburg, Kai ^a   Fujinaga, Ichiro ^a   McAdams, Stephen ^a  

^a MCGILL UNIVERSITY   (Canada)

Author keywords

audio analysis; evaluation; information retrieval; instrument classification; timbre perception

Indexed keywords

EID: 84959207560     PISSN: 09298215     EISSN: 17445027     Source Type: Journal    
DOI: 10.1080/09298215.2015.1132737     Document Type: Article

References (100)

1. T.R.Agus,, C.Suied,, S.J.Thorpe,, & D.Pressnitzer, (2012). Fast recognition of musical sounds based on timbre. The Journal of the Acoustical Society of America, 131(5), 4124–4133.
2. T.R.Agus,, S.J.Thorpe,, & D.Pressnitzer, (2010). Rapid formation of robust auditory memories: Insights from noise. Neuron, 66, 610–618.
3. V.Alluri,, & P.Toiviainen, (2009). Exploring perceptual and acoustical correlates of polyphonic timbre. Music Perception, 27(3), 223–241.
4. J.Andén,, & S.Mallat, (2014). Deep scattering spectrum. IEEE Transactions on Signal Processing, 62(16), 4114–4128.
5. F.G.Ashby, (Ed.). (1992). Multidimensional models of categorization. Multidimensional models of perception and cognition (pp. 449–483). Hillsdale, NJ: Lawrence Erlbaum Associates.
6. J.-J.Aucouturier,, & E.Bigand, (2012). Mel Cepstrum & Ann Ova: The difficult dialog between MIR and music cognition. In F.Gouyon, P.Herrera, L.G.Martins, & M.Müller (Eds.), Proceedings of the 13th International Society for Music Information Retrieval Conference, Porto, Portugal, Oct 8–12, 2012 (pp. 397–402). Porto, Portugal: FEUP Ediç oes.
7. J.-J.Aucouturier,, & E.Bigand, (2013). Seven problems that keep MIR from attracting the interest of cognition and neuroscience. Journal of Intelligent Information Systems, 41(3), 483–497.
8. J.G.A.Barbedo,, & G.Tzanetakis, (2011). Musical instrument classification using individual partials. IEEE Transactions on Audio, Speech, and Language Processing, 19(1), 111–122.
9. M.Barthet,, P.Depalle,, R.Kronland-Martinet,, & S.Ystad, (2010). Acoustical correlates of timbre and expressiveness in clarinet performance. Music Perception, 28(2), 135–153.
10. M.Barthet,, P.Guillemain,, R.Kronland-Martinet,, & S.Ystad, (2010). From clarinet control to timbre perception. Acta Acustica united with Acustica, 96(4), 678–689.
11. C.J.Berry,, D.R.Shanks,, M.Speekenbrink,, & R.N.A.Henson, (2012). Models of recognition, repetition priming, and fluency: Exploring a new framework. Psychological Review, 119(1), 40–79.
12. H.Bozdogan, (1987). Model selection and Akaike’s information criterion (AIC): The general theory and its analytical extensions. Psychometrika, 52(3), 345–370.
13. J.Bruna,, & S.Mallat, (2013). Audio texture synthesis with scattering moments. arXiv preprint. Retrieved from http://arxiv.org/pdf/1311.0407.pdf
14. A.Caclin,, S.McAdams,, B.K.Smith,, & S.Winsberg, (2005). Acoustic correlates of timbre space dimensions: A confirmatory study using synthetic tones. The Journal of the Acoustical Society of America, 118(1), 471–482.
15. T.Dau, (2008). Auditory processing models. In D.Havelock, S.Kuwano, & M.Vorländer (Eds.), Handbook of signal processing in acoustics, Vol. 1 (pp. 175–196). New York: Springer.
16. T.Dau,, B.Kollmeier,, & A.Kohlrausch, (1997a). Modeling auditory processing of amplitude modulation. I. Detection and masking with narrow-band carriers. The Journal of the Acoustical Society of America, 102(5), 2892–2905.
17. T.Dau,, B.Kollmeier,, & A.Kohlrausch, (1997b). Modeling auditory processing of amplitude modulation. II. Spectral and temporal integration. The Journal of the Acoustical Society of America, 102(5), 2906–2919.
18. T.Dau,, D.Püschel,, & A.Kohlrausch, (1996). A quantitative model of the "effective" signal processing in the auditory system. I. Model structure. The Journal of the Acoustical Society of America, 99(6), 3615–3622.
19. T.Elliott,, L.Hamilton,, & F.Theunissen, (2013). Acoustic structure of the five perceptual dimensions of timbre in orchestral instrument tones. The Journal of the Acoustical Society of America, 133(1), 389–404.
20. T.Elliott,, & F.Theunissen, (2009). The modulation transfer function for speech intelligibility. PLoS Computational Biology, 5(3), e1000302.
21. D.P.Ellis, (1999). Using knowledge to organize sound: The prediction-driven approach to computational auditory scene analysis and its application to speech/nonspeech mixtures. Speech Communication, 27(3), 281–298.
22. A.Eronen, (2001). Comparison of features for musical instrument recognition. In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz, NY, USA, Oct 21-–24, 2001 (pp. 19–22). Piscataway, NJ: IEEE.
23. S.Essid,, G.Richard,, & B.David, (2006). Musical instrument recognition by pairwise classification strategies. IEEE Transactions on Audio, Speech, and Language Processing, 14(4), 1401–1412.
24. A.Flexer, (2014). On inter-rater agreement in audio music similarity. In H.Wang, Y.Yang, & J.H.Lee (Eds.), Proceedings of the 15th International Society for Music Information Retrieval Conference, Taipei, Taiwan, Oct 27-–31, 2014 (pp. 245–250). Vienna, Austria: Arthur Flexer.
25. C.Fritz,, A.F.Blackwell,, I.Cross,, J.Woodhouse,, & B.C.Moore, (2012). Exploring violin sound quality: Investigating English timbre descriptors and correlating resynthesized acoustical modifications with perceptual properties. The Journal of the Acoustical Society of America, 131(1), 783–794.
26. J.Fritz,, S.David,, & S.Shamma, (2013). Attention and dynamic, task-related receptive field plasticity in adult auditory cortex. In Y.E.Cohen, A.N.Popper, & R.R.Fay (Eds.), Neural correlates of auditory cognition, Vol. 45 (pp. 251–291). New York: Springer.
27. J.Fritz,, S.Shamma,, M.Elhilali,, & D.Klein, (2003). Rapid task-related plasticity of spectrotemporal receptive fields in primary auditory cortex. Nature Neuroscience, 6(11), 1216–1223.
28. Z.Fu,, G.Lu,, K.M.Ting,, & D.Zhang, (2011). A survey of audio-based music classification and annotation. IEEE Transactions on Multimedia, 13(2), 303–319.
29. I.Fujinaga, (1998). Machine recognition of timbre using steady-state tone of acoustic musical instruments. In M.Simoni (Ed.), Proceedings of the International Computer Music Conference, Ann Arbor, MI, USA, Oct 1–6, 1998 (pp. 207–210). San Francisco, CA: International Computer Music Association.
30. I.Fujinaga,, & K.MacMillan, (2000). Realtime recognition of orchestral instruments. In I.Zannos (Ed.), Proceedings of the International Computer Music Conference, Berlin, Germany, Aug 27-Sep 1, 2000 (pp. 141–143). San Francisco, CA: International Computer Music Association.
31. T.Gadermaier,, & C.Reuter, (2014). Strukturelle Merkmale von Blasinstrumentenspektrenâ–die Schumannschen Klangfarbengesetze aus heutiger Sicht [Structural features of brass instruments – Schumann’s laws of tone colour from a contemporary perspective]. In B.Kollmeier, M.Blau, & S.van de Par (Eds.), Fortschritte der Akustik – Tagungsband der 40. DAGA, Oldenburg, Germany, Mar 10–13, 2014 (pp. 48–49), Berlin, Germany. Deutsche Gesellschaft f\"{u}r Akustik e.V.
32. D.Giannoulis,, E.Benetos,, M.Rossignol,, M.Lagrange,, & M.D.Plumbley, (2013). Detection and classification of acoustic scenes and events: An IEEE AASP challenge. In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Paltz NY, USA, Oct 20-–23, 2013 (pp. 1–4). Piscataway, NJ: IEEE.
33. B.L.Giordano,, & S.McAdams, (2010). Sound source mechanics and musical timbre perception: evidence from previous studies. Music Perception, 28(2), 155–168.
34. J.L.Golubock,, & P.Janata, (2013). Keeping timbre in mind: Working memory for complex sounds that can’t be verbalized. Journal of Experimental Psychology: Human Perception and Performance, 39(2), 399–412.
35. J.M.Grey, (1975). An exploration of musical timbre (Unpublished doctoral dissertation). CCRMA: Stanford University, USA.
36. J.M.Grey, (1977). Multidimensional perceptual scaling of musical timbres. The Journal of the Acoustical Society of America, 61(5), 1270–1277.
37. J.M.Grey,, & J.W.Gordon, (1978). Perceptual effects of spectral modifications on musical timbres. The Journal of the Acoustical Society of America, 63(5), 1493–1500.
38. T.L.Griffiths, (2015). Manifesto for a new (computational) cognitive revolution. Cognition, 135, 21–23.
39. S.Handel, (1995). Timbre perception and auditory object identification. In B.C.J.Moore (Ed.), Hearing Handbook of Perception and Cognition (pp. 425–461). San Diego, CA: Academic Press.
40. P.Boyer,, A.Klapuri,, & M.Davy, (2006). Automatic classification of pitched musical instrument sounds. Signal processing methods for music transcription (pp. 163–200). New York: Springer.
41. A.B.Horner,, J.W.Beauchamp,, & R.H.So, (2011). Evaluation of mel-band and MFCC-based error metrics for correspondence to discrimination of spectrally altered musical instrument sounds. Journal of the Audio Engineering Society, 59(5), 290–303.
42. E.J.Humphrey,, J.P.Bello,, & Y.LeCun, (2012). Moving beyond feature design: Deep architectures and automatic feature learning in music informatics. In F.Gouyon, P.Herrera, L.G.Martins, & M.Müller (Eds.), Proceedings of the 13th International Society for Music Information Retrieval Conference, Porto, Portugal, Oct 8-–12, 2012 (pp. 403–408). Porto, Portugal: FEUP Ediç oes.
43. P.Iverson,, & C.L.Krumhansl, (1993). Isolating the dynamic attributes of musical timbre. Journal of the Acoustical Society of America, 94(5), 2595–2603.
44. A.M.Jacobs,, & J.Grainger, (1994). Models of visual word recognition: Sampling the state of the art. Journal of Experimental Psychology, 20(6), 1311–1334.
45. M.L.Jepsen,, S.D.Ewert,, & T.Dau, (2008). A computational model of human auditory signal processing and perception. The Journal of the Acoustical Society of America, 124(1), 422.
46. C.Joder,, S.Essid,, & G.Richard, (2009). Temporal integration for audio classification with application to musical instrument classification. IEEE Transactions on Audio, Speech, and Language Processing, 17(1), 174–186.
47. R.A.Kendall,, E.C.Carterette,, & J.M.Hajda, (1999). Perceptual and acoustical features of natural and synthetic orchestral instrument tones. Music Perception, 16(3), 327–363.
48. J.Krimphoff,, S.McAdams,, & S.Winsberg, (1994). Caractérisation du timbre des sons complexes. II. Analyses acoustiques et quantification psychophysique. Le. Journal de Physique IV, 4(C5), 625–628.
49. C.L.Krumhansl, (1989). Why is musical timbre so hard to understand? In S.Nielzén & O.Olsson (Eds.), Structure and perception of electroacoustic sound and music, Vol. 846 (pp. 43–53). Amsterdam: Excerpta Medica.
50. J.B.Kruskal, (1964). Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis. Psychometrika, 29(1), 1–27.
51. S.Lakatos, (2000). A common perceptual space for harmonic and percussive timbres. Perception and Psychophysics, 62(7), 1426–1439.
52. S.-A.Lembke, (2014). When timbre blends musically: perception and acoustics underlying orchestration and performance (Unpublished doctoral dissertation). Canada: McGill University.
53. S.Mallat, (2012). Group invariant scattering. Communications on Pure and Applied Mathematics, 65(10), 1331–1398.
54. J.Marozeau,, A.de Cheveigné,, S.McAdams,, & S.Winsberg, (2003). The dependency of timbre on fundamental frequency. The Journal of the Acoustical Society of America, 114(5), 2946–2957.
55. D.Marr, (2010). Vision: A computational approach. Cambridge, MA: MIT Press.
56. A.Marsden, (2012). Interrogating melodic similarity: A definitive phenomenon or the product of interpretation? Journal of New Music Research, 41(4), 323–335.
57. K.D.Martin, Sound-source recognition: A theory and computational model. (Unpublished doctoral dissertation), Massachusetts Institute of Technology, USA.
58. S.McAdams, (1993). Recognition of sound sources and events. In S.McAdams & E.Bigand (Eds.), Thinking in sound: The cognitive psychology of human audition (pp. 146–198). Oxford: Oxford University Press.
59. S.McAdams, (2013). Musical timbre perception. In D.Deutsch (Ed.), The psychology of music (3rd ed.). (pp. 35–67). San Diego, CA: Academic Press.
60. S.McAdams,, J.W.Beauchamp,, & S.Meneguzzi, (1999). Discrimination of musical instrument sounds resynthesized with simplified spectrotemporal parameters. The Journal of the Acoustical Society of America, 105(2), 882–897.
61. S.McAdams,, V.Roussarie,, A.Chaigne,, & B.L.Giordano, (2010). The psychomechanics of simulated sound sources: Material properties of impacted thin plates. The Journal of the Acoustical Society of America, 128(3), 1401–1413.
62. S.McAdams,, S.Winsberg,, S.Donnadieu,, G.De Soete,, & J.Krimphoff, (1995). Perceptual scaling of synthesized musical timbres: Common dimensions, specificities, and latent subject classes. Psychological Research, 58(3), 177–192.
63. J.McDermott,, & E.P.Simoncelli, (2011). Sound texture perception via statistics of the auditory periphery: Evidence from sound synthesis. Neuron, 71, 926–940.
64. N.Misdariis,, A.Minard,, P.Susini,, G.Lemaitre,, S.McAdams,, & E.Parizet, (2010). Environmental sound perception: Metadescription and modeling based on independent primary studies. EURASIP Journal on Audio, Speech, and Music Processing, 2010(1), Article ID 362013
65. M.Müller,, D.P.Ellis,, A.Klapuri,, & G.Richard, (2011). Signal processing for music analysis. IEEE Journal of Selected Topics in Signal Processing, 5(6), 1088–1110.
66. J.I.Myung,, Y.Tang,, & M.A.Pitt, (2009). Evaluation and comparison of computational models. Methods in Enzymology, 454, 287–304.
67. A.Y.Ng, (1997). Preventing "overfitting" of cross-validation data. In D.H.Fisher (Ed.), Proceedings of the 14th International Conference on Machine Learning, Nashville, TN, USA, July 8–-12, 1997 (pp. 245–253). San Francisco, CA: Morgan Kaufmann.
68. F.Pachet,, & J.-J.Aucouturier, (2004). Improving timbre similarity: How high is the sky? Journal of Negative Results in Speech and Audio Sciences, 1(1), 1–13.
69. E.Pampalk,, A.Flexer,, & G.Widmer, (2005). Improvements of audio-based music similarity and genre classification. In Proceedings of the 6th International Society for Music Information Retrieval Conference, London, UK, Sep 11–-15, 2005 (pp. 628–633). London: Queen Mary, University of London.
70. K.Patil,, & M.Elhilali, (2013). Task-driven attentional mechanisms for auditory scene recognition. In Proceedings of the 2013 IEEE International Conference on Acoustic, Speech and Signal Processing (ICASSP), Vancouver, BC, Canada, Mar 26–-31, 2013 (pp. 828–832). Piscataway, NJ: IEEE.
71. K.Patil,, D.Pressnitzer,, S.Shamma,, & M.Elhilali, (2012). Music in our ears: The biological bases of musical timbre perception. PLOS Computational Biology, 8(11), e1002759.
72. R.D.Patterson,, E.Gaudrain,, & T.C.Walters, (2010). The perception of family and register in musical tones. In M.Riess Jones, R.R.Fay, & A.Popper. (Eds.), Music perception. New York: Springer. (pp. 13–50)
73. G.Peeters, (2003). Automatic classification of large musical instrument databases using hierarchical classifiers with inertia ratio maximization. In Proceedings of the 115th Audio Engineering Society Convention, New York, NY, USA, Oct 10–-13, 2003 (pp. 1–14). New York: Audio Engineering Society.
74. G.Peeters,, B.L.Giordano,, P.Susini,, N.Misdariis,, & S.McAdams, (2011). The Timbre Toolbox: Extracting audio descriptors from musical signals. The Journal of the Acoustical Society of America, 130(5), 2902–2916.
75. R.Plomp, (1970). Timbre as a multidimensional attribute of complex tones. In R.Plomp & G.F.Smoorenburg (Eds.), Frequency analysis and periodicity detection in hearing (pp. 397–414). Leiden: Suithoff.
76. K.R.Popper, (1963). Conjectures and refutations: The growth of scientific knowledge. London: Routledge and Kegan Paul.
77. C.Reuter, (2002). Klangfarbe und instrumentation. Bern: Peter Lang.
78. G.Richard,, S.Sundaram,, & S.Narayanan, (2013). An overview on perceptually motivated audio indexing and classification. Proceedings of the IEEE, 101(9), 1939–1954.
79. C.Saitis,, G.P.Scavone,, C.Fritz,, & B.L.Giordano, (2015). Effect of task constraints on the perceptual evaluation of violins. Acta Acustica united with Acustica, 101(2), 382–393.
80. S.Sattath,, & A.Tversky, (1977). Additive similarity trees. Psychometrika, 42(3), 319–345.
81. E.Schubert,, & J.Wolfe, (2006). Does timbral brightness scale with frequency and spectral centroid? Acta Acustica united with Acustica, 92(5), 820–825.
82. G.Schwarz, (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464.
83. R.Shepard, (1962). The analysis of proximities: Multidimensional scaling with an unknown distance function. I. Psychometrika, 27(2), 125–140.
84. R.N.Shepard, (1987). Toward a universal law of generalization for psychological science. Science, 237(4820), 1317–1323.
85. W.Siedlecki,, & J.Sklansky, (1989). A note on genetic algorithms for large-scale feature selection. Pattern Recognition Letters, 10(5), 335–347.
86. J.Štěpánek,, & Z.Otčenášek, (2004, March). Interpretation of violin spectrum using psychoacoustic experiments. Paper presented at the International Symposium on Musical Acoustics (ISMA2004), Nara, Japan.
87. J.Štěpánek,, V.Syrový,, Z.Otčenášek,, C.Taesch, C.h., & J.Angster, (2005, August). Spectral features influencing perception of pipe organ sounds. Paper presented at the Forum Acusticum, Budapest, Hungary.
88. B.L.Sturm, (2012). Two systems for automatic music genre recognition: What are they really recognizing? In C.C.S.Liem (Ed.), Proceedings of the 2nd International ACM Workshop on Music Information Retrieval with User-Centered and Multimodal Strategies, Nara, Japan, Oct 29 - Nov 2, 2012 (pp. 69–74). New York: ACM.
89. B.L.Sturm, (2013). Classification accuracy is not enough. Journal of Intelligent Information Systems, 41(3), 371–406.
90. B.L.Sturm, (2014). A simple method to determine if a music information retrieval system is a horse. IEEE Transactions on Multimedia, 16(6), 1636–1644.
91. B.L.Sturm,, & P.Noorzad, (2012). On automatic music genre recognition by sparse representation classification using auditory temporal modulations. In M.Aramaki, M.Barthet, R.Kronland-Martinet, & S.Ystad (Eds.), Proceedings of the 9th International Symposium on Computer Music Modelling and Retrieval, London, UK, Jun 19–22, 2012 (pp. 379–394). Heidelberg: Springer.
92. P.Susini,, G.Lemaitre,, & S.McAdams, (2012). Psychological measurement for sound description and evaluation. In B.Berglund, G.B.Rossi, J.T.Townsend, & L.R.Pendrill (Eds.), Measurement with persons: Theory, methods, and implementation areas (pp. 227–253). New York: Psychology Press.
93. H.Terasawa,, J.Berger,, & S.Makino, (2012). In search of a perceptual metric for timbre: Dissimilarity judgments among synthetic sounds with MFCC-derived spectral envelopes. Journal of the Audio Engineering Society, 60(9), 674–685.
94. A.Tversky, (1977). Features of similarity. Psychological Review, 84(4), 327–352.
95. H.von Helmholtz, (1875). On the sensations of tone as a physiological basis for the study of music. London: Longmans, Green.
96. D.L.Wessel, (1973). Psychoacoustics and music: A report from Michigan State University. PACE: Bulletin of the Computer Arts Society, 30, 1–2.
97. D.L.Wessel,, D.Bristow,, & Z.Settel, (1987). Control of phrasing and articulation in synthesis. In J.Beauchamp (Ed.), Proceedings of the International Computer Music Conference, Urbana-Champaign, IL, USA, Aug 23–26, 1987 (pp. 108–116). San Francisco, CA: Computer Music Association.
98. G.A.Wiggins (2009). Semantic gap?? Schemantic schmap!! Methodological considerations in the scientific study of music. Proceedings of the 11th IEEE International Symposium on Multimedia, San Diego, CA, USA, Dec 14–-16, 2009 (pp. 447–482). Piscataway, NJ: IEEE.
99. S.Winsberg,, & G.De Soete, (1993). A latent class approach to fitting the weighted Euclidean model. CLASCAL. Psychometrika, 58(2), 315–330.
100. S.Wun,, A.Horner,, & B.Wu, (2014). Effect of spectral centroid manipulation on discrimination and identification of instrument timbres. Journal of the Audio Engineering Society, 62(9), 575–583.