Perception and modeling of affective qualities of musical instrument sounds across pitch registers

Frontiers in Psychology

Volumn 8, Issue FEB, 2017, Pages

  McAdams, Stephen ^a   Douglas, Chelsea ^a   Vempala, Naresh N ^b  

^a MCGILL UNIVERSITY   (Canada)

^b RYERSON UNIVERSITY   (Canada)

Author keywords

Emotion; Energy arousal; Musical instruments; Musical timbre; Pitch register; Preference; Tension arousal; Valence

Indexed keywords

EID: 85014045186     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2017.00153     Document Type: Article

References (58)

1. Adler, S. (2002). Study of Orchestration, 3rd Edn. New York, NY: W. W. Norton.
2. Baayen, R. H., Davidson, D. J., and Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. J. Mem. Lang. 59, 390-412. doi: 10.1016/j.jml.2007.12.005
3. Barr, D. J., Levy, R., Scheepers, C., and Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: keep it maximal. J. Mem. Lang. 68, 255-278. doi: 10.1016/j.jml.2012.11.001
4. Bates, D., Maechler, M., Bolker, B., and Walker, S. (2014). lme4: Linear Mixed-Effects Models Using Eigen and S4. R Package Version 1.0-5. Available online at: http://CRAN.R-project.org/package=lme4
5. Bigand, E., Vieillard, S., Madurell, F., Marozeau, J., and Dacquet, A. (2005). Multidimensional scaling of emotional responses to music: the effect of musical expertise and of the duration of the excerpts. Cogn. Emot. 19, 1113-1139. doi: 10.1080/02699930500204250
6. Bishop, C. M. (1996). Neural Networks for Pattern Recognition. Oxford, UK: Oxford University Press.
7. Coutinho, E., and Cangelosi, A. (2011). Musical emotions: predicting second-by-second subjective feelings of emotion from low-level psychoacoustic features and physiological measurements. Emotion 11, 921-937. doi: 10.1037/a0024700
8. Eerola, T., Ferrer, R., and Alluri, V. (2012). Timbre and affect dimensions: evidence from affect and similarity ratings and acoustic correlates of isolated instrument sounds. Music Percept. 30, 49-70. doi: 10.1525/mp.2012.30.1.49
9. Eerola, T., Friberg, A., and Bresin, R. (2013). Emotional expression in music: contribution, linearity, and additivity of primary musical cues. Front. Psychol. 4:487. doi: 10.3389/fpsyg.2013.00487
10. Eerola, T., Lartillot, O., and Toiviainen, P. (2009). "Prediction of multidimensional emotional ratings in music from audio using multivariate regression models," in Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR 2009) (Kobe), 621-626.
11. Filipic, S., Tillmann, B., and Bigand, E. (2010). Judging familiarity and emotion from very brief musical excerpts. Psychon. Bull. Rev. 17, 335-341. doi: 10.3758/PBR.17.3.335
12. Fox, J., and Weisberg, S. (2011). An R Companion to Applied Regression 2nd Edn. Thousand Oaks, CA: Sage.
13. Frick, R. W. (1985). Communicating emotion: the role of prosodic features. Psychol. Bull. 97, 412-429. doi: 10.1037/0033-2909.97.3.412
14. Gabrielsson, A., and Lindström, E. (2010). "The influence of musical structure on emotional expression," in Handbook of Music and Emotion: Theory, Research, Application, eds P. N. Juslin and J. A. Sloboda (New York, NY: Oxford University Press), 367-400.
15. Geladi, P., and Kowalski, B. R. (1986). Partial least-squares regression: a tutorial. Anal. Chim. Acta 185, 1-17. doi: 10.1016/0003-2670(86)80028-9
16. Hailstone, J. C., Omar, R., Henley, S. M., Frost, C., Kenward, M. G., and Warren, J. D. (2009). It's not what you play, it's how you play it: timbre affects perception of emotion in music. Q. J. Exp. Psychol. 62, 2141-2155. doi: 10.1080/17470210902765957
17. Haykin, S. (2008). Neural Networks and Learning Machines. Upper Saddle River, NJ: Prentice Hall.
18. Hexler.net (2011). TouchOSC: Modular OSC and MIDI Control Surface for iPhone, iPod Touch, and iPad. Available online at: http://hexler.net/
19. Holmes, P. A. (2011). An exploration of musical communication through expressive use of timbre: the performer's perspective. Psychol. Music 40, 301-323. doi: 10.1177/0305735610388898
20. Huron, D. (2008). A comparison of average pitch height and interval size in major-and minor-key themes: evidence consistent with affect-related pitch prosody. Empir. Musicol. Rev. 3, 59-63. Available online at: http://hdl.handle.net/1811/31940
21. Huron, D., Anderson, N., and Shanahan, D. (2014). You can't play a sad song on the banjo: acoustic factors in the judgment of instrument capacity to convey sadness. Empir. Musicol. Rev. 9, 29-41. doi: 10.18061/emr.v9i1.4085
22. Huron, D., Kinney, D., and Precoda, K. (2006). Influence of pitch height on the perception of submissiveness and threat in musical passages. Empir. Musicol. Rev. 1, 170-177. Available online at: http://hdl.handle.net/1811/24068
23. Ilie, G., and Thompson, W. F. (2006). A comparison of acoustic cues in music and speech for three dimensions of affect. Music Percept. 23, 319-329. doi: 10.1525/mp.2006.23.4.319
24. ISO (2004). Acoustics-Reference Zero for the Calibration of Audiometric Equipment-Part 8: Reference Equivalent Threshold Sound Pressure Levels for Pure Tones and Circumaural Earphones (ISO 389-8). Technical report, International Organization for Standardization, Geneva.
25. Juslin, P. N., and Laukka, P. (2003). Communication of emotions in vocal expression and music performance: different channels, same code? Psychol. Bull. 129:770. doi: 10.1037/0033-2909.129.5.770
26. Juslin, P. N., and Laukka, P. (2004). Expression, perception, and induction of musical emotions: a review and a questionnaire study of everyday listening. J. New Music Res. 33, 217-238. doi: 10.1080/0929821042000317813
27. Juslin, P. N., and Västfjäll, D. (2008). Emotional responses to music: the need to consider underlying mechanisms. Behav. Brain Sci. 31, 559-575. doi: 10.1017/S0140525X08005293
28. Kazazis, S., Esterer, N., Depalle, P., and McAdams, S. (2016). "Testing the robustness of the Timbre Toolbox and the MIRtoolbox," in Proceedings of the 14th International Conference for Music Perception and Cognition, ed T. Zanto (San Francisco, CA: University of California), 24.
29. Kim, Y. E., Schmidt, E. M., Migneco, R., Morton, B. G., Richardson, P., Scot, J., et al. (2010). "Music emotion recognition: a state of the art review," in Proceedings of the 11th International Society for Music Information Retrieval Conference (ISMIR 2010) (Utrecht), 255-266.
30. Krumhansl, C. L. (2002). Music: a link between cognition and emotion. Curr. Dir. Psychol. Sci. 11, 45-50. doi: 10.1111/1467-8721.00165
31. Kumar, S., Forster, H. M., Bailey, P., and Griffiths, T. D. (2008). Mapping unpleasantness of sounds to their auditory representation. J. Acoust. Soc. Am. 124, 3810-3817. doi: 10.1121/1.3006380
32. Lartillot, O., and Toiviainen, P. (2007). "A Matlab toolbox for musical feature extraction from audio," in Proceedings of the 10th International Conference on Digital Audio Effects (DAFx) (Bordeaux), 237-244.
33. Lenth, R. V. (2013). lsmeans: Least-Squares Means. R Package Version 2.00-4. Available online at: https://CRAN.R-project.org/package=lsmeans (Accessed 18 October, 2016).
34. Marozeau, J., de Cheveigné, A., McAdams, S., and Winsberg, S. (2003). The dependency of timbre on fundamental frequency. J. Acoust. Soc. Am. 114, 2946-2957. doi: 10.1121/1.1618239
35. Martin, F. N., and Champlin, C. A. (2000). Reconsidering the limits of normal hearing. J. Am. Acad. Audiol. 11, 64-66.
36. McAdams, S. (1993). "Recognition of sound sources and events," in Thinking in Sound: The Cognitive Psychology of Human Audition, eds S. McAdams and E. Bigand (Oxford, UK: Oxford University Press), 146-198.
37. McAdams, S., and Goodchild, M. (forthcoming). "Musical structure: sound timbre," in Routledge Companion to Music Cognition, eds R. Ashley R. Timmers (New York, NY: Routledge).
38. McAdams, S., Winsberg, S., Donnadieu, S., De Soete, G., and Krimphoff, J. (1995). Perceptual scaling of synthesized musical timbres: common dimensions, specificities, and latent subject classes. Psychol. Res. 58, 177-192. doi: 10.1007/BF00419633
39. Milne, L. (1995). "Feature selection using neural networks with contribution measures," in AI'95, Australian Joint Conferences on Artificial Intelligence, Canberra. Singapore, World Scientific.
40. Moore, B. C., and Glasberg, B. R. (1983). Suggested formulae for calculating auditory-filter bandwidths and excitation patterns. J. Acoust. Soc. Am. 74, 750-753. doi: 10.1121/1.389861
41. Peeters, G. (2004). A Large Set of Audio Features for Sound Description (Similarity and Classification) in the CUIDADO Project. Technical Report, IRCAM, Paris, France.
42. Peeters, G., Giordano, B. L., Susini, P., Misdariis, N., and McAdams, S. (2011). The timbre toolbox: extracting audio descriptors from musical signals. J. Acoust. Soc. Am. 130, 2902-2916. doi: 10.1121/1.3642604
43. Peretz, I., Gagnon, L., and Bouchard, B. (1998). Music and emotion: perceptual determinants, immediacy, and isolation after brain damage. Cognition 68, 111-141. doi: 10.1016/S0010-0277(98)00043-2
44. Risset, J.-C., and Wessel, D. L. (1999). "Exploration of timbre by analysis and synthesis." in The Psychology of Music, ed D. Deutsch (San Diego, CA: Academic Press), 113-169.
45. Rumelhart, D. E., Hinton, G. E., and Williams, R. J. (1986). Learning representations by back-propagating errors. Nature 323, 533-536. doi: 10.1038/323533a0
46. Rumsey, F., Zielinski, S., Kassier, R., and Bech, S. (2005). On the relative importance of spatial and timbral fidelities in judgments of degraded multichannel audio quality. J. Acoust. Soc. Am. 118, 968-976. doi: 10.1121/1.1945368
47. Russo, F. A., Vempala, N. N., and Sandstrom, G. M. (2013). Predicting musically induced emotions from physiological inputs: linear and neural network models. Front. Psychol. 4:468. doi: 10.3389/fpsyg.2013.00468
48. Scherer, K. R., and Oshinsky, J. S. (1977). Cue utilization in emotion attribution from auditory stimuli. Motiv. Emot. 1, 331-346. doi: 10.1007/BF00992539
49. Schimmack, U., and Grob, A. (2000). Dimensional models of core affect: a quantitative comparison by means of structural equation modeling. Eur. J. Pers. 14, 325-345. doi: 10.1002/1099-0984(200007/08)14:43.0.CO;2-I
50. Schimmack, U., and Reisenzein, R. (2002). Experiencing activation: energetic arousal and tense arousal are not mixtures of valence and activation. Emotion 2, 412-417. doi: 10.1037/1528-3542.2.4.412
51. Schubert, E. (1999). Measuring emotion continuously: validity and reliability of the two-dimensional emotion-space. Aust. J. Psychol. 51, 154-165. doi: 10.1080/00049539908255353
52. Sloboda, J. A., and O'Neill, S. A. (2001). "Emotions in everyday listening to music," in Music and Emotion: Theory and Research, eds P. N. Juslin and J. A. Sloboda (Oxford, UK: Oxford University Press), 415-430.
53. Smith, J. O. (2007). Mathematics of the Discrete Fourier Transform (DFT): With Audio Applications. Stanford, CA: W3K Publishing.
54. Vempala, N. N., and Russo, F. A. (2013). "Exploring cognitivist and emotivist positions of musical emotion using neural network models," in Proceedings of the 12th International Conference on Cognitive Modeling, eds R. L. West and T. C. Stewart (Ottawa, ON), 257-262.
55. Vienna Symphonic Library GmbH (2011). Vienna Symphonic Library. Available online at: http://vsl.co.at
56. Weber, R. (1991). "The continuous loudness judgement of temporally variable sounds with an "analog" category procedure," in Fifth Oldenburg Symposium on Psychological Acoustics, eds A. Schick, J. Hellbruck, and R. Weber (Oldenburg: BIS), 267-294.
57. West, B. T., Welch, K. B., and Galecki, A. T. (2006). Linear Mixed Models: A Practical Guide Using Statistical Software. Boca Raton, FL: Chapman & Hall/CRC Press.
58. Wold, S., Sjöström, M., and Eriksson, L. (2001). PLS-regression: a basic tool of chemometrics. Chemometr. Intell. Lab. Syst. 58, 109-130. doi: 10.1016/S0169-7439(01)00155-1