Analyzing correspondence between sound objects and body motion

ACM Transactions on Applied Perception

Volumn 10, Issue 2, 2013, Pages

  Nymoen, Kristian ^a   Godøy, Rolf Inge ^a   Jensenius, Alexander Refsum ^a   Torresen, Jim ^a  

^a UNIVERSITY OF OSLO   (Norway)

Author keywords

Music related motion; Sound tracing

Indexed keywords

ANALYSIS TECHNIQUES; CANONICAL CORRELATIONS; MOTION CAPTURE DATA; MULTIDIMENSIONAL DATA; MUSIC-RELATED MOTION; PATTERN RECOGNITION CLASSIFIERS; PERCEPTUAL FEATURE; SOUND-TRACING;

PATTERN RECOGNITION;

EXPERIMENTS;

EID: 84885636283     PISSN: 15443558     EISSN: 15443965     Source Type: Journal    
DOI: 10.1145/2465780.2465783     Document Type: Article

References (50)

1. BREGMAN, A. S. 1990. Auditory Scene Analysis: The Perceptual Organization of Sound. MIT Press, Cambridge, MA.
2. BURGER, B., THOMPSON, M., LUCK, G., SAARIKALLIO, S., AND TOIVIAINEN, P. 2012. Music moves us: Beat-related musical features influence regularity of music-induced movement. In Proceedings of the International Conference on Music Perception and Cognition. 183-187.
3. CAMURRI, A., LAGERLÖF, I., AND VOLPE, G. 2003. Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques. Int. J. Hum.-Comput. Stud. 59, 1-2, 213-225.
4. CARAMIAUX, B., BEVILACQUA, F., AND SCHNELL, N. 2010. Towards a gesture-sound cross-modal analysis. In Gesture in Embodied Communication and Human-Computer Interaction, S. Kopp and I. Wachsmuth, Eds., Lecture Notes in Computer Science Series, vol. 5934, Springer, Berlin, 158-170.
5. CLARKE, E. F. 2005. Ways of Listening: An Ecological Approach to the Perception of Musical Meaning. Oxford University Press, New York.
6. DUDA, R. O., HART, P. E., AND STORK, D. G. 2000. Pattern Classification. Wiley-Interscience, New York.
7. EITAN, Z. AND GRANOT, R. Y. 2006. How music moves: Musical parameters and listeners' images of motion. Music Percept. 23, 3, 221-248.
8. GALANTUCCI, B., FOWLER, C., AND TURVEY, M. 2006. The motor theory of speech perception reviewed. Psychonom. Bull. Rev. 13, 3, 361-377.
9. GODØY, R. I. 2006. Gestural-sonorous objects: Embodied extensions of Schaeffer's conceptual apparatus. Org. Sound 11, 02, 149-157.
10. GODØY, R. I. 2010a. Gestural affordances of musical sound. In Musical Gestures: Sound, Movement, and Meaning, R. I. Godøy and M. Leman, Eds., Routledge, New York, 103-125.
11. GODØY, R. I. 2010b. Images of sonic objects. Org. Sound 12, 54-62.
12. GODØY, R. I., HAGA, E., AND JENSENIUS, A. R. 2006. Exploring music-related gestures by sound-tracing a preliminary study. In Proceedings of the 2nd ConGAS International Symposium on Gesture Interfaces for Multimedia Systems.
13. GODØY, R. I. AND JENSENIUS, A. R. 2009. Body movement in music information retrieval. In Proceedings of the 10th International Society for Music Information Retrieval Conference. 45-50.
14. GREY, J. M. 1977. Multidimensional perceptual scaling of musical timbres. J. Acoust. Soc. Amer. 61, 5, 1270-1277.
15. HAGA, E. 2008. Correspondences between music and body movement. Ph.D. thesis, University of Oslo.
16. HOLM, S. 1979. A simple sequentially rejective multiple test procedure. Scand. J. Stat. 65-70.
17. HOTELLING, H. 1936. Relations between two sets of variates. Biometrika 28, 3/4, 321-377.
18. HURON, D., DAHL, S., AND JOHNSON, R. 2009. Facial expression and vocal pitch height: Evidence of an intermodal association. Emp. Music. Rev. 4, 3, 93-100.
19. KOHLER, E., KEYSERS, C., UMILTA, A., FOGASSI, L., GALLESE, V., AND RIZZOLATTI, G. 2002. Hearing sounds, understanding actions: Action representation in mirror neurons. Science 297, 5582, 846-848.
20. KOZAK.M.,NYMOEN, K., AND GODØY, R. I. 2011. The effects of spectral features of sound on gesture type and timing. In Proceedings of the 9th International Gesture Workshop. E. Efthimiou and G. Kouroupetroglou, Eds., 20-24.
21. KUSSNER, M. 2012. Creating shapes: Musicians' and non-musicians' visual representations of sound. In Proceedings of 4th International Conferance of Students of Systematic Musicology. U. Seifert and J. Wewers, Eds., epOs-Music.
22. LARTILLOT, O., TOIVIAINEN, P., AND EEROLA, T. 2008. A matlab toolbox for music information retrieval. In Data Analysis, Machine Learning and Applications, C. Preisach, H. Burkhardt, L. Schmidt-Thieme, and R. Decker, Eds., Studies in Classification, Data Analysis, and Knowledge Organization., Springer, Berlin, 261-268.
23. LEMAN, M. 2008. Embodied Music Cognition and Mediation Technology. The MIT Press, Cambridge, MA.
24. LEMAN, M., VERMEULEN, V., DE VOOGDT, L., TAELMAN, J.,MOELANTS, D., AND LESAFFRE, M. 2004. Correlation of gestural musical audio cues and perceived expressive qualities. In Gesture-Based Communication in Human-Computer Interaction, A. Camurri and G. Volpe, Eds., Lecture Notes in Computer Science Series, vol. 2915, Springer, Berlin, 40-54.
25. LIBERMAN, A. M. AND MATTINGLY, I. G. 1985. The motor theory of speech perception revised. Cognition 21, 1, 1-36.
26. MCADAMS, S.,WINSBERG, S.,DONNADIEU, S., SOETE, G., AND KRIMPHOFF, J. 1995. Perceptual scaling of synthesized musical timbres: Common dimensions, specificities, and latent subject classes. Psych. Res. 58, 177-192.
27. MERER, A., YSTAD, S., KRONLAND-MARTINET, R., AND ARAMAKI, M. 2008. Semiotics of sounds evoking motions: Categorization and acoustic features. In Computer Music Modeling and Retrieval. Sense of Sounds, R. Kronland-Martinet, S. Ystad, and K. Jensen, Eds., Lecture Notes in Computer Science Series, vol. 4969, Springer-Verlag, Berlin, 139-158.
28. MIERSWA, I., WURST, M., KLINKENBERG, R., SCHOLZ, M., AND EULER, T. 2006. Yale: Rapid prototyping for complex data mining tasks. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '06). L. Ungar, M. Craven, D. Gunopulos, and T. Eliassi-Rad, Eds., ACM, 935-940.
29. NYMOEN, K., CARAMIAUX, B., KOZAK, M., AND TORRESEN, J. 2011a. Analyzing sound tracings: a multimodal approach to music information retrieval. In Proceedings of the 1st International ACM Workshop on Music Information Retrieval with User-Centered and Multimodal Strategies (MIRUM '11). ACM, New York, 39-44.
30. NYMOEN, K., GLETTE, K., SKOGSTAD, S. A., TORRESEN, J., AND JENSENIUS, A. R. 2010. Searching for cross-individual relationships between sound and movement features using an SVM classifier. In Proceedings of the International Conference on New Interfaces for Musical Expression. 259-262.
31. NYMOEN, K., SKOGSTAD, S. A., AND JENSENIUS, A. R. 2011b. Soundsaber - a motion capture instrument. In Proceedings of the International Conference on New Interfaces for Musical Expression. 312-315.
32. NYMOEN, K., TORRESEN, J., GODØY, R., AND JENSENIUS, A. R. 2012. A statistical approach to analyzing sound tracings. In Speech, Sound and Music Processing: Embracing Research in India, S. Ystad, M. Aramaki, R. Kronland-Martinet, K. Jensen, and S. Mohanty, Eds., Lecture Notes in Computer Science Series, vol. 7172, Springer, Berlin, 120-145.
33. PEETERS, G., GIORDANO, B., SUSINI, P., MISDARIIS, N., AND MCADAMS, S. 2011. The timbre toolbox: Extracting audio descriptors from musical signals. Acoust. Soc. Amer. 130, 5, 2902-2916.
34. PELLEGRINO, G., FADIGA, L., FOGASSI, L.,GALLESE, V., AND RIZZOLATTI, G. 1992. Understanding motor events: A neurophysiological study. Exp. Brain Res. 91, 176-180.
35. PIRHONEN, A. 2007. Semantics of sounds and images - Can they be paralleled? In Proceedings of the 13th International Conference on Auditory Display, 319-325.
36. REPP, B. H. 1995. Quantitative effects of global tempo on expressive timing in music performance: Some perceptual evidence. Music Percep. 13, 1, 39-57.
37. ROSENBAUM, D. 2001. Human Motor Control. Academic Press, San Diego.
38. SCHAEFFER, P. 1966. Traité des Objets Musicaux. Éditions du Seuil, Paris.
39. SCHUBERT, E. 2002. Correlation analysis of continuous emotional response to music. Musicæ Scientiæ Spec.Is. 2001-02, 213-236.
40. STEIN, B. E., AND MEREDITH, M. A. 1993. The Merging of the Senses. The MIT Press, Cambridge, MA.
41. STYNS, F., VAN NOORDEN, L.,MOELANTS, D., AND LEMAN, M. 2007. Walking on music. Hum Movement Sci 26, 5, 769-785.
42. VAN NOORDEN, L. 1975. Temporal coherence in the perception of tone sequences. Ph.D. thesis, Technical University Eindhoven.
43. VAN NORT, D. 2009. Instrumental Listening: Sonic gesture as design principle. Organ. Sound 14, 2, 177-187.
44. VASSILAKIS, P. 2001. Perceptual and physical properties of amplitude fluctuation and their musical significance. Ph.D. thesis, University of California, Los Angeles.
45. VINES, B. W.,KRUMHANSL, C. L.,WANDERLEY, M. M., AND LEVITIN, D. J. 2006. Cross-modal interactions in the perception of musical performance. Cognition 101, 1, 80-113.
46. VROOMEN, J. AND DE GEDLER, B. 2000. Sound enhances visual perception: Cross-modal effects on auditory organization on vision. J. Exper. Psych.: Human Percept. Perf. 26, 5, 1583-1590.
47. WALKER, B. N. 2000. Magnitude estimation of conceptual data dimensions for use in sonification. Ph.D. thesis, Rice University, Houston, TX.
48. WALKER, R. 1987. The effects of culture, environment, age, and musical training on choices of visual metaphors for sound. Percept. Psych. 491-502.
49. WESSEL, D. L. 1979. Timbre space as a musical control structure. Comput. Music J. 3, 2, 45-52.
50. ZATORRE, R. 2005. Music, the food of neuroscience? Nature 434, 312-315.