A hierarchical approach for the design of gesture-to-sound mappings

Proceedings of the 9th Sound and Music Computing Conference, SMC 2012

Volumn , Issue , 2012, Pages

  Françoise, Jules ^a   Caramiaux, Baptiste ^a   Bevilacqua, Frédéric ^a  

^a Université Pierre et Marie Curie   (France)

Author keywords

[No Author keywords available]

Indexed keywords

FLEXIBLE STRUCTURES; HIDDEN MARKOV MODELS;

GESTURE SEGMENTATION; HIERARCHICAL APPROACH; HIERARCHICAL HIDDEN MARKOV MODEL; LEARNING PHASE; MAPPING SYSTEMS; SOUND SYNTHESIS; SYNTHESIS TECHNIQUES; TEMPORAL MAPPINGS;

MAPPING;

EID: 84905196786     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (30)

1. A. Hunt, M. M. Wanderley, and M. Paradis, "The importance of parameter mapping in electronic instrument design," in Proceedings of the 2002 conference on New interfaces for musical expression. National University of Singapore, 2002, pp. 1-6.
2. J. Rovan, M. M. Wanderley, S. Dubnov, and P. Depalle, "Instrumental Gestural Mapping Strategies as Expressivity Determinants in Computer Music Performance," in Kansei, The Technology of Emotion. Proceedings of the AIMI International Workshop., 1997, pp. 68-73.
3. D. Van Nort, M. M. Wanderley, and P. Depalle, "On the Choice of Mappings Based on Geometric Properties," in Proceedings of the 2004 Conference on New Interfaces for Musical Expression, Hamamatsu, Japan, 2004.
4. M. A. Lee and D. Wessel, "Connectionist models for real-time control of synthesis and compositional algorithms," in Proceedings of the International Computer Music Conference (ICMC), San Jose, CA, 1992, pp. 277-280.
5. J. Chadabe, "The Limitations of Mapping as a Structural Descriptive in Electronic Instruments," in Proceedings of the 2002 conference on New interfaces for musical expression. National University of Singapore, 2002, pp. 1-5.
6. S. Fels and G. Hinton, "Glove-talk: A neural network interface between a data-glove and a speech synthesizer," Neural Networks, IEEE Transactions on, vol. 4, no. 1, pp. 2-8, 1993.
7. A. Cont, T. Coduys, and C. Henry, "Real-time Gesture Mapping in Pd Environment using Neural Networks," in Proceedings of the 2004 conference on New interfaces for musical expression. National University of Singapore, 2004, pp. 39-42.
8. F. Bevilacqua, R. Müller, and N. Schnell, "MnM: a Max/MSP mapping toolbox," in Proceedings of the 2005 conference on New interfaces for musical expression, 2005.
9. F. Bevilacqua, B. Zamborlin, A. Sypniewski, N. Schnell, F. Guédy, and N. Rasamimanana, "Continuous realtime gesture following and recognition," in Embodied Communication and Human-Computer Interaction, volume 5934 of Lecture Notes in Computer Science. Springer Verlag, 2010, pp. 73-84.
10. N. Gillian and R. Knapp, "A Machine Learning Toolbox For Musician Computer Interaction," in Proceedings of the 2011 Conference on New Interfaces for Musical Expression, 2011.
11. R. Fiebrink, "Real-time human interaction with supervised learning algorithms for music composition and performance," Ph.D. dissertation, Faculty of Princeton University, 2011.
12. J. Malloch, S. Sinclair, and M. M. Wanderley, "A Network-Based Framework for Collaborative Development and Performance of Digital Musical Instruments," in Computer Music Modeling and Retrieval. Sense of Sounds. Springer, 2008, pp. 401-425.
13. J. Malloch, S. Sinclair, A. Hollinger, and M. Wanderley, Input Devices and Music Interaction, ser. Springer Tracts in Advanced Robotics. Springer Berlin/Heidelberg, 2011, vol. 74, pp. 67-83.
14. B. Caramiaux, F. Bevilacqua, and N. Schnell, "Towards a gesture-sound cross-modal analysis," Gesture in Embodied Communication and Human-Computer Interaction, vol. 5934 of Lecture Notes in Computer Science, pp. 158-170, 2010.
15. D. J. Merrill and J. A. Paradiso, "Personalization, expressivity, and learnability of an implicit mapping strategy for physical interfaces," in Proc of CHI 2005 Conference on Human Factors in Computing Systems. Citeseer, 2005, p. 2152.
16. R. Fiebrink, P. R. Cook, and D. Trueman, "Playalong mapping of musical controllers," in In Proceedings of the International Computer Music Conference (ICMC), 2009.
17. F. Bevilacqua and N. Schnell, "Wireless sensor interface and gesture-follower for music pedagogy," Proceedings of the 7th international conference on New interfaces for musical expression, pp. 124-129, 2007.
18. F. Bevilacqua, N. Schnell, and N. Rasamimanana, "Online Gesture Analysis and Control of Audio Processing," in Musical Robots and Interactive Multimodal Systems. Springer, 2011, pp. 127-142.
19. S. Jordà, "Digital Lutherie: Crafting musical computers for new musics performance and improvisation," Ph.D. dissertation, UPF, 2005.
20. S. Jordà, "On Stage: the Reactable and other Musical Tangibles go Real," International Journal of Arts and Technology, vol. 1, pp. 268-287, 2008.
21. B. Caramiaux, "Etudes sur la relation geste-son en performance musicale," PhD Dissertation, Ircam Université Pierre et Marie Curie, 2012.
22. R. I. Godoy, A. R. Jensenius, and K. Nymoen, "Chunking in music by coarticulation," Acta Acustica united with Acustica, vol. 96, no. 4, pp. 690-700, 2010.
23. J. Bloit, N. Rasamimanana, and F. Bevilacqua, "Modeling and segmentation of audio descriptor profiles with segmental models," Pattern Recognition Letters, vol. 31, no. 12, pp. 1507-1513, Sep. 2010.
24. B. Caramiaux, M. M. Wanderley, and F. Bevilacqua, "Segmenting and Parsing Instrumentalist's Gestures," Journal of New Music Research, vol. 41, no. 1, pp. 13-29, 2012.
25. L. R. Rabiner, "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition," Proceedings of the IEEE, vol. 77, no. 2, pp. 257-286, 1989.
26. S. Fine, Y. Singer, and N. Tishby, "The hierarchical hidden Markov model: Analysis and applications," Machine learning, vol. 32, no. 1, pp. 41-62, 1998.
27. K. P. Murphy and M. A. Paskin, "Linear Time Inference in Hierarchical HMMs," in Advances in Neural Information Processing Systems, 2001.
28. J. Françoise, "Realtime Segmentation and Recognition of Gestures using Hierarchical Markov Models," Master's Thesis, Université Pierre et Marie Curie, Ircam, 2011.
29. N. Rasamimanana, F. Bevilacqua, N. Schnell, E. Fléty, and B. Zamborlin, "Modular Musical Objects Towards Embodied Control Of Digital Music Real Time Musical Interactions," Proceedings of the fifth international conference on Tangible, embedded, and embodied interaction, pp. 9-12, 2011.
30. N. Schnell, A. Röbel, D. Schwarz, G. Peeters, and R. Borghesi, "Mubu & friends assembling tools for content based real-time interactive audio processing in max/msp," in Proceedings of International Computer Music Conference, Montreal, 2009.