Virtual Reality Musical Instruments: State of the Art, Design Principles, and Future Directions

Computer Music Journal

Volumn 40, Issue 3, 2016, Pages 22-40

  Serafin, Stefania ^a   Erkut, Cumhur ^a   Kojs, Juraj ^b   Nilsson, Niels C ^a   Nordahl, Rolf ^a  

^a AALBORG UNIVERSITY   (Denmark)

^b UNIVERSITY OF MIAMI   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER MUSIC; HELMET MOUNTED DISPLAYS; VIRTUAL REALITY;

DESIGN PRINCIPLES; EVALUATION METHODS; HEAD MOUNTED DISPLAYS; IMMERSIVE VISUALIZATION; LOW COST TECHNOLOGY; NEW INTERFACES FOR MUSICAL EXPRESSIONS; SOFTWARE SIMULATION; VISUAL COMPONENTS;

MUSICAL INSTRUMENTS;

EID: 85006054928     PISSN: 01489267     EISSN: 15315169     Source Type: Journal    
DOI: 10.1162/COMJ_a_00372     Document Type: Article

References (75)

1. Argelaguet, F., et al. 2016. "The Role of Interaction in Virtual Embodiment: Effects of the Virtual Hand Representation." In IEEE Virtual Reality. doi:10.1109/VR.2016.7504682.
2. Begault, D. 1994. 3-D Sound for Virtual Reality and Multimedia. Cambridge, Massachusetts: Academic Press.
3. Berthaut, F., M. Desainte-Catherine, and M. Hachet. 2011. "Interacting with 3D Reactive Widgets for Musical Performance." Journal of New Music Research 40(3):253-263.
4. Berthaut, F., V. Zappi, and D. Mazzanti. 2014. "Scenography of Immersive Virtual Musical Instruments." In IEEE VR Workshop: Sonic Interaction in Virtual Environments, pp. 19-24.
5. Berthaut, F., et al. 2013. "Rouages: Revealing the Mechanisms of DigitalMusical Instruments to the Audience." In Proceedings of the International Conference on New Interfaces for Musical Expression. Available online at hal.archives-ouvertes.fr/hal-00807049v1/document. Accessed August 2016.
6. Bødker, S. 2006. "When Second Wave HCI Meets Third Wave Challenges." In Proceedings of the Nordic Conference on Human-Computer Interaction: Changing Roles. Available online at dl.acm.org/citation.cfm?id=1182476. Accessed August 2016, subscription required.
7. Bødker, S. 2015. "Third-Wave HCI, 10 Years Later: Participation and Sharing." Interactions 22(5):24-31.
8. Bouchard, S., et al. 2011. "Exploring New Dimensions in the Assessment of Virtual Reality Induced Side Effects." Journal of Computer and Information Technology 1(3):20-32.
9. Bowman, D., et al. 2004. 3D User Interfaces: Theory and Practice. Boston, Massachusetts: Addison-Wesley.
10. Cabral, M., et al. 2015. "Crosscale: A 3D Virtual Musical Instrument Interface." In IEEE Symposium on 3D User Interfaces, pp. 199-200.
11. Cadoz, C., A. Luciani, and J. L. Florens. 1993. "CORDISANIMA: A Modeling and Simulation System for Sound and Image Synthesis; The General Formalism." Computer Music Journal 17(1):19-29.
12. Chafe, C. 2005. "Oxygen Flute: A Computer Music Instrument That Grows." Journal of New Music Research 34(3):219-226.
13. Cook, P. 2001. "Principles for Designing Computer Music Controllers." In Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 3-6.
14. Cook, P. R. 2002. Real Sound Synthesis for Interactive Applications. Boca Raton, Florida: CRC.
15. Cook, P. R. 2009. "Re-Designing Principles for Computer Music Controllers: A Case Study of SqueezeVox Maggie." In Proceedings of the International Conference on New Interfaces for Musical Expression, volume 9, pp. 218-221.
16. Cruz-Neira, C., D. J. Sandin, and T. A. De Fanti. 1993. "Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE." In Proceedings of the Annual Conference on Computer Graphics and Interactive Techniques, pp. 135-142.
17. Davis, S., K. Nesbitt, and E. Nalivaiko. 2014. "A Systematic Review of Cybersickness." In Proceedings of the Conference on Interactive Entertainment, pp. 1-9. Available online at dl.acm.org/citation.cfm?id=2677780. Accessed August 2016, subscription required.
18. Erkut, C., A. Jylhä, and R. Discioglu. 2011. "A Structured Design and Evaluation Model with Application to Rhythmic Interaction Displays." In Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 477-480.
19. Fernandes, A. S., and S. K. Feiner. 2016. "Combating VR Sickness through Subtle Dynamic Field-of-View Modification." In IEEE Symposium on 3D User Interfaces, pp. 201-210.
20. Fillwalk, J. 2015. "ChromaChord: A Virtual Musical Instrument." In IEEE Symposium on 3D User Interfaces, pp. 201-202.
21. Ford, C. 2010. "Musical Presence: Towards a New Philosophy of Music." Contemporary Aesthetics 8. Available online at hdl.handle.net/2027/spo.7523862.0008.020. Accessed August 2016.
22. Gallagher, S. 2000. "Philosophical Conceptions of the Self: Implications for Cognitive Science." Trends in Cognitive Sciences 4(1):14-21.
23. Gelineck, S., et al. 2005. "Virtual Reality Instruments Capable of Changing Dimensions in Real-Time." In Proceedings of the International Conference on Enactive Interfaces. Available online at www.researchgate.net/publication/255634646. Accessed August 2016.
24. Hamilton, R. 2009. "Building Interactive Networked Musical Environments Using q3osc." In Proceedings of the AES 35th International Conference: Audio for Games, pp. 1-6.
25. Heilbrun, A. 1989. "An Interview with Jaron Lanier." Whole Earth Review (64):108-119.
26. Herrera, G., R. Jordan, and L. Vera. 2006. "Agency and Presence: A Common Dependence on Subjectivity?" Presence 15(5):539-552.
27. Jaffe, D. A., and J. O. Smith. 1983. "Extensions of the Karplus-Strong Plucked-String Algorithm." Computer Music Journal 7(2):56-69.
28. Johnston, A. 2011. "Beyond Evaluation: Linking Practice and Theory in New Musical Interface Design." In Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 280-283.
29. Johnston, A., L. Candy, and E. Edmonds. 2008. "Designing and Evaluating Virtual Musical Instruments: Facilitating Conversational User Interaction." Design Studies 29(6):556-571.
30. Jordà, S. 2003. "Interactive Music Systems for Everyone: Exploring Visual Feedback as a Way for Creating More Intuitive, Efficient, and Learnable Instruments." In Proceedings of the Stockholm Music Acoustics Conference, pp. 681-684.
31. Jordà, S. 2004. "Instruments and Players: Some Thoughts on Digital Lutherie." Journal of New Music Research 33(3):321-341.
32. Jylhä, A., et al. 2011. "Design and Evaluation of Rhythmic Interaction with an Interactive Tutoring System." Computer Music Journal 35(2):36-48.
33. Karjalainen, M., and T. Mäki-Patola. 2004. "Physics-Based Modeling of Musical Instruments for Interactive Virtual Reality." In IEEE Workshop on Multimedia Signal Processing, pp. 223-226.
34. Karjalainen, M., et al. 2006. "Virtual Air Guitar." Journal of the Audio Engineering Society 54(10):964-980.
35. Kilteni, Konstantina, Ilias Bergstrom, and Mel Slater. 2013. "Drumming in Immersive Virtual Reality: The Body Shapes the Way We Play." IEEE Transactions on Visualization and Computer Graphics 19(4):597-605.
36. Kohlrausch, A., and S. van de Par. 1999. "Auditory-Visual Interaction: From Fundamental Research in Cognitive Psychology to (Possible) Applications." In International Society for Optics and Photonics (SPIE) Proceedings on Human Vision and Electronic Imaging IV, pp. 34-44. Available online at dx.doi.org/10.1117/12.348440. Accessed August 2016, subscription required.
37. Kojs, J. 2013. "Virtual Reality and Sonic Cyberspaces: Augmentation, Hybridisation and Abstraction." International Journal of Arts and Technology 6(2):167-180.
38. Kojs, J., S. Serafin, and C. Chafe. 2007. "Cyberinstruments via Physical Modeling Synthesis: Compositional Applications." Leonardo Music Journal 17:61-66.
39. LaViola, J. J. 2000. "A Discussion of Cybersickness in Virtual Environments." ACM SIGCHI Bulletin 32(1):47-56.
40. Leonard, J., et al. 2013. "A Virtual Reality Platform for Musical Creation: GENESIS-RT." In Proceedings of the 2013 International Symposium on Computer Music Multidisciplinary Research, pp. 346-371.
41. Levin, G. 2000. "Painterly Interfaces for Audiovisual Performance." PhD dissertation, Massachusetts Institute of Technology, Media Arts and Sciences, Cambridge, Massachusetts.
42. Levy, P. 1998. Becoming Virtual: Reality in the Digital Age. Cambridge, Massachusetts: Da Capo.
43. Lombard, M., and T. Ditton. 1997. "At the Heart of It All: The Concept of Presence." Journal of Computer-Mediated Communication 3(2). Available online at onlinelibrary.wiley.com/doi/10.1111/j.1083-6101.1997.tb00072.x/full. Accessed August 2016.
44. Maki-Patola, T., and P. Hämäl äinen. 2004. "Effect of Latency on Playing Accuracy of Two Gesture Controlled Continuous Sound Instruments without Tactile Feedback." In Proceedings of the International Conference on Digital Audio Effects, pp. 11-16.
45. Mäki-Patola, T., et al. 2005. "Experiments with Virtual Reality Instruments." In Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 11-16.
46. Marshall, M. T., and M. M. Wanderley. 2011. "Examining the Effects of Embedded Vibrotactile Feedback on the Feel of a Digital Musical Instrument." In Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 399-404.
47. Meehan,M. 2001. "Physiological Reaction as an Objective Measure of Presence in Virtual Environments." PhD dissertation, University of North Carolina at Chapel Hill, Department of Computer Science.
48. Moore, A. G., et al. 2015. "Wedge: A Musical Interface for Building and Playing Composition-Appropriate Immersive Environments." In Proceedings of the IEEE Symposium on 3D User Interfaces, pp. 205-206.
49. Morreale, F., A. De Angeli, and S. O'Modhrain. 2014. "Musical Interface Design: An Experience-Oriented Framework." In Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 467-472.
50. Mulder, A. G. 1998. "Design of Virtual Three-Dimensional Instruments for Sound Control." PhD dissertation, Simon Fraser University, School of Kinesiology, Vancouver, British Columbia.
51. Nowak, K. L., and F. Biocca. 2003. "The Effect of the Agency and Anthropomorphism on Users' Sense of Telepresence, Copresence, and Social Presence in Virtual Environments." Presence 12(5):481-494.
52. O'Modhrain, S. 2001. "Playing by Feel: Incorporating Haptic Feedback into Computer-Based Musical Instruments." PhD dissertation, Stanford University, Department of Music.
53. O'Modhrain, S. 2011. "A Framework for the Evaluation of DigitalMusical Instruments." Computer Music Journal 35(1):28-42.
54. O'Modhrain, S., et al. 2000a. "Incorporating Haptic Feedback into Interfaces for Music Applications." In Proceedings of the World Automation Congress, (pages unnumbered).
55. O'Modhrain, S., et al. 2000b. "Qualitative and Quantitive Assessment of a Virtual Bowed String Instrument." In Proceedings of the International Computer Music Conference, pp. 66-69.
56. Paine, G. 2009. "Towards Unified Design Guidelines for New Interfaces for Musical Expression." Organised Sound 14(02):142-155.
57. Poupyrev, I., et al. 1996. "The Go-Go Interaction Technique: Non-Linear Mapping for Direct Manipulation in VR." In Proceedings of the ACM Symposium on User Interface Software and Technology, pp. 79-80.
58. Preece, J., et al. 1994. "Interview with Ben Shneiderman." Available online at www.id-book.com /downloads/BenShneiderman.pdf. Accessed September 2016.
59. Riccio, G. E., and T. A. Stoffregen. 1991. "An Ecological Theory of Motion Sickness and Postural Instability." Ecological Psychology 3(3):195-240.
60. Rodet, X., et al. 2005. "Study of Haptic and Visual Interaction for Sound and Music Control in the Phase Project." In Proceedings of the InternationalConference on New Interfaces for Musical Expression, pp. 109-114.
61. Russolo, L. [1913] 1967. The Art of Noise: Futurist Manifesto, 1913. R. Filliou, trans.New York: Something Else.
62. Serafin, S., et al. 2006. "Synthesis and Control of Everyday Sounds Reconstructing Russolo's Intonarumori." In Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 240-245.
63. Serafin, S., et al. 2015. "The Digital Intonarumori." In IEEE Symposium on 3D User Interfaces, pp. 207-208.
64. Slater, M. 2009. "Place Illusion and Plausibility Can Lead to Realistic Behaviour in Immersive Virtual Environments." Philosophical Transactions of the Royal Society B: Biological Sciences 364(1535):3549-3557.
65. Slater, M., D.-P. Pertaub, and A. Steed. 1999. "Public Speaking in Virtual Reality: Facing an Audience of Avatars." IEEE Computer Graphics and Applications 19(2):6-9.
66. Smith, J. O. 2010. Physical Audio Signal Processing for Virtual Musical Instruments and Audio Effects. Palo Alto, California: W3K Publishing.
67. Sodhi, R., et al. 2013. "AIREAL: Interactive Tactile Experiences in Free Air." ACM Transactions on Graphics 32(4):134.
68. Sutherland, I. E. 1965. "The Ultimate Display." In Information Processing: Proceedings of the IFIP Congress, pp. 506-508.
69. Välimäki, V., and T. Takala. 1996. "Virtual Musical Instruments: Natural Sound Using Physical Models." Organised Sound 1(02):75-86.
70. Wanderley, M. M., and N. Orio. 2002. "Evaluation of Input Devices for Musical Expression: Borrowing Tools from HCI." Computer Music Journal 26(3):62-76.
71. Wang, G. 2014. "Principles of Visual Design for Computer Music." In Proceedings of the Joint International Computer Music Conference and the Sound and Music Computing Conference, pp. 391-396.
72. Wessel, D., and M.Wright. 2002. "Problems and Prospects for IntimateMusical Control of Computers." Computer Music Journal 26(3):11-22.
73. Xiao, X., B. Tome, and H. Ishii. 2014. "Andante: Walking Figures on the Piano Keyboard to VisualizeMusical Motion." In Proceedings of the International Conference on New Interfaces for Musical Expression, pp. 629-632.
74. Zhai, P. 1998. Get Real: A Philosophical Adventure in Virtual Reality. Lanham, Maryland: Rowman and Littlefield.
75. Zielasko, D., et al. 2015. "Cirque des Bouteilles: The Art of Blowing on Bottles." In IEEE Symposium on 3D User Interfaces, pp. 209-210.