Understanding gesture expressivity through muscle sensing

ACM Transactions on Computer-Human Interaction

Volumn 21, Issue 6, 2015, Pages

  Caramiaux, Baptiste ^a   Donnarumma, Marco ^a   Tanaka, Atau ^a  

^a UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Human factors

Indexed keywords

DESIGN; ELECTROMYOGRAPHY; HUMAN ENGINEERING;

EXPRESSIVE GESTURES; GESTURAL INTERACTION; HUMAN BODIES; MECHANOMYOGRAPHY; MUSCLE ACTIVITIES; PILOT STUDIES; SPACE AND TIME; SPATIAL AND TEMPORAL VARIATION;

MUSCLE;

EID: 84921472965     PISSN: 10730516     EISSN: 15577325     Source Type: Journal    
DOI: 10.1145/2687922     Document Type: Article

References (61)

1. Armando B. Barreto, Scott D. Scargle, and Malek Adjouadi. 2000. A practical EMG-based human-computer interface for users with motor disabilities. Journal of Rehabilitation Research and Development 37, 1, 53-64.
2. Daniel T. Barry, Timothy Hill, and Dukjin Im. 1992. Muscle fatiguemeasured with evokedmuscle vibrations. Muscle and Nerve 15, 3, 303-309.
3. Travis W. Beck, Terry J. Housh, Glen O. Johnson, Joseph P. Weir, Joel T. Cramer, Jared W. Coburn, and Moh H. Malek. 2004. Mechanomyographic and electromyographic time and frequency domain responses during submaximal to maximal isokinetic muscle actions of the biceps brachii. European Journal of Applied Physiology 92, 3, 352-359. DOI:http://dx.doi.org/10.1007/s00421-004-1110-9
4. Antonio Camurri, BarbaraMazzarino, and Matteo Ricchetti. 2004. Multimodal analysis of expressive gesture in music and dance performances. In Gesture-Based Communication in Human-Computer Interaction, Antonio Camurri and Gualtiero Volpe (Eds.). Lecture Notes in Computer Science, Vol. 2915. Springer, 20-39. http://link.springer.com/chapter/10.1007/978-3-540-24598-8-3.
5. Baptiste Caramiaux, Frederic Bevilacqua, and Atau Tanaka. 2013. Beyond recognition: Using gesture variation for continuous interaction. In Proceedings of CHI'13 Extended Abstracts on Human Factors in Computing Systems (CHI EA'13). ACM, New York, NY, 2109-2118. DOI:http://dx.doi.org/10.1145/2468356.2468730
6. Baptiste Caramiaux, Nicola Montecchio, Atau Tanaka, and Frédéric Bevilacqua. 2014. Adaptive gesture recognition with variation estimation for interactive systems. ACM Transactions on Interactive Intelligent Systems 4, 4.
7. Justine Cassell. 2000. Embodied Conversational Agents. MIT Press, Cambridge, MA.
8. Claudio Castellini and Patrick van der Smagt. 2009. Surface EMG in advanced hand prosthetics. Biological Cybernetics 100, 1, 35-47.
9. Diane Chi, Monica Costa, Liwei Zhao, and Norman Badler. 2000. The EMOTE model for effort and shape. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH' 00). ACM, New York, New York, 173-182. DOI:http://dx.doi.org/10.1145/344779.352172
10. Enrico Costanza, Samuel A. Inverso, and Rebecca Allen. 2005. Toward subtle intimate interfaces for mobile devices using an EMG controller. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 481-489.
11. Enrico Costanza, Samuel A. Inverso, Rebecca Allen, and Pattie Maes. 2007. Intimate interfaces in action: Assessing the usability and subtlety of EMG-based motionless gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 819-828.
12. Scott Day. 2002. Important Factors in Surface EMG Measurement. Technical Report. Retrieved December 1, 2014, from http://www.andrewsterian.com/214/EMG-measurement-and-recording.pdf.
13. Christopher Dobrian and Daniel Koppelman. 2006. The 'E' in NIME: Musical expression with new computer interfaces. In Proceedings of the International Conference on New Interfaces for Musical Expression (NIME'06). 277-282.
14. Marco Donnarumma. 2012. Incarnated sound in music for flesh II. Defining gesture in biologically informed musical performance. Leonardo Electronic Almanac (Touch and Go) 18, 3, 164-175. Available at http://www.leoalmanac.org/vol18-no3-incarnated-sound/.
15. Marco Donnarumma, Baptiste Caramiaux, and Atau Tanaka. 2013a. Body and space: Combining modalities for musical expression. Work-in-Progress Accepted at the Conference on Tangible and Embedded Interaction (TEI). 1-4.
16. Marco Donnarumma, Baptiste Caramiaux, and Atau Tanaka. 2013b.Muscular interactions combining EMG and MMG sensing for musical practice. In Proceedings of the International Conference on New Interfaces for Musical Expression (NIME'13). 128-131.
17. David Efron. 1941. Gesture, Race and Culture. Mouton.
18. Stephen H. Fairclough. 2009. Fundamentals of physiological computing. Interacting with Computers 21, 1, 133-145.
19. Dario Farina, Ning Jiang, Hubertus Rehbaum, Ales Holobar, Bernhard Graimann, Hans Dietl, and Oskar Aszmann. 2014. The extraction of neural information from the surface EMG for the control of upper-limb prostheses: Emerging avenues and challenges. IEEE Transactions on Neural Systems and Rehabilitation Engineering: A Publication of the IEEE Engineering in Medicine and Biology Society 22, 4, 797-809. DOI:http://dx.doi.org/10.1109/TNSRE.2014.2305111
20. Sarah Fdili Alaoui. 2012. Analyse du geste dansé et retours visuels par modèles physiques: apport des qualités de mouvement à l'interaction avec le corps entier. Ph.D. Dissertation. University of Paris XI, Orsay, France.
21. Sarah Fdili Alaoui, Baptiste Caramiaux,Marcos Serrano, and Frédéric Bevilacqua. 2012. Movement qualities as interactionmodality. In Proceedings of the Designing Interactive Systems Conference. ACM, New York, NY, 761-769.
22. Matthias Felleisen. 1991. On the expressive power of programming languages. Science of Computer Programming 17, 1-3, 35-75. DOI:http://dx.doi.org/10.1016/0167-6423(91)90036-W
23. David Hofmann. 2014. Myoelectric Signal Processing for Prosthesis Control. Ph.D. Dissertation. University of Göttingen, Göttingen, Germany.
24. Alexander Refsum Jensenius. 2014. To gesture or not? An analysis of terminology in NIME proceedings 2001-2013. In Proceedings of the International Conference on New Interfaces for Musical Expression (NIME'14). 217-220. http://www.nime.org/proceedings/2014/nime2014-351.pdf.
25. FrankW. Jobe, James E. Tibone, Jacquelin Perry, and Diane Moynes. 1983. An EMG analysis of the shoulder in throwing and pitching.Apreliminary report. American Journal of Sports Medicine 11, 1, 3-5. Available at http://www.ncbi.nlm.nih.gov/pubmed/6829838.
26. Sergi Jordà. 2005. Digital Lutherie: Crafting Musical Computers for New Musics' Performance and Improvisation. Ph.D. Dissertation. Unversitat Pompeu Fabra, Barcelona, Spain.
27. Eugenijus Kaniusas. 2012. Biomedical Signals and Sensors I. Linking Physiological Phenomena and Biosignals. Springer, Berlin, Heidelberg. DOI:http://dx.doi.org/10.1007/978-3-642-24843-6
28. Adam Kendon. 2004. Gesture: Visible Action as Utterance. Cambridge University Press.
29. Heloyse Uliam Kuriki and Fábio Mícolis De Azevedo. 2012. The relationship between electromyography and muscle force. In EMG Methods for Evaluating Muscle and Nerve Function, M. Schwartz (Ed.). InTech, 31-54.
30. Gordon Kurtenbach and Eric A. Hulteen. 1990. Gestures in human-computer communication. In The Art of Human-Computer Interface Design (1990), B. Laurel (Ed.). Addison-Wesley, 309-317.
31. Rudolf Laban. 1963. Modern Educational Dance. MacDonald and Evans, London, England.
32. Pascal Madeleine, Prem Bajaj, Gisela Sjøgaard, and Lars Arendt-Nielsen. 2001. Mechanomyography and electromyography force relationships during concentric, isometric and eccentric contractions. Journal of Electromyography and Kinesiology 11, 2, 113-21. Available at http://www.ncbi.nlm.nih.gov/pubmed/11228424.
33. Helena Mentis and Carolina Johansson. 2013. Seeing movement qualities. In Proceedings of the International Conference on Human Factors in Computing Systems (SIGCHI'13). 3375-3384. DOI:http://dx.doi.org/10.1145/2470654.2466462
34. Ilona Miko. 2008. Phenotype variability: Penetrance and expressivity. Nature Education 1, 1, 137.
35. Axel Mulder. 1996. Hand Gestures for HCI. Hand Centered Studies of Human Movement Project Technical Report 96-1. School of Kinesiology, Simon Fraser University, Burnaby, British Columbia.
36. Claudio Orizio, Fabio Esposito, Valeria Sansone, Giovanni Parrinello, Giovanni Meola, and Arsenio Veicsteinas. 1997. Muscle surfacemechanical and electrical activities in myotonic dystrophy. Electromyography and Clinical Neurophysiology 37, 4, 231-239.
37. Claudio Orizio, Renza Perini, Bertrand Diemont, and Arsenio Veicsteinas. 1992. Muscle sound and electromyogram spectrum analysis during exhausting contractions in man. European Journal of Applied Physiology and Occupational Physiology 65, 1, 1-7. Available at http://www.ncbi.nlm.nih.gov/pubmed/1505534.
38. Gascia Ouzounian, R. Benjamin Knapp, Eric Lyon, and R. Luke DuBois. 2012. To be inside someone else's dream: On Music for Sleeping & Waking Minds. In Proceedings of the International Conference on New Interfaces for Musical Expression (NIME'12). 1-6.
39. Caroline Palmer. 1997. Music performance. Annual Review of Psychology 48, 1, 115-138.
40. Geoffroy Peeters. 2004. A Large Set of Audio Features for Sound Description. CUIDADO Project. 1-25.
41. Catherine Pelachaud. 2009. Studies on gesture expressivity for a virtual agent. Speech Communication 51, 7, 630-639. DOI:http://dx.doi.org/10.1016/j.specom.2008.04.009
42. Sharon R. Perry-Rana, Terry J. Housh, Glen O. Johnson, Anthony J. Bull, Joseph M. Berning, and Joel T. Cramer. 2002. MMG and EMG responses during fatiguing isokinetic muscle contractions at different velocities. Muscle and Nerve 26, 3, 367-373.
43. William Putnam and R. Benjamin Knapp. 1993. The use of the electromyogram in a man-machine interface. In Proceedings of the Virtual Reality and Persons with Disabilities Conference. 1-6.
44. Terence D. Sanger. 2007. Bayesian filtering of myoelectric signals. Journal of Neurophysiology 97, 1839-1845. DOI:http://dx.doi.org/10.1152/jn.00936.2006
45. T. Scott Saponas, Desney S. Tan, Dan Morris, Ravin Balakrishnan, Jim Turner, and James A. Landay. 2009. Enabling always-available input with muscle-computer interfaces. In Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology. ACM, New York, NY, 167-176.
46. T. Scott Saponas, Desney S. Tan, Dan Morris, Jim Turner, and James A. Landay. 2010. Making musclecomputer interfaces more practical. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, NY, 851-854.
47. George N. Saridis and Thomas P. Gootee. 1982. EMG pattern analysis and classification for a prosthetic arm. IEEE Transactions on Biomedical Engineering 6, 403-412.
48. Tanja Schultz and Michael Wand. 2010. Modeling coarticulation in EMG-based continuous speech recognition. Speech Communication 52, 4, 341-353. DOI:http://dx.doi.org/10.1016/j.specom.2009.12.002
49. Hugo Silva, Carlos Carreiras, Andre Lourenco, and Ana Fred. 2013. Off-the-person electrocardiography. In Proceedings of the International Congress on Cardiovascular Technologies (CARDIOTECHNIX). 99-106.
50. Hugo Silva, Andre Lourenco, Ana Fred, and Raul Martins. 2014. BIT: Biosignal Igniter Toolkit. Computer Methods and Programs in Biomedicine 115, 1, 20-32.
51. Hugo Silva, Reinhold Scherer, Joana Sousa, and Ana Londral. 2013. Towards improving the usability of electromyographic interfaces. In Converging Clinical and Engineering Research on Neurorehabilitation. Biosystems and Biorobotics, Vol. 1. Springer, 437-441.
52. Jorge Silva and TomChau. 2003. Coupledmicrophone-accelerometer sensor pair for dynamic noise reduction in MMG signal recording. Electronics Letters 39, 21, 1496-1498.
53. Jorge Silva,Winfried Heim, and Tom Chau. 2004. MMG-based classification of muscle activity for prosthesis control. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society. 968-971. DOI:http://dx.doi.org/10.1109/IEMBS.2004.1403322
54. Jorge Silva, Winfried Heim, and Tom Chau. 2005. A self-contained, mechanomyography-driven externally powered prosthesis. Archives of Physical Medicine and Rehabilitation 86, 10, 2066-2070.
55. Atau Tanaka and R. Benjamin Knapp. 2002. Multimodal interaction in music using the electromyogram and relative position sensing. In Proceedings of the International Conference on New Interfaces for Musical Expression (NIME'02). 1-6.
56. Mihai T. Tarata. 2003. Mechanomyography versus electromyography, in monitoring the muscular fatigue. BioMedical Engineering OnLine, 1-10. Available at http://www.biomedcentral.com/content/pdf/1475-925X-2-3.pdf.
57. Mihai T. Tarata. 2009. The electromyogram and mechanomyogram in monitoring neuromuscular fatigue: Techniques, results, potential use within the dynamic effort. In Proceedings of the 7th International Conference on Measurement. 67-77.
58. Paolo Viviani and Tamar Flash. 1995. Minimum-jerk, two-thirds power law, and isochrony: Converging approaches to movement planning. Journal of Experimental Psychology. Human Perception and Performance 21, 1, 32-53. Available at http://www.ncbi.nlm.nih.gov/pubmed/7707032.
59. Harald G. Wallbott. 1998. Bodily expression of emotion. European Journal of Social Psychology 28, 6, 879-896.
60. Kevin R. Wheeler and Charles C. Jorgensen. 2003. Gestures as input: Neuroelectric joysticks and keyboards. IEEE Pervasive Computing 2, 2, 56-61.
61. Jacob O. Wobbrock, Meredith Ringel Morris, and Andrew D. Wilson. 2009. User-defined gestures for surface computing. In Proceedings of the 27th International Conference on Human Factors in Computing Systems (CHI'09). ACM, New York, NY, 1083-1092. DOI:http://dx.doi.org/10.1145/1518701.1518866