A framework for hand gesture recognition based on accelerometer and EMG sensors

IEEE Transactions on Systems, Man, and Cybernetics Part A:Systems and Humans

Volumn 41, Issue 6, 2011, Pages 1064-1076

  Zhang, Xu ^a,b   Chen, Xiang ^a   Li, Yun ^a   Lantz, Vuokko ^c   Wang, Kongqiao ^c   Yang, Jihai ^a  

^a UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA   (China)

^b REHABILITATION INSTITUTE OF CHICAGO   (United States)

^c NOKIA RESEARCH CENTER   (United States)

Author keywords

Acceleration; electromyography; hand gesture recognition; hidden Markov models (HMMs)

Indexed keywords

CHINESE SIGN LANGUAGE; CONTROL COMMAND; DECISION LEVEL FUSION; EMG SIGNAL; END POINTS; GESTURE-BASED INTERACTION; HAND GESTURE; HAND-GESTURE RECOGNITION; INTERACTIVE SYSTEM; MULTI-CHANNEL; MULTI-STREAM HIDDEN MARKOV MODEL; SIGN LANGUAGE RECOGNITION; THREE AXIS ACCELEROMETERS;

ACCELEROMETERS; DECISION TREES; HIDDEN MARKOV MODELS; SENSORS; SPEECH RECOGNITION; USER INTERFACES;

GESTURE RECOGNITION;

EID: 80054822470     PISSN: 10834427     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCA.2011.2116004     Document Type: Article

References (38)

1. S. Mitra and T. Acharya, "Gesture recognition: A survey," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 37, no. 3, pp. 311-324, May 2007.
2. J. Mäntyjärvi, J. Kela, P. Korpipää, and S. Kallio, "Enabling fast and effortless customisation in accelerometer based gesture interaction," in Proc. 3rd Int. Conf. Mobile Ubiquitous Multimedia, New York, 2004, pp. 25-31.
3. T. Pylvänäinen, "Accelerometer based gesture recognition using continuous HMMs," in Proc. Pattern Recog. Image Anal., LNCS 3522, 2005, pp. 639-646.
4. E. Costanza, S. A. Inverso, and R. Allen, "Toward subtle intimate interfaces for mobile devices using an EMG controller," in Proc. SIGCHI Conf. Human Factors Comput. Syst. , Portland, OR, Apr. 2-7, 2005, pp. 481-489.
5. M. Asghari Oskoei and H. Hu, "Myoelectric control systems-A survey," Biomed. Signal Process. Control, vol. 2, no. 4, pp. 275-294, Oct. 2007.
6. D. M. Sherrill, P. Bonato, and C. J. De Luca, "A neural network approach to monitor motor activities," in Proc. 2nd Joint EMBS/BMES Conf., Houston, TX, 2002, vol. 1, pp. 52-53.
7. T. Starner, J. Weaver, and A. Pentland, "Real-time American Sign Language recognition using desk and wearable computer based video," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 12, pp. 1371-1375, Dec. 1998.
8. T. Shanableh, K. Assaleh, and M. Al-Rousan, "Spatio-temporal featureextraction techniques for isolated gesture recognition in Arabic Sign Language," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 37, no. 3, pp. 641-650, Jun. 2007.
9. G. Fang, W. Gao, and D. Zhao, "Large vocabulary sign language recognition based on fuzzy decision trees," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 34, no. 3, pp. 305-314, May 2004.
10. K. Assaleh, T. Shanableh, M. Fanaswala, H. Bajaj, and F. Amin, "Visionbased system for continuous Arabic Sign Language recognition in user dependent mode," in Proc. 5th Int. Symp. Mechatron. Appl., Amman, Jordan, 2008, pp. 1-5.
11. K. R. Wheeler, M. H. Chang, and K. H. Knuth, "Gesture-based control and EMG decomposition," IEEE Trans. Syst.,Man, Cybern. C, Appl. Rev., vol. 36, no. 4, pp. 503-514, Jul. 2006.
12. T. S. Saponas, D. S. Tan, D. Morris, and R. Balakrishnan, "Demonstrating the feasibility of using forearm electromyography for muscle-computer interfaces," in Proc. 26th SIGCHI Conf. Human Factors Comput. Syst., Florence, Italy, 2008, pp. 515-524.
13. A. Wilson and S. Shafer, "Between u and i: XWand: UI for intelligent spaces," in Proc. SIGCHI Conf. Human Factors Comput. Syst., Ft. Lauderdale, FL, Apr. 2003, pp. 545-552.
14. H. Brashear, T. Starner, P. Lukowicz, and H. Junker, "Using multiple sensors for mobile sign language recognition," in Proc. 7th IEEE ISWC, 2003, pp. 45-52.
15. X. Chen, X. Zhang, Z. Y. Zhao, J. H. Yang, V. Lantz, and K. Q. Wang, "Hand gesture recognition research based on surface EMG sensors and 2D-accelerometers," in Proc. 11th IEEE ISWC, 2007, pp. 11-14.
16. J. Kim, S. Mastnik, and E. André, "EMG-based hand gesture recognition for realtime biosignal interfacing," in Proc. 13th Int. Conf. Intell. User Interfaces, Gran Canaria, Spain, 2008, pp. 30-39.
17. J. Kela, P. Korpipää, J. Mäntyjärvi, S. Kallio, G. Savino, L. Jozzo, and S. D. Marca, "Accelerometer-based gesture control for a design environment," Pers. Ubiquitous Comput., vol. 10, no. 5, pp. 285-299, Jul. 2006.
18. K. Englehart, B. Hudgins, and P. A. Parker, "A wavelet-based continuous classification scheme for multifunction myoelectric control," IEEE Trans. Biomed. Eng., vol. 48, no. 3, pp. 302-310, Mar. 2001.
19. Y. Huang, K. Englehart, B. Hudgins, and A. D. C. Chan, "A Gaussian mixture model based classification scheme for myoelectric control of powered upper limb prostheses," IEEE Trans. Biomed. Eng., vol. 52, no. 11, pp. 1801-1811, Nov. 2005.
20. A. V. Nefian, L. Liang, X. Pi, X. Liu, C. Mao, and K. Murphy, "A coupled HMM for audio-visual speech recognition," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., Orlando, FL, 2002, pp. 2013-2016.
21. H. Manabe and Z. Zhang, "Multi-stream HMM for EMG-based speech recognition," in Proc. 26th Annu. Int. Conf. IEEE EMBS, San Francisco, CA, 2004, pp. 4389-4392.
22. M. Gurban, J. P. Thiran, T. Drugman, and T. Dutoit, "Dynamic modality weighting for multi-stream HMMs in audio-visual speech recognition," in Proc. 10th Int. Conf. Multimodal Interfaces, Chania, Greece, 2008, pp. 237-240.
23. V. E. Kosmidou, L. J. Hadjileontiadis, and S. M. Panas, "Evaluation of surface EMG features for the recognition of American Sign Language gestures," in Proc. IEEE 28th Annu. Int. Conf. EMBS, New York, Aug. 2006, pp. 6197-6200.
24. R. N. Khushaba and A. Al-Jumaily, "Channel and feature selection in multifunction myoelectric control," in Proc. IEEE 29th Annu. Int. Conf. EMBS, Lyon, France, Aug. 2007, pp. 5182-5185.
25. X. Hu and V. Nenov, "Multivariate AR modeling of electromyography for the classification of upper arm movements," Clinical Neurophysiol., vol. 115, no. 6, pp. 1276-1287, Jun. 2004.
26. X. Chen, Q. Li, J. Yang, V. Lantz, and K. Wang, "Test-retest repeatability of surface electromyography measurement for hand gesture," in Proc. 2nd ICBBE, Shanghai, China, 2008, pp. 1923-1926.
27. R.O.Duda, P. E. Hart, and D. G. Stork, Pattern Classification, Second ed. New York: Wiley, 2001, Section 10.4.4.
28. C. Liu and H. Wechsler, "Robust coding schemes for indexing and retrieval from large face databases," IEEE Trans. Image Process., vol. 9, no. 1, pp. 132-137, Jan. 2000.
29. L. Rabiner and B. Jaung, Fundamentals of Speech Recognition. Englewood Cliffs, NJ: Prentice-Hall, 1993.
30. G. Gravier, S. Axelrod, G. Potamianos, and C. Neti, "Maximum entropy and MCE based HMM stream weight estimation for audio-visual ASR," in Proc. ICASSP, 2002, p. 853.
31. S. Tamura, K. Iwano, and S. Furui, "A stream-weight optimization method for multi-stream HMMs based on likelihood value normalization," in Proc. ICASSP, Philadelphia, PA, Mar. 2005, pp. 468-472.
32. J. Kim, J. Wagner, M. Rehm, and E. André, "Bi-channel sensor fusion for automatic sign language recognition," in Proc. IEEE Int. Conf. Automat. Face Gesture Recog., Amsterdam, The Netherlands, 2008, pp. 1-6.
33. C. Vogler and D.Metaxas, "ASL recognition based on a coupling between HMMs and 3D motion analysis," in Proc. 6th Int. Conf. Comput. Vis., Bombay, India, 1999, pp. 363-369.
34. V. E. Kosmidou and L. J. Hadjileontiadis, "Sign language recognition using intrinsic mode sample entropy on sEMG and accelerometer data," IEEE Trans. Biomed. Eng., vol. 56, no. 12, pp. 2879-2890, Dec. 2009.
35. D. Kelly, R. Delannoy, and J. Mc Donald, "A framework for continuous multimodal sign language recognition," in Proc. ICMI-MLMI, Cambridge, MA, 2009, pp. 351-358.
36. Y. Kessentini, T. Paquet, and A. M. Ben Hamadou, "Off-line handwritten word recognition using multi-stream hidden Markov models," Pattern Recognit. Lett., vol. 31, no. 1, pp. 60-70, Jan. 2010.
37. C. Zhu and W. Sheng, "Wearable sensor-based hand gesture and daily activity recognition for robot-assisted living," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, pp. 1-5, Jan. 2011. DOI: 10.1109/TSMCA.2010.2093883.
38. S. Gao and P. Yu, Atlas of Human Anatomy (Revision). Shanghai, China: Shanghai Sci. & Tech. Publ., 1998, ch. 2, (in Chinese).