A sign-component-based framework for Chinese sign language recognition using accelerometer and sEMG data

IEEE Transactions on Biomedical Engineering

Volumn 59, Issue 10, 2012, Pages 2695-2704

  Li, Yun ^a   Chen, Xiang ^a   Zhang, Xu ^b   Wang, Kongqiao ^c   Wang, Z Jane ^d  

^a UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA   (China)

^b REHABILITATION INSTITUTE OF CHICAGO   (United States)

^c NOKIA RESEARCH CENTER   (United States)

^d UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Accelerometer (ACC); hidden Markov model (HMM); sign component; sign language recognition (SLR); surface electromyography

Indexed keywords

CHINESE SIGN LANGUAGE RECOGNITION; CLASSIFICATION ACCURACY; COMPONENT CLASSIFIERS; COMPONENT LEVELS; DATA SEGMENTATION; DECISION LEVELS; ELECTROMYOGRAPHIC; HAND SHAPE; PRE-PROCESSING OPERATIONS; RECOGNITION PERFORMANCE; SIGN COMPONENT; SIGN LANGUAGE; SIGN LANGUAGE RECOGNITION; SUBWORDS; SURFACE ELECTROMYOGRAPHY;

ACCELEROMETERS; BUILDING COMPONENTS; HIDDEN MARKOV MODELS;

GESTURE RECOGNITION;

ACCELEROMETER; ACCURACY; ARTICLE; CHINESE; CLASSIFICATION; CONTROLLED STUDY; ELECTROMYOGRAM; HAND; HUMAN; LINGUISTICS; MOVEMENT (PHYSIOLOGY); ORIENTATION; SIGN LANGUAGE; SIGN LANGUAGE RECOGNITION;

ACCELEROMETRY; ADULT; ARM; ELECTROMYOGRAPHY; FEMALE; HAND; HUMANS; MALE; MARKOV CHAINS; PATTERN RECOGNITION, AUTOMATED; SIGN LANGUAGE; SIGNAL PROCESSING, COMPUTER-ASSISTED;

EID: 84866541121     PISSN: 00189294     EISSN: 15582531     Source Type: Journal    
DOI: 10.1109/TBME.2012.2190734     Document Type: Article

References (32)

1. S. C. W. Ong and S. Ranganath, "Automatic sign language analysis: A survey and the future beyond lexical meaning," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 6, pp. 873-891, Jun. 2005. (Pubitemid 40889839)
2. L. Ding and A. M. Martinez, "Modelling and recognition of the linguistic components in American sign language," Image Vis. Comput., vol. 27, no. 12, pp. 1826-1844, Nov. 2009.
3. G. Fang,W. Gao, and D. Zhao, "Large vocabulary sign language recognition based on fuzzy decision trees," IEEE Trans. Syst., Man, Cybern. A, Syst. Humans, vol. 34, no. 3, pp. 305-314, May 2004.
4. D. Kelly, R. Delannoy, J. Mc Donald, and C. Markham, "A framework for continuous multimodal sign language recognition," in Proc. Int. Conf. Multimodal Interfaces, Cambridge, MA, 2009, pp. 351-358.
5. Y. Li, X. Chen, J. Tian, X. Zhang, K. Wang, and J. Yang, "Automatic recognition of sign language subwords based on portable accelerometer and EMG sensors," in Proc. Int. Conf. Multimodal Interfaces - Workshop Mach. Learn. Multimodal Interaction, Beijing, China, 2010, pp. 1-7.
6. J. Kela, P. Korpipää, J. Mäntyjärvi, S. Kallio, G. Savino, L. Jozzo, and S. D. Marca, "Accelerometer-based gesture control for a design environment," Pers. Ubiquitous Comput., vol. 10, no. 5, pp. 285-299, Jul. 2006. (Pubitemid 44151449)
7. M. Asghari Oskoei and H. Hu, "Myoelectric control systems - A survey," Biomed. Signal Process. Control, vol. 2, no. 4, pp. 275-294, Oct. 2007.
8. R. H. Liang and M. Ouhyoung, "A real-time continuous gesture recognition system for sign language," in Proc. 3rd IEEE Int. Conf. Automat. Face Gesture Recognit., Apr. 1998, pp. 558-567.
9. T. Starner, J.Weaver, and A. Pentland, "Real-timeAmerican sign language recognition using desk and wearable computer based video," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 12, pp. 1371-1375, Dec. 1998. (Pubitemid 128741382)
10. X. Chen, X. Zhang, Z. Y. Zhao, J. H. Yang, V. Lantz, and K. Q. Wang, "Hand gesture recognition research based on surface EMG sensors and 2D-accelerometers," in Proc. 11th IEEE Int. Symp. Wearable Comput., 2007, pp. 11-14. (Pubitemid 351424259)
11. J. Kim, J.Wagner, M. Rehm, and E. André, "Bi-channel sensor fusion for automatic sign language recognition," in Proc. IEEE Int. Conf. Automat. Face Gesture Recognit., Amsterdam, The Netherlands, 2008, pp. 1-6.
12. V. E. Kosmidou and L. J. Hadjileontiadis, "Sign language recognition using intrinsic mode sample entropy on sEMG and accelerometer data," IEEE Trans. Biomed. Eng., vol. 56, no. 12, pp. 2879-2890, Dec. 2009.
13. S. H. Roy, M. S. Cheng, S. S. Chang, J. Moore, G. De Luca, S. H. Nawab, andC. J.De Luca, "A combined sEMG and accelerometer system formonitoring functional activity in stroke," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 17, no. 6, pp. 585-594, 2009.
14. Department for Education and Employment in China Disabled Persons' Federation, China Association of the Deaf, Chinese Sign Language (Revised edition). Beijing, China: HuaXia Press, 2003.
15. C.Wang,W. Gao, and S. Shan, "An approach based on phonemes to large vocabulary Chinese sign language recognition," in Proc. IEEE Int. Conf. Automat. Face Gesture Recognit., Washington, DC, 2002, pp. 411-416.
16. X. Zhang, X. Chen, Y. Li, V. Lantz, K.Wang, and J. Yang, "A framework for hand gesture recognition based on accelerometer and EMG sensors," IEEE Trans. Syst.,Man, Cybern. A, Syst. Humans, vol. 41, no. 6, pp. 1064-1076, Nov. 2011.
17. C.Vogler andD.Metaxas, "Aframework for recognizing the simultaneous aspects of American sign language," Comput. Vis. Image Understand., vol. 81, no. 3, pp. 358-384, 2001. (Pubitemid 32415981)
18. X. Zhang, X. Chen, Z. Zhao, Y. Tu, J. Yang, V. Lantz, and K. Wang, "Research on gesture definition and electrode placement in pattern recognition of hand gesture action SEMG," in Proc. Int. Conf.Med. Biometrics, 2008, pp. 33-40. (Pubitemid 351153200)
19. X. Li, P. Zhou, and A. S. Aruin, "Teager-Kaiser energy operation of surface EMG improves muscle activity onset detection," Ann. Biomed. Eng., vol. 35, no. 9, pp. 1532-1538, Sep. 2007. (Pubitemid 350092430)
20. X. Chen,V.Lantz, K.Wang, Z. Zhao,X.Zhang, and J.Yang, "Feasibility of building robust surface electromyography-based hand gesture interfaces," in Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., Minneapolis, MN, 2009, pp. 2983-2986.
21. B. Hudgins, P. A. Parker, and R. N. Scott, "A new strategy for multifunction myoelectric control," IEEE Trans. Biomed. Eng., vol. 40, no. 1, pp. 82-94, Jan. 1993. (Pubitemid 24060692)
22. Y. Huang, K. B. Englehart, B. Hudgins, and A. D. C. Chan, "A Gaussian mixture model based classification scheme for myoelectric control of powered upper limb prostheses," IEEE Trans. Biomed. Eng., vol. 52, no. 11, pp. 1801-1811, Nov. 2005. (Pubitemid 41578800)
23. P. Zhou, M. M. Lowery, K. B. Englehart, H. Huang, G. Li, L. Hargrove, J. P. A. Dewald, and T. A. Kuiken, "Decoding a new neural-machine interface for control of artificial limbs," J. Neurophysiol., vol. 98, no. 5, pp. 2974-2982, 2007. (Pubitemid 350127955)
24. C. Liu and H.Wechsler, "Robust coding schemes for indexing and retrieval from large face databases," IEEE Trans. Image Process., vol. 9, no. 1, pp. 132-137, Jan. 2000. (Pubitemid 30562090)
25. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd ed. New York: Wiley, 2001.
26. D. A. Reynolds, "An overview of automatic speaker recognition technology," in Proc. Int. Conf. Acoust., Speech, Signal Process., Orlando, FL, 2002, vol. 4, pp. 4072-4075.
27. A. S. Malegaonkar, A. M. Ariyaeeinia, and P. Sivakumaran, "Efficient speaker change detection using adapted Gaussian mixture models," IEEE Trans. Speech Audio Process., vol. 15, no. 6, pp. 1859-1869, Aug. 2007.
28. V. E. Kosmidou, P. C. Petrantonakis, and L. J. Hadjileontiadis, "Enhanced sign language recognition using weighted intrinsic-mode entropy and signer's level of deafness," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 41, no. 6, pp. 1531-1543, Dec. 2011.
29. K. Assaleh, T. Shanableh, M. Fanaswala, H. Bajaj, and F. Amin, "Vision based system for continuous Arabic sign language recognition in user dependent mode," in Proc. 5th Int. Symp. Mechatron. Appl., Amman, Jordan, 2008, pp. 1-5.
30. T. S. Saponas, D. S. Tan, D. Morris, and R. Balakrishnan, "Demonstrating the feasibility of using forearm electromyography for muscle-computer interfaces," in Proc. 26th Annu. SIGCHI Conf. Human Factors Comput. Syst., Florence, Italy, 2008, pp. 515-524.
31. R. L. Mandryk, M. S. Atkins, and K. M. Inkpen, "A continuous and objective evaluation of emotional experience with interactive play environments," in Proc. SIGCHI Conf. Human Factors Comput. Syst., New York, NY, 2006, pp. 1027-1036. (Pubitemid 44032197)
32. X. Zhang, X. Chen, V. Lantz, J. Yang, and K. Wang, "Exploration on the feasibility of building muscle-computer interfaces using neck and shoulder motions," in Proc. Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., Minneapolis, MN, 2009, pp. 7018-7021.