Dynamic affine-invariant shape-appearance handshape features and classification in sign language videos

Journal of Machine Learning Research

Volumn 14, Issue , 2013, Pages 1627-1663

  Roussos, Anastasios ^a   Theodorakis, Stavros ^b   Pitsikalis, Vassilis ^b   Maragos, Petros ^b  

^a QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

^b NATIONAL TECHNICAL UNIVERSITY OF ATHENS   (Greece)

Author keywords

Affine invariant shape appearance model; Feature extraction; Handshape classification; Landmarks free shape representation; Static and dynamic priors

Indexed keywords

AFFINE TRANSFORMATIONS; DYNAMIC PRIORS; HAND CONFIGURATION; HAND-SHAPE FEATURES; LINEAR COMBINATIONS; SHAPE REPRESENTATION; SIGN LANGUAGE; SIGN RECOGNITION;

EXPERIMENTS;

FEATURE EXTRACTION;

EID: 84884247128     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (58)

1. U. Agris, J. Zieren, U. Canzler, B. Bauer, and K. F. Kraiss. Recent developments in visual sign language recognition. Universal Access in the Information Society, 6:323-362, 2008.
2. T. Ahmad, C. J. Taylor, and T. F. Lanitis, A. Cootes. Tracking and recognising hand gestures, using statistical shape models. Image and Visual Computing, 15(5):345-352, 1997.
3. A. Argyros and M. Lourakis. Real time tracking of multiple skin-colored objects with a possibly moving camera. In Proceedings of the European Conference on Computer Vision, 2004.
4. V. Athitsos and S. Sclaroff. An appearance-based framework for 3d hand shape classification and camera viewpoint estimation. In Proceedings of the International Conference on Automatic Face and Gesture Recognition, pages 45-52, 2002.
5. S. Baker and I. Matthews. Lucas-kanade 20 years on: A unifying framework: Part 1. Technical report, Carnegie Mellon University, 2002.
6. S. Baker, R. Gross, and I. Matthews. Lucas-kanade 20 years on: A unifying framework: Part 4. Technical report, Carnegie Mellon University, 2004.
7. B. Bauer and K. F. Kraiss. Towards an automatic sign language recognition system using subunits. In Proceedings of the International Gesture Workshop, volume 2298, pages 64-75, 2001.
8. H. Birk, T. B. Moeslund, and C. B. Madsen. Real-time recognition of hand alphabet gestures using principal component analysis. In Proceedings of the Scandinavian Conference Image Analysis, 1997.
9. A. Blake and M. Isard. Active Contours. Springer, 1998.
10. R. Bowden and M. Sarhadi. A nonlinear model of shape and motion for tracking fingerspelt american sign language. Image and Visual Computing, 20:597-607, 2002.
11. P. Buehler, M. Everingham, and A. Zisserman. Learning sign language by watching TV (using weakly aligned subtitles). In Proceedings of the Conference on Computer Vision and Pattern Recognition, 2009.
12. J. Cai and A. Goshtasby. Detecting human faces in color images. Image and Visual Computing, 18:63-75, 1999.
13. F.-S. Chen, C.-M. Fu, and C.-L. Huang. Hand gesture recognition using a real-time tracking method and hidden markov models. Image and Visual Computing, 21(8):745-758, 2003.
14. S. Conseil, S. Bourennane, and L. Martin. Comparison of Fourier descriptors and Hu moments for hand posture recognition. In Proceedings of the European Conference on Signal Processing, 2007.
15. T. F. Cootes and C. J. Taylor. Statistical models of appearance for computer vision. Technical report, University of Manchester, 2004.
16. Y. Cui and J. Weng. Appearance-based hand sign recognition from intensity image sequences. Computer Vision and Image Understanding, 78(2):157-176, 2000.
17. L. Davies, David and W. Bouldin, Donald. A cluster separation measure. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1:224-227, April 1979.
18. J.-W. Deng and H. T. Tsui. A novel two-layer PCA/MDA scheme for hand posture recognition. In Proceedings of the International Conference on Pattern Recognition, volume 1, pages 283-286, 2002.
19. DictaSign. Greek sign language corpus. http://www.sign-lang.uni-hamburg. de/dicta-sign/portal, 2012.
20. L. Ding and A. M. Martinez. Modelling and recognition of the linguistic components in american sign language. Image and Visual Computing, 27(12):1826-1844, 2009.
21. P. Dreuw, J. Forster, T. Deselaers, and H. Ney. Efficient approximations to model-based joint tracking and recognition of continuous sign language. In Proceedings of the International Conference on Automatic Face and Gesture Recognition, Sep. 2008.
22. I. L. Dryden and K. V. Mardia. Statistical Shape Analysis. John Wiley and Sons, 1998.
23. W. Du and J. Piater. Hand modeling and tracking for video-based sign language recognition by robust principal component analysis. In Proceedings of the ECCV Workshop on Sign, Gesture and Activity, September 2010.
24. A. Farhadi, D. Forsyth, and R. White. Transfer learning in sign language. In Proceedings of the Conference on Computer Vision and Pattern Recognition, pages 1-8. IEEE, 2007.
25. H. Fillbrandt, S. Akyol, and K.-F. Kraiss. Extraction of 3D hand shape and posture from images sequences from sign language recognition. In Proceedings of the International Workshop on Analysis and Modeling of Faces and Gestures, pages 181-186, 2003.
26. R. Gross, I. Matthews, and S. Baker. Generic vs. person specific active appearance models. Image and Visual Computing, 23(12):1080-1093, 2005.
27. T. Hanke. HamNoSys Representing sign language data in language resources and language processing contexts. In Proceedings of the International Conference on Language Resources and Evaluation, 2004.
28. M.-K. Hu. Visual pattern recognition by moment invariants. IEEE Transactions on Information Theory, 8(2):179-187, February 1962.
29. C.-L. Huang and S.-H. Jeng. A model-based hand gesture recognition system. Machine Vision and Application, 12(5):243-258, 2001.
30. P. Kakumanu, S. Makrogiannis, and N. Bourbakis. A survey of skin-color modeling and detection methods. Pattern Recognition, 40(3):1106-1122, Mar. 2007.
31. E. Learned-Miller. Data driven image models through continuous joint alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(2):236-250, 2005.
32. S. Liwicki and M. Everingham. Automatic recognition of fingerspelled words in British sign language. In Proceedings of the CVPR Workshop on Human Communicative Behavior Analysis, 2009.
33. P. Maragos. Morphological Filtering for Image Enhancement and Feature Detection, chapter The Image and Video Processing Handbook. Elsevier, 2005.
34. I. Matthews and S. Baker. Active appearance models revisited. International Journal of Computer Vision, 60(2):135-164, 2004.
35. C. Neidle. Signstream annotation: Addendum to conventions used for the american sign language linguistic research project. Technical report, 2007.
36. C. Neidle and C. Vogler. A new web interface to facilitate access to corpora: development of the ASLLRP data access interface. In Proceedings of the International Conference on Language Resources and Evaluation, 2012.
37. E. J. Ong, H. Cooper, N. Pugeault, and R. Bowden. Sign language recognition using sequential pattern trees. In Proceedings of the Conference on Computer Vision and Pattern Recognition, pages 2200-2207. IEEE, 2012.
38. Y. Peng, A. Ganesh, J. Wright, W. Xu, and Y. Ma. RASL: Robust alignment by sparse and low-rank decomposition for linearly correlated images. In Proceedings of the Conference on Computer Vision and Pattern Recognition, 2010.
39. V. Pitsikalis, S. Theodorakis, and P. Maragos. Data-driven sub-units and modeling structure for continuous sign language recognition with multiple cues. In LREC Workshop Repr. & Proc. SL: Corpora and SL Technologies, 2010.
40. L. R. Rabiner and R. W. Schafer. Introduction to digital speech processing. Foundations and Trends in Signal Processing, 1 (1-2):1-194, 2007.
41. A. Roussos, S. Theodorakis, V. Pitsikalis, and P. Maragos. Affine-invariant modeling of shapeappearance images applied on sign language handshape classification. In Proceedings of the International Conference on Image Processing, Sep. 2010a.
42. A. Roussos, S. Theodorakis, V. Pitsikalis, and P. Maragos. Hand tracking and affine shapeappearance handshape sub-units in continuous sign language recognition. In Proceedings of the ECCV Workshop on Sign, Gesture and Activity, September 2010b.
43. J. Sherrah and S. Gong. Resolving visual uncertainty and occlusion through probabilistic reasoning. In Proceedings of the British Machine Vision Conference, pages 252-261, 2000.
44. P. Soille. Morphological Image Analysis: Principles and Applications. Springer, 2004.
45. T. Starner, J. Weaver, and A. Pentland. Real-time american sign language recognition using desk and wearable computer based video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(12):1371-1375, Dec. 1998.
46. B. Stenger, A. Thayananthan, P. H. S Torr, and R. Cipolla. Model-based hand tracking using a hierarchical bayesian filter. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(9):1372-1384, Sep. 2006.
47. G. J. Sweeney and A. C. Downton. Towards appearance-based multi-channel gesture recognition. In Proceedings of the International Gesture Workshop, pages 7-16, 1996.
48. N. Tanibata, N. Shimada, and Y. Shirai. Extraction of hand features for recognition of sign language words. In Proceedings of the International Conference on Vision Interface, pages 391-398, 2002.
49. J. Terrillon, M. Shirazi, H. Fukamachi, and S. Akamatsu. Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images. In Proceedings of the International Conference on Automatic Face and Gesture Recognition, pages 54-61, 2000.
50. A. Thangali, J. P. Nash, S. Sclaroff, and C. Neidle. Exploiting phonological constraints for handshape inference in asl video. In Proceedings of the Conference on Computer Vision and Pattern Recognition, pages 521-528. IEEE, 2011.
51. S. Theodorakis, V. Pitsikalis, and P. Maragos. Advances in dynamic-static integration of movement and handshape cues for sign language recognition. In Proceedings of the International Gesture Workshop, 2011.
52. S. Theodorakis, V. Pitsikalis, I. Rodomagoulakis, and P. Maragos. Recognition with raw canonical phonetic movement and handshape subunits on videos of continuous sign language. In Proceedings of the International Conference on Image Processing, 2012.
53. M. Viola and M. J. Jones. Fast multi-view face detection. In Proceedings of the Conference on Computer Vision and Pattern Recognition, 2003.
54. C. Vogler and D. Metaxas. Parallel hidden markov models for american sign language recognition. In Proceedings of the International Conference on Computer Vision, volume 1, pages 116-122, 1999.
55. Y. Wu and T. S. Huang. View-independent recognition of hand postures. In Proceedings of the Conference on Computer Vision and Pattern Recognition, volume 2, pages 88-94, 2000.
56. M.-H. Yang, N. Ahuja, and M. Tabb. Extraction of 2d motion trajectories and its application to hand gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(8):1061-1074, Aug. 2002.
57. S. Young, D. Kershaw, J. Odell, D. Ollason, V. Woodland, and P. Valtchevand. The HTK Book. Entropic Ltd., 1999.
58. J. Zieren, N. Unger, and S. Akyol. Hands tracking from frontal view for vision-based gesture recognition. In Pattern Recognition, LNCS, pages 531-539, 2002.