Facial expressions in American sign language: Tracking and recognition

Pattern Recognition

Volumn 45, Issue 5, 2012, Pages 1877-1891

  Nguyen, Tan Dat ^a   Ranganath, Surendra ^b  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b INDIAN INSTITUTE OF TECHNOLOGY GANDHINAGAR   (India)

Author keywords

American sign language (ASL); Bayesian tracking; Facial expression recognition; Facial feature tracking; Hidden Markov Models (HMM); KLT tracker; Probabilistic Principal Component Analysis (PPCA); Support Vector Machine (SVM)

Indexed keywords

AMERICAN SIGN LANGUAGE; BAYESIAN TRACKING; FACIAL EXPRESSION RECOGNITION; FACIAL FEATURE TRACKING; HIDDEN MARKOV MODELS (HMM); KLT TRACKER; PROBABILISTIC PRINCIPAL COMPONENT ANALYSIS; SUPPORT VECTOR;

GESTURE RECOGNITION; HIDDEN MARKOV MODELS; HUMAN COMPUTER INTERACTION; PRINCIPAL COMPONENT ANALYSIS; SUPPORT VECTOR MACHINES;

FACE RECOGNITION;

EID: 84855896310     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2011.10.026     Document Type: Article

References (52)

1. B.L. Loeding, S. Sarkar, A. Parashar, A.I. Karshmer, Progress in automated computer recognition of sign language, in: International Conference on Computers for Handicapped Persons, Paris, France, 2004, pp. 10791087.
2. S.C. Ong, and S. Ranganath Automatic sign language analysis: a survey and the future beyond lexical meaning IEEE Transactions on Pattern Analysis and Machine Intelligence 27 6 2005 873 891
3. U. Canzler, T. Dziurzyk, Extraction of non manual features for video-based sign language recognition, in: IAPR Workshop on Machine Vision Applications, Nara, Japan, 2002, pp. 318321.
4. C. Vogler, and S. Goldenstein Facial Movement Analysis in ASL Journal on Universal Access in the Information Society 6 4 2008 363 374
5. T.J. Castelli, M. Betke1, C. Neidle, Facial feature tracking and occlusion recovery in American sign language, in: The Sixth International Workshop on Pattern Recognition in Information Systems, Paphos, Cyprus, 2006, pp. 8190.
6. U. von Agris, J. Zieren, U. Canzler, B. Bauer, and K.-F. Kraiss Recent developments in visual sign language recognition Universal Access in the Information Society 6 4 2008 323 362
7. B. Bridges, and M. Metzger Deaf Tend Your 1996 Calliope Press Silver Spring, MD
8. C. Baker, and D. Cokely American Sign Language: A Teachers Resource Text on Grammar and Culture 1980 Clerc Books, Gallaudet University Press Washington, DC, USA
9. M. Pantic, and L.J. Rothkrantz Expert system for automatic analysis of facial expression Image and Vision Computing 18 11 2000 881 905
10. B. Fasel, and J. Luettin Automatic facial expression analysis: a survey Pattern Recognition 36 1 2003 259 275
11. M.S. Bartlett, J.C. Hager, P. Ekman, and T.J. Sejnowski Measuring Facial Expressions by Computer Image Analysis Psychophysiology 36 1999 253 263
12. M.J. Black, and Y. Yacoob Recognizing facial expressions in image sequences using local parameterized models of image motion International Journal of Computer Vision 25 1 1997 23 48
13. P. Ekman, et al., Facial Action Coding System, A Human Face, 2002 (Chapters 1 and 2).
14. D. Terzopoulos, and K. Waters Analysis and synthesis of facial image sequences using physical and anatomical models IEEE Transactions on Pattern Analysis and Machine Intelligence 15 6 1993 569 579
15. I.A. Essa, and A. Pentland Coding, analysis, interpretation and recognition of facial expressions IEEE Transactions on Pattern Analysis and Machine Intelligence 19 7 1998 757 763
16. D. DeCarlo, and D. Metaxas Optical flow constraints on deformable models with applications to face tracking International Journal of Computer Vision 38 2 2000 99 127
17. I. Cohen, N. Sebe, A. Garg, L. Chen, and T. Huang Facial expression recognition from video sequences: temporal and static modeling Computer Vision and Image Understanding 91 12 2003 160 187 (special issue on Face Recognition)
18. A.L. Yuille, P.W. Hallinan, and D.S. Cohen Feature extraction from faces using deformable templates International Journal of Computer Vision 8 2 1992 99 111
19. T.F. Cootes, C.J. Taylor, D.H. Cooper, and J. Graham Active shape models - their training and application Computer Vision and Image Understanding 61 1 1995 38 59
20. Y. li Tian, T. Kanade, and J.F. Cohn Recognizing action units for facial expression analysis IEEE Transactions on Pattern Analysis and Machine Intelligence 23 2 2001 97 115
21. S. Romdhani, S. Gong, A. Psarrou, A multi-view nonlinear active shape model using kernel PCA, in: British Machine Vision Conference, Nottingham, UK, 1999, pp. 483492.
22. Y. Chang, C. Hu, R. Feris, and M. Turk Manifold based analysis of facial expression Image and Vision Computing 24 6 2006 605 614
23. F. Tsalakanidou, and S. Malassiotis Real-time 2D3D facial action and expression recognition Pattern Recognition 43 2010 1763 1775
24. Y. Tong, J. Chen, and Q. Ji A unified probabilistic framework for spontaneous facial action modeling and understanding IEEE Transactions on Pattern Analysis and Machine Intelligence 32 2010 258 273
25. M.E. Tipping, and C.M. Bishop Probabilistic principal component analysis Journal of the Royal Statistical Society, Series B 61 3 1999 611 622
26. B.D. Lucas, T. Kanade, An iterative image registration technique with an application to stereo vision, in: International Joint Conference on Artificial Intelligence, vol. 2, British Columbia, Canada, 1981, pp. 674679.
27. H.T. Nguyen, and Q. Ji Spatio-temporal context for robust multitarget tracking IEEE Transactions on Pattern Analysis and Machine Intelligence 29 1 2007 52 64
28. C. Neidle, J. Nash, N. Michael, D. Metaxas, A method for recognition of grammatically significant head movements and facial expressions, developed through use of a linguistically annotated video corpus, in: Language and Logic Workshop, Formal Approaches to Sign Languages, European Summer School in Logic, Language, and Information, Bordeaux, France, 2009.
29. A. Kanaujia, Y. Huang, D. Metaxas, Tracking Facial Features Using Mixture of Point Distribution Models, in: Indian Conference on Computer Vision, Graphics and Image Processing, Madurai, India, 2006, pp. 492503.
30. A. Kanaujia, D.N. Metaxas, Large scale learning of active shape models, in: IEEE International Conference on Image Processing, San Antonio, TX, USA, 2007, pp. 265268.
31. D. Metaxas, A. Kanaujia, Z. Li, Dynamic tracking of facial expressions using adaptive, overlapping subspaces, in: International Conference on Computational Science, Beijing, China, 2007, pp. 11141121.
32. I. Patras, and E.R. Hancock Coupled prediction-classification for robust visual tracking IEEE Transactions on Pattern Analysis and Machine Intelligence 32 2010 1553 1567
33. D. Cristinacce, T. Cootes, Feature detection and tracking with constrained local models, in: British Machine Vision Conference, Edinburgh, UK, 2006, pp. 838928.
34. L. Ding, and A.M. Martinez Features versus context: an approach for precise and detailed detection and delineation of faces and facial features IEEE Transactions on Pattern Analysis and Machine Intelligence 32 11 2010 2022 2038
35. Y. Tong, Q. Ji, Multiview facial feature tracking with a multi-modal probabilistic model, in: International Conference on Pattern Recognition, Hong Kong, 2006, pp. 307310.
36. F. Zhou, F. De la Torre, J.F. Cohn, Unsupervised discovery of facial events, in: IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 2010, pp. 25742581.
37. T. Simon, M.H. Nguyen, F. De La Torre, J.F. Cohn, Action unit detection with segment-based SVMs, in: IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA, 2010, pp. 27372744.
38. T.F. Cootes, G.J. Edwards, C.J. Taylor, Active appearance models, in: European Conference on Computer Vision, Freiburg, Germany, 1998, pp. 484498.
39. M.E. Tipping, and C.M. Bishop Mixtures of probabilistic principal component analysers Neural Computation 11 2 1999 443 482
40. G. Hamerly, C. Elkan, Learning the k in k-means, in: Neural Information Processing System, British Columbia, Canada, 2003, pp. 281288.
41. M.A. Stephens EDF statistics for goodness of fit and some comparisons Journal of the American Statistical Association 69 347 1974 730 737
42. L. Rabiner, and B.-H. Juang Fundamentals of Speech Recognition 1993 Prentice-Hall Englewood Cliffs, NJ, USA
43. B.E. Boser, I.M. Guyon, and V.N. Vapnik A training algorithm for optimal margin classifiers The Fifth Annual Workshop on Computational Learning Theory 1992 ACM Press Pittsburgh, PA, USA 144 152
44. C. Cortes, and V. Vapnik Support-vector networks Machine Learning 20 3 1995 273 297
45. M. Bienvenu, B. Colonomos, The face of American Sign Language, Videotape, Sign Media Inc.
46. C. Neidle, S. Sclaroff, and V. Athitsos SignStream ™: a tool for linguistic and computer vision research on visual-gestural language data Behavior Research Methods, Instruments, and Computers 33 2001 311 320
47. C. Neidle SignStream ™: a database tool for research on visual-gestural language Journal of Sign Language and Linguistics 4 12 2002 203 214
48. C. Mayer, M. Wimmer, and B. Radig Adjusted pixel features for robust facial component classification Image and Vision Computing 28 2010 762 771
49. M.B. Stegmann, B.K. Ersboll, and R. Larsen FAME - a flexible appearance modelling environment IEEE Transactions on Medical Imaging 22 10 2003 1319 1331
50. K. Murphy, Hidden Markov Model Toolbox for Matlab, 〈 http://www.cs.ubc.ca/∼murphyk/Software/HMM/hmm.html 〉.
51. C.-C. Chang, C.-J. Lin, LIBSVM: A Library for Support Vector Machines, Software Available at: 〈 http://www.csie.ntu.edu.tw/∼cjlin/libsvm 〉 (2001).
52. T. Kanade, Y. Tian, J.F. Cohn, Comprehensive database for facial expression analysis, in: IEEE International Conference on Automatic Face and Gesture Recognition, Grenoble, France, 2000, pp. 4653.