SPATIAL, TEMPORAL, and SEMANTIC MODELS for American SIGN LANGUAGE GENERATION: IMPLICATIONS for GESTURE GENERATION

International Journal of Semantic Computing

Volumn 2, Issue 1, 2008, Pages 21-45

  Huenerfauth, Matt ^a  

^a CITY UNIVERSITY OF NEW YORK   (United States)

Author keywords

American sign language; classifier predicates; multimodal systems; natural language generation; spatial models; temporal coordination

Indexed keywords

NATURAL LANGUAGE PROCESSING SYSTEMS; SEMANTICS; SOFTWARE PROTOTYPING;

AMERICAN SIGN LANGUAGE; MULTIMODAL SYSTEM; NATURAL LANGUAGE GENERATION; SPATIAL MODELS; TEMPORAL COORDINATION;

HUMAN COMPUTER INTERACTION;

EID: 77954444990     PISSN: 1793351X     EISSN: 17937108     Source Type: Journal    
DOI: 10.1142/S1793351X08000336     Document Type: Article

References (42)

1. C. Neidle, D. Kegl, D. MacLaughlin, B. Bahan and R. G. Lee, The Syntax of American Sign Language: Functional Categories and Hierarchical Structure, MIT Press, Cambridge, MA, 2000.
2. S. Liddell, Grammar, Gesture, and Meaning in American Sign Language, Cambridge University Press, Cambridge, UK, 2003.
3. R. E. Mitchell, T. A. Young, B. Bachleda and M. A. Karchmer, How many people use ASL in the United States? Why estimates need updating, Sign Language Studies 6(3) (2006).
4. J. Holt, Demographic, Stanford Achievement Test-8th Edition for Deaf and Hard of Hearing Students: Reading Comprehension Subgroup Results, American Annals of the Deaf 138(2) (1993) 172-175.
5. M. Huenerfauth and V. L. Hanson, Sign language in the interface: Access for deaf signers, in The Universal Access Handbook, ed. C. Stephanidis, Lawrence Erlbaum Associates, Mahwah, NJ, to appear.
6. S. Liddell, An Investigation into the Syntactic Structure of ASL, Ph.D. dissertation, University of California, San Diego, 1977.
7. T. Supalla, Morphology of verbs of motion and location, in Proceedings of the Second National Symposium on Sign Language Research and Teaching, eds. F. Caccamise and D. Hicks, National Association for the Deaf, Silver Spring, MD, 1978, pp. 27-45.
8. A. Kendon, Gesture: Visible Action as Utterance, Cambridge University Press, UK, 2004.
9. D. McNeill, Hand and Mind, The Chicago University Press, Chicago, 1992.
10. S. Liddell, Sources of meaning in ASL classifier predicates, in Perspectives on Classifier Constructions in Sign Languages, ed. K. Emmorey, Workshop on Classifier Constructions, La Jolla/San Diego, California, 2003.
11. B. L. Loeding, S. Sarkar, A. Parashar and A. I. Karshmer, Progress in automated computer recognition of Sign Language, in Proceedings of ICCHP 2004, Lecture Notes in Computer Science 3118, ed. K. Miesenberger, Springer-Verlag, Berlin, 2004, pp. 1079-1087.
12. M. Huenerfauth, Generating American Sign Language Classifier Predicates for English-To-ASL Machine Translation, Ph.D. dissertation, Computer and Information Science, University of Pennsylvania, Philadelphia, PA, 2006.
13. I. Marshall and É. Sáfár, Extraction of semantic representations from syntactic SMU link grammar linkages, in Proceedings of Recent Advances in Natural Language Processing (RANLP), ed. G. Angelova, Tzigov Chark, Bulgaria, September, 2001, pp. 154-159.
14. S. Morrissey and A. Way, Lost in translation: The problems of using mainstream MT evaluation metrics for sign language translation, in Proc. of the 5th SALTMIL Workshop on Minority Languages, Language Resources and Evaluation Conference, LREC-2006, 2006.
15. L. van Zijl, South African sign language machine translation project, in Proceedings of the 8th international ACM SIGACCESS Conference on Computers and Accessibility (ASSETS-2006), Portland, Oregon, USA, 2006.
16. J. Morford and J. MacFarlane, Frequency characteristics of American Sign Language, Sign Language Studies 3(2) (2003).
17. M. Huenerfauth, Spatial representation of classifier predicates for machine translation into American Sign Language, in Proceedings of the Workshop on Representation and Processing of Signed Languages, Language Resources and Evaluation Conference, (LREC-2004), 2004.
18. N. Badler, R. Bindiganavale, J. Allbeck, W. Schuler, L. Zhao, S. Lee, H. Shin and M. Palmer, Parameterized action representation and natural language instructions for dynamic behavior modification of embodied agents, in Proceedings of AAAI Spring Symposium, 2000.
19. R. Coyne and R. Sproat, WordsEye: An automatic text-to-scene conversion system, in Proceedings of the 28th International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH-2001), Los Angeles, CA, USA, 2001, pp. 487-496.
20. Y. Liu, Interactive Reach Planning for Animated Characters Using Hardware Acceleration, Ph.D. dissertation, Computer Science, University of Pennsylvania, PA, 2003.
21. L. Zhao, Y. Liu and N. I. Badler, Applying empirical data on upper torso movement to real-time collision-free reach tasks, in Proceedings of the SAE Digital Human Modeling Conference, Iowa City, 2005.
22. P. Tepper, S. Kopp and J. Cassell, Content in context: Generating language and iconic gesture without a gestionary, in Proc. of the Workshop on Balanced Perception &Action in ECAs, Autonomous Agents &Multiagent Systems Conference (AAMAS-04), New York, 2004.
23. S. Kopp, P. Tepper and J. Cassell, Towards integrated microplanning of language and iconic gesture for multimodal output, in Proceedings of the International Conference on Multimodal Interfaces, State College, PA, USA, 2004.
24. M. Evers, M. Theune and J. Karreman, Which way to turn? Guide orientation in virtual way finding, in Proceedings of the Workshop on Embodied Language Processing, Annual Meeting of the Association of Computational Linguistics, Prague, Czech Republic, 2007.
25. H. vanWelbergen, A. Nijholt, D. Reidsma and J. Zwiers, Presenting in virtual worlds: Towards an architecture for a 3D presenter explaining 2D-presented information, IEEE Intelligent Systems 21(5) (2006) 47-53.
26. E. André, The generation of multimedia presentations, in A Handbook of Natural Language Processing: Techniques and Applications for the Processing of Language as Text, eds. R. Dale, H. Moisl and H. Somers, Marcel Dekker, 2000, pp. 305-327.
27. M. E. Foster, M. White, A. Setzer and R. Catizone, Multimodal generation in the COMIC dialogue system, in Proceedings Annual Meeting of the Association for Computational Linguistics (ACL-2005), Interactive Poster and Demonstration Sessions, 2005.
28. J. Cassell, M. Stone and H. Yan, Coordination and context-dependence in the generation of embodied conversation, in Proceedings of the International Natural Language Generation Conference, Mitzpe Ramon, Israel, 2000.
29. S. Liddell and R. Johnson, American Sign Language: The phonological base, in Sign Language Studies 64 (1989) 195-277.
30. D. Brentari, Theoretical Foundations of American Sign Language Phonology, Ph.D. dissertation, University of Chicago, Chicago, IL, 1990.
31. G. Coulter (ed.), Current Issues in ASL phonology, Phonetics and Phonology, Vol. 3, Academic Press, New York, NY, 1993.
32. V. Sutton, The signwriting literacy project, in Proceedings of the Impact of Deafness on Cognition AERA Conference, San Diego, CA, 1998.
33. D. Newkirk, SignFont Handbook, Emerson and Associates, San Diego, CA, 1987.
34. S. Prillwitz, R. Leven, H. Zienert, T. Hanke and J. Henning, HamNo-Sys. Version 2.0; Hamburg notation system for sign languages, introductory guide, in International Studies on Sign Language and Communication of the Deaf, Vol. 5, Signum, Hamburg, Germany, 1989.
35. R. Elliott, J. Glauert, V. Jennings and J. Kennaway, An overview of the SiGML notation and SiGML signing software system, in Proceedings of the Workshop on the Representation and Processing of Signed Languages, 4th International Conference on Language Resources and Evaluation (LREC-2004), Lisbon, Portugal, 2004.
36. M. Huenerfauth, Representing coordination and non-coordination in American Sign Language animations, Behaviour &Information Technology 25(4) (2006) 285-295.
37. M. Huenerfauth, American Sign Language generation: Multimodal NLG with multiple linguistic channels, in Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL-2005), Student Research Workshop, Ann Arbor, USA, 2005.
38. C. Neidle, S. Sclaroff and V. Athitsos, SignStreamTM: A tool for linguistic and computer vision research on visual-gestural language data, Behavior Research Methods, Instruments, and Computers 33(3) (2001) 311-320.
39. C. Martell, FORM: An Experiment in the Annotation of the Kinetics of Gesture Doctoral dissertation, Computer and Information Science, University of Pennsylvania, PA, 2005.
40. C. Martell, Corpus-based gesture analysis: An extension of the FORM dataset for the automatic detection of phase in a gesture, Int. J. Semantic Computing 1(4) (2007).
41. S. Bird and M. Lieberman, A formal framework for linguistic annotation, Speech Communication 33(1,2) (2000) 23-60.
42. M. Huenerfauth, L. Zhou, E. Gu and J. Allbeck, Evaluation of American Sign Language generation by native ASL signers, ACM Transactions on Accessible Computing 1(1) (2008).