Multi-modal interfaces for interaction-communication between hearing and visually impaired individuals: Problems & issues

Proceedings - International Conference on Tools with Artificial Intelligence, ICTAI

Volumn 2, Issue , 2007, Pages 522-530

  Bourbakis, Nikolaos ^a,c   Esposito, Anna ^b   Kabraki, D ^c  

^a WRIGHT STATE UNIVERSITY   (United States)

^b SECOND UNIVERSITY OF NAPLES   (Italy)

^c AIIS Inc   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; CYBERNETICS; FLOW INTERACTIONS;

BLIND PERSON; CHALLENGING PROBLEM; HUMAN INTERACTIONS; INTERNATIONAL CONFERENCES; MULTI-MODAL INTERFACES; SIGN LANGUAGES; VISUALLY IMPAIRED;

COMMUNICATION;

EID: 48649094632     PISSN: 10823409     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICTAI.2007.178     Document Type: Conference Paper

References (71)

1. N.G.Bourbakis and D.Kavraki, Tyflos - an intelligent assistant for navigation of visually impaired people, Proc. IEEE Symp. on BIBE-01, pp. 230-235, Bethesda MD, Nov. 2001
2. D.Kavraki and N. Bourbakis, Koufos - an intelligent assistant for deaf people's independence, AIIS-Technical Report TR-03, 1993, 12 pages.
3. D.Kavraki and N.Bourbakis, Intelligent assistants for disable people's independence: a case study, Proc. IEEE Symp.on Intelligent Systems, pp. 337-344, Nov. 1996, MD, also AIIS-TR-1993,10 pages
4. G.Dagnelie and R.Massof, Toward an artificial eye, IEEE Spectrum, May 1996, pp.20-29
5. Williams E. 1977. Experimental comparisons of face-to-face and mediated communication: a review. Psychological Bulletin, 84, 963-976.
6. J.Wyatt and J.Rizzo, Ocular implants for the blind, IEEE Spectrum, May 1996,pp.47-53
7. R.Normann, E.Maynard, K.Guillory and D.Warren, Cortical implants for the blind, IEEE Spectrum, May 1996, pp. 54-59
8. N.Bourbakis, SPN representation on spoken natural language, AIIS TR-1997
9. Ekman P. 1992a. Facial expression of emotion: new findings, new questions. Psychological Science, 3, 34-38.
10. Ekman P. 1992b. An argument for basic emotions. Cognition and Emotion, 6, 169-200.
11. M. Hammett, D. Maurer, M. Neel, D. Wheeler, N. Bourbakis, Text Image to Speech Processing, GMU, ECE, TR-09-1988
12. Esposito A., Duncan S., Quek F., 2002. Holds as gestural correlated to empty and filled pauses. In Proc. of ICSLP 2002, vol. 1, pp. 541-544, Colorado.
13. Kendon A. , 1986. Current issues in the study of gesture. In The Biological Foundations of Gestures: Motor and Semiotic Aspects, Nespoulous J.L. et al. (Eds.), pp. 23-47, LEA Publishers, Hillsdale, New Jersey, London.
14. Klasmeyer G., Sendlmeier W. F. 1995. Objective voice parameters to characterize the emotional content in speech. In Proc. Of ICPhS 1995, K. Elenius, P. Branderudf ed.s, vol.1,182-185, Arne Strömbergs Grafiska.
15. Krauss R., Morrel-Samuels P., Colasante C., 1991. Do conversational hand gestures communicate?. Jour. of Personality and Social Psychology, 61(5), 743-754.
16. McNeill D., 1992. Hand and Mind: What Gesture Reveal about Thought. University of Chicago Press.
17. Pittam J., Scherer K. R. 1993. Vocal expression and communication of emotion. In Handbook of Emotion, M. Lewis, J. M. Haviland ed.s, 185-197, Guilford Press.
18. A.Moghaddamzadeh & N.Bourbakis, "Segmentation Of Color Images With Highlights And Shadows Using Fuzzy Reasoning", SPIE Conf. Electronic Imaging, pp.300-310, 1995.
19. Weeks, A.R. and Hague G.E., "Color Segmentation In The HSI Color Space Using The K-Means Algorithm", Proceedings of the SPIE - The International Society for Optical Engineering, vol. 3026, pp.143-54, 1997.
20. P.Yuan, Recognizing 3D view of objects using the L-G graph, PhD thesis, 2002, Dept. ECE, SUNY
21. Hu MK, "Visual Pattern Recognition by Moments Invariants", IRE Transaction of Information Theory, IT. 8, pp. 179-187, 1962.
22. A. Fernadez-Caballero et. al., Spatio-temporal shape building from image sequences using lateral interaction in acculative computation, Pattern recognition Journal, 36,2003, pp.1131-42
23. T.R.Gowrishakar and N.Bourbakis, Specifications for the development of a knowledge-based image interpretation system", Int. J. on Engr. Applications of Artificial Intelligence, vol.3, No.1,1990,pp.79-90.
24. Esposito A. , McCullough K. E., Quek F., 2001. Disfluencies in gesture: gestural correlates to filled and unfilled speech pauses. In Proc. of IEEE Workshop on Cues in Communication, Hawai, Dec. 9, 2001.
25. A. Esposito and N.Bourbakis, Speech/sound conversation into NL text and into images/icons, ATRC-TR-2006.
26. N.Bourbakis, Emulating human visual perception for measuring differences in images using an SPN graph approach, IEEE T-SMC, 32,2,191-201, 2002
27. N.Bourbakis, J.Gattiker and G.Bebis, Representing and interpreting human activity and events from video, IJAIT, vol.12,1,2003
28. J.Gattiker and N.Bourbakis, An SPN scheme for functional and structural representation of knowledge, Proc. Int. SEKE, June1995, MD
29. N.Bourbakis, Converting Sign Language into NL descriptions using SPN graphs, IJAIT, vol.15,6, 2008
30. T. Kanade A stereo machine for video-rate dense depth mapping and its applications, Int. Conf-CVPR, June 1996, pp. 196-202
31. I.Haritaoglu, et. al. "Real-time surveillance of people and their activities, IEEE T-PAMI-22,8, 2000, 809-830.
32. N. Bourbakis, Independent and "free-hands" Navigation in 3D Space for Blind and Visually Impaired Individuals , World Congress for Blind, Oct. 6-8, 2005, Baltimore MD.
33. Blauert, J. (1994). Spatial hearing: The psychophysics of human sound localization. Cambridge, MA: MIT Press.
34. Eide, F.F. (2003). Sensory Integration: Current concepts and practical implications. Sensory Integration Special Interest Section Quarterly, 26(3)1-3.
35. Lewkowicz, D.J., & Lickliter, R. (Eds.) (1994). The development of intersensory perception: Comparative perspectives. Hillsdale, NJ: Lawrence Erlbaum and Associates.
36. Stein, B., & Meredith, M.A. (1993). The merging of the senses. Boston, MA: The MIT Press.
37. J.Yamato, J.Ohya and Kenichiro, "Recognizing human action in time-sequential images using Hidden Markov Model", IEEE Conf Computer Vision, 1992, pp.379-385
38. C.Bregler, "Probabilistic Recognition of Human Actions", UCB-TR-May-1996, p.28
39. N. Bourbakis, "A Rule-Based Scheme For Synthesis of Texture Images", Int. IEEE Conf. on systems, Man and Cybernetics, Fairfax, VA, pp. 999-1003, 1987.
40. R.Cowie, et. al., Emotion recognition in human computer interaction, IEEE Signal Processing Magazine, vol.18, 32-80, 2001.
41. Pavlidis Theodosios, "Structural Pattern Recognition", Berlin, New York: Springer-Verlag, 1977.
42. K. S. Fu and J. K. Mu, "A Survey on Image Segmentation", Pattern Recognition, 13, pp. 3-16, 1981.
43. N. Bourbakis, A 2D Vibration Array for Sensing Dynamic Changes and 3D space for Blinds' Navigation, IEEE Symp. On BIBE-05, Minneapolis, MN, Oct. 19-21, 2005.
44. World Congress for Blind, Alternative Perception Projects, Oct. 6-8, 2005, Baltimore, National Federation for Blind, MD.
45. A. Esposito and N. Bourbakis, "The role of timing on the speech perception and production processes and its effects on language impaired individuals", Proc. Int. IEEE Symposium on BIBE-06, Oct. 16-18, 2006, WDC, pp. 348-356
46. N. Bourbakis, A. Esposito, D. Kavraki, "Learning and Understanding Emotional Behavior Patterns from People with Disabilities: A case Study", Proc. Int. IEEE Symp. on BIBE-06, Oct. 16-18, 2006, WDC, pp. 357-369
47. N. Bourbakis and G. Bebis, "Associating behavioral patterns of events from video", Proc. IEEE Conf. on TAI-06, WDC, Nov. 13-15, 2006
48. P. Kakumanu and N.Bourbakis, "Recognizing facial expressions using Local-Global graphs for Blind", Proc. IEEE Int. Conf. on TAI-06, WDC, Nov. 13-15, 2006
49. Sutton, S., Novick, D., Cole, R.A., and Fanty, M. Building 10,000 spoken-dialogue systems. Proceedings of the ICSLP, Philadelphia, PA, October 1996.
50. Cole, R., Sutton, S., Yan, Y., Vermeulen, P., and Fanty, M. Accessible technology for interactive systems: A new approach to spoken language research, Proceedings of the Int. Conference on Acoustics, Speech and Signal Processing, Seattle, WA, May 1998.
51. Ron Cole. Tools for research and education in speech science. In Proceedings of the International Conference of Phonetic Sciences, San Francisco, CA, Aug 1999.
52. Patrick Stone. Revolutionizing Language Instruction in Oral Deaf Education. In Proceedings of the International Conference of Phonetic Sciences, San Francisco, CA, Aug 1999.
53. Beskow, J. (1995). Rule-based Visual Speech Synthesis. Proceedings of Eurospeech '95, Madrid, Spain.
54. Beskow, J.,Dahlquist, M., Granström, B., Lundeberg, M., Spens, K-E & Öhman, T. (1997). The Teleface project - Multimodal Speech Communication for the Hearing Impaired. Proceedings of Eurospeech '97, Rhodos, Greece.
55. Thomas F. McCaul Video-Based Telecommunications Technology and the Deaf Community, http://www.aceinfo.net.au/Resources/Research/vri.html
56. Kaasinen Eija Adaptable appearance and functionality in communication software for speech impaired people ERCIM NEWS (1997) Nr: 28, pp. 9 - VTT Information Technology.
57. Stephen Brewster and Lorna M. Brown. Tactons: structured tactile messages for non-visual information display. AUIC '04: Proceedings of the fifth conference on Australasian user interface}, 15-23, Australian Computer Society, Inc , 2004.
58. Wall, S. and Brewster, S.A. Feeling What You Hear: Tactile Feedback for Navigation of Audio Graphs. In Proceedings of ACM CHI 2006 (Montreal, Canada), ACM Press Addison-Wesley, pp 1123-1132
59. Huang X., Acero A., Hon H., Ju Y., Liu J., Mederith S., Plumpe M. (1997). Recent Improvements on Microsoft's Trainable Text-to-Speech System - Whistler. Proceedings of ICASSP97 (2): 959-934.
60. Hallahan W. (1996). DECtalk Software: Text-to-Speech Technology and Implementation. Digital Technical Journal. 〈http://www.digital.com/ DTJ01〉.
61. Klatt D. (1987) Review of Text-to-Speech Conversion for English. Journal of the Acoustical Society of America, JASA vol. 82 (3), pp.737-793.
62. Averbuch et alt. (1987) Experiments with the TANGORA 20.000 Word Speech Recognizer, in ICASSP, 701-704.
63. Rabiner L. R., Levinson S. E., Sondhi M. M., (1983). On the Application of Vector Quantization and Hidden Markov Models to Speaker Independent Isolated Word Recognition, Bell Syst. Tech. Journal, vol 62, n.4, 1075-1105.
64. A. Esposito, M. Marinaro. Some Notes on Nonlinearities of Speech. In G. Chollet et al (Eds.) Nonlinear Speech Modeling and Applications, Lectures Notes in Computer Science, vol. 3445, pp: 1-14, Springer Verlag, ISBN 3-540-27441-3, 2005.
65. A. Esposito, G. Aversano, 2005. Text Independent Methods for Speech Segmentation. In G. Chollet et al. (eds.) Nonlinear Speech Modeling and Applications. Lectures Notes in Computer Science, vol. 3445, pp: 261-290, Springer Verlag, ISBN 3-540-27441-3, 2005.
66. D. Dakopoulos and N. Bourbakis, A comparative survey on navigation devices for blind, IEEE Symp. on Research on Assistive Technology, OH, April 2007
67. A. Esposito, et. al., Proc. of the NATO Int. Workshop on Verbal and non-verbal Communication, Vietri, Italy, Sept. 2006, IOS Pub., vol. 18, 2007
68. N.Bourbakis, An SPNG based method for image to NL text conversion, PR Journal,
69. N.Bourbakis, D. Kavraki and A. Esposito, "Koufos-An intelligent prototype system for deaf individuals", Assistive Technology Magazine
70. N.Bourbakis and A. Esposito, "3D Vibration patterns providing sensation of pitch, tone, volume for deaf", Int. Conf. on Assistive Technologies, April, 2008.
71. D. Dakopoulos S. Bodhu and N. Bourbakis, A 2D Vibration Array as an Assistive Device for Visually Impaired, Int. IEEE Symp. on BIBE-07, Boston, Oct. 14-17, 2007.