Semi-synchronous speech and pen input for mobile user interfaces

Speech Communication

Volumn 53, Issue 3, 2011, Pages 283-291

  Shinoda, Koichi ^a   Watanabe, Yasushi ^a,b   Iwata, Kenji ^a,c   Liang, Yuan ^a   Nakagawa, Ryuta ^a,d   Furui, Sadaoki ^a  

^a TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^b NEC CORPORATION   (Japan)

^c TOSHIBA CORPORATION   (Japan)

^d ASAHI KASEI CORPORATION   (Japan)

Author keywords

Adaptation; Handwritten character recognition; Multi modal recognition; Speech recognition; User interfaces

Indexed keywords

ADAPTATION; ASYNCHRONICITY; HAND WRITTEN CHARACTER RECOGNITION; MOBILE ENVIRONMENTS; MOBILE USER INTERFACE; MULTI-MODAL; MULTI-MODAL RECOGNITION; NOISY SPEECH; PEN INPUT; RECOGNITION ACCURACY; RECOGNITION ALGORITHM; RECOGNITION PERFORMANCE; SEGMENT-BASED; SPEECH AND PEN INPUT; SUBJECTIVE TESTS; TIME LAG;

ALGORITHMS; CHARACTER RECOGNITION; GLOBAL SYSTEM FOR MOBILE COMMUNICATIONS; USER INTERFACES;

SPEECH RECOGNITION;

EID: 79551501502     PISSN: 01676393     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.specom.2010.10.001     Document Type: Article

References (16)

1. Ban, H.; Miyajima, C.; Itou, K.; Takeda, K.; Itakura, F.; 2004. Speech recognition using synchronization between speech and finger tapping. In: Proc. INTERSPEECH 2004 - ICSLP, Jeju, Korea.
2. S. Hayamizu, S. Itahashi, T. Kobayashi, and T. Takezawa Design and creation of speech and text corpora of dialogue IEICE Trans. Inform. Systems Vol. E76-D 1 1993 17 22
3. Hui, P.Y.; Meng, H.M.; 2006. Joint interpretation of input speech and pen gestures for multimodal human-computer interaction. In: Proc. INTERSPEECH 2006 - ICSLP, Pittsburgh, Pennsylvania, pp. 1197-1200.
4. S. Itahashi A noise database and Japanese common speech data corpus, J Acoust. Soc. Jpn. 47 12 1991 951 953 in Japanese
5. Itou, K.; Yamamoto, M.; Takeda, K.; Takezawa, T.; Matsuoka, T.; Kobayashi, T.; Shikano, K.; Itahashi, S.; 1998. The design of the newspaper-based Japanese large vocabulary continuous speech recognition corpus. In: Proc. ICSLP 1998, pp. 3261-3264.
6. Japanese dictation toolkit: free software repository for automatic speech recognition. Available from: < http://winnie.kuis.kyoto-u.ac.jp/dictation/ >.
7. Lee, A.; Kawahara, T.; Shikano, K.; 2001. Julius - an open source real-time large vocabulary recognition engine. In: Proc. EUROSPEECH 2001, pp. 1691-1694.
8. C.J. Leggetter, and P.C. Woodland Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models Comput. Speech Lang. 9 2 1995 171 185
9. Nakagawa, M.; Higashiyama, T.; Yamanaka, Y.; Sawada, S.; Tu, L.V.; Akiyama, K. 1995. Collection and utilization of on-line handwritten character patterns sampled in a sequence of sentences without any writing instructions. Technical Report of IEICE, Vol. 95, No. 278, pp. 43-48. (in Japanese).
10. Nakai, N.; Akira, N.; Shimodaira, H.; Sagayama, S.; 2001. Substroke approach to HMM-based on-line kanji handwriting recognition. In: Proc. ICDAR 2001, pp. 491-495.
11. S. Tamura, K. Iwano, and S. Furui Multi-modal speech recognition using optical-flow analysis for lip images J. VLSI Signal Process. Systems Signal Image Video Technol. 36 2-3 2004 117 124
12. The Mainichi Newspapers. Available from: .
13. Watanabe, Y.; Iwata, K.; Nakagawa, R.; Shinoda, K.; Furui, S.; 2007. Semi-synchronous speech and pen input. In: Proc. ICASSP 2007, Vol. 4, pp. 409-412.
14. Watanabe, Y.; Shinoda, K.; Furui, S.; 2008. Time-lag adaptation for semi-synchronous speech and pen input. In: Proc. INTERSPEECH 2008, pp. 2675-2678.
15. L. Wu, S.L. Oviatt, and P.R. Cohen Multimodal integration-a statistical view IEEE Trans. Multimedia 1 4 1999 334 341
16. Zhou, X.; Tian, Y.; Zhou, J.; Soong, F.K.; Dai, B.; 2006. Improved Chinese character input by merging speech and handwriting recognition hypotheses. In: Proc. ICASSP 2006, Vol. 1, Toulouse, France, pp. 609-612.