Speech and multimodal interaction in mobile search

IEEE Signal Processing Magazine

Volumn 28, Issue 4, 2011, Pages 40-49

  Feng, Junlan ^a   Johnston, Michael ^a   Bangalore, Srinivas ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

INTERACTIVE COMPUTER SYSTEMS; PERSONAL COMPUTERS;

HIGH SPEED WIRELESS NETWORKS; MOBILE SEARCH; MULTI-MODAL INTERACTIONS;

MOBILE TELECOMMUNICATION SYSTEMS;

EID: 85032751148     PISSN: 10535888     EISSN: None     Source Type: Journal    
DOI: 10.1109/MSP.2011.941073     Document Type: Article

References (39)

1. A. Acero, N. Bernstein, R. Chambers, Y. Ju, X. Li, J. Odell, P. Nguyen, O. Scholz, and G. Zweig, "Live search for mobile: Web services by voice on the cellphone," in Proc. ICASSP, 2008, pp. 5256-5259.
2. E. André, "Natural language in multimedia/multimodal systems," in Handbook of Computational Linguistics, R. Mitkov, Ed. London, U.K.: Oxford Univ. Press, 2002, pp. 650-669.
3. M. Bacchiani, F. Beaufays, J. Schalkwyk, M. Schuster, and B. Strope, "Deploying GOOG-411: Early lessons in data, measurement, and testing," in Proc. ICASSP, 2008, pp. 5260-5263.
4. S. Bangalore and M. Johnston, "Robust understanding in multimodal interfac-es," Comput Linguist., vol. 35, no. 3, pp. 345-397, 2009.
5. S. Bangalore and T. Mishra, "Tightly coupling speech recognition and search," in Proc. NAACL-HLT, 2009, pp. 281-284.
6. J. R. Bellegarda, "Statistical language model adaptation: Review and perspectives," Speech Commun., vol. 42, pp. 93-108, 2004.
7. E. Bocchieri and D. Caseiro, "Use of geographical meta-data in ASR language and acoustic models," in Proc. ICASSP, 2010, pp. 5118-5121.
8. J. Chai, P. Hong, and M. Zhou, "Combining semantic and temporal constraints for multimodal integration in conversation systems," in Proc. HLT-NAACL 2003 Workshop on Research Directions in Dialog Processing, Edmonton, Alberta, Canada, 2003, pp. 1-3.
9. C. Chelba, J. Schalkwyk, T. Brants, V. Ha, B. Harb, W. Neveitt, C. Parada, and P. Xu, "Query language modeling for voice search," in Proc. 2010 IEEE Workshop on Spoken Language Technology, pp. 127-132.
10. P. Cohen, "The role of natural language in a multimodal interface," in Proc. 5th Annu. ACM Symp. User Interface Software and Technology, 1992, pp. 143-149.
11. P. Ehlen and M. Johnston, "Location grounding in multimodal local search," in Proc. ICMI-MLMI, Beijing, China, 2010, pp. 32:1-32:4.
12. J. Feng and S. Bangalore, "Effects of word confusion networks on voice search," in Proc. EACL, 2009, pp 238-245.
13. L. Galescu and J. Allen, "Hierarchical statistical language models: Experiments on in-domain adaptation," in Proc. ICSLP, 2000, vol. 1, pp. 186-189.
14. Y. Gao, B. Ramabhadran, J. Chen, and M. Picheny, "Innovative approaches for large vocabulary name recognition," in Proc. ICASSP 2001, pp. 53-56. (Pubitemid 32839186)
15. A. Ghosha, M. Jansche, S. Khudanpur, M. Riley, and M. Ulinski, "Web-derived pronunciations," in Proc. ICASSP, 2009, pp. 4289-4292.
16. C. V. Heerden, J. Schalkwyk, and B. Strope, "Language modeling for what-with-where on GOOG-411," in Proc. Interspeech, 2009, pp. 991-994.
17. M. Johnston, "Multimodal unification-based parsing," in Proc. ACL, Montreal, Canada, 1998, pp. 624-630.
18. M. Johnston, "Building multimodal applications with EMMA," in Proc. ICMI-MLMI, 2009, pp. 47-54.
19. R. Jones, B. Rey, and O. Madani, "Generating query substitutions," in Proc. WWW, 2006, pp. 387-396. (Pubitemid 46946631)
20. D. Jurafsky and J. Martin, Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Englewood Cliffs, NJ: Prentice-Hall, 2008.
21. M. Kamvar, M. Kellar, R. Patel, and Y. Xu, "Computers and iPhones and mobile phones, oh my! A logs-based comparison of search users on different devic-es," in Proc. WWW, 2009, pp. 801-810.
22. F. Kubala, R. Schwartz, R. Stone, and R. Weischedel, "Named entity extrac-tion from speech," in Proc. DARPA Broadcast News Transcription and Understanding Workshop, 1998, pp. 287-292.
23. C. Leggetter and P. Woodland, "Flexible speaker adaptation using maximum likelihood linear regression," in Proc. ARPA Spoken Language Technology Workshop. San Francisco, CA: Morgan Kaufmann, 1995, pp. 110-115.
24. X. Li, P. Nguyen, G. Zweig, and D. Bohus, "Leveraging multiple query logs to improve language models for spoken query recognition," in Proc. ICASSP, 2009, pp. 3713-3716.
25. L. Mangu, E. Brill, and A. Stolcke, "Finding consensus in speech recogni-tion: Word error minimization and other applications of confusion networks," Comput. Speech Lang., pp. 373-400, 2000.
26. S. Oviatt and R. V. Gent, "Error resolution during multimodal human-com-puter interaction," in Proc. ICSLP, 1996, pp. 93-129.
27. S. L. Oviatt, "Multimodal interactive maps: Designing for human perform-ance," Hum. Comput. Interact., pp. 93-129, 1997.
28. T. Paek, B. Thiesson, Y.-C. Ju, and B. Lee, "Search vox: Leveraging multimo-dal refinement and partial knowledge for mobile voice search," in Proc. 21st Annu. Symp. User Interface Software and Technology (UIST), Monterey, CA. AAAI Press, 2008, pp. 141-150.
29. R. Pieraccini, K. Dayanidhi, J. Bloom, J.-G. Dahan, M. Phillips, B. R. Goodman, and K. V. Prasad, "Multimodal conversational systems for automobiles," Commun. ACM, vol. 47, no. 1, pp. 47-49, 2004.
30. J. Schalkwyk, D. Beeferman, F Beaufays, B. Byrne, C. Chelba, M. Cohen, M. Kamvar, and B. Strope, "Google search by voice: A case study," in Visions of Speech: Exploring New Voice Apps in Mobile Environments, Call Centers and Clinics, A. Weinstein, Ed. Heidelberg: Springer-Verlag, 2010, pp. 127-132.
31. Y Song, Y Wang, Y Ju, M. Seltzer, I. Tashev, and A. Acero, "Voice search of structured media data," in Proc. ICASSP, 2009 pp. 3941-3944.
32. A. Stent, I. Zeljković, D. Caseiro, and J. Wilpon, "Geo-centric language models for local business voice search," in Proc. NAACL, 2009, pp. 389-396.
33. B. Su hm, B. Myers, and A. Waibel, "Multimodal error correction for speech user interfaces," ACM Trans. Hum. Comput Interact., vol. 8, no. 1, pp. 60-98, Mar. 2001.
34. B. Ta n and F. Peng, "Unsupervised query segmentation using generative lan-gauge models and Wikipedia," in Proc. WWW, 2008, pp. 347-356.
35. J. Te evan, S. T. Dumais, and E. Horvitz, "Personalizing search via automated analysis of interests and activities," in Proc. SIGIR. ACM Press, 2005, pp. 449-456.
36. J. Ve rgo, "A statistical approach to multimodal natural language interaction," in Proc. AAAI Workshop on Representations for Multimodal Human-Computer Interaction, Madison, WI, 1998, pp. 81-85.
37. M. We ster, "Pronunciation modeling for ASR: Knowledge-based and data-derived methods," Comput Speech Lang., vol. 17, no. 1, pp. 69-85, 2003.
38. R. W. White and R. A. Roth, Exploratory Search: Beyond the Query-Response Paradigm Synthesis Lectures on Information Concepts, Retrieval, and Services). San Rafael, CA: Morgan & Claypool, 2009.
39. P. Zh an and A. Waibel, "Vocal tract length normalization for large vocabulary continuous speech recognition," CMU Computer Science Technical Reports, Tech. Rep., 1997.