Integration of a voice recognition system in a social robot

Cybernetics and Systems

Volumn 42, Issue 4, 2011, Pages 215-245

  Alonso Martin F ^a   Salichs, Miguel A ^a  

^a UNIVERSIDAD CARLOS III DE MADRID   (Spain)

Author keywords

ASR; audition system; automatic speech recognition; computing confidence score; dialogue; human computer interaction; human robot interaction; maggie; microphone system; natural language processing; natural language understanding; personal robot; robot audition; social robot; speech recognition; voice recognition

Indexed keywords

ASR; AUDITION SYSTEM; AUTOMATIC SPEECH RECOGNITION; CONFIDENCE SCORE; DIALOGUE; HUMAN-COMPUTER; MAGGIE; MICROPHONE SYSTEM; NATURAL LANGUAGE PROCESSING; NATURAL LANGUAGE UNDERSTANDING; PERSONAL ROBOT; ROBOT AUDITION; SOCIAL ROBOTS;

AUDIO SYSTEMS; AUDITION; COMPUTATIONAL LINGUISTICS; HUMAN COMPUTER INTERACTION; HUMAN ROBOT INTERACTION; KNOWLEDGE MANAGEMENT; MAN MACHINE SYSTEMS; NATURAL LANGUAGE PROCESSING SYSTEMS; SEMANTICS; SPEECH PROCESSING;

SPEECH RECOGNITION;

EID: 79958703093     PISSN: 01969722     EISSN: 10876553     Source Type: Journal    
DOI: 10.1080/01969722.2011.583593     Document Type: Article

References (50)

1. Barber, R. and Salichs, Ma. 2002. A new human based architecture for intelligent autonomous robots. In Intelligent autonomous vehicles 2001 (IAV 2001): a proceedings volume from the 4th IFAC Symposium, Sapporo, Japan, 5-7 September 2001, 81. Pergamon. http://scholar.google.es/scholar?cluster= 10839062160608396845&hl=es&as-sdt=20000
2. Bennett, C., Llitjós, A. F., Shriver, S., Rudnicky, A. I., and Black, A. W. 2002. Building VoiceXML-based applications. In Seventh International Conference on Spoken Language Processing, 2-5. Citeseer. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.73.228&rep= rep1&type=pdf
3. Bischoff, R. and Graefe, V. 2004. HERMES - a versatile personal robotic assistant. Proceedings of the IEEE 92, no. 11:1759-1779. doi:10.1109/JPROC.2004. 835381. http://ieeexplore.ieee.org/xpl/freeabs-all.jsp?arnumber=1347457 (Pubitemid 40890670)
4. Breazeal, C. 2003. Emotive qualities in lip-synchronized robot speech. Advanced Robotics 17, no. 2(May):97-113. doi:10.1163=156855303321165079. http://www.ingentaconnect.com/content/vsp/arb/2003/00000017/00000002/art00003
5. Chan, S. W. K. 1995. Inferences in natural language understanding. In Proceedings of 1995 IEEE International Conference on Fuzzy Systems. The International Joint Conference of the Fourth IEEE International Conference on Fuzzy Systems and The Second International Fuzzy Engineering Symposium, 935-940. IEEE. doi:10.1109=FUZZY.1995.409794. http://ieeexplore.ieee.org/xpl/freeabs- all.jsp?arnumber=409794
6. Chetty, V. 2009. Microsoft's Project Natal for Xbox 360 (July). http://www.suite101.com/content/microsofts-project-natal-for-xbox-360-a129412
7. Dalmasso, E., Castaldo, F., Laface, P., Colibro, D., and Vair, C. Loquendo - Speaker recognition evaluation system. In Acoustics, Speech and Signal Processing, ICASSP 2009. IEEE International Conference on, 4213-4216. Taipei. doi:10.1109/ICASSP.2009.4960558. http://ieeexplore.ieee.org/xpl/freeabs- all.jsp?arnumber=4960558.
8. Favre, B., Benoit, B., and Hakkani-Tur, D. 2010a. Evaluation of semantic role labeling and dependency parsing of automatic speech recognition output. 2010 IEEE International Conference on Acoustics, Speech and Signal Processing 1:5342-5345. doi:10.1109=ICASSP.2010.5494946. http://ieeexplore.ieee.org/lpdocs/ epic03/wrapper.htm?arnumber=5494946.
9. - 2010b. Evaluation of semantic role labeling and dependency parsing of automatic speech recognition output. In 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, 5342-5345. Dallas (USA): IEEE. doi:10.1109/ICASSP.2010.5494946. http://ieeexplore.ieee.org/xpl/freeabs-all.jsp? arnumber=5494946.
10. Fry, J., Asoh, H., and Matsui, T. 1998. Natural dialogue with the Jijo-2 office robot. In Intelligent Robots and Systems, 1998. Proceedings., 1998 IEEE/RSJ International Conference on, 1278-1283. Victoria (Canada). doi:10.1109/IROS. 1998.727475. http://ieeexplore.ieee.org/xpl/freeabs-all.jsp? arnumber=727475
11. Gorostiza, J., Barber, R., Khamis, A., Malfaz, M., Pacheco, R., Rivas, R., Corrales, A., Delgado, E., and Salichs, M. 2006. Multimodal Human-Robot Interaction Framework for a Personal Robot. In ROMAN 2006 - The 15th IEEE International Symposium on Robot and Human Interactive Communication, 39-44. Hatfield (UK): IEEE, September. doi:10.1109/ROMAN.2006.314392. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4107783
12. Hegel, F., Muhl, C., Wrede, B., Hielscher-Fastabend, M., and Sagerer, G. 2009. Understanding Social Robots. 2009 Second International Conferences on Advances in Computer-Human Interactions. IEEE, February. doi:10.1109=ACHI. 2009. 51. http://ieeexplore.ieee.org/xpl/freeabs-all.jsp?arnumber=4782510
13. Huang, J, Ohnishi, N., and Sugie, N. 1997. Building ears for robots: Sound localization and separation. Artificial Life and Robotics 1, no. 4(December):157-163. doi:10.1007=BF02471133. http://www.springerlink.com/ content/upw68k6138152679/
14. Huang, X. D., Lee, K. F., Hon, H. W., and Hwang, M. Y. 1991. Improved acoustic modeling with the SPHINX speech recognition system. In Acoustics, Speech, and Signal Processing, 1991. ICASSP-91., 1991 International Conference on, 345-348. Toronto (Canada): IEEE. doi:10.1109/ICASSP.1991.150347. http://ieeexplore.ieee.org/xpl/freeabs-all.jsp?arnumber=150347
15. Ishi, C., Matsuda, S., Kanda, T., Jitsuhiro, T., Ishiguro, H., Nakamura, S., and Hagita, N. 2006. Robust Speech Recognition System for Communication Robots in Real Environments. In 2006 6th IEEE-RAS International Conference on Humanoid Robots, 340-345. Genoa (Italy): IEEE, December. doi:10.1109/ICHR.2006. 321294. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4115624
16. Jansen, B. and Belpaeme, T. 2006. A computational model of intention reading in imitation. Robotics and Autonomous Systems 54, no. 5(May):394-402. doi:10.1016/j.robot.2006.01.006. http://linkinghub.elsevier.com/retrieve/pii/ S0921889006000194
17. Junlan, F. 2010. A general framework for building natural language understanding modules in voice search. In 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, 5362-5365. Dallas: IEEE, March. doi:10.1109/ICASSP.2010.5494951. http://ieeexplore.ieee.org/xpl/freeabs-all.jsp? arnumber=5494951
18. Kibria, S. and Hellström, T. 2007. Voice user interface in robotics - common issues and problems. aass.oru.se. http://www.aass.oru.se/Research/ Learning/publications/2007/Kibria-Hellstrom-2007-CS07-Voice-User-Interface-in- Robotics-Common-Issues-and-Problems.pdf
19. Kim, H. and Choi, J. 2007. Human-robot interaction in real environments by audiovisual integration. International Journal of Control 5(1):61-69. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.126.8455&rep= rep1&type=pdf (Pubitemid 46198189)
20. Kim, K., Jeong, M., and Lee, G. G. 2007. Improving Speech Recognition Using Semantic and Reference Features in a Multimodal Dialog System. In RO-MAN 2007 - The 16th IEEE International Symposium on Robot and Human Interactive Communication, 416-420. Jeju Island (Korea): IEEE. doi:10.1109/ROMAN. 2007.4415120. http://ieeexplore.ieee.org/xpl/freeabs-all.jsp?arnumber=4415120
21. Lecouteux, B., Nocera, P., and Linares, G. 2010. Semantic cache model driven speech recognition. 2010 IEEE International Conference on Acoustics, Speech and Signal Processing. IEEE. doi:10.1109/ICASSP.2010.5495642. http://ieeexplore.ieee.org/xpl/freeabs-all.jsp?arnumber=5495642
22. Li, J. and Wang, K. 1993. Natural language understanding based on background knowledge. In Proceedings of TENCON'93. IEEE Region 10 International Conference on Computers, Communications and Automation, 460-462. Beijing: IEEE. doi:10.1109/TENCON.1993.320026. http://ieeexplore.ieee.org/xpl/freeabs-all.jsp? arnumber=320026
23. Lin, F. and Weng, F. 2010. Computing confidence score of any input phrases for a spoken dialog system. In 2010 IEEE Spoken Language Technology Workshop, 295-300. Berkeley, California (USA): IEEE, December. doi:10.1109/SLT.2010. 5700867. http://ieeexplore.ieee.org/xpl/freeabs-all.jsp? arnumber=5700867
24. Llisterri, J., Carbó, C, Machuca, M., De la Mota, C, Riera, M., and Rios, A. 2003. El papel de la fonética en el desarrollo de las tecnologías del habla. In Memorias de las VII Jornadas de Linguística. Cadiz (Spain): Servicio de Publicaciones de la Universidad de Cádiz. http://liceu.uab.es/~joaquim/speech-technology/UNAM-03/UNAM03- Guion-Bib.pdf
25. Lopes L. Seabra and Belpaeme, T. 2008. Beyond the individual: new insights on language, cognition and robots. Connection Science 204(December):231-237. doi:10.1080/09540090802518661. http://www.informaworld. com/openurl?genre=article&doi=10.1080/09540090802518661&magic=crossref) D404A21C5BB053405B1A640AFFD44AE3
26. Mitsunaga, N., Miyashita, T., Ishiguro, H., Kogure, K., and Hagita, N. 2006. Robovie-IV: A Communication Robot Interacting with People Daily in an Office. In Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on, 5066-5072. Beijing: IEEE. doi:10.1109/IROS.2006.282594. http://ieeexplore.ieee.org/xpl/freeabs-all.jsp?arnumber=4059225 (Pubitemid 46928732)
27. Nakadai, K., Lourens, T., Okuno, H. G., and Kitano, H. 2000. Active Audition for Humanoid. In Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence, 832-839. AAAI Press. http://portal.acm.org/citation. cfm?id=647288.723417
28. Nakadai, K., Okuno, H. G., Nakajima, H., Hasegawa, Y., and Tsujino, H. 2008. An Open Source Software System For Robot Audition HARK and Its Evaluation.
29. Niklfeld, G. and Finan, R. 2001. Architecture for adaptive multimodal dialog systems based on VoiceXML. In Proceedings of EuroSpeech, 1-4. Scandinavia: Association for Computational Linguistics Morristown. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.21.7658&rep= rep1&type=pdf
30. Nyberg, E., Mitamura, T., Placeway, P., and Duggan, M. 2002. DialogXML: Extending VoiceXML for Dynamic Dialog Management. In Proceedings of the second international conference on Human Language Technology Research, 298-302. San Diego (California): Morgan Kaufmann Publishers Inc.
31. Oh, S., Viswanathan, V., and Papamichalis, P. 1992. Hands-free voice communication in an automobile with a microphone array. In ICASSP-92:1992 IEEE International Conference on Acoustics, Speech, and Signal Processing, 281-284. San Francisco: IEEE. doi:10.1109/ICASSP.1992.225916. http://ieeexplore.ieee.org/ xpl/freeabs-all.jsp?arnumber=225916
32. Okuno Hiroshi G. 2007. Design and implementation of a robot audition system for automatic speech recognition of simultaneous speech. In IEEE Workshop on Automatic Speech Recognition & Understanding (ASRU), 111-116. Kyoto (Japan): IEEE. doi:10.1109/ASRU.2007.4430093. http://ieeexplore.ieee.org/lpdocs/ epic03/wrapper.htm?arnumber=4430093
33. Paolo, B. and Mosso, S. 2005. Loquendo Speech Technologies and multimodality.
34. Rivas, R., Corrales, A., Barber, R., and Salichs, M. A. 2007. Robot Skill Abstraction for AD Architecture. 6th IFAC Symposium on Intelligent Autonomous Vehicles. http://roboticslab.uc3m.es/publications/iav07-AD.pdf
35. Roy, N., Pineau, J., and Thrun, S. 1998. Spoken Dialog Management for Robots. Management.
36. -. 2000. Spoken dialogue management using probabilistic reasoning. Proceedings of the 38th Annual Meeting on Association for Computational Linguistics - ACL'00. Morristown, NJ, USA: Association for Computational Linguistics, October. doi:10.3115/1075218.1075231. http://portal.acm.org/ citation.cfm?id=1075218.1075231.
37. Sakagami, Y., Watanabe, R., Aoyama, C., Matsunaga, S., Higaki, N., and Fujimura, K. 2002. The intelligent ASIMO: system overview and integration. In Intelligent Robots and Systems, 2002. IEEE/RSJ International Conference on, 2478-2483. Laussane: IEEE. doi:10.1109/IRDS.2002.1041641. http://ieeexplore. ieee.org/xpl/freeabs-all.jsp?arnumber=1041641 (Pubitemid 35438181)
38. Shuyin, I. Toptsis, Li, S., Wrede, B., and Fink, G. A. 2004. A Multi-modal Dialog System for a Mobile Robot. In Int. Conf. on Spoken Language Processing, 273-276. Jeju Island (Korea): IEEE. http://citeseerx.ist.psu.edu/ viewdoc/summary? doi=10.1.1.59.9237
39. Sofge, D., Trafton, J. G., Cassimatis, N., Perzanowski, D., Magdalena Bugajska, William Adams, and Alan Schultz. Human-robot collaboration and cognition with an autonomous mobile robot. Artificial Intelligence.
40. Takahashi, T., Nakadai, K., Komatani, K., Ogata, T., and Okuno, H. G. 2010. Improvement in listening capability for humanoid robot HRP-2. In Robotics and Automation (ICRA), 2010 IEEE International Conference on, 470-475. Anchorage: IEEE. doi:10.1109/ROBOT.2010.5509830. http://ieeexplore.ieee.org/xpl/ freeabs-all.jsp?arnumber=5509830
41. Tam, Y., Mizoguchi AI, H., Sasaki, Y., and Kagami, S. Three Ring Microphone Array for 3D Sound Localization and Separation for Mobile Robot Audition.
42. Tamai, Y., Sasaki, Y., Kagami, S., and Mizoguchi, H. 2005. Three Ring Microphone Array for 3D Sound Localization and Separation for Mobile Robot Audition. In IEEE/RSJ International Conference on Intelligent Robots and Systems, 903-908. Edmonton (Canada): IEEE. doi:10.1109/IROS.2005.1545095. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1545095 (Pubitemid 43896422)
43. Tanaka, N., Ogawa, T., Akagiri, K., and Kobayashi, T. 2010. DEVELOPMENT OF ZONAL BEAMFORMER AND ITS APPLICATION TO ROBOT AUDITION. In Signal Processing, 1:1529-1533. http://www.eurasip.org/Proceedings/Eusipco/Eusipco2010/Contents/ papers/1569292345.pdf
44. Valin, J.-M., Rouat, J., and Michaud, F. 2004. Enhanced robot audition based on microphone array source separation with post-filter. In 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2123-2128. Sendai (Japan): IEEE. doi:10.1109/IROS.2004.1389723. http://ieeexplore.ieee.org/ xpl/freeabs-all.jsp?arnumber=1389723 (Pubitemid 40275913)
45. -. Enhanced robot audition based on microphone array source separation with post-filter. 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No. 04CH37566). IEEE. doi:10.1109/IROS.2004. 1389723. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1389723.
46. Valverde-Albacete, F. J. and Pardo, J. M. 1996. A multi-level lexical-semantics based language model design for guided integrated continuous speech recognition. In Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP'96, 224-227. Philadelphia (USA): IEEE. doi:10.1109/ICSLP.1996.607082. http://ieeexplore.ieee.org/xpl/freeabs-all.jsp? arnumber=607082
47. Walker, D. E. 1976. Speech Understanding Through Syntactic and Semantic Analysis. IEEE Transactions on Computers C-25, no. 4(April):432-439. doi:10.1109/TC.1976.1674625. http://ieeexplore.ieee.org/xpl/freeabs-all.jsp? arnumber=1674625
48. Wallis, P. 2010. A robot in the kitchen (July 15):25-30. http://portal.acm.org/citation.cfm?id=1870559.1870564
49. Yoshida, T., Kazuhiro, N., and Okuno, H. G. 2009. Automatic speech recognition improved by two-layered audio-visual integration for robot audition. In 2009 9th IEEE-RAS International Conference on Humanoid Robots, 604-609. Paris (France): IEEE, December. doi:10.1109/ICHR.2009.5379586. http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5379586
50. Yu, X., Zhou, F., Zhang, F., and Yang, B. 2009. Intelligent Decision Support System Based on Natural Language Understanding. In 2009 International Conference on Management and Service Science, 1-4. Beijing (China): IEEE, September. doi:10.1109/ICMSS.2009.5302806. http://ieeexplore.ieee.org/xpl/ freeabs-all.jsp?arnumber=5302806