Out-of-domain utterance detection using classification confidences of multiple topics

IEEE Transactions on Audio, Speech and Language Processing

Volumn 15, Issue 1, 2007, Pages 150-161

  Lane, Ian ^a,b   Kawahara, Tatsuya ^a,c   Matsui, Tomoko ^a,d   Nakamura, Satoshi ^a  

^a ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

^b KYOTO UNIVERSITY   (Japan)

^c KYOTO UNIVERSITY   (Japan)

^d INSTITUTE OF STATISTICAL MATHEMATICS   (Japan)

Author keywords

Confidence measures; Out of domain (OOD) utterance detection; Speech recognition; Spoken language understanding; Topic classification; Topic clustering

Indexed keywords

CONFIDENCE MEASURES; OUT-OF-DOMAIN (OOD) UTTERANCE DETECTION; SPOKEN LANGUAGE UNDERSTANDING; TOPIC CLASSIFICATION; TOPIC CLUSTERING;

ERROR DETECTION; FACE RECOGNITION; LINGUISTICS; SPEECH ANALYSIS;

SPEECH RECOGNITION;

EID: 64149119118     PISSN: 15587916     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASL.2006.876727     Document Type: Article

References (33)

1. V. Zue, S. Seneff, J. Glass, J. Polifroni, C. Pao, T. Hazen, and L. Hetherington, "JUPITER: A telephone-based conversational interface for weather information," IEEE Trans. Speech Audio Process., vol. 8, no. 1, pp. 1-8, Jan. 2000.
2. S. Seneff and J. Polifroni, "A new restaurant guide conversational system: Issues in rapid prototyping for specialized domains," in Proc. ICSLP, 1996, vol. 2, pp. 665-668.
3. J. Glass, G. Flammia, D. Goodine, M. Phillips, J. Polifroni, S. Sakai, S. Seneff, and V. Zue, "Multilingual spoken-language understanding in the MIT Voyager system," Speech Commun., vol. 17, pp. 1-18, 1995.
4. M.Walker, J. Aberdeen, J. Boland, E. Bratt, J. Garofolo, L. Hirschman, A. Le, S. Lee, S. Narayanan, K. Papineni, B. Pellom, J. Polifroni, A. Potamianos, P. Prabhu, A. Rudnicky, G. Sanders, S. Seneff, D. Stallard, and S. Whittaker, "DARPA Communicator dialog travel planning systems: The June 2000 data collection," in Proc. Eurospeech, 2001, pp. 1371-1374.
5. A. Gorin, G. Riccardi, and J. Wright, "How may I help you?," Speech Commun., vol. 23, pp. 113-127, 1997.
6. P. Durston, M. Farrell, D. Attwater, J. Allen, H.-K. J. Kuo, M. Afify, E. Fosler-Lussier, and C.-H. Lee, "OASIS natural language call steering trial," in Proc. Eurospeech, 2001, vol. 2, pp. 1323-1326.
7. C.-H. Lee, B. Carpenter,W. Chou, J. Chu-Carroll, W. Reichl, A. Saad, and Q. Zhou, "On natural language call routing," Speech Commun., vol. 31, no. 4, pp. 309-320, 2000.
8. E. W. Wahlster, VERBMOBIL: Foundations of Speech-to-Speech Translation. Berlin, Germany: Springer, 2000.
9. A. Lavie, F. Metze, R. Cattoni, E. Costantini, S. Burger, D. Gates, C. Langley, K. Laskowski, L. Levin, K. Peterson, T. Schultz, A. Waibel, D.Wallace, J. McDonough, H. Soltau, G. Lazzari, N. Mana, F. Pianesi, E. Pianta, L. Besacier, H. Blanchon, D. Vaufreydaz, and L. Taddei, "A multi-perspective evaluation of the NESPOLE! speech-to-speech translation system," in Proc. ACL, Workshop on Speech-to-Speech Translation, 2002, pp. 121-128.
10. T. Takezawa, T. Morimoto, Y. Sagisaka, N. Campbell, H. Iida, F. Sugaya, A. Yokoo, and S. Yamamoto, "A Japanese-to-English speech translation system: ATR-MATRIX," in Proc. ICSLP, 1998, pp. 957-960.
11. T. Hanzen, J. Polifroni, and S. Seneff, "Recognition confidence scoring for use in speech understanding systems," Comput. Speech Lang., vol. 16, no. 1, pp. 49-67, 2002.
12. C. Ma, M. Randolph, and J. Drish, "A support vector machines-based rejection technique for speech recognition," in Proc. ICASSP, 2001, vol. 1, pp. 381-384.
13. R. Sukkar and C. Lee, "Vocabulary independent discriminative utterance verification for non-keyword rejection in subword based speech recognition," IEEE Trans. Speech Audio Process., vol. 4, no. 6, pp. 420-429, Nov. 1996.
14. R. San-Segundo, B. Pellom, and W. Ward, "Confidence measures for dialogue management in the CU communicator system," in Proc. ICASSP, 2000, vol. 2, pp. 1237-1240.
15. K. Komatani and T. Kawahara, "Generating effective confirmation and guidance using two-level confidence measures for dialogue systems," in Proc. ICSLP, 2000, pp. 648-651.
16. S. Deerwester, S. Dumais, G. Furnas, T. Landauer, and R. Harshman, "Indexing by latent semantic analysis," J. Amer. Soc. Inform. Sci., vol. 41, pp. 391-407, 1990.
17. S. Cox and B. Shahshahani, "A comparison of some different techniques for vector based call-routing," in Proc. Eurospeech, 2001, vol. 4, pp. 2337-2340.
18. T. Joachims, "Text categorization with support vector machines," in Proc. Eur. Conf. Machine Learning, 1998, pp. 137-147.
19. P. Haffner, G. Tur, and J. Wright, "Optimizing SVMs for complex call classification," in Proc. ICASSP, 2003, vol. 1, pp. 632-635.
20. Y. Wang, A. Acero, C. Chelba, B. Frey, and L. Wong, "Combination of statistical and rule-based approaches for spoken language understanding," in Proc. ICSLP, 2002, pp. 609-612.
21. G. Tur and D. Hakkani-Tur, "Exploiting unlabeled utterances for spoken language understanding," in Proc. Eurospeech, 2003, pp. 2793-2796.
22. I. Zitouni, H.-K. J. Kuo, and C.-H. Lee, "Combination of boosting and discriminative training for natural language call steering systems," in Proc. ICASSP, 2002, vol. 1, pp. 25-28.
23. P. Liu, H. Jiang, and I. Zitouni, "Discriminative training of naive bayes classifiers for natural language call routing," in Proc. ICSLP, 2004, pp. 1589-1592.
24. P. Langley, W. Iba, and K. Thompson, "An analysis of Bayesian classifiers," in Proc. Conf. Artificial Intelligence, 1992, pp. 223-228.
25. H.-K. J. Kuo and C.-H. Lee, "Discriminative training of natural language call routers," IEEE Trans. Speech Audio Process., vol. 11, no. 1, pp. 24-35, Jan. 2003.
26. I. Lane, T. Kawahara, T. Matsui, and S. Nakamura, "Dialogue speech recognition by combining hierarchical topic classification and Language model switching," IEICE Trans. Inform. Syst., vol. E88-D, no. 3, pp. 446-454, 2005.
27. I. Lane and T. Kawahara, "Incorporating dialogue context and topic clustering in out-of-domain detection," in Proc. ICASSP, 2005, vol. 1, pp. 1045-1048.
28. S. Katagiri, C.-H. Lee, and B.-H. Juang, "New discriminative training algorithms based on the generalized probabilistic descent method," in Proc. IEEE Int. Workshop on Neural Networks for Signal Process., 1991, pp. 299-308.
29. T. Takezawa, A. Nishino, K. Takashima, T. Matsui, and G. Kikui, "An experimental system for collecting machine-translation aided dialogues," in Forum on Information Technology (Japanese), 2003, vol. 2, pp. 161-162.
30. T. Takezawa, M. Sumita, F. Sugaya, H. Yamamoto, and S. Yamamoto, "Towards a broad-coverage bilingual corpus for speech translation of travel conversations in the real world," in Proc. Int. Conf. on Langauge Resources and Evaluation, 2002, pp. 147-152.
31. I. Lane, T. Kawahara, T. Matsui, and S. Nakamura, "Out-of-domain detection based on confidence measures from multiple topic classification," in Proc. ICASSP, 2004, vol. 1, pp. 757-760.
32. A. Lee, T. Kawahara, and K. Shikano, "Julius- an open source real-time large vocabulary recognition engine," in Proc. Eurospeech, 2001, pp. 1691-1694.
33. B. Carlson, "Unsupervised topic clustering of switchboard speech messages," in Proc. ICASSP, 1996, pp. 315-318.