Towards developing general models of usability with PARADISE

Natural Language Engineering

Volumn 6, Issue 3-4, 2000, Pages 363-378

  Walker, Marilyn ^a   Kamm, Candace ^a   Litman, Diane ^a  

^a AT AND T LABS RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84974698591     PISSN: 13513249     EISSN: 14698110     Source Type: Journal    
DOI: 10.1017/s1351324900002503     Document Type: Article

References (25)

1. Baggia, P., Castagneri, G. and Danieli, M. (1998) Field trials of the italian arise train timetable system. Interactive Voice Technology for Telecommunications Applications, IVTTA, pp. 97–102.
2. Bernsen, N. O., Dybkjaer, H. and Dybkjaer, L. (1996) Principles for the design of cooperative spoken human-machine dialogue. Int. Conference on Spoken Language Processing (ICSLP 96), pp. 729–732.
3. Brieman, L., Friedman, J. H., Olshen, R. A. and Stone, C. J. (1984) Classification and Regression Trees. Wadsworth and Brooks, Monterey, CA.
4. Danieli, M. and Gerbino, E. (1995) Metrics for evaluating dialogue strategies in a spo-ken language system. Proc. AAAI Spring Symposium on Empirical Methods in Discourse Interpretation and Generation, pp. 34–39.
5. Hirschman, L., Dahl, D. A., McKay, D. P., Norton, L. M. and Linebarger, M. C. (1990) Beyond class A: A proposal for automatic evaluation of discourse. Proc. Speech and Natural Language Workshop, pp. 109–113.
6. Hirschman, L. and Pao, C. (1993) The cost of errors in a spoken language system. Proc. Third European Conference on Speech Communication and Technology, pp. 1419–1422.
7. Jack, M., Foster, J. C. and Stentiford, F. W. (1992) Intelligent dialogues in automated telephone services. Int. Conference on Spoken Language Processing (ICSLP), pp. 715–718. Kamm, C., Litman, D. and Walker, M. A. (1998) From novice to expert: The effect of tutorials on user expertise with spoken dialogue systems. Proc. International Conference on Spoken Language Processing (ICSLP98).
8. Kamm, C., Narayanan, S., Dutton, D. and Ritenour, R. (1997) Evaluating spoken dialog systems for telecommunication services. 5th European Conference on Speech Technology and Communication, EUROSPEECH 91, pp. 2203–2206.
9. Larsen, L. B. (1999) Combining objective and subjective data in evaluation of spoken dialogues. ESC A Workshop on Interactive Dialogue in Multi-Modal Systems, pp. 89–92.
10. Litman, D. J. and Pan, S. (1999) Empirically Evaluating an Adaptable Spoken Dialogue System. Proc. 1th International Conference on User Modeling.
11. Litman, D. J., Pan, S. and Walker, M. A. (1998) Evaluating Response Strategies in a Web-Based Spoken Dialogue Agent. Proc. ACL/CODING 98: 36th Annual Meeting of the Association of Computational Linguistics, pp. 780–787.
12. Litman, D. J., Walker, M. A. and Kearns, M. J. (1999) Automatic detection of poor speech recognition at the dialogue level. Proc. 37th Annual Meeting of the Association of Computational Linguistics 3ACL99), pp. 309–316.
13. Love, S., Dutton, R. T., Foster, J. C., Jack, M. A. and Stentiford, F. W. M. (1994) Identifying salient usability attributes for automated telephone services. Int. Conference on Spoken Language Processing (ICSLP), pp. 1307–1310.
14. Pallett, D. S. (1985) Performance assessment of automatic speech recognizers. J. Res. Nat. Bureau of Standards, 90: 371–387.
15. Polifroni, J., Seneff, S., Glass, J. and Hazen, T. (1998) Evaluation methodology for a telephone-based conversational system. Proc. 1st International Conference on Language Resources and Evaluation, Granada, Spain, pp. 42–50.
16. Ralston, J. V., Pisoni, D. B. and Mullennix, J. W. (1995) Perception and comprehension of speech. In: A. K. Syrdal, R. W. Bennet and S. L. Greenspan, editors, Applied Speech Technology, pp. 233–287. CRC Press.
17. Shriberg, E., Wade, E. and Price, P. (1992) Human-machine problem solving using spoken language systems (SLS): Factors affecting performance and user satisfaction. Proc. DARPA Speech and NL Workshop, pp. 49–54.
18. Sparck-Jones, K. and Galliers, J. R. (1996) Evaluating Natural Language Processing Systems. Springer-Verlag.
19. Walker, M., Fromer, J., Di Fabbrizio, G., Mestel, C. and Hindle, D. (1998a) What can I say?: Evaluating a spoken language interface to email. Proc. Conference on Computer Human Interaction (CHI 98), pp. 582–589.
20. Walker, M., Hindle, D., Fromer, J., Di Fabbrizio, G. and Mestel, C. (1997a) Evaluating competing agent strategies for a voice email agent. Proc. European Conference on Speech Communication and Technology, EUROSPEECH97.
21. Walker, M., Kamm, C. and Boland, J. (2000) Developing and testing general models of spoken dialogue system performance. Proc. Language Resources and Evaluation Conference, LREC-2000.
22. Walker, M. A. (2000) An application of reinforcement learning to dialogue strategy selection in a spoken dialogue system for email. J. Artificial Intelligence Research, 12: 387–416.
23. Walker, M. A., Fromer, J. C. and Narayanan, S. (1998b) Learning optimal dialogue strategies: A case study of a spoken dialogue agent for email. Proc. 36th Annual Meeting of the Association of Computational Linguistics, COLING/ACL 98, pp. 1345–1352.
24. Walker, M. A., Hirschman, L. and Aberdeen, J. (2000) Evaluation for DARPA communicator spoken dialogue systems. In Proc. 2nd International Conference on Language Resources and Evaluation, Athens, Greece.
25. Walker, M. A., Litman, D., Kamm, C. A. and Abella, A. (1997b) PARADISE: A gen-eral framework for evaluating spoken dialogue agents. Proc. 35th Annual Meeting of the Association of Computational Linguistics, ACL/EACL 91, pp. 271–280.