Assessing the usability of a dialogue management system designed in the framework of a rapid dialogue prototyping methodology

Acta Acustica united with Acustica

Volumn 90, Issue 6, 2004, Pages 1096-1111

  Rajman, Martin ^a   Bui, Trung H ^a   Rajman, Andréa ^a   Seydoux, Florian ^a   Trutnev, Alex ^a   Quarteroni, Silvia ^a  

^a EPFL   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER SIMULATION; CORRELATION METHODS; ITERATIVE METHODS; MATHEMATICAL MODELS; PROJECT MANAGEMENT; RAPID PROTOTYPING; SERVERS; USER INTERFACES;

DIALOGUE MANAGEMENT SYSTEM; EXTERNAL FIELD TESTS; RAPID DIALOGUE PROTOTYPING METHODOLOGY (RDPM); TASK MODELS;

SPEECH COMMUNICATION;

EID: 11244294054     PISSN: 16101928     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (35)

1. M. F. McTear: Spoken Dialogue Technology: Enabling the Conversational User Interface. ACM Computing Surveys 34 (2002) 90-169.
2. R. Catizone, A. Setzer, Y. Wilks: State of the Art in Dialogue Management. - In: Deliverable D5.1 of COMIC Project, www.hcrc.ed.ac.uk/comic/documents/. 2002.
3. G. E. Churcher, E. S. Atwell, C. Souter: Dialogue Management Systems: a Survey and Overview. Tech. Rept. University of Leeds, Leeds, UK, 1997.
4. M. Rajman, A. Rajman, F. Seydoux, A. Trutnev: Assessing the usability of a dialogue management system designed in the framework of a rapid dialogue prototyping methodology. First ISCA Tutorial Research Workshop on Auditory Quality of Systems, Akademie Mont-Cenis (April 2003).
5. N. Fraser, N. Gilbert: Simulating Speech Systems. Computer Speech and Language 5 (1991) 81-89.
6. R. van Kommer, M. Rajman, H. Bourlard: Heading Towards Virtual-Commerce Portals. Comtec 9 (2000) 10-13.
7. D. Jurafsky, C. Wooters, G. Tajchman, J. Segal, E. Fosler, N. Morgan: The Berkeley Restaurant Project. Proceedings of the International Conference on Spoken Language Processing (ICSLP), Yokohama, Japan, 1994, 2139-2142.
8. R. A. Cole, D. Novick, M. Fanty, S. Sutton, B. Hansen, D. Burnett: Rapid prototyping of spoken language systems: The year 2000 census project. Proceedings of the International Symposium on Spoken Dialogue (ISSD), Tokyo, Japan, 1993.
9. E. Bilange: Dialogue personne-machine, modélisation et réalisation informatique. Langue, Raisonnement, Calcul, Hermès, Paris, France, 1992.
10. N. Dahlbäck, A. Jönsson, L. Ahrenberg: Wizard of Oz studies - why and how. Knowledge Based Systems 6 (1993) 258-266.
11. P. Geutner, F. Steffens, D. Manstetten: Design of the VICO Spoken Dialogue System: Evaluation of User Expectations by Wizard-of-Oz experiments. Third International Conference on Language Resources and Evaluation (LREC), Las Palmas, Spain, 2002.
12. O. Daly-Jones, N. Bevan, C. Thomas: Wizard-of-Oz prototyping. - In: Handbook of User-Centred Design, www.ejeisa.com/nectar/inuse/6.2/3-3.htm. 1999.
13. S. Boyce, A. Gorin: User Interface Issues for Natural Spoken Dialog Systems. Proceedings of the International Symposium on Spoken Dialogue (ISSD), Philadelphia, USA, October 1996, 65-68.
14. C. Dudda: Evaluierung eines natürlichen Dialogssystems für Restaurantauskünfte. 2001. Diploma Thesis (unpublished), Institut für Kommunikationsakustik, Ruhr-Universität, Bochum, Deutschland.
15. M. Rajman, A. Rozenknop, A. Trutnev: Speech recognition simulation and its application for Wizard of Oz experiments. Fourth International Conference on Language Resources and Evaluation (LREC), Lisbon, Portugal, 2004.
16. R. A. Cole, J. Mariani, H. Uzkoreit, A. Zaewe, V. Zue: Survey of the state of the art in human language technology. Cambridge University Press, 1996, Ch. Chapter 13. Evaluation.
17. M. Walker, C. Kamm, J. Boland: Developing and Testing General Models of Spoken Dialogue Systems Performance. Second International Conference on Language Resources and Evaluation (LREC), Athens, Greece, jun 2000.
18. D. Maulsby, S. Greenberg, R. Mander: Prototyping an Intelligent Agent through Wizard of Oz. Proceedings of the Conference on Human Factors in Computing Systems (INTERCHI), Amsterdam, Netherlands, 1993, ACM Press, 277-284.
19. J. Nogier: Utilisabilité du logiciel-Test d'utilisabilité. - In: Conseil en utilisabilité web & logiciel, http://www.usabilis.com/pratique/test.htm.
20. J. Kirakowski: Questionnaire in Usability Engineering. A List of Frequently Asked Questions (3rd. Ed.). - In: http://www.ucc.ie/hfrg/resources/qfaq1.html.
21. E. Hartikainen, P. Boda: Results and Observations from the Evaluation of a Telephony-Based Conversational Information Retrieval System. First ISCA Tutorial and Research Workshop on Auditory Quality of Systems, Akademie Mont-Cenis, Germany, 2003, 134-139.
22. K. S. Hone, G. Robert: Towards a Toll for the Subjective Assessment of Speech System Interface. Natural Language Engineering 6 (2000) 287-303.
23. J. Gauvain, S. Bennacef, L. Devillers, L. Lamel, S. Rosset: Spoken Language Component of the MASK Kiosk. Human Comfort and Security of Information Systems (1997) 93-103. Springer.
24. L. Dybkjaer, N. O. Bernsen: D3.8a DISC Dialogue Engineering Best Practice Methodology. - In: DISC Deliverables, http://www.disc2.dk. 1999.
25. W. Minker: Evaluation Methodologies for Interactive Speech Systems. Proceedings of the First International Conference on Language Ressources and Evaluation (LREC), Granada, Spain, May 1998, 199-206.
26. F. Néel, G. Chollet, L. Lamel, W. Minker, A. Constantinescu: Reconnaissance et compréhension de la Parole: Évaluation et Applications. - In: Fondements et Perspectives en Traitement automatique de la parole. AUPELF-UREF, 1998.
27. M. Walker, C. Kamm, D. Litman: Towards Developing General Models of Usability with PARADISE. Natural Language Engineering 6 (2000) 363-377.
28. L. Lamel: Spoken language dialog system development and evaluation at LIMSI. Proceedings of the International Symposium on Spoken Dialogue (ISSD), Sydney, Australia, 1998, 9-17.
29. M. Rajman, A. Rajman: Le système InfoVox: méthodologie d'évaluation. Tech. Rept. EPFL, Lausanne, Switzerland, 2004.
30. N. Berthier: Les techniques d'enquêtes en sciences sociales. Méthode et exercices corrigés. Armand Colin/HER, Paris, France, 2000.
31. R. Ghiglione, B. Matalon: Les enquêtes sociologiques. Théories et pratique. Armand Colin, Paris, France, 1978.
32. A. Davison: Statistical models. Cambridge University Press, Cambridge, UK, 2004.
33. H. Keselman, A. Othman, R. Wilcox, K. Fradette: The New and Improved Two-Sample t Test. Psychological Science 15(1) (2004) 47-51.
34. B. Welch: The significance of the difference between two means when the population variances are unequal. Biometrika 29 (1938) 350-362.
35. S. Whittaker, M. Walker, J. Moore: Fish or Fowle: A Wizard of Oz Evaluation of Dialogue Strategies in the Restaurant Domain. Proceedings of the Third International Conference on Language Ressources and Evaluation (LREC), Las Palmas, Spain, 2002, 1602-1609.