DARPA communicator evaluation: Progress from 2000 to 2001

7th International Conference on Spoken Language Processing, ICSLP 2002

Volumn , Issue , 2002, Pages 273-276

  Walker, Marilyn ^a   Rudnicky, Alex ^a   Aberdeen, John ^a   Bratt, Elizabeth Owen ^a   Garofolo, John ^a   Hastie, Helen ^a   Le, Audrey ^a   Pellom, Bryan ^a   Potamianos, Alex ^a   Passonneau, Rebecca ^a   Prasad, Rashmi ^a   Roukos, Salim ^a   Sanders, Greg ^a   Seneff, Stephanie ^a   Stallard, Dave ^a  

^a BBN TECHNOLOGIES   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPLEX TASK; CONVERSATIONAL INTERACTION; DATA COLLECTION; EVALUATION METHODOLOGIES; NOCV1; PROGRAM GOALS; SPOKEN DIALOG SYSTEMS; TRAVEL PLANNING;

EID: 85009268761     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (12)

1. J. Aberdeen. Darpa communicator logfile standard, 2000. http://fofoca.mitre.org/logstandard.
2. H. Bonneau-Maynard, L. Devillers, and S. Rosset. Predictive performance of dialog systems. In Language Resources and Evaluation Conference, 2000.
3. C. A. Kamm. User interfaces for voice applications. In Roe and Wilpon, eds, Voice Communication between Humans and Machines, pp. 422-442. National Academy Press, 1995.
4. R. Keeney and H. Raiffa. Decisions with Multiple Objectives: Preferences and Value Tradeoffs. John Wiley, 1976.
5. L. Lamel and S. Rosset. Considerations in the design and evaluation of spoken language dialog systems. In ICSLP, 2000.
6. W. J. M. Levelt and G. Kempen. Semantic and syntactic aspects of remembering sentences: A review of some recent contintental research. In A. Kennedy and A. Wilkes, editors, Studies in Long Term Memory. John Wiley, 1975.
7. J. Polifroni, L. Hirschman, S. Seneff, and V. Zue. Experiments in evaluating interactive spoken language systems. In Proc. of the DARPA Speech and NL Workshop, pp. 28-33, 1992.
8. J. D. Sachs. Recognition memory for syntactic and semantic aspects of connected discourse. PhD thesis, University of California Berkeley, 1967.
9. E. Shriberg, E. Wade, and P. Price. Human-machine problem solving using spoken language systems (SLS): Factors affecting performance and user satisfaction. In Proc. of the DARPA Speech and NL Workshop, pp. 49-54, 1992.
10. M. Walker, J. Aberdeen, J. Boland, E. Bratt, J. Garofolo, L. Hirschman, A. Le, S. Lee, S. Narayanan, K. Papineni, B. Pellom, J. Polifroni, A. Potamianos, P. Prabhu, A. Rudnicky, G. Sanders, S. Seneff, D. Stallard, and S. Whittaker. DARPA Communicator dialog travel planning systems: The June 2000 data collection. In EUROSPEECH 2001.
11. M. Walker, A. Rudnicky, R. Prasad, J. Aberdeen, E. Bratt, J. Garofolo, H. Hastie, A. Le, B. Pellom, A. Potamianos, R. Passonneau, S. Roukos, G. Sanders, S. Seneff, and D. Stallard. Darpa Communicator: Cross-system results for the 2001 evaluation. In ICSLP 2002.
12. M. A. Walker, C. A. Kamm, and D. J. Litman. Towards developing general models of usability with PARADISE. Natural Language Engineering: Special Issue on Best Practice in Spoken Dialogue Systems, 2000.