Influence of training on direct and indirect measures for the evaluation of multimodal systems

Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH

Volumn , Issue , 2009, Pages 300-303

  Seebode, Julia ^a   Schaffer, Stefan ^a   Wechsung, Ina ^a   Metze, Florian ^b  

^a TECHNISCHE UNIVERSITÄT BERLIN   (Germany)

^b Carnegie Mellon University ^*  (United States)

Author keywords

Multimodal interfaces; Usability evaluation methods

Indexed keywords

DATA SOURCE; DIRECT AND INDIRECT MEASURES; EVALUATION METHOD; INDIRECT MEASUREMENTS; MULTI-MODAL; MULTI-MODAL INTERFACES; MULTIMODAL SYSTEM; PERCEIVED QUALITY; USABILITY EVALUATION METHODS;

INTERACTIVE COMPUTER SYSTEMS; SPEECH COMMUNICATION;

USER INTERFACES;

EID: 70450162199     PISSN: None     EISSN: 19909772     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (16)

1. Kirakowski, J. & Corbett, M.: SUMI: The Software Usability Measurement Inventory, In: British Journal of Educational Technology, 24(3), pp. 210-212, 1993.
2. Hone, K. and Graham, R. (2001) Subjective assessment of speech-system interface usability, In: Proceedings of Eurospeech 2001, Volume 3, 2001.
3. Naumann, A. & Wechsung, I.: Developing Usability Methods for Multimodal Systems: The Use of Subjective and Objective Measures. In: Proceedings of the International Workshop on Meaningful Measures: Valid Useful User Experience Measurement (VUUM), pp. 8-12, 2008.
4. Annett, J.: Subjective Rating Scales: Science or Art? Ergonomics, 45, pp. 966-987, 2002.
5. Krosnick, J. A.: Survey research. Annual Review of Psychology, 50, pp. 537-567, 1999.
6. Engelbrecht, K. P., Möller, S., Schleicher, R. & Wechsung, I.: Analysis of PARADISE Models for Individual Users of a Spoken Dialog System, In: Elektronische Sprachsignalverarbeitung. Tagungsband der 19. Konferenz, DE-Frankfurt a. M., A. Lacroix (ed.), TUDpress, DE-Dresden, 2008.
7. Walker, M., Kamm, C. & Litman, D.: Towards developing general models of usability with PARADISE. In Natural Language Engineering, 6, pp. 363-377, Cambridge University Press, 2000.
8. Sturm, J. A.: On the Usability of Multimodal Interaction for Mobile Access to Information Services, PhD Thesis, 2005.
9. Wechsung I. & Naumann, A.: Established Usability Evaluation Methods for Multimodal Systems: A Comparison of Standardized Usability Questionnaires, In: Proceedings of PIT 08, pp 276-284, Heidelberg: Springer, in press.
10. Sauro, J. & Kindlund, E.: A method to standardize usability metrics into a single score. In: Proceedings of CHI 2005, pp 401 - 409, ACM Press, 2005.
11. Krämer, N. C. & Nitschke, J.: Ausgabemodalitäten im Vergleich: Verändern sie das Eingabeverhalten der Benutzer? [Output modalities in comparison: Do they change user's input behaviour?] In R. Marzi, V. Karavezyris, H.-H. Erbe & K.-P. Timpe (Hrsg.), Bedienen und Verstehen. 4. Berliner Werkstatt Mensch-Maschine-Systeme. Düsseldorf: VDI-Verlag, 2002.
12. Möller, S.: Messung und Vorhersage der Effizienz bei der Interaktion mit Sprachdialogdiensten [Measuring and predicting efficiency for the interaction with speech dialogue systems], In S. Langer & W. Scholl (Eds.), Fortschritte der Akustik - DAGA 2006. DEGA, Berlin, 2006.
13. Hornbæk, K. & Law, E. L.: Meta-analysis of correlations among usability measures, In: Proceedings of CHI 2007, pp. 617-626, ACM Press, 2007.
14. Metze, F., Wechsung, I., Schaffer, S., Seebode, J. & Möller, S.: Reliable Evaluation of Multimodal Dialogue Systems. In: Proceedings of International Conference on Human-Computer Interaction, 2009 (to appear).
15. Hassenzahl, M., Burmester, M., & Koller, F.: AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität [A questionnaire for measuring perceived hedonic and pragmatic quality]. In: J. Ziegler & G. Szwillus (Hrsg.), Mensch & Computer 2003. Interaktion in Bewegung . Stuttgart, Leipzig: B.G. Teubner, 2003.
16. Mandryk, R. L., Inkpen, K. M., & Calvert, T. W.: Using psychophysiological techniques to measure user experience with entertainment technologies, In: Behaviour & Information Technology 2006, volume 25, pp. 141-158, 2006.