Attuning speech-enabled interfaces to user and context for inclusive design: Technology, methodology and practice

Universal Access in the Information Society

Volumn 8, Issue 2, 2009, Pages 109-122

  Neerincx, Mark A ^a,b   Cremers, Anita H M ^a   Kessens, Judith M ^a   van Leeuwen, David A ^a   Truong, Khiet P ^a  

^a TNO   (Netherlands)

^b DELFT UNIVERSITY OF TECHNOLOGY   (Netherlands)

Author keywords

Multimodal interaction; Speech technology; Universal access; User experience engineering

Indexed keywords

BEST PRACTICE; CONTEXT DEPENDENT; DESIGN RATIONALE; INCLUSIVE DESIGN; INFORMATION AND COMMUNICATION TECHNOLOGIES; INTERFACE PROTOTYPE; MULTIMODAL INTERACTION; MULTIMODAL USER INTERFACE; OLDER ADULTS; SPEECH TECHNOLOGY; TEST PROCESS; UNIVERSAL ACCESS; USER EXPERIENCE ENGINEERING; USER GROUPS; USER NEED;

AUDITION; INTERACTIVE COMPUTER SYSTEMS; TECHNOLOGY; USER INTERFACES;

SPEECH COMMUNICATION;

EID: 67349277164     PISSN: 16155289     EISSN: 16155297     Source Type: Journal    
DOI: 10.1007/s10209-008-0136-x     Document Type: Article

References (74)

1. Ang, J., Dhillon, R., Krupski, A., Shriberg, E., Stolcke, A.: Prosody-based automatic detection of annoyance and frustration in Human-Computer Dialog. In: Proceedings of the ICSLP International Conference on Spoken Language Processing, Denver, Colorado, September 2002, pp. 2037-2040
2. Banse, R., Scherer, K.R.: Acoustic profiles in vocal emotion expression. J Pers Soc Psychol 70, 614-636 (1996)
3. Batliner, A., Steidl, S., Hacker, C., Nöth, E., Niemann, H.: Tales of tuning - prototyping for automatic classification of emotional user states. In: Proceedings of Interspeech, Lisbon, Portugal, September 2005, pp. 489-492
4. Bennett, C.L., Black, A.W.: Blizzard Challenge 2006: Results. Blizzard Challenge 2006 Workshop, Pittsburgh, PA, September 2006
5. Blanson Henkemans, O.A., Neerincx, M.A., Lindenberg, J., van der Mast, C.A.P.GS: SuperAssist: supervision of patient self-care and medical adherence. In: Proceedings of the 16th Triennial Congress of the International Ergonomics Association (CD-rom), pp. 3637-3643. Elsevier, Amsterdam (2006)
6. Bradley, M.M., Lang, P.J.: Affective reactions to acoustic stimuli. Psychophysiology 37, 204-215 (2000)
7. Buisine, S., Martin, J.-C.: The effects of speech-gesture cooperation in animated agents' behavior in multimedia presentations. Interact Comput 19, 484-493 (2007)
8. Campbell, N.: A language-resources approach to emotion (2006). Corpora for the analysis of expressive speech. In: Proceedings of the 5th International Conference on Language Resources and Evaluation LREC, Genoa, Italy, May 2006
9. Campbell, N.: The JST/CREST Expressive Speech Processing project, introductory web pages at: http://feast.atr.jp
10. Carbonell, N.: AmTient multimodality: towards advancing computer accessibility and assisted living. Universal Access Informat Soc 5(1), 96-104 (2006)
11. Chan, F.Y., Khalid, H.M.: Is talking to an automated teller machine natural and fun? Ergonomics 46, 1386-1407 (2003)
12. Charpentier, F., Moulines, E.: Pitch-synchronous waveform processing techniques for text-to-speech synthesis. Proc ICSLP 2, 13-19 (1989)
13. Cowie, R., Douglas-Cowie, E., Savvidou, S., McMahon, E., Sawey, M., SchroAer, M.: FEELTRACE: an instrument for recording perceived emotion in real time. In: Proceedings of ISCA ITRW Workshop on Speech and Emotion, ISCA, Belfast, Northern Ireland, 2000, pp. 19-24
14. Coyle, D., Doherty, G., Matthews, M., Sharry, J.: Computers in talk-based mental health interventions. Interact Comput 19, 545-562 (2007)
15. Cremers, A.H.M., Neerincx, M.A.: Personalisation meets accessibility: towards the design of individual user interfaces for all. In: User-Centered Interaction Paradigms for Universal Access in the Information Society. Lecture Notes in Computer Science, pp. 119-124. Springer, Berlin (2004)
16. Czaja, S.J., Lee, C.C.: The impact of aging on access to technology. Universal Access Informat Soc 5, 341-349 (2007)
17. Davis, K.H., Biddulph, R., Balashek, S.: Automatic speech recognition of spoken digits. J Acoust Soc Am 24(6), 637-642 (1952)
18. De Silva, L.C., Miyasato, T., Nakatsu, R.: Facial emotion recognition using multi-modal information. In: Proceedings of the ICICS International Conference on Information, Communications and Signal Processing, Singapore, 9-12 September 1997, pp. 397-401
19. Douglas-Cowie, E., Devillers, L., Martin, J., Cowie, R., Savvidou, S., Abrilian, S., Cox, C.: Multimodal daFabases of everyday emotion: facing up to complexity. Proceedings of Interspeech, Lisbon, Portugal, September 2005, pp. 813-816
20. Douglas-Cowie, E., Cowie, R., Schröder, M.: A new emotion database: considerations, sources and scope. In: Proceedings of the ISCA Workshop on Speech and Emotion, ICSA, Belfast, Northern Ireland, 2000, pp. 39-44
21. Dudley, H.: The vocoder. Bell Labs Rec 17, 122-126 (1939)
22. Dutoit, T., Pagel, V., Pierret, N., Bataiile, F., van der Vrecken, O.: The MBROLA project: Towards a set of high quality speech synthesizers of use for non commercial purposes. In: Proceedings of ICSLP'96, Philadelphia, October 1996
23. Ekman, P., Friesen, W.VA: Facial action coding system: a technique for the measurement of facial movement. Consulting Psychologists Press, Palo Alto (1978)
24. ETSI EG 202 116 V 1.2.1: Human Factors (HF); Guidelines for ICT Products and Services; "Design for All". Sophia Antipolis Cedex, ETSI (2002)
25. Fiscus, J., Ajot, J., Garofolo, J.: The Rich Transcription 2007 Meeting Recognition Evaluation, The Joint Proceedings of the CLEAR 2007 and RT 2007 Evaluation Workshops, Lecture Notes in Computer Science, vol. 4625. Springer, Berlin (2007)
26. Fogg, B.J.: PersuUsive technology: using computers to change what we think and do. Morgan Kaufmann Publishers, Amsterdam (2003)
27. Foulke, E.: A survey of the acceptability of rapid speech. New Outlook Blind 60, 261-265 (1966)
28. Grimm, M., Kroschel, K., Narayanan, S.: Support vector regression for automatic recognition of spontaneous emotions in speech. In: IEEE International Conference on Acoustics, Speech and Signal Processing 2007 (ICASSP 2007), Honolulu, 2007. Publication Date: 15-20 April 2007
29. Hinckley, K.: Input technologies and techniques. In: Jacko, J.A., Sears, A. (eds.) The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, Chapter 7. Lawrence Erlbaum, Mahwah (2003)
30. Hone, K.S., Graham, R., Maguire, M.C., Baber, C., Johnson, G.I.: Speech technology for automatic teller machines: An investigation of user attitude and performance. Ergonomics 41, 962-981 (1998)
31. Janse, E.: Production and perception of fast speech. PhD Thesis, University of Utrecht, The Netherlands (2003)
32. Johnson, G.I., Coventry, L.: "You talking to me?" Exploring voice in self-service user interfaces. Int J Hum Comput Interact 13, 161-186 (2001)
33. Lange, de V.: ICat as personal assistant for diabetic children. MSc Thesis, Delft University of Technology, Delft, The Netherlands (2007)
34. Lemmetty, S.: Review of speech synthesis technology. Master's Thesis, University of Helsinki, Electrical and Communications Engineering (1999)
35. Lindenberg, J., Neerincx, M.A.: The need for a 'universal accessibility' engineering tool. In: ACM SIGCAPH Computers and the Physically Handicapped, Issue 69, pp. 14-17. ACM Press, New York (2001)
36. Lindenberg, J., Pasman, W., Kranenborg, K., Stegeman, J., Neerincx, M.A.: Improving service matching and selection in ubiquitous computing environments: A user study. Pers Ubiquit Comput 11, 59-68 (2007)
37. Looije, R., Cnossen, F., Neerincx, M.A.: Incorporating guidelines for health assistance into a socially intelligent robot. In: Proceedings of the 15th IEEE International Symposium on Robot and Human Interactive Communication (Ro-Man 2006), University of Hertfordshire, Hatfield, UK, 6-8 September 2006, pp. 515-520
38. Maguire, M.: Methods to support human-centred design. Int J Hum Comput Stud 55, 587-634 (2001)
39. Martin, A.F., Le, A.N.: Current state of language recognition: NIST 2005 evaluation results. In: Proceedings of Odyssey 2006 Speaker and Language Recognition Workshop, San Juan (2006)
40. McGilloway, S.,Cowie, R., Douglas-Cowie, E., Gielen, S., Westerdijk, M., Stroeve, S.: Approaching automatic recognition of eAotion from voice: a rough benchmark. In: Proceedings of the ISCA Workshop on Speech and Emotion, ICSA, Belfast, Northern Ireland, 2000, pp. 207-212
41. Merkx, P.A.B., Truong, K.P., Neerincx, M.A.: Inducing and measuring emotion through a multiplayer first-person shooter computer game. In: van den Herik, H.J., Uiterwijk, J.W.H.M., Winands, M.H.M., Schadd, M.P.D. (eds.) Proceedings of the Computer Games Workshop 2007, Amsterdam, The Netherlands, MICC Technical Report Series, ISSN 0922-8721, number 07-06, pp. 231-242 (2007)
42. Müller, C. (ed.): Speaker Classification I & II. Lecture Notes in Computer Science, vols. 4343 and 4441, ISBN 978-3-540-74186-2. Springer, Heidelberg (2007)
43. Neerincx, M.A., Lindenberg, J.: Situated cognitive engineering for complex task environments. In: Schraagen, J.M.C., Militello, L., Ormerod, T., Lipshitz, R. (eds.) Natural Decision Making & Macrocognition, pp. 373-390. Ashgate Publishing Limited, Aldershot (2008)
44. Neerincx, M.A., Lindenberg, J., Grootjen, M.: Accessibility on the Job: Cognitive Capacity Driven Personalization. HCII 2005, vol. 7: Universal Access in HCI: Exploring New Interaction Environments (10 pages). MIRA Digital Publishing, St Louis (2005)
45. Nwe, T.L., Foo, S.W., De Silva, L.C.: Speech emotion recognition using hidden Markov models. Speech Commun 41, 603-623 (2003)
46. Ordelman, R.J.F., de Jong, F., Huijbregts, M.A.H., van Leeuwen, D.A.: Robust audio indexing for Dutch spoken-word collections. In: Proceedings of the XVIth International Conference of the Association for History and Computing (AHC2005), Amsterdam, The Netherlands, 14-17 September 2005, pp. 215-223
47. Oviatt, S.: User-centered modeling and evaluation of multimodal interfaces. Proc IEEE 91(9), 1457-1468 (2003)
48. Pallett, D.: A look at NIST's bePchmark ASR tests: past, present, and future. In: Proceedings of ASRU, St Thomas, US Virgin Islands, 2003, pp. 483-488
49. Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)
50. Russell, J.A.: A circumplex model of affect. J Pers Soc Psychol 39(6), 1161-1178 (1980)
51. Schlossberg, H.: Three dimensions of emotion. Psychol Rev 61, 81-88 (1954)
52. Schröder, M., Cowie, R., Douglas-Cowie, E., Westerdijk, M., Gielen, S.: Acoustic correlates of emotion dimensions in view of speech synthesis. In: Proceedings of Eurospeech 2001, Aalborg, September 2001, pp. 87-90
53. Schröder, M.: SpeeAh and emotion research: an overview of research frameworks and a dimensional approach to emotional speech synthesis. PhD Thesis, PHONUS 7, Research Report of the Institute of Phonetics, Saarland University (2004)
54. Schuller, B., Müller, R., Lang, M., Rigoll, G.: Speaker independent emotion recognition by early fusion of acoustic and linguistic features within ensembles. In: Proceedings of Interspeech, Lisbon, Portugal, September 2005, pp. 805-809
55. Scopelliti, M., Giuliani, M.V., Fornara, F.: Robots in a domestic setting: A psychological approach. Universal Access Informat Soc 4, 146-155 (2005)
56. Sim, G., MacFarlMne, S., Read, J.: All work and no play: measuring fun, usability, and learning in software for children. Comput Educ 46, 235-248 (2006)
57. Stary, C., Stephanidis, C. (eds.): User-Centered Interaction Paradigms for Universal Access in the Information Society. Lecture Notes in Computer Science. Springer, Berlin (2004)
58. Steiniger, S., Schiel, F., Dioubina, O., Raubold, S.: Development of user-state conventions for the multimodal corpus in SmartKom. In: Proceedings of the Workshop on Multimodal Resources and Multimodal Systems Evaluation, Las Palmas, Spain, 2002, pp. 33-37
59. Stephanidis, C. (edC): User interfaces for all: concepts, methods, and tools. Lawrence Erlbaum, London (2001)
60. Stephanidis, C.: Adaptive techniques for universal access. User Model User-Adapt Interact 11, 159-179 (2001)
61. Streefkerk, J.W., van Esch-Bussemakers, M.P., Neerincx, M.A., Looije, R.: Evaluating Context-Aware Mobile User Interfaces. In: Lumsden, J. (ed.) Handbook of Research on User Interface Design and Evaluation for Mobile Technology, Chapter XLV, pp. 756-776. IGI Global, Hershey (2008)
62. Truong, K.P, van Leeuwen, D.A., Neerincx, M.A.: Unobtrusive multimodal emotion detection in adaptive interfaces: Speech and facial expressions. In: Schmorrow, D.D., Reeves, L.M. (eds.) Foundations of Augmented Cognition, 3rd edn., LNAI 4565 proceedings, pp. 354-363, ISBN 978-3-540-73215-0 (2007)
63. Van der Pijl, D.J., Cremers, A.H.M., Soede, M.: Personalized PDA accessibility for intellectually disabled persons: Concept guidelines based on the development of an Electronic Travel Companion. In: HCII 2005, vol. 7: Universal Access in HCI: Exploring New Interaction Environments. MIRA Digital Publishing, St Louis (2005)
64. Van Erp, J.B.F., Kooi, F.L., Bronkhorst, A.W., van Leeuwen, D.L., van Esch, M.P., van WAjngaarden, S.J.: Multimodal interfaces: a framework based on modality appropriateness. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, October 2006, pp. 1542-1546
65. van Leeuwen, D.A.: Consumer off-the-shelf (COTS) product and service evaluation. In: Gibbon, D., Mertins, I., Moore, R. (eds.) Handbook of Multimodal and Spoken Dialogue Systems, Resources, Terminology and Product Evaluation, Chapter 3. Kluwer, Boston (2000)
66. van Leeuwen, D.A., Martin, A.F., Przybocki, M.A., Bouten, J.S.: NIST and TNO-NFI evaluations of automatic speaker recognition, Computer. Speech Language 20, 128-158 (2006)
67. Van Wijngaarden, S.J., Smeele, P.M.T., Steeneken, H.J.M.: A new method for testing communication efficiency and user acceptability of speech communication channels. In: Proceedings of Eurospeech 2001, Aalborg, September 2001, pp. 1675-1678
68. Ververidis, D., Kotropolos, C.: Automatic speech classification to five emotional states based on gender information. In: Proceedings of Eusipco, Vienna, Austria, 6-10 September 2004, pp. 341-344
69. Vidrascu, L., Devillers, L.: Detection of real-life emotions in call centers. In: Proceedings of Interspeech, Lisbon, Portugal, September 2005, pp. 1841-1844
70. Williams, U., SSevens, K.N.: Emotions and speech: some acoustical correlates. JASA 52, 1238-1250 (1972)
71. Wilting, J., Krahmer, E., Swerts, M.: Real vs. acted emotional speech. In: Proceedings of Interspeech, Pittsburgh, September 2006
72. World Wide Web Consortium. Web Content Accessibility Guidelines 1.0; User Agent Accessibility Guidelines 1.0. http://www.w3.org
73. Yildirim, S., Bulut, M., Lee, C.M., Kazemzadeh, A.: An acoustic study of emotions expressed in speech. In: Proceedings of ICSLP, Jeju Island, Korea, 4-8 October 2004, pp. 2193-2196
74. Zekveld, A., Kramer, S.E., Kessens, J.M., Vlaming, M.S.M.G., Houtgast, T.: The Benefit obtained from visually displayed text from an automatic speech recogniser during listening to speech presented in noise. Ear Hear (2008, accepted for publication)