ECHOES: An intelligent serious game for fostering social communication in children with autism

Information Sciences

Volumn 264, Issue , 2014, Pages 41-60

  Bernardini, Sara ^a   Porayska Pomsta, Kaśka ^b   Smith, Tim J ^c  

^a KING'S COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY COLLEGE LONDON   (United Kingdom)

^c UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Artificial intelligence planning; Autism; Autonomous intelligent agent; Pedagogical agent; Social communication; Virtual social partner

Indexed keywords

EID: 84894470129     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2013.10.027     Document Type: Article

References (80)

1. B. Abirached, Y. Zhang, J. Aggarwal, B. Tamersoy, T. Fernandes, J. Miranda, V. Orvalho, Improving communication skills of children with asds through interaction with virtual characters, in: 2011 IEEE 1st International Conference on Serious Games and Applications for Health (SeGAH), 2011, pp. 1-4.
2. A. Alcorn, H. Pain, G. Rajendran, T. Smith, O. Lemon, K. Porayska-Pomsta, M.E. Foster, K. Avramides, C. Frauenberger, and S. Bernardini Social communication between virtual characters and children with autism Proceedings of the 15th International Conference on Artificial Intelligence in Education AIED'11 2011 Springer-Verlag Berlin/Heidelberg 7 14
3. American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorders, fourth ed., Text Revision (DSM-IV-TR), 2000.
4. A. Anwar, M. Rahman, S. Ferdous, S. Anik, S. Ahmed, A computer game based approach for increasing fluency in the speech of the autistic children, in: 11th IEEE International Conference on Advanced Learning Technologies (ICALT), 2011, pp. 17-18.
5. K. Avramides, S. Bernardini, M.E. Foster, C. Frauenberger, L. Kossyvaki, and M. Mademtzi State-of-the-art in tel to support social communication skill development in children with autism: a multi-disciplinary review International Journal of Technology Enhanced Learning (IJTEL) 4 5/6 2012 359 372
6. W. Bainbridge, J. Hart, E. Kim, and B. Scassellati The benefits of interactions with physically present robots over video-displayed agents International Journal of Social Robotics 3 1 2010 41 52
7. E. Barakova, G. van Wanrooij, R. van Limpt, and M. Menting Using an emergent system concept in designing interactive games for autistic children Proceedings of the 6th International Conference on Interaction Design and Children IDC '07 2007 ACM New York, NY, USA 73 76
8. S. Baron-Cohen Mindblindness 1995 MIT Press Cambridge MA
9. A. Battocchi, F. Pianesi, D. Tomasini, M. Zancanaro, G. Esposito, P. Venuti, A. Ben Sasson, E. Gal, and P.L. Weiss Collaborative puzzle game: a tabletop interactive game for fostering collaboration in children with autism spectrum disorders (asd) Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces ITS '09 2009 ACM New York, NY, USA 197 204
10. R. Beaumont, and K. Sofronoff A multi-component social skills intervention for children with asperger syndrome: the junior detective training program Journal of Child Psychology and Psychiatry 49 2008 743 753
11. S. Bernardini, K. Porayska-Pomsta, Planning-based social partners for children with autism, in: Proc. of the Twenty Third International Conference on Automated Planning and Scheduling (ICAPS-13), 2013.
12. S. Bernardini, K. Porayska-Pomsta, H. Sampath, Designing an intelligent partner for social communication in autism, in: Proc. of the Ninth Annual AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment (AIIDE-13), 2013.
13. S. Bernardini, K. Porayska-Pomsta, J.T. Smith, K. Avramides, Building autonomous social partners for autistic children, in: Proc. of the 12th International Conference on Intelligent Virtual Agents (IVA-12), Lecture Notes in Computer Science, vol. 7502, 2012, pp. 46-52.
14. A. Bosseler, and D. Massaro Development and evaluation of a computer-animated tutor for vocabulary and language learning in children with autism Journal of Autism and Developmental Disorders 33 6 2003 653 672
15. S. Cobb Virtual environments supporting learning and communication in special needs education Topics in Language Disorders 27 3 2007 211 225
16. K. Dautenhahn, and I. Werry Towards interactive robots in autism therapy: background, motivation and challenges Pragmatics and Cognition 12 1 2004 1 35
17. M. Davis, N. Otero, K. Dautenhahn, C. Nehaniv, S. Powell, Creating a software to promote understanding about narrative in children with autism: reflecting on the design of feedback and opportunities to reason, in: IEEE 6th International Conference on Development and Learning, 2007 (ICDL 2007), July 2007, pp. 64 -69.
18. G. Dawson, S. Rogers, J. Munson, M. Smith, J. Winter, J. Greenson, A. Donaldson, and J. Varley Randomized, controlled trial of an intervention for toddlers with autism: the early start denver model Pediatrics 125 1 2010 17 23
19. S. De Freitas, Learning in immersive worlds, Tech. rep., Joint Information Systems Committee, Bristol, 2006. < http://www.jisc.ac.uk/ elioutcomes.html > (accessed 20.10.12).
20. J. Dias, and A. Paiva Feeling and reasoning: a computational model for emotional characters Progress in Artificial Intelligence Lecture Notes in Computer Science vol. 3808 2005 Springer Berlin/Heidelberg 127 140
21. S.L. Finkelstein, A. Nickel, L. Harrison, E.A. Suma, T. Barnes, cMotion: a new game design to teach emotion recognition and programming logic to children using virtual humans, in: Proceedings of the 2009 IEEE Virtual Reality Conference, 2009, pp. 249-250.
22. C. Frauenberger, J. Good, and W. Keay-Bright Phenomenology, a framework for participatory design Proceedings of the 11th Biennial Participatory Design Conference PDC '10 2010 ACM New York, NY, USA 187 190
23. M. Frutos, I. Bustos, B. Zapirain, A. Zorrilla, Computer game to learn and enhance speech problems for children with autism, in: 16th International Conference on Computer Games (CGAMES), 2011, pp. 209-216.
24. O. Golan, E. Ashwin, Y. Granader, S. McClintock, K. Day, V. Leggett, and S. Baron-Cohen A multi-component social skills intervention for children with asperger syndrome: the junior detective training program Journal of Autism and Developmental Disorders 40 2010 269 279
25. M. Goodwin Enhancing and accelerating the pace of autism research and treatment: the promise of developing innovative technology Focus on Autism and Other Developmental Disabilities 23 2 2008 125 128
26. O. Grynszpan, J.-C. Martin, and J. Nadel Multimedia interfaces for users with high functioning autism: an empirical investigation International Journal on Human-Computer Studies 66 8 2008 628 639
27. H. Hoffman Virtual reality therapy Scientific American Megazine 2004 58 65
28. I.M. Hopkins, M.W. Gower, T.A. Perez, D.S. Smith, and F.R. Amthor et al. Avatar assistant: improving social skills in students with an asd through a computer-based intervention Journal of Autism and Developmental Disorders 41 11 2011 1543 1555
29. M.E. Hoque, J.K. Lane, R.E. Kaliouby, M.S. Goodwin, R.W. Picard, Exploring speech therapy games with children on the autism spectrum, in: Proceedings of the 10th Annual Conference of the International Speech Communication Association, INTERSPEECH 2009, 2009, pp. 1455-1458.
30. G. Iarocci, and J. McDonald Sensory integration and the perceptual experience of persons with autism Journal of Psychiatric Research 3 2006 181 197
31. C. Kidd, C. Breazeal, Effect of a robot on user perceptions, in: Proceedings. 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 4, 2004, pp. 3559-3564.
32. H. Kozima, M. Michalowski, and C. Nakagawa Keepon: a playful robot for research, therapy, and entertainment International Journal of Social Robotics 1 1 2009 3 18
33. M. Kriegel, R. Aylett, Emergent narrative as a novel framework for massively collaborative authoring, in: Proceedings of the 8th International Conference on Intelligent Virtual Agents, 2008, pp. 73-80.
34. M.Y. Lim, J. Dias, R. Aylett, and A. Paiva Creating adaptive affective autonomous NPCs Journal of Autonomous Agents and Multi-Agent Systems 24 2 2012 287 311
35. O.I. Lovaas Behavioral treatment and normal educational and intellectual functioning in young autistic children Journal of Consulting and Clinical Psychology 55 1 1987 3 9
36. D. Massaro Embodied agents in language learning for children with language challenges Proceedings of the 10th International Conference on Computers Helping People with Special Needs ICCHP'06 2006 Springer-Verlag Berlin/Heidelberg 809 816
37. ASC-Inclusion, 2012. < http://asc-inclusion.eu/ > (accessed 20.10.12).
38. Brigadoon, John Lester, 2012. < http://braintalk.blogs.com/brigadoon > (accessed 20.10.12).
39. Cospatial, 2012. < http://cospatial.fbk.eu/ > (accessed 20.10.12).
40. ECHOES, 2013. < http://echoes2.org/ > (accessed 22.07.13).
41. Medical Research Council, Autism - Research Review, 2001. < http://www.mrc.ac.uk/Utilities/Documentrecord/index.htm?d=MRC002394 > (accessed 20.10.12).
42. National Autistic Society, What is autism? 2012. < http://www.autism.org.uk/about-autism.aspx > (accessed 20.10.12).
43. Second Life, 2012. < http://secondlife.com/ > (accessed 20.10.12).
44. Shape, 2012. < http://www.birmingham.ac.uk/research/activity/ education/shape > (accessed 15.07.13).
45. SHARE-IT, 2013. < http://shareitproject.wordpress.com/ > (accessed 15.07.13).
46. TurboSquid, Andy, 2012. < http://www.turbosquid.com/3d-models/human- kid-child-3d-model/544104 > (accessed 20.10.12).
47. M. Milne, M. Luerssen, T. Lewis, R. Leibbrandt, D. Powers, Development of a virtual agent based social tutor for children with autism spectrum disorders, in: Proc. of the International Joint Conference on Neural Networks, 2010, pp. 1-9.
48. P. Mitchell, S. Parsons, and A. Leonard Using virtual environments for teaching social understanding to 6 adolescents with autistic spectrum disorders Journal of Autism and Developmental Disorders 37 3 2007 589 600
49. M. Moore, and S. Calvert Brief report: vocabulary acquisition for children with autism: teacher or computer instruction Journal of Autism and Developmental Disorders 30 4 2000 359 362
50. D. Murray Autism and information technology: therapy with computers D.F. Publishers, Autism and Learning: A Guide to Good Practice 1997 Brookes 100 117
51. T. Myers, L. Swan-Kremeier, S. Wonderlich, K. Lancaster, and J. Mitchell The use of alternative delivery systems and new technologies in the treatment of patients with eating disorders International Journal of Eating Disorders 36 2 2004 123 143
52. H.A.M. Noor, F. Shahbodin, N.C. Pee, Serious game for autism children: review of literature, in: Proceedings of the International Conference on Computer Games, Multimedia and Allied Technology (ICCGMAT '12), 2012.
53. A. Ortony, G.L. Clore, and A. Collins The Cognitive Structure of Emotions 1988 Cambridge University Press
54. M. Pantic, R. Cowie, F. D'ericco, D. Heylen, M. Mehu, C. Pelachaud, I. Poggi, M. Schroder, and A. Vinciarelli Social Signal Processing: The Research Agenda 2011 511 538
55. M. Pantic, A. Nijholt, A. Pentland, and T. Huang Human-centred intelligent human computer interaction (HCI2): how far are we from attaining it? International Journal on Autonomous and Adaptive Communications Systems 1 2 2008 168 187
56. M. Pantic, and L.J.M. Rothkrantz Towards an affect-sensitive multimodal human-computer interaction Special Issue on Multimodal Human-Computer Interaction (HCI) 91 2003 1370 1390
57. S. Parsons, and S. Cobb State-of-the-art of virtual reality technologies for children on the autism spectrum European Journal of Special Needs Education 26 3 2011 355 366
58. S. Parsons, and P. Mitchell The potential of virtual reality in social skills training for people with autistic spectrum disorders Journal of Intellectual Disability Research 46 5 2002 430 443
59. S. Parsons, P. Mitchell, and A. Leonard The use and understanding of virtual environments by adolescents with autistic spectrum disorders Journal of Autism and Developmental Disorders 34 4 2004 449 466
60. S. Parsons, P. Mitchell, and A. Leonard Do adolescents with autistic spectrum disorders adhere to social conventions in virtual environments? Autism 9 1 2005 95 117
61. K. Porayska-Pomsta, K. Anderson, S. Bernardini, K. Guldberg, T. Smith, L. Kossivaki, S. Hodgings, I. Lowe, Building intelligent authorable serious game for autistic children and their carers, in: Proceedings of the 10th International Conference on Advances in Computer Entertainment (ACE 2013), 2013.
62. K. Porayska-Pomsta, C. Frauenberger, H. Pain, G. Rajendran, T. Smith, R. Menzies, M.E. Foster, A. Alcorn, S. Wass, S. Bernardini, K. Avramides, W. Keay-Bright, J. Chen, A. Waller, K. Guldberg, J. Good, and O. Lemon Developing technology for autism: an interdisciplinary approach Personal Ubiquitous Computing 16 2 2011 117 127
63. B. Prizant, A. Wetherby, E. Rubin, and A. Laurent The scerts model: a transactional, family-centered approach to enhancing communication and socioemotional ability in children with autism spectrum disorder Infants and Young Children 16 4 2003 296 316
64. B. Prizant, A. Wetherby, E. Rubin, A. Laurent, and P. Rydell The SCERTS®Model: A Comprehensive Educational Approach for Children with Autism Spectrum Disorders 2006 Brookes
65. C. Putnam, L. Chong, Software and technologies designed for people with autism: what do users want? in: 10th International ACM SIGACCESS Conference on Computers and Accessibility, 2008, pp. 3-8.
66. M.M. Rahman, S. Ferdous, S.I. Ahmed, and A. Anwar Speech development of autistic children by interactive computer games Interactive Technology and Smart Education 8 4 2011 208 223
67. M. Riedl, C.J. Saretto, and R.M. Young Managing interaction between users and agents in a multi-agent storytelling environment Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems AAMAS '03 2003 ACM New York, NY, USA 741 748
68. S.J. Rogers, and L.A. Vismara Evidence-based comprehensive treatments for early autism Journal of Clinical Child and Adolescent Psychology 37 1 2008 8 38
69. S. Russell, and P. Norvig Artificial Intelligence: A Modern Approach second ed. 2003 Prentice Hall
70. C.A. Smith, and R.S. Lazarus Emotion and adaptation I. Vlahavas, D. Vrakas, Handbook of Personality: Theory and Research 1990 Guilford New York 609 637 Chapter 23, L.A. Pervin (Ed.)
71. A. Tartaro, and J. Cassell Using virtual peer technology as an intervention for children with autism J. Lazar, Universal Usability: Designing Computer Interfaces for Diverse User Populations 2006 John Wiley and Sons, Ltd. New York 231 262 Chapter 8
72. A. Tartaro, J. Cassell, Playing with virtual peers: bootstrapping contingent discourse in children with autism, in: Proc. ICLS 2008, 2008, pp. 382-389.
73. J. Wainer, D. Feil-Seifer, D. Shell, M. Mataric, The role of physical embodiment in human-robot interaction, in: The 15th IEEE International Symposium on Robot and Human Interactive Communication, 2006, pp. 117-122.
74. M. Walker, and A. Armfield What is the makaton vocabulary? Special Education: Forward Trends 8 3 1981 19 20
75. L. Wiggs, and G. Stores Sleep patterns and sleep disorders in children with autistic spectrum disorders: insights using parent report and actigraphy Developmental Medicine and Child Neurology 46 2004 372 380
76. C. Williams, B. Wright, G. Callaghan, and B. Coughlan Do children with autism learn to read more readily by computer assisted instruction or traditional book methods? Autism 6 1 2002 71 91
77. L. Wing, The Autism Spectrum, Constable, London, 1996.
78. D. Wood, J. Murphy, K. Center, C. Russ, R. McLay, D. Reeves, J. Pyne, R. Shilling, J. Hagan, and B. Wiederhold Combat related post-traumatic stress disorder: a multiple case report using virtual reality graded exposure therapy with physiological monitoring Studies in Health Technology and Informatics 132 2008 556 561
79. M. Wooldridge, and N.R. Jennings Intelligent agents: theory and practice Knowledge Engineering Review 10 2 1995 115 152
80. G. Yannakakis, and J. Hallam Modeling and augmenting game entertainment through challenge and curiosity International Journal on Artificial Intelligence Tools 16 6 2007 981 999