Online affect detection and robot behavior adaptation for intervention of children with autism

IEEE Transactions on Robotics

Volumn 24, Issue 4, 2008, Pages 883-896

  Liu, Changchun ^a   Conn, Karla ^a   Sarkar, Nilanjan ^a   Stone, Wendy ^b,c  

^a VANDERBILT UNIVERSITY   (United States)

^b VANDERBILT UNIVERSITY MEDICAL CENTER   (United States)

^c VANDERBILT UNIVERSITY   (United States)

Author keywords

Autism intervention; Closed loop human robot interaction (HRI); Physiological sensing

Indexed keywords

FLOW INTERACTIONS; GAME THEORY;

AFFECTIVE STATES; AUTISM INTERVENTION; CLOSED-LOOP HUMAN-ROBOT INTERACTION (HRI); PHYSIOLOGICAL SENSING;

ROBOTS;

EID: 50649096770     PISSN: 15523098     EISSN: None     Source Type: Journal    
DOI: 10.1109/TRO.2008.2001362     Document Type: Article

References (52)

1. American Psychiatric Association, Diagnostic and Statistical Manual of Mental Disorders: DSM-IV-TR, 4th ed. Washington, DC: Amer. Psychiatr. Assoc., 2000.
2. CDC, "Prevalence of autism spectrum disorders - Autism and developmental disabilities monitoring network, 14 sites, United States, 2002," MMWR Surveill Summ., vol. 56, pp. 12-28, Feb. 2007.
3. S. J. Rogers, "Empirically supported comprehensive treatments for young children with autism," J. Clin. Child Psychol., vol. 27, pp. 168-179, Jun. 1998.
4. M. Rutter, "Autism: Its recognition, early diagnosis, and service implications," J. Dev. Behav. Pediatr., vol. 27, pp. S54-S58, Apr. 2006.
5. L. Tarkan, "Autism therapy is called effective, but rare," New York Times, Oct. 21, 2002.
6. V. Bernard-Opitz, N. Sriram, and S. Nakhoda-Sapuan, "Enhancing social problem solving in children with autism and normal children through computer-assisted instruction," J. Autism Dev. Disord., vol. 31, pp. 377-384, Aug. 2001.
7. S. Parsons and P. Mitchell, "The potential of virtual reality in social skills training for people with autistic spectrum disorders," J. Intellect. Disabil. Res., vol. 46, pp. 430-443, Jun. 2002.
8. K. Dautenhahn and I. Werry, "Towards interactive robots in autism therapy: Background, motivation and challenges," Pragmatics Cogn. vol. 12, pp. 1-35, 2004.
9. F. Michaud and C. Theberge-Turmel, "Mobile robotic toys and autism," in Socially Intelligent Agents: Creating Relationships With Computers and Robots, K. Dautenhahn, A. H. Bond, L. Canamero, and B. Edmonds, Eds. Norwell, MA: Kluwer, 2002, pp. 125-132.
10. G. Pioggia, R. Igliozzi, M. Ferro, A. Ahluwalia, F. Muratori, and D. De Rossi, "An android for enhancing social skills and emotion recognition in people with autism," IEEE Trans. Neural Syst. Rehabil. Eng., vol. 13, no. 4, pp. 507-515, Dec. 2005.
11. B. Robins, P. Dickerson, P. Stribling, and K. Dautenhahn, "Robot-mediated joint attention in children with autism: A case study in robot-human interaction," Interact. Stud., vol. 5, no. 2, pp. 161-198, 2004.
12. H. Kozima, C. Nakagawa, and Y. Yasuda, "Interactive robots for communication-care: A case-study in autism therapy," in Proc. IEEE Int. Workshop Robot Hum. Interactive Commun., Nashville, TN, 2005, pp. 341-346.
13. B. Scassellati, "Quantitative metrics of social response for autism diagnosis," in Proc. IEEE Int. Workshop Robot Hum. Interactive Commun., Nashville, TN, 2005, pp. 585-590.
14. T. Fong, I. Nourbakhsh, and K. Dautenhahn, "A survey of socially interactive robots," Robot. Auton. Syst., vol. 42, no. 3/4, pp. 143-166, 2003.
15. R. W. Picard, Affective Computing. Cambridge, MA: MIT Press, 1997.
16. J. Seip, Teaching the Autistic and Developmentally Delayed: A Guide for Staff Training and Development. Delta, BC: Author, 1996.
17. L. Ernsperger, Keys to Success for Teaching Students With Autism. Arlington, TX: Future Horizons, 2003.
18. S. Wieder and S. Greenspan, "Can children with autism master the core deficits and become empathetic, creative, and reflective?," J. Dev. Learn. Disord., vol. 9, pp. 1-29, 2005.
19. M. S. Bartlett, G. Littlewort, I. Fasel, and J. R. Movellan, "Real time face detection and facial expression recognition: Development and applications to human computer interaction," presented at the Comput. Vis. Pattern Recognit. Workshop, Madison, WI, 2003.
20. C. M. Lee and S. S. Narayanan, "Toward detecting emotions in spoken dialogs," IEEE Trans. Speech Audio Process., vol. 13, no. 2, pp. 293-303, Mar. 2005.
21. P. R. D. Silva, M. Osano, A. Marasinghe, and A. P. Madurapperuma, "A computational model for recognizing emotion with intensity for machine vision applications," IEICE Trans., vol. 89-D, no. 7, pp. 2171-2179, 2006.
22. F. Nasoz, K. Alvarez, C. L. Lisetti, and N. Finkelstein, "Emotion recognition from physiological signals for presence technologies," Int. J. Cognit., Technol., Work, Spec. Issue Presence, vol. 6, no. 1, pp. 1-32, 2003.
23. D. Kulic and E. Croft, "Physiological and subjective responses to articulated robot motion," Robotica, vol. 25, pp. 13-27, Jan./Feb. 2007.
24. R. L. Mandryk and M. S. Atkins, "A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies," Int. J. Hum.-Comput. Stud., vol. 65, pp. 329-347, 2007.
25. J. Groden, M. S. Goodwin, M. G. Baron, G. Groden, W. F. Velicer, L. P. Lipsitt, S. G. Hofmann, and B. Plummer, "Assessing cardiovascular responses to stressors in individuals with autism spectrum disorders," Focus Autism Other Dev. Disabil., vol. 20, pp. 244-252, 2005.
26. M. M. Bradley, "Emotion and motivation," in Handbook of Psychophysiology, J. T. Cacioppo, L. G. Tassinary, and G. Berntson, Eds. New York: Cambridge Univ. Press, 2000, pp. 602-642.
27. P. Rani, C. Liu,N. Sarkar, and E.Vanman, "An empirical study of machine learning techniques for affect recognition in human-robot interaction," Pattern Anal. Appl., vol. 9, pp. 58-69, May 2006.
28. P. Rani, N. Sarkar, C. A. Smith, and L. D. Kirby, "Anxiety detecting robotic system-Towards implicit human-robot collaboration," Robotica vol. 22, pp. 85-95, Jan./Feb. 2004.
29. P. Agrawal, C. Liu, and N. Sarkar, "Interaction between human and robot: An affect-inspired approach," Interact. Stud., vol. 9, no. 2, pp. 230-257, 2008.
30. D. B. Shalom, S. H. Mostofsky, R. L. Hazlett, M. C. Goldberg, R. J. Landa, Y. Faran, D. R. McLeod, and R. Hoehn-Saric, "Normal physiological emotions but differences in expression of conscious feelings in children with high-functioning autism," J. Autism Dev. Disord., vol. 36, pp. 395-400, Apr. 2006.
31. R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz, and J. G. Taylor, "Emotion recognition in human-computer interaction," IEEE Signal Process. Mag., vol. 18, no. 1, pp. 32-80, Jan. 2001.
32. R. M. Brown, L. R. Hall, R. Holtzer, S. L. Brown, and N. L. Brown, "Gender and video game performance," Sex Roles, vol. 36, pp. 793-812, 1997.
33. A. Gillott, F. Furniss, and A. Walter, "Anxiety in high-functioning children with autism," Autism, vol. 5, pp. 277-286, Sep. 2001.
34. L. A. Ruble and D. M. Robson, "Individual and environmental determinants of engagement in autism," J. Autism Dev. Disord., vol. 37, pp. 1457-1468, 2007.
35. K. Vansteelandt, I. Van Mechelen, and J. B. Nezlek, "The co-occurrence of emotions in daily life: A multilevel approach," J. Res. Pers., vol. 39, no. 3, pp. 325-335, 2005.
36. J. I. Lacey and B. C. Lacey, "Verification and extension of the principle of autonomic response-stereotypy," Amer. J. Psychol., vol. 71, pp. 50-73, Mar. 1958.
37. K. Dautenhahn, I. Werry, T. Salter, and R. T. Boekhorst, "Towards adaptive autonomous robots in autism therapy: Varieties of interactions," in Proc. IEEE Int. Symp. Comput. Intell. Robot. Autom., Kobe, Japan, 2003, pp. 577-582.
38. M. D. Lloyd and M. D. Leota, Peabody Picture Vocabulary Test (PPVTIII), 3rd ed. Circle Pines, MN: Amer. Guid. Serv., 1997.
39. A. Pecchinenda and C. A. Smith, "The affective significance of skin conductance activity during a difficult problem-solving task," Cognit. Emotion, vol. 10, no. 5, pp. 481-504, 1996.
40. J. T. Cacioppo, G. G. Berntson, J. T. Larsen, K. M. Poehlmann, and T. A. Ito, "The psychophysiology of emotion," in Handbook of Emotions, M. Lewis and J. M. Haviland-Jones, Eds. New York: Guilford, 2000, pp. 173-191.
41. J. F. Papillo and D. Shapiro, "The cardiovascular system," in Principles of Psychophysiology: Physical, Social, and Inferential Elements, J. T. Cacioppo and L. G. Tassinary, Eds. New York: Cambridge Univ. Press, 1990, pp. 456-512.
42. C. Iani, D. Gopher, and P. Lavie, "Effects of task difficulty and invested mental effort on peripheral vasoconstriction," Psychophysiology, vol. 41, no. 5, pp. 789-798, 2004.
43. M. E. Dawson, A. M. Schell, andD. L. Filion, "The electrodermal system," in Principles of Psychophysiology: Physical, Social, and Inferential Elements, J. T. Cacioppo and L. G. Tassinary, Eds. New York: Cambridge Univ. Press, 1990, pp. 295-324.
44. V. N. Vapnik, Statistical Learning Theory. New York: Wiley-Interscience, 1998.
45. C. J. C. Burges, "A tutorial on support vector machines for pattern recognition," Data Min. Knowl. Discov., vol. 2, pp. 121-167, Jun. 1998.
46. M. A. Wiering, "QV(χ)-learning:ANewon-policy reinforcement learning algorithm," in Proc. 7th Eur. Workshop Reinf. Learn., Napoli, Italy, Oct. 2005, pp. 17-18.
47. M. Pantic and L. J.M. Rothkrantz, "Toward an affect-sensitive multimodal human-computer interaction," Proc. IEEE, vol. 91, no. 9, pp. 1370-1390, Sep. 2003.
48. R. A. Barkley, Attention Deficit Hyperactivity Disorder: A Handbook for Diagnosis and Treatment, 2nd ed. New York: Guilford, 1998.
49. E. Hill, S. Berthoz, and U. Frith, "Brief report: Cognitive processing of own emotions in individuals with autistic spectrum disorder and in their relatives," J. Autism Dev. Disord., vol. 34, no. 2, pp. 229-235, 2004.
50. S. Siegel and J.N. J.Castellan, Nonparametric Statistics for the Behavioral Sciences. New York: McGraw-Hill, 1988.
51. R. Jafari, F. Dabiri, P. Brisk, and M. Sarrafzadeh, "Adaptive and fault tolerant medical vest for life critical medical monitoring," in Proc. ACM Symp. Appl. Comput., Santa Fe, NM, 2005, pp. 272-279.
52. C. J. C. H. Watkins and P. Dayan, "Q-learning," Mach. Learn., vol. 8, pp. 279-292, May 1992.