Joint attention by gaze interpolation and saliency

IEEE Transactions on Cybernetics

Volumn 43, Issue 3, 2013, Pages 829-842

  Yücel, Zeynep ^a   Salah, Albert Ali ^b   Meriçli, Çetin ^c   Mericļi, Tekin ^b   Valenti, Roberto ^d   Gevers, Theo ^d,e  

^a ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

^b BOGAZICI UNIVERSITY   (Turkey)

^c Carnegie Mellon University   (United States)

^d UNIVERSITY OF AMSTERDAM   (Netherlands)

^e UNIVERSITAT AUTÒNOMA DE BARCELONA   (Spain)

Author keywords

Developmental robotics; Gaze following; Head pose estimation; Joint visual attention; Saliency; Selective attention

Indexed keywords

DEVELOPMENTAL ROBOTICS; GAZE FOLLOWING; HEAD POSE ESTIMATION; SALIENCY; SELECTIVE ATTENTION; VISUAL ATTENTION;

IMAGE RECOGNITION;

INTERPOLATION;

ALGORITHM; ARTIFICIAL INTELLIGENCE; ATTENTION; AUTOMATED PATTERN RECOGNITION; BIOMIMETICS; EYE FIXATION; HUMAN; INTERPERSONAL COMMUNICATION; MAN MACHINE INTERACTION; PHYSIOLOGY; PROCEDURES; ROBOTICS; ARTICLE; METHODOLOGY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; ATTENTION; BIOMIMETICS; COMMUNICATION; FIXATION, OCULAR; HUMANS; MAN-MACHINE SYSTEMS; PATTERN RECOGNITION, AUTOMATED; ROBOTICS;

EID: 84890437287     PISSN: 21682267     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCB.2012.2216979     Document Type: Article

References (63)

1. M. Asada, K. MacDorman, H. Ishiguro, and Y. Kuniyoshi, "Cognitive developmental robotics as a new paradigm for the design of humanoid robots," Robot. Auton. Syst., vol. 37, no. 2/3, pp. 185-193, Nov. 2001.
2. P.Mundy and L. Newell, "Attention, joint attention, and social cognition," Curr. Dir. Psychol. Sci., vol. 16, no. 5, pp. 269-274, Oct. 2007.
3. R. Rao, A. Shon, and A. Meltzoff, "A Bayesian model of imitation in infants and robots," in Imitation and Social Learning in Robots, Humans, and Animals: Behavioural, Social and Communicative Dimensions, K. Dautenhahn and C. Nehaniv, Eds. Cambridge, U.K.: Cambridge Univ. Press, 2004, pp. 217-247.
4. V. Corkum and C. Moore, "Development of joint visual attention in infants," in Joint Visual Attention: Its Origins and Role in Development, C. Moore and P. J. Dunham, Eds. Hillsdale, NJ: Erlbaum, 1995, pp. 61-84.
5. A. Meltzoff, R. Brooks, A. Shon, and R. Rao, ""Social" robots are psychological agents for infants: A test of gaze following," Neural Netw., vol. 23, no. 8/9, pp. 966-972, Nov. 2010.
6. Z. Yücel and A. A. Salah, "Head pose and neural network based gaze direction estimation for joint attention modeling in embodied agents," in Proc. Annu. Meet. Cogn. Sci. Soc., 2009, pp. 3139-3144.
7. J. Stahl, "Amplitude of human head movements associated with horizontal saccades," Exp. Brain Res., vol. 126, no. 1, pp. 41-54, Apr. 1999.
8. D. Hansen and Q. Ji, "In the eye of the beholder: A survey of models for eyes and gaze," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 3, pp. 478-500, Mar. 2010.
9. P. Ravindra, S. De Silva, K. Tadano, S. Lambacher, S. Herath, and M. Higashi, "Unsupervised approach to acquire robot joint attention," in Proc. Int. Conf. Auton. Robots Agents, 2009, pp. 601-606.
10. L. Fletcher, G. Loy, N. Barnes, and A. Zelinsky, "Correlating driver gaze with the road scene for driver assistance systems," Robot. Auton. Syst., vol. 52, no. 1, pp. 71-84, Jul. 2005.
11. S. Lee, J. Jo, H. Jung, K. Park, and J. Kim, "Real-time gaze estimator based on driver's head orientation for forward collision warning system," IEEE Trans. Intell. Transp. Syst., vol. 12, no. 1, pp. 254-267, Mar. 2011.
12. S. Ba and J. Odobez, "Multiperson visual focus of attention from head pose and meeting contextual cues," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 1, pp. 101-116, Jan. 2011.
13. C. Peters, S. Asteriadis, and K. Karpouzis, "Investigating shared attention with a virtual agent using a gaze-based interface," J. Multimodal User Interfaces, vol. 3, no. 1/2, pp. 119-130, Mar. 2010.
14. B. Scassellati, "Imitation and mechanisms of joint attention: A developmental structure for building social skills on a humanoid robot," in Computation for Metaphors, Analogy, and Agents, C. Nehaniv, Ed. Berlin, Germany: Springer-Verlag, 1999, pp. 176-195.
15. S. Langton, H. Honeyman, and E. Tessler, "The influence of head contour and nose angle on the perception of eye-gaze direction," Percept. Psychophys., vol. 66, no. 5, pp. 752-771, Jul. 2004.
16. G. Butterworth and N. Jarrett, "What minds have in common is space: Spatial mechanisms serving joint visual attention in infancy," Brit. J. Dev. Psychol., vol. 9, no. 1, pp. 55-72, Mar. 1991.
17. M. Begum and F. Karray, "Visual attention for robotic cognition: A survey," IEEE Trans. Auton. Mental Dev., vol. 3, no. 1, pp. 92-105, Mar. 2011.
18. Y. Nagai, M. Asada, and K. Hosoda, "Developmental learning model for joint attention," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2002, pp. 932-937.
19. Y. Nagai, K. Hosoda, A. Morita, and M. Asada, "A constructive model for the development of joint attention," Connect. Sci., vol. 15, no. 4, pp. 211-229, Dec. 2003.
20. S. Baluja and D. Pomerleau, "Non-intrusive gaze tracking using artificial neural networks," in Proc. Adv. Neural Inf. Process. Sys., 1994, pp. 753-760.
21. Y. Sugano, Y. Matsushita, Y. Sato, and H. Koike, "An incremental learning method for unconstrained gaze estimation," in Proc. Eur. Conf. Comput. Vis., 2008, pp. 656-667.
22. Y. Sugano, Y. Matsushita, and Y. Sato, "Calibration-free gaze sensing using saliency maps," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2010, pp. 2667-2674.
23. J. Chen and Q. Ji, "Probabilistic gaze estimation without active personal calibration," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2011, pp. 609-616.
24. H. Sumioka, K. Hosoda, Y. Yoshikawa, and M. Asada, "Acquisition of joint attention through natural interaction utilizing motion cues," Adv. Robot., vol. 21, no. 9, pp. 983-999, Sep. 2007.
25. Y. Nagai, "Joint attention development in infant-like robot based on head movement imitation," in Proc. Int. Symp. Imitation Animals Artifacts, 2005, pp. 87-96.
26. E. Murphy-Chutorian and M. M. Trivedi, "Head pose estimation in computer vision: A survey," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 4, pp. 607-626, Apr. 2009.
27. M. Hoffman, D. Grimes, A. Shon, and R. Rao, "A probabilistic model of gaze imitation and shared attention," Neural Netw., vol. 19, no. 3, pp. 299-310, Apr. 2006.
28. A. Shon, D. Grimes, C. Baker, M. Hoffman, S. Zhou, and R. Rao, "Probabilistic gaze imitation and saliency learning in a robotic head," in Proc. IEEE Int. Conf. Robot. Auton., 2005, pp. 2865-2870.
29. C. Bard, M. Fleury, and J. Paillard, "Different patterns in aiming accuracy for head-movers and non-head movers," in The Head-Neck Sensory Motor System, A. Berthoz, W. Graf, and P. Vidal, Eds. London, U.K.: Oxford Univ. Press, 1992, pp. 582-586.
30. C. Peters and A. Qureshi, "A head movement propensity model for animating gaze shifts and blinks of virtual characters," Comput. Graph., vol. 34, no. 6, pp. 677-687, Dec. 2010.
31. A. Shon, J. Storz, A. Meltzoff, and R. Rao, "A cognitive model of imitative development in humans and machines," Int. J. Hum. Robot., vol. 4, no. 2, pp. 387-406, Jun. 2007.
32. A. Meltzoff and M. Moore, "Explaining facial imitation: A theoretical model," Early Dev. Parenting, vol. 6, no. 3/4, pp. 179-192, Sep.-Dec.1997.
33. H. Goossens and A. Opstal, "Human eye-head coordination in two dimensions under different sensorimotor conditions," Exp. Brain Res., vol. 114, no. 3, pp. 542-560, May 1997.
34. E. Freedman, "Coordination of the eyes and head during visual orienting," Exp. Brain Res., vol. 190, no. 4, pp. 369-387, Oct. 2008.
35. B. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," in Proc. Int. Joint Conf. Artif. Intell., 1981, vol. 2, pp. 674-679.
36. M. Hayhoe, M. Land, and A. Shrivastava, "Coordination of eye and hand movements in a normal environment," Invest. Ophthalmol. Vis. Sci, vol. 40, no. 4, p. S380, Mar. 1999.
37. J. Triesch, H. Jasso, and G. Deák, "Emergence of mirror neurons in a model of gaze following," Adapt. Behav., vol. 15, no. 2, pp. 149-165, Jun. 2007.
38. L. Itti, N. Dhavale, and F. Pighin, "Photorealistic attention-based gaze animation," in Proc. IEEE Int. Conf. Multimedia Expo, 2006, pp. 521-524.
39. J. Xiao, T.Moriyama, T. Kanade, and J. Cohn, "Robust full-motion recovery of head by dynamic templates and re-registration techniques," Int. J. Imaging Syst. Technol., vol. 13, no. 1, pp. 85-94, Oct. 2003.
40. C. Gordon, T. Churchill, C. Clauser, B. Bradtmiller, J. McConville, I. Tebbetts, and R. Walker, "Anthropometric Survey of US Army Personnel: Methods and Summary Statistics," US Army Natick Res. Dev. Eng. Center, Natick, MA, Tech. Rep., 1989.
41. P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2001, vol. 1, pp. 511-518.
42. R. Valenti, Z. Yucel, and T. Gevers, "Robustifying eye center localization by head pose cues," in Proc. IEEE Conf. CVPR, 2009, pp. 612-618.
43. L. Piccardi, B. Noris, G. Schiavone, F. Keller, C. Von Hofsten, and A. Billard, "Wearcam: A head mounted wireless camera for monitoring gaze attention and for the diagnosis of developmental disorders in young children," in Proc. IEEE Int. Symp. Robot Human Interactive Commun., 2007, pp. 594-598.
44. R. Stiefelhagen, J. Yang, and A. Waibel, "Modeling focus of attention for meeting indexing," in Proc. Int. Conf. Multimedia, 1999, pp. 3-10.
45. M. Walters, K. Dautenhahn, R. Te Boekhorst, K. Koay, C. Kaouri, S. Woods, C. Nehaniv, D. Lee, and I. Werry, "The influence of subjects' personality traits on personal spatial zones in a human-robot interaction experiment," in Proc. IEEE Int. Workshop Robot Human Interactive Commun., 2005, pp. 347-352.
46. E. T. Hall, The Hidden Dimension. New York: Doubleday, 1963.
47. C. Rasmussen and C. Williams, Gaussian Processes for Machine Learning. Cambridge, MA: MIT Press, 2006.
48. S. Calinon and A. Billard, "Incremental learning of gestures by imitation in a humanoid robot," in Proc. ACM/IEEE Int. Conf. Human-Robot Interact., 2007, pp. 255-262.
49. C. Breazeal, "Social interactions in HRI: The robot view," IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 34, no. 2, pp. 181-186, May 2004.
50. Y. Nagai, K. Hosoda, A. Morita, and M. Asada, "Emergence of joint attention based on visual attention and self learning," in Proc. Int. Symp. Adapt. Motion Animals Mach., 2003, pp. 1-6.
51. Y. Ma and H. Zhang, "Contrast-based image attention analysis by using fuzzy growing," in Proc. ACM Int. Conf. Multimedia, 2003, pp. 374-381.
52. C. Breazeal, A. Edsinger, P. Fitzpatrick, and B. Scassellati, "Active vision for sociable robots," IEEE Trans. Syst., Man, Cybern. A, Syst., Humans, vol. 31, no. 5, pp. 443-453, Sep. 2001.
53. L. Itti, C. Koch, and E. Niebur, "Amodel of saliency-based visual attention for rapid scene analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
54. T. Judd, K. Ehinger, F. Durand, and A. Torralba, "Learning to predict where humans look," in Proc. IEEE Int. Conf. Comput. Vis., 2009, pp. 2106-2113.
55. N. Bruce and J. Tsotsos, "Saliency, attention, and visual search: An information theoretic approach," J. Vision, vol. 9, no. 3, pp. 1-24, Mar. 2009.
56. J. Han, K. Ngan, M. Li, and H. Zhang, "Unsupervised extraction of visual attention objects in color images," IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 1, pp. 141-145, Jan. 2006.
57. T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, X. Tang, and H. Shum, "Learning to detect a salient object," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 2, pp. 353-367, Feb. 2011.
58. A. Treisman and G. Gelade, "A feature-integration theory of attention," Cogn. Psychol., vol. 12, no. 1, pp. 97-136, Jan. 1980.
59. N. Bruce and J. Tsotsos, "Saliency based on information maximization," Proc. Adv. Neural Inf. Process. Syst., vol. 18, pp. 155-162, 2006.
60. B. Vincent, T. Troscianko, and I. Gilchrist, "Investigating a space-variant weighted salience account of visual selection," Vision Res., vol. 47, no. 13, pp. 1809-1820, Jun. 2007.
61. B. Tatler, M. Hayhoe, M. Land, and D. Ballard, "Eye guidance in natural vision: Reinterpreting salience," J. Vision, vol. 11, no. 5, May 2011.
62. C. Bishop, Neural Networks for Pattern Recognition. London, U.K.: Oxford Univ. Press, 2005.
63. B. Schauerte, J. Richarz, and G. Fink, "Saliency-based identification and recognition of pointed-at objects," in Proc. IEEE/RJS Int. Conf. Intell. Robots Syst., 2010, pp. 4638-4643.