First-person vision

Proceedings of the IEEE

Volumn 100, Issue 8, 2012, Pages 2442-2453

  Kanade, Takeo ^a   Hebert, Martial ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

Computer vision; eye tracking; object recognition; quality of life technologies

Indexed keywords

COMPUTER VISION; OBJECT RECOGNITION; SECURITY SYSTEMS;

FIRST-PERSON VISIONS; HEAD MOTION; PROTOTYPE SYSTEM; QUALITY OF LIFE; RESEARCH CHALLENGES;

EYE TRACKING;

EID: 84864121758     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2012.2200554     Document Type: Conference Paper

References (82)

1. O. Aghazadeh, J. Sullivan, and S. Carlsson, "Novelty detection from an ego-centric perspective," in Proc. Comput. Vis. Pattern Recognit., 2011, pp. 3297-3304.
2. M. Agrawal, K. Konolige, and L. Iocchi, "Real-time detection of independent motion using stereo," in Proc. IEEE Workshop Motion Video Comput., 2005, pp. 207-214.
3. J. Aloimonos, "Purposive and qualitative active vision," in Proc. 10th Int. Conf. Pattern Recognit., 1990, pp. 346-360.
4. J. Aloimonos, I.Weiss, and A. Bandyopadhyay, "Active vision," Int. J. Comput. Vis., vol. 1, no. 4, pp. 333-356, 1988.
5. P. Bach-y-Rita and K. A. Kaczmarek, "Tongue placed tactile output device," USA, Patent 6 430 459, 2002.
6. H. Badino and T. Kanade, "A head-wearable short-baseline stereo system for the simultaneous estimation of structure and motion," presented at the Mach. Vis. Appl., Nara, Japan, Jun. 13, 2011.
7. H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, "SURF: Speeded up robust features," Comput. Vis. Image Understand., vol. 110, no. 3, pp. 346-359, 2008.
8. B. Boots and G. J. Gordon, "Predictive state temporal difference learning," Adv. Neural Inf. Process. Syst., vol. 23, pp. 271-279, 2010.
9. B. Boots, S. M. Siddiqi, and G. J. Gordon, "Closing the learning-planning loop with predictive state representations," in Proc. Robot., Sci. Syst. VI, 2010, pp. 36-43.
10. B. Clarkson and A. Pentland, "Unsupervised clustering of ambulatory audio and video," in Proc. Int. Conf. Acoust. Speech Signal Process., 1999, pp. 3037-3040.
11. A. Collet and S. Srinivasa, "Efficient multi-view object recognition and full pose estimation," in Proc. IEEE Int. Conf. Robot. Autom., May 2010, pp. 2050-2055.
12. A. Collet, D. Berenson, S. S. Srinivasa, and D. Ferguson, "Object recognition and full pose registration from a single image for robotic manipulation," in Proc. IEEE Int. Conf. Robot. Autom., 2009, pp. 48-55.
13. A. Collet, S. Srinivasa, and M. Hebert, "Structure discovery in multi-modal data a region-based approach," in Proc. IEEE Int. Conf. Robot. Autom., May 2011, pp. 5695-5702.
14. CMU Kitchen Data Set. Online. Available: http://kitchen.cs.cmu.edu
15. M. Devyver, A. Tsukada, and T. Kanade, "A wearable device for first person vision," presented at the Int. Symp. Quality Life Technol., Toronto, Canada, Jun. 6-7, 2011.
16. A. Duchowski, "A breadth-first survey of eye-tracking applications," Behav. Res. Meth., vol. 34, pp. 455-470, 2002.
17. A. Ess, ". Leibe, K. Schindler, and L. Van Gool, BMoving obstacle detection in highly dynamic scenes," in Proc. Int. Conf. Robot. Autom., 2009, pp. 56-63.
18. A. Fathi, X. Ren, and J. Rehg, "Learning to recognize objects in egocentric activities," in Proc. Comput. Vis. Pattern Recognit., 2011, pp. 3281-3288.
19. A. Fathi, A. Farhadi, and J. Rehg, "Understanding Egocentric Activities," in Proc. Int. Conf. Comput. Vis., 2011, pp. 407-414.
20. U. Franke, C. Rabe, H. Badino, and S. Gehrig, "6D-vision: Fusion of stereo and motion for robust environment perception," in Proc. Deutsche Arbeitsgemeinschaft für Mustererkennung, 2005, pp. 216-223. (Pubitemid 41520657)
21. A. Godfrey, R. Conway, D. Meagher, and G. O. Laighin, "Direct measurement of human movement by accelerometry," Med. Eng. Phys., vol. 30, no. 10, pp. 1364-1386, 2008.
22. P. W. Hallinan, Recognizing Human Eye, ". C. Vemuri, Ed. Bellingham, WA: SPIE, 1991, vol. 1570, ch. 1, pp. 214-226.
23. M. Hayhoe and D. Ballard, BEye movements in natural behavior," Trends Cogn. Sci., vol. 9, no. 4, pp. 188-194, 2005.
24. E. Hsiao, A. Collet, and M. Hebert, "Making specific features less discriminative to improve point-based 3D object recognition," in Proc. Comput. Vis. Pattern Recognit., Jun. 2010, pp. 2653-2660.
25. T. Hynh, U. Blanke, and B. Schiele, "Scalable recognition of daily activities with wearable sensors," in Proc. Location Context Awareness, 2007, pp. 50-67.
26. H. Jegou, C. Schmid, H. Harzallah, and J. Verbeek, "Accurate image search using the contextual dissimilarity measure," IEEE Trans. Pattern Anal. Machine Intell., vol. 32, no. 1, pp. 2-11, Jan. 2009.
27. M. A. Just and P. A. Carpenter, "Eye fixations and cognitive processes," Cogn. Psychol., vol. 8, pp. 441-480, 1976.
28. T. Kanade, "First-person inside-out vision," presented at the IEEE Workshop Egocentric Vis., FL, Miami, Jun.20, 2009.
29. H. Kang, A. Efros, M. Hebert, and T. Kanade, "Image composition for object pop-out," in Proc. IEEE Workshop 3D Represent. Recognit., 2009, pp. 681-688.
30. H. Kang, A. A. Efros, M. Hebert, and T. Kanade, "Image matching in large scale indoor environment," in Proc. Workshop Egocentric Vis., 2009, pp. 33-40.
31. H. Kang, M. Hebert, and T. Kanade, "Image matching with distinctive visual vocabulary," in Proc. Workshop Appl. Comput. Vis., Jan. 2011, pp. 402-409.
32. N. Kern, ". Schiele, and A. Schmidt, BMulti-sensor, FActivity context detection for wearable computing," in Proc. Eur. Symp. Ambient Intell., 2003, pp. 220-232. (Pubitemid 137631976)
33. K. Kitani, T. Obake, Y. Sato, and A. Sugimoto, "Fast unsupervised ego-action learning for first-person sports videos," in Proc. Comput. Vis. Pattern Recognit., 2010, pp. 3241-3248.
34. I.-K. Jung and S. Lacroix, "High resolution terrain mapping using low altitude aerial stereo imagery," in Proc. Int. Conf. Comput. Vis., 2003, pp. 946-951.
35. H. Kang, M. Hebert, and T. Kanade, "Discovering object instances from scenes of daily living," in Proc. Int. Conf. Comput. Vis., 2011, pp. 762-769.
36. N. Katayama and S. Satoh, "Distinctiveness-sensitive nearest neighbor search for efficient similarity retrieval of multimedia information," in Proc. IEEE Int. Conf. Data Eng., 2001, pp. 493-502. (Pubitemid 32457152)
37. M. H. Law, A. Topchy, and A. K. Jain, "Clustering with soft and group constraints," Structural Syntact. Stat. Pattern Recognit., pp. 662-670, 2004.
38. J. Lester, T. Choudhury, and G. Borriello, "A practical approach to recognizing physical activities," in Proc. Pervasive Comput., 2006, pp. 1-16. (Pubitemid 44044748)
39. D. Li, J. Babcock, and D. J. Parkhurst, "Open eyes: A low-cost head-mounted eye-tracking solution," in Proc. Symp. Eye Tracking Res. Appl., 2006, pp. 95-100. (Pubitemid 44758656)
40. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, pp. 91-110, 2004.
41. X. Lu and R. Manduchi, "Detection and localization of curbs and stairways using stereo vision," in Proc. Int. Conf. Robot. Autom., 2005, pp. 4648-4654. (Pubitemid 46084512)
42. W. Mayol-Cuevas, "Wearable visual robots," Ph.D. dissertation, Dept. Comput. Sci., Univ. Oxford, Oxford, U.K., 2004.
43. W. Mayol and D. W. Murray, "Wearable hand activity recognition for event summarization," in Proc. Int. Semantic Web Conf., 2005, pp. 122-129. (Pubitemid 44867504)
44. B. Noris, J.-B. Keller, and A. Billard, "A wearable gaze tracking system for children in unconstrained environments," Comput. Vis. Image Understand., vol. 115, no. 4, pp. 476-486, 2011.
45. N. Oliver, E. Horvitz, and A. Garg, "Layered representations for human activity recognition," in Proc. Int. Conf. Multimodal Interact., 2002, pp. 3-9.
46. D. Patterson, D. Fox, H. Kautz, and M. Philipose, "Fine-grained activity recognition by aggregating abstract object usage," in Proc. Int. Semantic Web Conf., 2005, pp. 44-51. (Pubitemid 44867493)
47. A. Pentland, "Looking at people: Sensing for ubiquitous and wearable computing," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 1, pp. 107-119, Jan. 2000. (Pubitemid 32132957)
48. J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Object retrieval with large vocabularies and fast spatial matching," in Proc. Comput. Vis. Pattern Recognit., 2007, pp. 1-8.
49. Pittsburgh Pattern Recognition. Online. Available: http://www.pittpatt. com/
50. R. Rao and D. Ballard, "An active vision architecture based on iconic representations," Artif. Intell., vol. 78, no. 1-2, pp. 461-505, Oct. 1995.
51. C. Rao, C., A. Gritai, M. Shah, and T. F. Syeda-Mahmood, "View-invariant alignment and matching of video sequences," in Proc. Int. Conf. Comput. Vis., 2003, pp. 939-945.
52. N. Ratliff, ". Ziebart, K. Peterson, J. A. Bagnell, M. Hebert, A. Dey, and S. Srinivasa, BInverse optimal heuristic control for imitation learning," in Proc. 12th Int. Conf. Artif. Intell. Stat., Apr. 2009, DOI:10.1.1.154.6602.
53. X. Ren and M. Philipose, "Egocentric recognition of handled objects benchmark and analysis," in Proc. Egovision Workshop, Miami, FL, 2009, pp. 1-8.
54. X. Ren and C. Gu, "Figure-ground segmentation improves handled object recognition in egocentric video," in Proc. Comput. Vis. Pattern Recognit., 2010, pp. 3137-3144.
55. R. D. Rimey and C. M. Brown, "Control of selective perception using Bayes nets and decision theory," Int. J. Comput. Vis., vol. 12, Special Issue on Active Vision, no. 2/3, pp. 173-208, Apr. 1993.
56. J. Sez and F. Escolano, "Stereo-based aerial obstacle detection for the visually impaired," presented at the ECCV Workshop Comput. Vis. Appl. Visually Impaired, Marseilles, France, Oct. 18, 2008.
57. S. M. Siddiqi, ". Boots, and G. J. Gordon, BReduced-rank hidden Markov models," in Proc. 13th Int. Conf. Artif. Intell. Stat., 2010, pp. 741-748.
58. S. Satinde, M. R. James, and M. R. Rudary, "Predictive state representations: A new theory for modeling dynamical systems," in Proc. Uncertainty Artif. Intell., 2004, pp. 512-518.
59. B. Schiele, H. Aoki, and A. Pentland, "Realtime personal positioning system for wearable computers," in Proc. 3rd IEEE Int. Symp. Wearable Comput., 1999, pp. 37-43. (Pubitemid 30538048)
60. B. Schiele, N. Oliver, T. Jebara, and A. Pentland, "An interactive computer vision systemVDyPERS: Dynamic personal enhanced reality system," in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst., 1999, pp. 51-65.
61. D. Sorokin, D. Berenson, S. Srinivasa, and M. Hebert, "People helping robots helping people: Crowd sourcing for grasping novel objects," in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst., 2010, pp. 2117-2122.
62. E. H. Spriggs, F. De la Torre Frade, and M. Hebert, "Temporal segmentation and activity classification from first-person sensing," in Proc. IEEE Workshop Egocentric Vis., 2009, pp. 17-24.
63. T. B. Starner, ". Schiele, and A. Pentland, BVisual context awareness in wearable computing," in Proc. Int. Symp. Wearable Comput., 1998, pp. 50-57.
64. T. Starner, J. Weaver, and A. Pentland, "Real-time American sign language recognition using desk and wearable computer based video," IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 12, pp. 1371-1375, Dec. 1998. (Pubitemid 128741382)
65. T. Stiefmeier, D. Roggen, G. Ogris, P. Lukowicz, and G. Tröster, "Wearable activity tracking in car manufacturing," IEEE Pervasive Comput., vol. 7, no. 2, pp. 42-50, 2008. (Pubitemid 351575121)
66. L. Sun, U. Klank, and M. Beetz, "EYEWATCHME: 3D hand and object tracking for inside out activity analysis," in Proc. Workshop Egocentric Vis., 2009, pp. 9-16.
67. S. Sundaram and W. Cuevas, "High level activity recognition using low resolution wearable vision," in Proc. Workshop Egocentric Vis., 2009, pp. 25-32.
68. A. Talukder and L. Matthies, "Real-time detection of moving objects from moving vehicles using dense stereo and optical flow," in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst., 2004, pp. 3718-3725. (Pubitemid 40276169)
69. E. Taralova, F. Dela Torre, and M. Hebert, "Source constrained clustering," in Proc. Int. Conf. Comput. Vis., 2011, pp. 1927-1934.
70. M. Torres, A. Collet, and S. Srinivasa, "MOPED: A scalable and low latency object recognition and pose estimation system," in Proc. Int. Conf. Robot. Autom., 2010, pp. 2050-2055.
71. S. Treuillet, E. Royer, T. Chateau,M. Dhome, and J.-M. Lavest, "Body mounted vision system for visually impaired outdoor and indoor way finding assistance," in Proc. Conf. Workshop Assistive Technol. People Vis. Hearing Impairments, 2007, pp. 25-30.
72. A. Tsukada, M. Devyver, and T. Kanade, "Illumination-free gaze estimation method for first-person vision wearable device," in Proc. Int. Conf. Comput. Vis., 2011, pp. 2084-2091.
73. M. Vester-Christensen, D. Leimberg, ". K. Ersbøll, and L. K. Hansen, BDeformable models for eye tracking," in Proc. Konferencei MønstergenkendelseogBilledanalyse, 2005, pp. 35-43.
74. K. Wagstaff, C. Cardie, S. Rogers, and S. Schroedl, "Constrained K-means clustering with background knowledge," in Proc. 18th Int. Conf. Mach. Learn., 2001, pp. 577-584.
75. S. Wang, W. Pentney, A. M. Popescu, T. Choudhury, and M. Philipose, "Commonsense based joint training of human activity recognizers," in Proc. Int. Joint Conf. Artif. Intell., 2007, pp. 2237-2242.
76. L. Young and D. Sheena, "Survey of eye movement recording methods," Behav. Res. Meth., vol. 7, pp. 397-429, 1975.
77. J. Zhang, S. Ong, and A. Nee, "Navigation systems for individuals with visual impairment: A survey," in Proc. Int. Conv. Rehab. Eng. Assistive Technol., 2008, pp. 159-162.
78. F. Zhou and F. De la Torre, "Canonical time warping for alignment of human behavior," in Proc. Neural Inf. Process. Syst., 2009, pp. 2286-2294.
79. F. Zhou and F. De la Torre, "Generalized time warping for multi-modal alignment of human motion," presented at the IEEE Conf. Comput. Vis. Pattern Recognit., 2012.
80. B. D. Ziebart, N. Ratliff, G. Gallagher, C. Mertz, K. Peterson, J. A. Bagnell, M. Hebert, A. K. Dey, and S. Srinivasa, "Planning-based prediction for pedestrians," in Proc. IEEE/RSJ Int. Conf. Intell. Robot. Syst., 2009, pp. 3931-3936.
81. B. Ziebart, J. A. Bagnell, and A. Dey, "Purposeful adaptive behavior modeling for robotics," in Proc. Neural Inf. Process. Syst., 2009.
82. B. D. Ziebart, J. Bagnell, and A. Dey, "Modeling interaction via the principle of maximum causal entropy," in Proc. Int. Conf. Mach. Learn., 2010, pp. 1255-1262.