Characterizing everyday activities from visual lifelogs based on enhancing concept representation

Computer Vision and Image Understanding

Volumn 148, Issue , 2016, Pages 181-192

  Wang, Peng ^a   Sun, Lifeng ^a   Yang, Shiqiang ^a   Smeaton, Alan F ^b   Gurrin, Cathal ^b  

^a TSINGHUA NATIONAL LABORATORY FOR INFORMATION SCIENCE AND TECHNOLOGY   (China)

^b DUBLIN CITY UNIVERSITY   (Ireland)

Author keywords

Activity classification; Assistive living; Lifelogging; SenseCam; Wearable camera

Indexed keywords

CAMERAS; GESTURE RECOGNITION; SEMANTICS; WEARABLE TECHNOLOGY;

ACTIVITY CLASSIFICATIONS; ASSISTIVE LIVING; LIFELOGGING; SENSECAM; WEARABLE CAMERAS;

RANDOM PROCESSES;

EID: 84971300531     PISSN: 10773142     EISSN: 1090235X     Source Type: Journal    
DOI: 10.1016/j.cviu.2015.09.014     Document Type: Article

References (56)

1. Placelab (MIT). Online: http://web.mit.edu/cron/group/house-n/placelab.html, 2015 (last accessed July, 2015).
2. R. Aly, D. Hiemstra, F. de Jong, and P. Apers Simulating the future of concept-based video retrieval under improved detector performance Multimed. Tools Appl. 60 1 2011 1 29
3. M. Blum, A.S. Pentland, and G. Tröster InSense: interest-based life logging IEEE Multimed. 13 4 2006 40 48 10.1109/MMUL.2006.87
4. M. Bukhin, and M. DelGaudio WayMarkr: acquiring perspective through continuous documentation Proceedings of the 5th International Conference on Mobile and Ubiquitous Multimedia, MUM'06 2006 ACM New York, NY, USA 9 http://doi.acm.org/10.1145/1186655.1186664
5. D. Byrne, A.R. Doherty, C.G. Snoek, G.J. Jones, and A.F. Smeaton Everyday concept detection in visual lifelogs: validation, relationships and trends Multimed. Tools Appl. 49 1 2010 119 144 10.1007/s11042-009-0403-8
6. M. Campbell, E. Haubold, S. Ebadollahi, D. Joshi, and M.R. Naphade IBM research TRECVid-2006 video retrieval system TRECVid Online Proceedings, TRECVid 2006 2006
7. A.R. Doherty, D. Byrne, A.F. Smeaton, G.J. Jones, and M. Hughes Investigating keyframe selection methods in the novel domain of passively captured visual lifelogs Proceedings of the 2008 International Conference on Content-based Image and Video Retrieval, CIVR'08 2008 ACM New York, NY, USA 259 268 10.1145/1386352.1386389
8. A.R. Doherty, N. Caprani, C. O'Conaire, V. Kalnikaite, C. Gurrin, N.E. O'Connor, and A.F. Smeaton Passively recognising human activities through lifelogging Comput. Hum. Behav. 27 5 2011 1948 1958
9. A.R. Doherty, and A.F. Smeaton Automatically segmenting lifelog data into events Proceedings of the 2008 Ninth International Workshop on Image Analysis for Multimedia Interactive Services, WIAMIS'08 2008 IEEE Computer Society Washington, DC, USA 20 23 http://dx.doi.org/10.1109/WIAMIS.2008.32
10. A.R. Doherty, and A.F. Smeaton Automatically augmenting lifelog events using pervasively generated content from millions of people Sensors 10 3 2010 1423 1446
11. J. Hamm, B. Stone, M. Belkin, and S. Dennis Automatic annotation of daily activity from smartphone-based multisensory streams Proceedings of the 4th International Conference on Mobile Computing, Applications, and Services, MobiCASE 2012, Seattle, WA, USA, October 11-12, 2012. 2012 328 342
12. S. Hodges, L. Williams, E. Berry, S. Izadi, J. Srinivasan, A. Butler, G. Smyth, N. Kapur, and K. Wood SenseCam: A retrospective memory aid Proceedings of the 8th International Conference on Ubicomp, Orange County, CA, USA 2006 177 193
13. T. Hori, and K. Aizawa Context-based video retrieval system for the life-log applications Proceedings of the 5th ACM SIGMM International Workshop on Multimedia Information Retrieval, MIR'03 2003 ACM New York, NY, USA 31 38 http://doi.acm.org/10.1145/973264.973270
14. J. Machajdik, A. Hanbury, A. Garz, and R. Sablatnig Affective computing for wearable diary and lifelogging systems: an overview Workshop of the Austrian Association for Pattern Recognition 2011
15. A. Jalal, and S. Kamal Real-time life logging via a depth silhouette-based human activity recognition system for smart home services Proceedings of the 11th IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2014 74 80 10.1109/AVSS.2014.6918647
16. A. Jalal, S. Kamal, and D. Kim A depth video sensor-based life-logging human activity recognition system for elderly care in smart indoor environments Sensors 14 7 2014 11735 11759
17. A. Jalal, Y. Kim, and D. Kim Ridge body parts features for human pose estimation and recognition from RGB-D video data Proceedings of the 2014 International Conference on Computing, Communication and Networking Technologies, ICCCNT 2014 1 6 10.1109/ICCCNT.2014.6963015
18. A. Jalal, N. Sharif, J.T. Kim, and T.-S. Kim Human activity recognition via recognized body parts of human depth silhouettes for residents monitoring services at smart homes Indoor Built Environ. 22 2013 271 279
19. A. Jalal, M. Uddin, and T.-S. Kim Depth video-based human activity recognition system using translation and scaling invariant features for life logging at smart home IEEE Trans. Consum. Electron. 58 3 2012 863 871 10.1109/TCE.2012.6311329
20. Y.-G. Jiang, Q. Dai, J. Wang, C.-W. Ngo, X. Xue, and S.-F. Chang Fast semantic diffusion for large-scale context-based image and video annotation IEEE Trans. Image Process. 21 6 2012 3080 3091 10.1109/TIP.2012.2188038
21. Y.-G. Jiang, J. Wang, S.-F. Chang, and C.-W. Ngo Domain adaptive semantic diffusion for large scale context-based video annotation Proceedings of the IEEE 12th International Conference on Computer Vision, ICCV 2009 IEEE 1420 1427 10.1109/ICCV.2009.5459295
22. Y. Jin, L. Khan, L. Wang, and M. Awad Image annotations by combining multiple evidence & WordNet Proceedings of the 13th ACM Conference on Multimedia, Singapore 2005 706 715 10.1145/1101149.1101305
23. C. Jose, P. Goyal, P. Aggrwal, and M. Varma Local deep kernel learning for efficient non-linear SVM prediction Proceedings of the 30th International Conference on Machine Learning, ICML'13 2013 486 494
24. P. Kelly, A.R. Doherty, A.F. Smeaton, C. Gurrin, and N.E. O'Connor The colour of life: novel visualisations of population lifestyles Proceedings of the International Conference on Multimedia, MM'10 2010 ACM New York, NY, USA 1063 1066 http://doi.acm.org/10.1145/1873951.1874150
25. L.S. Kennedy, and S.-F. Chang A reranking approach for context-based concept fusion in video indexing and retrieval Proceedings of the 6th ACM International Conference on Image and Video Retrieval, CIVR'07 2007 ACM 333 340 10.1145/1282280.1282331
26. M. Koskela, and A.F. Smeaton An empirical study of inter-concept similarities in multimedia ontologies Proceedings of the 6th ACM International Conference on Image and Video Retrieval, CIVR'07 2007 ACM New York, NY, USA 464 471 10.1145/1282280.1282348
27. Q. Le, T. Sarlós, and A. Smola Fastfood - approximating kernel expansions in loglinear time Proceedings of the International Conference on Machine Learning, ICML 2013
28. D. Lee, and H. Seung Learning the parts of objects by nonnegative matrix factorization Nature 401 1999 788 791
29. H. Lee, A.F. Smeaton, N.E. O'Connor, G.J.F. Jones, M. Blighe, D. Byrne, A.R. Doherty, and C. Gurrin Constructing a SenseCam visual diary as a media process Multimed. Syst. 14 6 2008 341 349 10.1007/s00530-008-0129-x
30. X. Li, D. Wang, J. Li, and B. Zhang Video search in concept subspace: a text-like paradigm Proceedings of the 6th ACM International Conference on Image and Video Retrieval, CIVR'07 2007 ACM New York, NY, USA 603 610 http://doi.acm.org/10.1145/1282280.1282366
31. S. Mann 'WearCam' (the wearable camera): personal imaging systems for long-term use in wearable tetherless computer-mediated reality and personal photo/videographic memory prosthesis Proceedings of the 2nd IEEE International Symposium on Wearable Computers, ISWC'98 1998 IEEE Computer Society Washington, DC, USA 124 131
32. S. Mann, J. Fung, C. Aimone, A. Sehgal, and D. Chen Designing EyeTap digital eyeglasses for continuous lifelong capture and sharing of personal experiences Proceedings of the CHI 2005 Conference on Computer Human Interaction 2005 ACM Press Portland, Oregon, USA
33. R. Mégret, V. Dovgalecs, H. Wannous, S. Karaman, J. Benois-Pineau, E.E. Khoury, J. Pinquier, P. Joly, R. André-Obrecht, Y. Gaëstel, and J.-F. Dartigues The IMMED project: wearable video monitoring of people with age dementia Proceedings of the International Conference on Multimedia, MM'10 2010 ACM New York, NY, USA 1299 1302 http://doi.acm.org/10.1145/1873951.1874206
34. M. Merler, B. Huang, L. Xie, G. Hua, and A. Natsev Semantic model vectors for complex video event recognition IEEE Trans. Multimed. 14 1 2012 88 101 10.1109/TMM.2011.2168948
35. G. O'Loughlin, S.J. Cullen, A. McGoldrick, S. O'Connor, R. Blain, S. O'Malley, and G.D. Warrington Using a wearable camera to increase the accuracy of dietary analysis Am. J. Prev. Med. 44 3 2013 297 301
36. G.-J. Qi, X.-S. Hua, Y. Rui, J. Tang, T. Mei, and H.-J. Zhang Correlative multi-label video annotation Proceedings of the 15th International Conference on Multimedia 2007 17 26 10.1145/1291233.1291245
37. A. Quattoni, S. Wang, L.-P. Morency, M. Collins, and T. Darrell Hidden conditional random fields IEEE Trans. Pattern Anal. Mach. Intell. 29 10 2007 1848 1852 http://doi.ieeecomputersociety.org/10.1109/TPAMI.2007.1124
38. S. Reddy, A. Parker, J. Hyman, J. Burke, D. Estrin, and M. Hansen Image browsing, processing, and clustering for participatory sensing: lessons from a DietSense prototype Proceedings of the 4th Workshop on Embedded Networked Sensors, EmNets'07 2007 ACM Press Cork, Ireland 13 17
39. B. Safadi, and G. Quénot Evaluations of multi-learner approaches for concept indexing in video documents Proceedings of international conference on Adaptivity, Personalization and Fusion of Heterogeneous Information, RIAO 2010 88 91
40. A. Sellen, A. Fogg, M. Aitken, S. Hodges, C. Rother, and K. Wood Do life-logging technologies support memory for the past? An experimental study using SenseCam Proceedings of Conference on Human Factors in Computing Systems, CHI 2007 2007 ACM Press New York, NY, USA 81 90
41. A. Shashua, and T. Hazan Non-negative tensor factorization with applications to statistics and computer vision Proceedings of the International Conference on Machine Learning, ICML 2005 792 799
42. A.R. Silva, S. Pinho, L.M. Macedo, and C.J. Moulin Benefits of SenseCam review on neuropsychological test performance Am. J. Prev. Med. 44 3 2013 302 307
43. A.F. Smeaton, P. Over, and W. Kraaij High level feature detection from video in TRECVid: a 5-year retrospective of achievements Ajay Divakaran, Multimedia Content Analysis, Theory and Applications 2008 Springer 151 174
44. J. Smith, M. Naphade, and A. Natsev Multimedia semantic indexing using model vectors Proceedings of International Conference on Multimedia and Expo, ICME'03 vol. 2 2003 II 445-8 10.1109/ICME.2003.1221649
45. C.G.M. Snoek, J.C. van Gemert, T. Gevers, B. Huurnink, D.C. Koelma, M. van Liempt, O. de Rooij, K.E.A. van de Sande, F.J. Seinstra, A.W.M. Smeulders, A.H.C. Thean, C.J. Veenman, and M. Worring The MediaMill TRECVid 2006 semantic video search engine Proceedings of the TRECVid Workshop, Gaithersburg, USA 2006
46. C.G.M. Snoek, B. Huurnink, L. Hollink, M. de Rijke, G. Schreiber, and M. Worring Adding semantics to detectors for video retrieval IEEE Trans. Multimed. 9 5 2007 975 986
47. S. Song, V. Chandrasekhar, N.-M. Cheung, S. Narayan, L. Li, and J.-H. Lim Activity recognition in egocentric life-logging videos C.V. Jawahar, S. Shan, Computer Vision - ACCV 2014 Workshops Volume 9010 of Lecture Notes in Computer Science 2015 Springer International Publishing 445 458 10.1007/978-3-319-16634-6-33
48. Y. Song, J. Tang, F. Liu, and S. Yan Body surface context: A new robust feature for action recognition from depth videos IEEE Trans. Circuits Syst. Video Technol. 24 6 2014 952 964 10.1109/TCSVT.2014.2302558
49. C.C. Tan, Y.-G. Jiang, and C.-W. Ngo Towards textually describing complex video contents with audio-visual concept classifiers Proceedings of the 19th ACM International Conference on Multimedia MM '11 2011 ACM New York, NY, USA 655 658 10.1145/2072298.2072411
50. C. Wang, F. Jing, L. Zhang, and H.-J. Zhang Image annotation refinement using random walk with restarts Proceedings of ACM International Conference on Multimedia, MM'06 2006 647 650 10.1145/1180639.1180774
51. C. Wang, F. Jing, L. Zhang, and H.-J. Zhang Content-based image annotation refinement Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2007 1 8 10.1109/CVPR.2007.383221
52. P. Wang, and A.F. Smeaton Aggregating semantic concepts for event representation in lifelogging Proceedings of the International Workshop on Semantic Web Information Management, SWIM'11 2011 ACM New York, NY, USA 8:1 8:6 10.1145/1999299.1999307
53. P. Wang, and A.F. Smeaton Semantics-based selection of everyday concepts in visual lifelogging Int. J. Multimed. Inf. Retr. 1 2 2012 87 101
54. P. Wang, and A.F. Smeaton Using visual lifelogs to automatically characterize everyday activities Inf. Sci. 230 2013 147 161 http://dx.doi.org/10.1016/j.ins.2012.12.028
55. P. Wang, A.F. Smeaton, and C. Gurrin Factorizing time-aware multi-way tensors for enhancing semantic wearable sensing X. He, S. Luo, D. Tao, C. Xu, J. Yang, M. Hasan, MultiMedia Modeling Volume 8935 of Lecture Notes in Computer Science 2015 Springer International Publishing 571 582 10.1007/978-3-319-14445-0-49
56. Y. Wu, B. Tseng, and J. Smith Ontology-based multi-classification learning for video concept detection Proceedings of IEEE International Conference on Multimedia and Expo, ICME Vol.2 2004 1003 1006