Discovering salient regions on 3D photo-textured maps: Crowdsourcing interaction data from multitouch smartphones and tablets

Computer Vision and Image Understanding

Volumn 131, Issue , 2015, Pages 28-41

  Johnson Roberson, Matthew ^a   Bryson, Mitch ^c   Douillard, Bertrand ^b   Pizarro, Oscar ^c   Williams, Stefan B ^c  

^a UNIVERSITY OF MICHIGAN   (United States)

^b CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^c UNIVERSITY OF SYDNEY   (Australia)

Author keywords

3D maps; Crowdsourcing; Gaze tracking; HMM; Mobile devices; Multitouch interaction; Smartphones; Visual saliency

Indexed keywords

AUTONOMOUS UNDERWATER VEHICLES; CAMERAS; HIDDEN MARKOV MODELS; MOBILE DEVICES; SMARTPHONES; TRACKING (POSITION); VISUALIZATION;

3D MAPS; CROWDSOURCING; GAZE-TRACKING; HMM; MULTI-TOUCH INTERACTIONS; VISUAL SALIENCY;

THREE DIMENSIONAL COMPUTER GRAPHICS;

EID: 84918832835     PISSN: 10773142     EISSN: 1090235X     Source Type: Journal    
DOI: 10.1016/j.cviu.2014.07.006     Document Type: Article

References (75)

1. Luis von Ahn, and Laura Dabbish Labeling images with a computer game Proceedings of the SIGCHI Conference on Human Factors in Computing Systems CHI '04 2004 ACM Vienna, Austria 319 326
2. Luis von. Aim, Ruoran Liu, and Manuel Blum Peekaboom: a game for locating objects in images Proceedings of the SIGCHI Conference on Human Factors in Computing Systems CHI '06 2006 ACM New York, NY, USA 55 64
3. B. Baccot, V. Charvillat, R. Grigoras, C. Plesca, Visual attention metadata from pictures browsing, in: Ninth International Workshop on Image Analysis for Multimedia Interactive Services, 2008 (WIAMIS '08), 2008, pp. 122-125.
4. Michael Bewley, Bertrand Douillard, Navid Nourani-Vatani Ariell Friedman, Oscar Pizarro, and Stefan Williams Automated species detection: an experimental approach to kelp detection from sea-floor AUV images Australasian Conference on Robotics and Automation (ACRA) 2012 2012 Australian Robotics and Automation Association (ARAA) Wellington, New Zealand 1 10
5. A. Borji, L. Itti, Exploiting local and global patch rarities for saliency detection, in: Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, Rhode Island, 2012, pp. 1-8.
6. D.C. Brabham Crowdsourcing as a model for problem solving: an introduction and cases Converg.: Int. J. Res. New Media Technol. 14 1 2008 75 90
7. Andrew P. Bradley, and Fred W.M. Stentiford Visual attention for region of interest coding in JPEG 2000 J. Vis. Commun. Image Represent. 14 3 2003 232 250
8. Axel Carlier, Vincent Charvillat, Wei Tsang Ooi, Romulus Grigoras, and Geraldine Morin Crowdsourced automatic zoom and scroll for video retargeting Proceedings of the International Conference on Multimedia MM '10 2010 ACM 201 210
9. Axel Carlier, Guntur Ravindra, Vincent Charvillat, and Wei Tsang Ooi Combining content-based analysis and crowdsourcing to improve user interaction with zoomable video Proceedings of the 19th ACM International Conference on Multimedia MM '11 2011 ACM Scottsdale, Arizona, USA 43 52
10. J. Carreira, C. Sminchisescu, Constrained parametric min-cuts for automatic object segmentation, in: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010, pp. 3241-3248.
11. James H. Clark Hierarchical geometric models for visible surface algorithms Commun. ACM 19 10 1976 547 554
12. Francesco Cricri, Kostadin Dabov, Mikko J. Roininen, Sujeet Mate, Igor D.D. Cur-cio, and Moncef Gabbouj Multimodal semantics extraction from user-generated videos Adv. MultiMedia 2012 2012 1:1-1:1
13. G. Deco, and B. Schurmann A hierarchical neural system with attentional top-down enhancement of the spatial resolution for object recognition Vis. Res. 40 20 2000 2845 2859
14. Jia Deng, Wei Dong, R. Socher, Li-Jia Li, Kai Li, Li Fei-Fei, ImageNet: a large-scale hierarchical image database, in: IEEE Conference on Computer Vision and Pattern Recognition, 2009 (CVPR 2009), 2009, pp. 248-255.
15. Sean R. Eddy What is a hidden Markov model? Nat. Biotech. 22 10 2004 1315 1316
16. Ian. Endres, and Derek. Hoiem Category independent object proposals Proceedings of the 11th European Conference on Computer Vision: Part V ECCV'10 2010 Springer-Verlag Heraklion, Crete, Greece 575 588
17. Fisher Danyel, Rob DeLine, Mary Czerwinski, and Steven Drucker Interactions with big data analytics Interactions 19 3 2012 50 59
18. Carolina Galleguillos, Peter Faymonville, Serge Belongie, BUBL: an effective region labeling tool using a hexagonal lattice, in: Workshop on Emergent Issues in Large Amounts of Visual Data, Kyoto, Japan, 2009.
19. Mark Hancock, Sheelagh Carpendale, and Andy Cockburn Shallow-depth 3d interaction: design and evaluation of one-, two- and three-touch techniques Proceedings of the SIGCHI Conference on Human Factors in Computing Systems CHI '07 2007 ACM San Jose, California, USA 1147 1156
20. Jonathan Harel, Christof Koch, and Pietro Perona Graph-based visual saliency Advances in Neural Information Processing Systems vol. 19 2007 MIT Press 545 552
21. Andrew Heyward, Jane Fromont, Christine Hanna Lydia Schonberg, Jamie Colquhoun, Ben Radford, and Oliver Gomez The sponge gardens of Ningaloo Reef, Western Australia Open Marine Biol. J. 4 2010 3 11
22. Xiaodi Hou, Liqing Zhang, Saliency detection: a spectral residual approach, in: IEEE Conference on Computer Vision and Pattern Recognition, 2007 (CVPR '07), 2007, pp. 1-8.
23. Xiaodi Hou, Jonathan Harel, and Christof Koch Image signature: highlighting sparse salient regions IEEE Trans. Pattern Anal. Machine Intell. 34 1 2012 194 201
24. Xiaodi Hou, J. Harel, and C. Koch Image signature: highlighting sparse salient regions IEEE Trans. Pattern Anal. Machine Intell. 34 1 2012 194 201
25. L. Itti, C. Koch, and E. Niebur A model of saliency-based visual attention for rapid scene analysis IEEE Trans. Pattern Anal. Machine Intell. 20 11 1998 1254 1259
26. Laurent Itti, Christol Koch, and Ernst Niebur A model of saliency-based visual attention for rapid scene analysis IEEE Trans. Pattern Anal. Machine Intell. 20 11 1998 1254 1259
27. Craig Richard Johnson, Establishment of the long-spined sea urchin (Gen-trostephanus rodgersii) in Tasmania: first assessment of potential threats to fisheries, English, School of Zoology, Tasmanian Aquaculture, and Fisheries Institute, Hobart, 2005, 79 p.
28. M. Johnson-Roberson, Oscar Pizarro, Stelan B. Williams, and Ian Mahon Generation and visualization of large-scale three-dimensional reconstructions from underwater robotic surveys J. Field Robotics 27 1 2010 21 51
29. M. Johnson-Roberson, M. Bryson, B. Douillard, O. Pizarro, S.B. Williams, Out-of-core efficient blending for underwater georelerenced textured 3D maps, in: 2013 Fourth International Conference on Computing for Geospatial Research and Application (COM.Geo), 2013, pp. 8-15.
30. Nicolas Kaufrnann, Thimo Schulze, Daniel Veit, More than frm and money, worker motivation in crowdsourcing-a study on mechanical turk, in: Proceedings of the Seventeenth Americas Conference on Information Systems, Detroit, MI, 2011.
31. Hyung Jin Kim, Inchan Kim, Ho Geun Lee, The success factors for app store-like platform businesses from the perspective of third-party developers: an empirical study based on a dual model framework, in: Weidenhiller: Erfolgsfaktoren mobiler Content-Apps im Ve.rgle.ich 131, 2010, p. 272.
32. Aniket Kittur, Boris Smus, Susheel Khamkar, and Robert E. Kraut Crowd-Forge: crowdsourcing complex work Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology UIST '11 2011 ACM Santa Barbara, California, USA 43 52
33. N. Kumar, A.C. Berg, P.N. Belhumeur, S.K. Nayar, Attribute and simile classifiers for face verification, in: 2009 IEEE 12th International Conference on Computer Vision, 2009, pp. 365-372.
34. Yong Jae Lee, J. Ghosh, K. Grauman, Discovering important people and objects for egocentric video summarization, in: 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012, pp. 1346-1353.
35. Bruno Levy, Sylvain Petitjean, Nicolas Ray, and Jerome Maillot Least squares conformal maps for automatic texture atlas generation ACM Trans. Graph. 21 3 2002 362 371
36. Feng Liu, and Michael Gleicher Automatic image retargeting with fisheye-view warping Proceedings of the 18th Annual ACM Symposium on User Interface Software and Technology UIST '05 2005 ACM Seattle, WA, USA 153 162
37. Tie Liu, Jian Sun, Nan-Ning Zheng, Xiaoou Tang, Heung-Yeung Shum, Learning to detect a salient object, in: IEEE Conference on Computer Vision and Pattern Recognition, 2007 (CVPR '07), 2007, pp. 1- -8.
38. Tie Liu, Nanning Zheng, Wei Ding, Zejian Yuan, Video attention: Learning to detect a salient object sequence, in: 19th International Conference on Pattern Recognition, 2008 (ICPR 2008), 2008, pp. 1-4.
39. David G. Lowe Distinctive image features from scale-invariant keypoints Int. J. Comput. Vis. 60 2 2004 91 110
40. Ian Mahon, Stefan Williams, Oscar Pizarro, and M. Johnson-Roberson Efficient view-based SLAM using visual loop closures IEEE Trans. Robot. 24 5 2008 1002 1014
41. Adam Marcus, Michael S. Bernstein, Osama Badar, David R. Karger, Samuel Madden, and Robert C. Miller Processing and visualizing the data in tweets SIGMOD Rec. 40 4 2012 21 27
42. A. Martinet, G. Casiez, L. Grisoni, The design and evaluation of 3D positioning techniques for multi-touch displays, in: 2010 IEEE Symposium on 3D User Interfaces (3DUI), 2010, pp. 115-118.
43. Anthony Martinet, Gery Casiez, and Laurent Grisoni The effect of DOF separation in 3D manipulation tasks with multi-touch displays Proceedings of the 17th ACM Symposium on Virtual Reality Software and Technology VRST '10 2010 ACM Hong Kong 111 118
44. Albert Julian Mayer, Virtual Texturing, MA thesis, Institute of Computer Graphics and Algorithms, Vienna University of Technology, Favoritenstrasse 9-11/186, A-1040 Vienna, Austria, 2010.
45. Olivier Meur, and Thierry Baccino Methods for comparing scanpaths and saliency maps: strengths and weaknesses Behav. Res. Methods 2012 1 16
46. Lisa Meyer, Nicole Hill, Peter Walsh, Neville Barrett, Methods for the processing and scoring of AUV digital imagery from South Eastern Tasmania, Tech. rep. IMAS - Fisheries, Aquaculture and Coasts Consultant Report, 2011.
47. Martin Mittring, and Crytek GmbH Advanced virtual texture topics ACM SIGGRAPH 2008 Games SIGGRAPH '08 2008 ACM Press Los Angeles, California 23 51
48. Thi Phuong Nghiem, Axel Carlier, Geraldine Morin, and Vincent Charvillat Enhancing online 3D products through crowdsourcing Proceedings of the ACM Multimedia 2012 Workshop on Crowdsourcing for Multimedia CrowdMM '12 2012 ACM Nara, Japan 47 52
49. E. Niebur, and C. Koch Computational architectures for attention R. Parasuraman, The Attentive Brain 1998 MIT Press Cambridge, MA 163 186 (Chapter 9)
50. Stefanie Nowak, and Stefan Ruger How reliable are annotations via crowdsourcing: a study about inter-annotator agreement for multi-label image annotation Proceedings of the International Conference on Multimedia Information Retrieval MIR '10 2010 ACM Philadelphia, Pennsylvania, USA 557 566
51. Nabil Ouerhani, Roman Von Wartburg, Heinz Hugh, and Rene Muri Empirical validation of the saliency-based model of visual attention Electron. Lett. Comput. Vis. Image Anal. 3 1 2004 13 24
52. Hyun Soo Park, Eakta Jain, and Yaser Sheikh 3D gaze concurrences from head-mounted cameras Adv. Neural Inform. Process. Syst. 2012
53. Claudio M. Privitera, and Lawrence W. Stark Algorithms for defining visual regions-of-interest: comparison with eye fixations IEEE Trans. Pattern Anal. Machine Intell. 22 9 2000 970 982
54. Lawrence R. Rabiner A tutorial on hidden Markov models and selected applications in speech recognition Proc. IEEE 77 2 1989 257 286
55. Vikas C. Raykar, Shipeng Yu, Linda H. Zhao, Gerardo Hermosillo Valadez, Charles Florin, Luca Bogoni, and Linda Moy Learning from crowds J. Machine Learn. Res. 99 2010 1297 1322
56. Joel Ross, Lilly Irani, M. Six Silberman, Andrew Zaldivar, and Bill Tomlinson Who are the crowdworkers?: shifting demographics in mechanical turk CHI '10 Extended Abstracts on Human Factors in Computing Systems CHI EA '10 2010 ACM Atlanta, Georgia, USA 2863 2872
57. M. Rudinac, P.P. Jonker, Saliency detection and object localization in indoor environments, in: 2010 20th International Conference on Pattern Recognition (ICPR), 2010, pp. 404-407.
58. Dmitry Rudoy, Dan B. Goldman, Eli Shechtman, Lihi Zelnik-Manor, Crowdsourcing gaze data collection, in: CoRR abs/1204.3367, 2012.
59. Bryan C. Russell, Antonio Torralba, Kevin P. Murphy, and William T. Freeman LabelMe: a database and web-based tool for image annotation Int. J. Comput. Vis. 77 1-3 2008 157 173
60. Dario D. Salvucci An integrated model of eye movements and visual encoding Cognit. Syst. Res. 1 4 2001 201 220
61. Alia Sheffer, Bruno Levy, Maxim Mogilnitsky, and Alexander Bogomyakov ABFH - h: fast and robust angle based flattening ACM Trans. Graph. 24 2 2005 311 330
62. Dan A. Smale Regional-scale benthic monitoring for ecosystem-based fisheries management (EBFM) using an autonomous underwater vehicle (AUV) ICES J. Marine Sci.: J. Conseil 2012
63. A. Sorokin, D. Forsyth, Utility data annotation with Amazon mechanical turk, in: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, 2008 (CVPRW'08), 2008, pp. 1-8.
64. D.M. Steinberg, A. Friedman, O. Pizarro, S.B. Williams, A Bayesian nonparametric approach to clustering data from underwater robotic surveys, in: International Symposium on Robotics Research, Flagstaff, AZ, 2011.
65. Ashish Thusoo, Zheng Shao, Suresh Anthony, Dhruba Borthakur, Namit Jain, Joy-deep Sen Sarma, Raghotham Murthy, and Hao Liu Data warehousing and analytics infrastructure at facebook Proceedings of the 2010 ACM SIGMOD International Conference on Management of data SIGMOD '10 2010 ACM Indianapolis, Indiana, USA 1013 1020
66. Bill Triggs, Philip F. McLauchlan, Richard I. Hartley, and Andrew W. Fitzgibbon Bundle adjustment - a modern synthesis Proceedings of the International Workshop on Vision Algorithms: Theory and Practice ICCV '99 2000 Springer-Verlag London, UK 298 372
67. J.K. Tsotsos, S.M. Culhane, W.Y. Kei Wai, Y. Lai, N. Davis, and F. Nuflo Modeling visual attention via selective tuning Artif. Intell. 78 1-2 1995 507 545
68. A. Viterbi Error bounds for convolutional codes and an asymptotically optimum decoding algorithm IEEE Trans. Inform. Theory 13 2 1967 260 269
69. Dirk Walther, and Christof Koch Modeling attention to salient proto-objects Neural Netw. 19 9 2006 1395 1407
70. Lloyd R. Welch Hidden Markov models and the Baum-Welch algorithm IEEE Inform. Theory Soc. Newslett. 2003 53
71. P. Welinder, P. Perona, Online crowdsourcing: rating annotators and obtaining cost-effective labels, in: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), 2010, pp. 25-32.
72. S.B. Williams, O. Pizarro, M.V. Jakuba, I. Mahon, S.D. Ling, C.R. Johnson, Repeated AUV surveying of urchin barrens in North Eastern Tasmania, in: 2010 IEEE International Conference on Robotics and Automation (ICRA), 2010. pp. 293-299.
73. S.B. Williams Monitoring of benthic reference sites using an autonomous underwater vehicle IEEE Robot. Automat. Mag. 19 1 2012 73 84
74. Mengdi Xu, Bingbing Ni, Jian Dong, Zhongyang Huang, Meng Wang, and Shuicheng Yan Touch saliency Proceedings of the 20th ACM International Conference on Multimedia, MM '12 2012 ACM Nara, Japan 1041 1044
75. Lingyun Zhang, Matthew H. Tong, Tim K. Marks, Honghao Shan, and Garrison W. Cottrell SUN: a Bayesian framework for saliency using natural statistics J. Vis. 8 7 2008 1 20 32