A Novel Methodology to Label Urban Remote Sensing Images Based on Location-Based Social Media Photos

Proceedings of the IEEE

Volumn 105, Issue 10, 2017, Pages 1926-1936

  Chi, Mingmin ^a   Sun, Zhongyi ^a   Qin, Yiqing ^a   Shen, Jinsheng ^a   Benediktsson, Jon Atli ^b  

^a FUDAN UNIVERSITY   (China)

^b UNIVERSITY OF ICELAND   (Iceland)

Author keywords

Big data; crowdsourcing; deep learning; remote sensing; social media

Indexed keywords

ARTIFICIAL INTELLIGENCE; BIG DATA; CHARGE COUPLED DEVICES; CROWDSOURCING; DEEP LEARNING; IMAGE RESOLUTION; LOCATION; PHOTOINTERPRETATION; REMOTE SENSING; SEMANTICS; SOCIAL NETWORKING (ONLINE);

AUTOMATIC CLASSIFICATION SYSTEMS; CLASSIFICATION RESULTS; DATA ANALYSIS TECHNIQUES; HIGH SPATIAL RESOLUTION; MULTIPLE SATELLITE SYSTEMS; QUALITATIVE EXPERIMENTS; SOCIAL MEDIA; URBAN REMOTE SENSING IMAGES;

IMAGE CLASSIFICATION;

EID: 85029173213     PISSN: 00189219     EISSN: 15582256     Source Type: Journal    
DOI: 10.1109/JPROC.2017.2730585     Document Type: Article

References (60)

1. J. M. Shapiro, "Smart cities: Quality of life, productivity, and the growth effects of human capital, " Rev. Econ. Stat., vol. 88, no. 2, pp. 324-335, 2006.
2. P. Gamba, "Human settlements: A global challenge for EO data processing and interpretation, " Proc. IEEE, vol. 101, no. 3, pp. 570-581, Mar. 2013.
3. M. T. Eismann, A. D. Stocker, and N. M. Nasrabadi, "Automated hyperspectral cueing for civilian search and rescue, " Proc. IEEE, vol. 97, no. 6, pp. 1031-1055, Jun. 2009.
4. S. B. Serpico, S. Dellepiane, G. Boni, G. Moser, E. Angiati, and R. Rudari, "Information extraction from remote sensing images for flood monitoring and damage evaluation, " Proc. IEEE, vol. 100, no. 10, pp. 2946-2970, Oct. 2012.
5. O. Nowakowski. (Oct. 2015). China Launches Four JILIN-1 Earth-Observing Satellites Into Space. [Online]. Available: http://www. spaceflightinsider. com/missions/earthscience/ china-launches-four-jilin-1-earthobserving-satellitesspace/# G47e1wHsF8rVeu08. 99
6. G. Cervone, E. Sava, Q. Huang, E. Schnebele, J. Harrison, and N. Waters, "Using twitter for tasking remote-sensing data collection and damage assessment: 2013 Boulder flood case study, " Int. J. Remote Sens., vol. 37, no. 1, pp. 100-124, Dec. 2015.
7. L. Barrington, S. Ghosh, M. Greene, and S. Harnoy, "Crowdsourcing earthquake damage assessment using remote sensing imagery, " Ann. Geophys., vol. 54, no. 6, pp. 680-687, 2011.
8. E. Schnebele, C. Oxendine, G. Cervone, C. M. Ferreira, and N. Waters, "Computational approaches for urban environments, " in Using Non-Authoritative Sources During Emergencies in Urban Areas. New York, NY, USA: Springer-Verlag, 2015, pp. 337-361, ch. 14.
9. C. Aggarwal and T. Abdelzaher, Managing and Mining Sensor Data. New York, NY, USA: Springer-Verlag, 2013, pp. 237-275.
10. R. Ganti, F. Ye, and H. Lei, "Mobile crowdsensing: Current state and future challenges, " IEEE Commun. Mag., vol. 49, no. 11, pp. 32-39, Nov. 2011.
11. C. Chen, Signal and Image Processing for Remote Sensing, 2nd ed. Boca Raton, FL, USA: CRC Press, Feb. 2012.
12. M. Chi, A. Plaza, J. A. Benediktsson, Z. Sun, J. Sheng, and Y. Zhu, "Big data for remote sensing: Challenges and opportunities, " Proc. IEEE, vol. 104, no. 11, pp. 2207-2219, Nov. 2016.
13. A. Babenko and V. Lempitsky, "Aggregating deep convolutional features for image retrieval, " CoRR, Oct. 2015.
14. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition, " CoRR, Sep. 2014.
15. Y. LeCun et al., "Backpropagation applied to handwritten zip code recognition, " Neural Comput., vol. 1, no. 4, pp. 541-551, 1989.
16. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks, " in Proc. 25th Int. Conf. Neural Inf. Process. Syst. (NIPS), 2012, pp. 1097-1105.
17. D. A. Cohn, Z. Ghahramani, and M. I. Jordan, "Active learning with statistical models, " J. Artif. Intell. Res., vol. 4, pp. 129-145, Mar. 1996.
18. M. M. Crawford, D. Tuia, and H. L. Yang, "Active learning: Any value for classification of remotely sensed data" Proc. IEEE, vol. 101, no. 3, pp. 593-608, Mar. 2013.
19. Z. Sun and M. Chi, "Superpixel-based active learning for the classification of hyperspectral images, " in Proc. 7th Workshop Hyperspectral Image Signal Process., Evol. Remote Sens., Tokyo, Japan, Jun. 2015.
20. M. Chi and L. Bruzzone, "Semisupervised classification of hyperspectral images by SVMs optimized in the primal, " IEEE Trans. Geosci. Remote Sens., vol. 45, no. 6, pp. 1870-1880, Jun. 2007.
21. O. Chapelle, B. Schölkopf, and A. Zien, Eds., Semi-Supervised Learning. Cambridge, MA, USA: MIT Press, 2006.
22. X. Zhu, "Semi-supervised learning literature survey, " Ph. D. dissertation, Dept. Comput. Sci., Univ. Wisconsin, Madison, WI, USA, 2008.
23. S. Tong and D. Koller, "Support vector machine active learning with applications to text classification, " J. Mach. Learn. Res., vol. 2, pp. 45-66, Mar. 2001.
24. B. Settles and M. Craven, "An analysis of active learning strategies for sequence labeling tasks, " in Proc. Conf. Empirical Methods Natural Lang. Process. (EMNLP), Stroudsburg, PA, USA, 2008, pp. 1070-1079.
25. A. J. Joshi, P. Fatih, and N. P. Papanikolopoulos, "Scalable active learning for multiclass image classification, " IEEE Trans. Softw. Eng., vol. 34, no. 11, pp. 2259-2273, Nov. 2012.
26. D. Angluin, "Queries and concept learning, " Mach. Learn., vol. 2, no. 4, pp. 319-342, 1987.
27. D. Cohn, L. Atlas, and R. Ladner, "Improving generalization with active learning, " Mach. Learn., vol. 15, no. 2, pp. 201-221, 1996.
28. D. D. Lewis and W. A. Gale, A Sequential Algorithm for Training Text Classifiers. London, U. K.: Springer-Verlag, 1994.
29. D. Tuia, M. Volpi, L. Copa, M. Kanevski, and J. Munoz-Mari, "A survey of active learning algorithms for supervised remote sensing image classification, " IEEE J. Sel. Topics Signal Process., vol. 5, no. 3, pp. 606-617, Jun. 2011.
30. M. M. Crawford, D. Tuia, and H. L. Yang, "Active learning: Any value for classification of remotely sensed data" Proc. IEEE, vol. 100, no. 3, pp. 593-608, Mar. 2013.
31. B. Settles, "Active learning literature survey, " Dept. Comput. Sci., Univ. Wisconsin, Madison, WI, USA, Tech. Rep., 1648, Jan. 2010.
32. C. Campbell, N. Cristianini, and A. J. Smola, "Query learning with large margin classifiers, " in Proc. 17th Int. Conf. Mach. Learn., San Francisco, CA, USA, 2000, pp. 111-118.
33. A. Vlachos, "A stopping criterion for active learning, " Comput. Speech Lang., vol. 22, no. 3, pp. 295-312, 2008.
34. B. Demir, C. Persello, and L. Bruzzone, "Batch-mode active-learning methods for the interactive classification of remote sensing images, " IEEE Trans. Geosci. Remote Sens., vol. 49, no. 3, pp. 1014-1031, Mar. 2011.
35. C. Cortes and V. Vapnik, "Support-vector networks, " Mach. Learn., vol. 20, no. 3, pp. 273-297, 1995.
36. Z. Sun and M. Chi, "Superpixel-based active learning for the classification of hyperspectral images, " in Proc. 7th Workshop Hyperspectral Image Signal Process., Evol. Remote Sens., Tokyo, Japan, Jun. 2015, pp. 1-41.
37. J. Howe, "The rise of crowdsourcing, " Wired Mag., vol. 14, no. 6, pp. 1-4, Jun. 2006.
38. D. C. Brabham, "Crowdsourcing as a model for problem solving: An introduction and cases, " Converg., Int. J. Res. Into New Media Technol., vol. 14, no. 1, pp. 75-90, 2008.
39. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "ImageNet: A large-scale hierarchical image database, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2009, pp. 248-255.
40. Amazon Turkey Market. [Online]. Available: https://www. mturk. com/mturk/welcome
41. O. Russakovsky et al., "ImageNet large scale visual recognition challenge, " Int. J. Comput. Vis., vol. 115, no. 3, pp. 211-252, Dec. 2015.
42. Imagenet Large Scale Visual Recognition Competition. [Online]. Available: http://www. image-net. org/challenges/LSVRC/
43. G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief nets, " Neural Comput., vol. 18, no. 7, pp. 1527-1554, 2006.
44. V. Ambati, S. Vogel, and J. Carbonell, "Active learning and crowd-sourcing for machine translation, " in Proc. 7th Int. Conf. Lang. Resour. Eval. (LREC), Valletta, Malta, May 2010, pp. 2169-2174.
45. J. Costa, C. Silva, M. Antunes, and B. Ribeiro, "On using crowdsourcing and active learning to improve classification performance, " in Proc. 11th Int. Conf. Intell. Syst. Des. Appl. (ISDA), Cordoba, Spain, 2011, pp. 469-474.
46. A. Brew, D. Greene, and P. Cunningham, "Using crowdsourcing and active learning to track sentiment in online media, " in Proc. 19th Eur. Conf. Artif. Intell. Conf. (ECAI), Amsterdam, The Netherlands, 2010, pp. 145-150.
47. M. Allahbakhsh, B. Benatallah, A. Ignjatovic, H. R. Motahari-Nezhad, E. Bertino, and S. Dustdar, "Quality control in crowdsourcing systems: Issues and directions, " IEEE Internet Comput., vol. 17, no. 2, pp. 76-81, Mar. 2013.
48. M. Lease, "On quality control and machine learning in crowdsourcing, " in Proc. 11th AAAI Conf. Human Comput. (AAAIWS), 2011, pp. 97-102.
49. Mashhadi and L. Capra, "Quality control for real-time ubiquitous crowdsourcing, " in Proc. 2nd Int. Workshop Ubiquitous Crowdsouring (UbiCrowd), New York, NY, USA, 2011, pp. 5-8.
50. T. Kanungo, D. M. Mount, N. S. Netanyahu, C. D. Piatko, R. Silverman, and A. Y. Wu, "An efficient k-means clustering algorithm: Analysis and implementation, " IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, pp. 881-892, Jul. 2002.
51. Y. Liu, D. Zhang, G. Lu, and W.-Y. Ma, "A survey of content-based image retrieval with high-level semantics, " Pattern Recognit., vol. 40, no. 1, pp. 262-282, 2007.
52. D. G. Lowe, "Object recognition from local scale-invariant features, " in Proc. Int. Conf. Comput. Vis., Kerkyra, Greece, Sep. 1999, pp. 1150-1159.
53. L. Fei-Fei and P. Perona, "A Bayesian hierarchical model for learning natural scene categories, " in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2. Jun. 2005, pp. 524-531.
54. H. Jégou, M. Douze, C. Schmid, and P. Pérez, "Aggregating local descriptors into a compact image representation, " in Proc. CVPR, Jun. 2010, vol. 238. no. 6, pp. 3304-3311.
55. F. Perronnin, J. Sánchez, and T. Mensink, "Improving the fisher kernel for large-scale image classification, " in Proc. Eur. Conf. Comput. Vis. (ECCV), Heraklion, Greece, Sep. 2010, pp. 119-133.
56. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection, " in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., San Diego, CA, USA, Jun. 2005, pp. 886-893.
57. R. Girshick, J. Donahue, T. Darrell, and J. Malik, "Rich feature hierarchies for accurate object detection and semantic segmentation, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Washington, DC, USA, Jun. 2014, pp. 580-587.
58. J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: Unified, real-time object detection, " CoRR, Jun. 2015.
59. Y. Jia et al., "Caffe: Convolutional architecture for fast feature embedding, " in Proc. 22nd ACM Int. Conf. Multimedia, Orlando, FL, USA, 2014, pp. 675-678.
60. L. Breiman, "Random forests, " Mach. Learn., vol. 45, no. 1, pp. 5-32, 2001.