Terabyte-scale image similarity search: Experience and best practice

Proceedings - 2013 IEEE International Conference on Big Data, Big Data 2013

Volumn , Issue , 2013, Pages 674-682

  Moise, Diana ^a   Shestakov, Denis ^a,b   Gudmundsson, Gylfi ^a   Amsaleg, Laurent ^c  

^a INRIA   (France)

^b AALTO UNIVERSITY   (Finland)

^c CAMPUS DE BEAULIEU   (France)

Author keywords

[No Author keywords available]

Indexed keywords

BIG DATA; IMAGE ANALYSIS; INFORMATION MANAGEMENT; OPEN SOURCE SOFTWARE;

BEST PRACTICES; DATA CHALLENGES; HADOOP FRAMEWORKS; IMAGE SIMILARITY; MAP-REDUCE; MAP-REDUCE PROGRAMMING; OPEN SOURCE IMPLEMENTATION; PRACTIONERS;

DATA HANDLING;

EID: 84893332207     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/BigData.2013.6691637     Document Type: Conference Paper

References (23)

1. http://gigaom.com/2012/10/17/ facebook-has-220-billion-of-your-photos-to- put-on-ice/.
2. J. Dean and S. Ghemawat, "MapReduce: simplified data processing on large clusters," Commun. ACM, vol. 51, no. 1, pp. 107-113, 2008.
3. The Apache Hadoop Project," http://www.hadoop.org.
4. K. Keahey and T. Freeman, "Science Clouds: Early experiences in Cloud computing for scientific applications," in Cloud Computing and Its Applications CCA, 2008.
5. D. Robinson, Amazon Web Services Made Simple: Learn how Amazon EC2, S3, SimpleDB and SQS Web Services enables you to reach business goals faster. Emereo Publishing, 2008.
6. The Hadoop MapReduce Framework," http://hadoop.apache. org/mapreduce/.
7. Twister-Iterative MapReduce," http://www. iterativeMapReduce.org/.
8. P. Mundkur, V. Tuulos, and J. Flatow, "Disco: a computing platform for large-scale data analytics," in Proc. of the 10th ACM SIGPLAN workshop on Erlang, 2011, pp. 84-89.
9. S. Owen, R. Anil, T. Dunning, and E. Friedman, Mahout in Action. Manning Publications, 2011.
10. H. Lejsek, F. H. Amundsson, B. T. J́onsson, and L. Amsaleg, "NV-Tree: An efficient disk-based index for approximate search in very large high-dimensional collections," IEEE Trans. Pattern Analysis and Machine Intelligence PAMI, vol. 31, pp. 869-883, 2009.
11. H. J́egou, F. Perronnin, M. Douze, J. Śanchez, P. Ṕerez, and C. Schmid, "Aggregating local image descriptors into compact codes," IEEE Trans. Pattern Analysis and Machine Intelligence PAMI, vol. 34, no. 9, pp. 1704-1716, 2012.
12. H. Lejsek, B. T. J́onsson, and L. Amsaleg, "NV-Tree: nearest neighbors at the billion scale," in ACM Proc. International Conference on Multimedia Retrieval ICMR, 2011, pp. 54:1-54:8.
13. H. Jegou, R. Tavenard, M. Douze, and L. Amsaleg, "Searching in one billion vectors: Re-rank with source coding," in IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP, 2011, pp. 861-864.
14. M. Douze, H. J́egou, H. Sandhawalia, L. Amsaleg, and C. Schmid, "Evaluation of GIST descriptors for web-scale image search," in ACM Proc. International Conference on Image and Video Retrieval CIVR, 2009, pp. 19:1-19:8.
15. M. Batko, F. Falchi, C. Lucchese, D. Novak, R. Perego, F. Rabitti, J. Sedmidubsḱy, and P. Zezula, "Building a web-scale image similarity search system," Multimedia Tools Appl., vol. 47, no. 3, pp. 599-629, 2010.
16. J. S. Hare, S. Samangooei, D. P. Dupplaw, and P. H. Lewis, "ImageTerrier: an extensible platform for scalable highperformance image retrieval," in ACM Proc. International Conference on Multimedia Retrieval ICMR, 2012, pp. 40:1-40:8.
17. K. Shvachko, H. Kuang, S. Radia, and R. Chansler, "The Hadoop Distributed File System," in IEEE Proc. Mass Storage Systems and Technologies MSST, 2010, pp. 1-10.
18. D. Moise, D. Shestakov, G. Gudmundsson, and L. Amsaleg, "Indexing and Searching 100M Images with Map-Reduce," in International Conference on Multimedia Retrieval ICMR, 2013, pp. 17-24.
19. D. Nister and H. Stewenius, "Scalable recognition with a vocabulary tree," in IEEE Proc. Computer Society Conference on Computer Vision and Pattern Recognition CVPR, 2006, pp. 2161-2168.
20. D. Shestakov, D. Moise, G. Gudmundsson, and L. Amsaleg, "Scalable high-dimensional indexing with Hadoop," in Content-Based Multimedia Indexing CBMI, 2013, pp. 207-212.
21. Y. J́egou, S. Lant́eri, J. Leduc, and all, "Grid'5000: a large scale and highly reconfigurable experimental Grid testbed." Intl. Journal of HPC Applications, vol. 20, no. 4, pp. 481-494, 2006.
22. F. Ahmad, S. T. Chakradhar, A. Raghunathan, and T. N. Vijaykumar, "Tarazu: optimizing mapreduce on heterogeneous clusters," SIGARCH Comput. Archit. News, vol. 40, no. 1, pp. 61-74, 2012.
23. A. Shinnar, D. Cunningham, V. Saraswat, and B. Herta, "M3R: increased performance for in-memory Hadoop jobs," Proc. VLDB Endow., vol. 5, no. 12, pp. 1736-1747, 2012.