Mammoth data in the cloud: Clustering social images

Advances in Parallel Computing

Volumn 23, Issue , 2013, Pages 231-246

  Qiu, Judy ^a   Zhang, Bingjing ^a  

^a INDIANA UNIVERSITY   (United States)

Author keywords

Collective Communication; Fast Kmeans Algorithm; High Dimension; Iterative MapReduce; Social Images

Indexed keywords

EID: 84895087448     PISSN: 09275452     EISSN: None     Source Type: Book Series    
DOI: 10.3233/978-1-61499-322-3-231     Document Type: Article

References (45)

1. Dubey, Pradeep. A Platform 2015 Model: Recognition, Mining and Synthesis Moves Computers to the Era of Tera. Compute-Intensive, Highly Parallel Applications and Uses. Volume 09 Issue 02. ISSN 1535-864X. February 2005.
2. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. Sixth Symp. on Operating System Design and Implementation, pp. 137-150, December 2004.
3. Apache Hadoop. http://hadoop.apache.org.
4. J.Ekanayake, H. Li, B. Zhang, T. Gunarathne, S. Bae, J. Qiu, G. Fox, Twister: A Runtime for iterative MapReduce, in Proceedings of the First International Workshop on MapReduce and its Applications of ACM HPDC 2010 conference June 20-25, 2010.
5. Yingyi Bu, Bill Howe, Magdalena Balazinska, and Michael D. Ernst. Haloop: Efficient Iterative Data Processing on Large Clusters. Proceedings of the VLDB Endowment, 3, September 2010.
6. J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, Q. Le, M. Mao, M. Ranzato, A. Senior, P. Tucker, K. Yang, A. Ng, Large Scale Distributed Deep Networks, in proceedings of NIPS 2012: Neural Information Processing Systems Conference.
7. MPI Forum, "MPI: A Message Passing Interface," in Proceedings of Supercomputing, 1993.
8. PolarGrid. http://polargrid.org/polargrid.
9. John Markoff, "Scientists See Promise in Deep-Learning Programs", New York Times, November 23, 2012.
10. Quoc V. Le et al., "Building High-level Features Using Large Scale Unsupervised Learning", in proceedings of International Conference in Machine Learning 2012.
11. Adam Coates, Andrej Karpathy, Andrew Y. Ng, "Emergence of Object-Selective Features in Unsupervised Feature Learning", in proceedings of NIPS 2012: Neural Information Processing Systems Conference.
12. Adam Coates and Andrew Y. Ng, "Learning Feature Representations with K-means", in proceedings of Neural Networks: Tricks of the Trade, Reloaded, Springer LNCS, 2012.
13. N. Dalal, B. Triggs. Histograms of Oriented Gradients for Human Detection. CVPR. 2005
14. J. B. MacQueen, Some Methods for Classification and Analysis of MultiVariate Observations, in Proc. of the fifth Berkeley Symposium on Mathematical Statistics and Probability. vol. 1, L. M. L. Cam and J. Neyman, Eds., ed: University of California Press, 1967.
15. Charles Elkan, Using the triangle inequality to accelerate k-means, in TWENTIETH INTERNATIONAL CONFERENCE ON MACHINE LEARNING, Tom Fawcett and Nina Mishra, Editors. August 21-24, 2003. Washington DC. pages. 147-153.
16. Jonathan Drake and Greg Hamerly, Accelerated k-means with adaptive distance bounds, in 5th NIPS Workshop on Optimization for Machine Learning. Dec 8th, 2012. Lake Tahoe, Nevada, USA,.
17. E. Chan, M. Heimlich, A. Purkayastha, and R. A. van de Geijn. Collective communication: theory, practice, and experience. Concurrency and Computation: Practice and Experience, 2007, vol 19, pp. 1749-1783.
18. Nikhil Jain, Yogish Sabharwal, Optimal Bucket Algorithms for Large MPI Collectives on Torus Interconnects, ICS '10 Proceedings of the 24th ACM International Conference on Supercomputing, 2010
19. R. Power and J. Li. Piccolo: building fast, distributed programs with partitioned tables. In OSDI '10, pages 1-14, 2010.
20. Yanfeng Zhang, Qixin Gao, Lixin Gao, and Cuirong Wang, "PrIter: A Distributed Framework for Prioritized Iterative Computations", in Proceeding of the 2nd ACM Symposium on Cloud Computing (ACM SOCC 2011)
21. ActiveMQ. http://activemq.apache.org/
22. S. Pallickara, G. Fox, NaradaBrokering: A Distributed Middleware Framework and Architecture for Enabling Durable Peer to-Peer Grids, Middleware 2003, 2003.
23. Ford L.R. Jr., Fulkerson D.R., Maximal Flow through a Network, Canadian Journal of Mathematics, 1956, pp.399-404.
24. K. Shvachko, H. Kuang, S. Radia, and R. Chansler, The Hadoop Distributed File System. IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), 2010
25. Mosharaf Chowdhury et al. Managing Data Transfers in Computer Clusters with Orchestra, Proceedings of the ACM SIGCOMM 2011 conference, 2011
26. T. Hoefler, C. Siebert, and W. Rehm. Infiniband Multicast A practically constant-time MPI Broadcast Algorithm for large-scale InfiniBand Clusters with Multicast. Proceedings of the 21st IEEE International Parallel & Distributed Processing Symposium. 2007
27. Watts J, van de Geijn R. A pipelined broadcast for multidimensional meshes. Parallel Processing Letters, 1995, vol.5, pp. 281-292.
28. T. Kielmann, H. Bal, S. Gorlatch, K. Verstoep, R. Hofman, Network performance-aware collective communication for clustered wide-area systems, Journal of Parallel Computing. PP. 1431-1446, Volume 27 Issue 11, October 2001.
29. Charles E. Leiserson, Fat-trees: universal networks for hardware efficient supercomputing, IEEE Transactions on Computers, vol. 34 , no. 10, Oct. 1985, pp. 892-901.
30. S. Kumar, Y. Sabharwal, R. Garg, P. Heidelberger, Optimization of All-to-all Communication on the Blue Gene/L Supercomputer, 37th International Conference on Parallel Processing, 2008
31. Open MPI, http://www.open-mpi.org
32. MPJ Express, http://mpj-express.org/
33. H. Mamadou T. Nanri, and K. Murakami. A Robust Dynamic Optimization for MPI AlltoAll Operation, IPDPS'09 Proceedings of IEEE International Symposium on Parallel & Distributed Processing, 2009
34. P. Balaji, A. Chan, R. Thakur, W. Gropp, and E. Lusk. Toward message passing for a million processes: Characterizing MPI on a massive scale Blue Gene/P. Computer Science-Research and Development, vol. 24, pp. 11-19, 2009.
35. M. Zaharia, M. Chowdhury, M. J. Franklin, S. Shenker, and I. Stoica. Spark: Cluster Computing with Working Sets. In HotCloud, 2010.
36. BitTorrent. http://www.bittorrent.com.
37. Yangdong Wang et al. Hadoop Acceleration Through Network Levitated Merge, International Conference for High Performance Computing, Networking, Storage and Analysis (SC'11), 2011
38. Infiniband Trade Association. http://www.infinibandta.org.
39. T. Bicer, D. Chiu, and G. Agrawal. MATE-EC2: A Middleware for Processing Data with AWS, Proceedings of the 2011 ACM international workshop on Many task computing on grids and supercomputers, 2011
40. EC2. http://aws.amazon.com/ec2/.
41. S3. http://aws.amazon.com/s3/.
42. Y. Zhang, Q. Gao, L. Gao, and C. Wang. imapreduce: A distributed computing framework for iterative computation. In DataCloud '11, 2011.
43. E. Elnikety, T. Elsayed, and H. Ramadan. iHadoop: Asynchronous Iterations for MapReduce, Proceedings of the 3rd IEEE International conference on Cloud Computing Technology and Science (CloudCom), 2011
44. B. Zhang, Y. Ruan, T. Wu, J. Qiu, A. Hugh, G. Fox. Applying Twister to Scientific Applications, Proceedings of the 2nd IEEE International conference on Cloud Computing Technology and Science (CloudCom), 2010
45. P. Hunt, M. Konar, F. P. Junqueira, and B. Reed, ZooKeeper: wait-free coordination for internet-scale systems, in USENIXATC'10: USENIX conference on USENIX annual technical conference, 2010, pp. 11-11.