TomusBlobs: Scalable data-intensive processing on Azure clouds

Concurrency and Computation: Practice and Experience

Volumn 28, Issue 4, 2016, Pages 950-976

  Costan, Alexandru ^a   Tudoran, Radu ^a   Antoniu, Gabriel ^a   Brasche, Goetz ^b  

^a INRIA   (France)

^b MICROSOFT   (United States)

Author keywords

Azure; big data; cloud computing; cloud storage; data intensive processing; MapReduce; scientific applications

Indexed keywords

BENCHMARKING; BIG DATA; CLOUD COMPUTING; DATA HANDLING; DIGITAL STORAGE; WINDOWS OPERATING SYSTEM;

AZURE; CLOUD STORAGES; DATA INTENSIVE; MAP-REDUCE; SCIENTIFIC APPLICATIONS;

STORAGE AS A SERVICE (STAAS);

EID: 84958914447     PISSN: 15320626     EISSN: 15320634     Source Type: Journal    
DOI: 10.1002/cpe.3034     Document Type: Conference Paper

References (30)

1. Hey T, Tansley S, Tolle KM, (eds). The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research: Redmond, WA, 2009.
2. The 1000 genomes project. (Available from: http://aws.amazon.com/fr/1000genomes/).
3. Azure. (Available from: http://www.windowsazure.com/en-us/).
4. HDFS. (Available from: http://hadoop.apache.org/hdfs/).
5. Carns P, Ligon WB, Ross RB, Thakur R,. PVFS: A parallel file system for linux clusters. In Proceedings of the 4th Annual Linux Showcase & Conference 2000, Atlanta, USA, 2000; 317-327.
6. Gunarathne T, Wu T-L, Qiu J, Fox G,. Mapreduce in the clouds for science. In Cloudcom10, 2010; 565-572.
7. Tudoran R, Costan A, Antoniu G, Soncu H,. TomusBlobs: Towards communication-efficient storage for MapReduce applications in Azure. In 12th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid'2012), Ottawa, Canada, 2012; 427-434.
8. Tudoran R, Costan A, Antoniu G,. MapIterativeReduce: A framework for reduction-intensive data processing on Azure clouds. In Third International Workshop on MapReduce and its Applications (MAPREDUCE'12), held in conjunction with ACM HPDC'12., Delft, Netherlands, 2012; 9-16.
9. Luo Y, Plale B,. Hierarchical MapReduce Programming Model and Scheduling Algorithms, 2012.
10. Tatebe O, Hiraga K, Soda N,. Gfarm grid file system. In New Generation Computing. Verlag Omsha: Tokyo, Japan, 2010; 257-275.
11. A-Brain. (Available from: http://www.msr-inria.inria.fr/Projects/a-brain).
12. Poline J, Lalanne C, Tenenhaus A, Duchesnay E, Thirion B, Frouin V,. Imaging genetics: Bio-informatics and bio-statistics challenges. In COMPSTAT, Vol. 1, Paris, 2010; 101-113.
13. Dorier M, Antoniu G, Cappello F, Snir M, Orf L,. Damaris: How to efficiently leverage multicore parallelism to achieve scalable, jitter-free I/O. In Cluster, 2012; 155-163.
14. Tudoran R, Costan A, Antoniu G, Bougé L,. A performance evaluation of Azure and nimbus clouds for scientific applications. In CloudCP 2012-2nd International Workshop on Cloud Computing Platforms, Held in Conjunction with the ACM SIGOPS Eurosys 12 Conference, Bern, Switzerland, 2012; 41-46.
15. Azure priceing. (Available from: https://www.windowsazure.com/en-us/pricing/details/).
16. Ostermann S, Iosup A, Yigitbasi N, Prodan R, Fahringer T, Epema DHJ,. A performance analysis of EC2 cloud computing services for scientific computing. In CloudComp'2009, Munich, Germany, 2009; 115-131.
17. Vecchiola C, Pandey S, Buyya R,. High-performance cloud computing: A view of scientific applications, pervasive systems, algorithms, and networksm. ISPAN9, Kaohsiung, Taiwan, 2009; 4-16.
18. Azure drives. (Available from: http://msdn.microsoft.com/en-us/library/windowsazure/ee924681.aspx).
19. Schwan P,. Lustre: Building a file system for 1000-node clusters. In Proceedings of the Linux Symposium 2003, Ottawa, Ontario, Canada, 2003; 380-386.
20. Eucalyptus. (Available from: http://www.eucalyptus.com/).
21. Keahey K, Freeman T, Bresnahan J, LaBissoniere D,. Cumulus: Open Source Storage Cloud for Science. In Sc2010 Poster, New Orleans, LA, USA, 2010; 25-32.
22. Amazon web service. (Available from: http://aws.amazon.com/).
23. Moise D, Carpen-Amarie A, Antoniu G, Bougé L,. A cost-evaluation of MapReduce applications in the cloud. In Proceedings of the Grid5000 School, Reims, France, 2011; 37-48.
24. Nicolae B, Antoniu G, Bougé L, Moise D, Carpen-Amarie A,. BlobSeer: Next generation data management for large scale infrastructures. Journal of Parallel and Distributed Computing 2011; 71 (2): 169-184.
25. Zaharia M, Chowdhury M, Franklin MJ, Shenker S, Stoica I,. Spark: cluster computing with working sets. In Hotcloud'10. USENIX: Berkeley, CA, 2010; 10-10.
26. Dave A, Lu W, Jackson J, Barga R,. Cloudclustering: Toward an iterative data processing pattern on the cloud. IPDPSW, 2011, Anchorage, Alaska, USA, May 2011; 1132-1137.
27. Bu Y, Howe B, Balazinska M, Ernst MD,. Haloop: Efficient iterative data processing on large clusters. Proceedings of the VLDB Endowment 2010; 3: 285-296.
28. Ekanayake J, Li H, Zhang B, Gunarathne T, Bae S-H, Qiu J, Fox G,. Twister: A runtime for iterative mapreduce. In HPDC '10, Chicago, Illinois, USA, 2010; 810-818.
29. Gunarathne T, Zhang B, Wu T-L, Qiu J,. Scalable parallel computing on clouds using twister4azure iterative mapreduce. Future Generation Computer Systems 2012; 29 (4): 1035-1048.
30. Zhang Y, Gao Q, Gao L, Wang C,. imapreduce: A distributed computing framework for iterative computation. In IPDPSW, 2011, Anchorage, Alaska, USA, May 2011; 1112-1121.