Scalable state management for scientific applications in the cloud

Proceedings - 2014 IEEE International Congress on Big Data, BigData Congress 2014

Volumn , Issue , 2014, Pages 204-211

  Li, Tonglin ^a   Raicu, Ioan ^a,b   Ramakrishnan, Lavanya ^c  

^a Illinois Institute of Technology   (United States)

^b ARGONNE NATIONAL LABORATORY   (United States)

^c LAWRENCE BERKELEY NATIONAL LABORATORY   (United States)

Author keywords

cloud computing; Data management; provenance; scientific computing; state management

Indexed keywords

CLOUD COMPUTING; ENVIRONMENTAL MANAGEMENT; NATURAL SCIENCES COMPUTING;

CLOUD COMPUTING ENVIRONMENTS; CLOUD ENVIRONMENTS; EXECUTION FRAMEWORK; EXPERIMENTAL FACILITIES; PROVENANCE; SCIENTIFIC APPLICATIONS; SCIENTIFIC SIMULATIONS; STATE MANAGEMENT;

INFORMATION MANAGEMENT;

EID: 84923853282     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/BigData.Congress.2014.37     Document Type: Conference Paper

References (34)

1. Frew, J., M Etzger, D., Slaughter, P. Automatic capture and reconstruction of computational provenance. Concurrency and Computation: Practice and Experience 20 (April 2008), 485-496
2. Muniswamy-Reddy, K.-K., H Olland, D. A., B Raun, U., Seltzer, M. Provenance-Aware storage systems. In Proceedings of the 2006 USENIX Annual Technical Conference
3. Peter Buneman and Wang-Chiew Tan. 2007. Provenance in databases. In Proceedings of the SIGMOD '07. ACM, New York, NY, USA, 1171-1173
4. Kiran-Kuma r Muniswamy-Reddy, Peter Macko, and Margo Seltzer. 2010. Provenance for the cloud. In Proceedings of FAST'10. USENIX Association, Berkeley, CA, USA, 15-14.
5. Yong Zhao, Mihael Hategan, Ben Clifford, Ian Foster, Gregor von Laszewski, Ioan Raicu, Tiberiu Stef-Praun, Mike Wilde. "Swift: Fast, Reliable, Loosely Coupled Parallel Computation ", IEEE Workshop on Scientific Workflows 2007
6. Jihie Kim, Ewa Deelman, Yolanda Gil, Gaurang Mehta, Varun Ratnakar. Provenance Trails in the Wings/Pegasus Workflow System. Concurrency and Computation: Practice and Experience, Special Issue on the First Provenance Challenge, 2007
7. Yogesh L. Simmhan, Beth Plale, Dennis Gannon, and Suresh Marru. 2006. Performance evaluation of the karma provenance framework for scientific workflows. In Proceedings of IPAW'06, Luc Moreau and Ian Foster (Eds.).Springer-Verlag, Berlin, Heidelberg, 222-236
8. Ioan Raicu, Ian Foster. " Many-Task Computing: Bridging the Gap between High Throughput Computing and High Performance Computing" , Computer Science Dept., University of Chicago, Doctorate Dissertation, March 2009
9. Y. Zhao, I. Raicu, S. Lu, X. Fei. " Opportunities and Challenges in Running Scientific Workflows on the Cloud" , IEEE CyberC 2011
10. Michael Wilde, Ioan Raicu, Allan Espinosa, Zhao Zhang, Ben Clifford, Mihael Hategan, Kamil Iskra, Pete Beckman, Ian Foster. "Extreme-scale scripting: Opportunities for large task-parallel applicatio ns on petascale computers", SciDAC 2009
11. Ioan Raicu,Ian Foster,Yong Zhao,Alex Szalay,Philip Little,Christopher M. Moretti Amitabh Chaudhary, Douglas Thain Towards Data Intensive Many-Task Computing " book chapter in "Data Intensive Distributed Computing: Challenges and Solutions for Large-Scale Information Management", IGI Global Publishers 2009
12. Jin Shao, Hao Wei, Qianxiang Wang, and Hong Mei. 2010. A Runtime Model Based Monitoring Approach for Cloud. In Proceedings of the 2010 IEEE 3rd International Conference on Cloud Computing (CLOUD '10). IEEE Computer Society, Washington, DC, USA, 313-320.
13. F. Barbon, P. Traverso, M. Pistore, and M. Trainotti, " Run-time monitoring of instances and classes of web service com-positions," in Web Services, 2006. ICWS '06. pp. 63-71
14. Matthew L Massie, Brent N Chun, David E Culler, The ganglia distributed monitoring system: design, implementation, and experience, Parallel Computing, Volume 30, Issue 7, July 2004, Pages 817-840
15. Devarshi Ghoshal and Lavanya Ramakrishnan. 2012. FRIEDA: Flexible Robust Intelligent Elastic Data Management in Cloud Environments. In Proceedings of SCC '12. IEEE Computer Society, Washington, DC, USA, 1096-1105
16. Rafail Ostrovsky and Boaz Patt-Shamir. 1999. Optimal and efficient clock synchronization under drifting clocks. In PODC '99. ACM, New York, NY, USA, 3-12
17. Lavanya Ram akrishnan, Pradeep K. Mantha, Yushu Yao, Richard S. Canon, " Evaluation of NoSQL and Array Databases for Scientific Applications".DataCloud Workshop 2013
18. Tonglin Li, Raman Verma, Xi Duan, Hui Jin, and Ioan Raicu. 2011. Exploring distributed hash tables in HighEnd computing. SIGMETRICS Perform. Eval. Rev. 39, 3 (December 2011), 128-130
19. Tonglin Li, Xiaobing Zhou, Kevin Brandstatter, Dongfang Zhao, Ke Wang, Anupam Rajendran, Zhao Zhang, and Ioan Raicu. 2013. ZHT: A Light-Weight Reliable Persistent Dynamic Scalable Zero-Hop Distributed Hash Table. In Proceedings of IPDPS '13. IEEE Computer Society, Washin gton, DC, USA, 775-787
20. Mattern, F. (October 1988), "Virtual Time and Global States of Distributed Systems " , in Cosnard, M., Proc. Workshop on Parallel and Distributed Algorithms: Elsevier, pp. 215-226
21. FutureGrid https://portal.futuregrid.org/, 2014
22. Amazon EC2 Cloud http://aws.amazon.com/ec2/, 2014
23. Yogesh L. Simmhan, Beth Plale, and Dennis Gannon. 2005. A survey of data provenance in e-science. SIGMOD Rec. 34, 3 (September 2005), 31-36. DOI=10.1145/1084805.1084812
24. Dongfang Zhao, Chen Shou, Tanu Malik and Ioan Raicu. Distributed data provenance for large-scale data-intensive computing. IEEE CLUSTER '13
25. You-Wei Cheah , Canon, R. , Plale, B. ; Ramakrishnan, L. Milieu: Lightweight and Configurable Big Data Provenance for Science, Big Data (BigData Congress), 2013
26. G. DeCandia, D. Hastorun, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, W. Vogels. "Dynamo: Amazon's Highly Available Key-Value Store. " SIGOPS Operating Systems Review, 2007
27. Riak, http://docs.basho.com/riak/latest/, 2013
28. Avinash Lakshman and Prashant Malik. 2010. Cassandra: a decentralized structured storage system. SIGOPS Oper. Syst. Rev. 44, 2 (April 2010), 35-40
29. CouchDB, http://couchdb.apache.org/, 2014
30. Rajesh Nishtala, Ha ns Fugal, Steven Grimm, Marc Kwiatkowski. 2013. Scaling Memcache at Facebook. In nsdi'13, USENIX Association, Berkeley, CA, USA, 385-398
31. James C. Corbett, Jeffrey Dean, Michael Epstein, etc. 2012. Spanner: Google's globally-distributed database. In proceeding of OSDI'12. USENIX Association, Berkeley, CA, USA, 251-264
32. Precision Time Protocol, http://www.ieee1588.com/, 2014
33. David L Mills (12 December 2010) Computer Network Time Synchronization: The Network Time Protocol. Taylor & Francis
34. Leslie Lamport. 1978. Time, clocks, and the ordering of events in a distributed system. Commun. ACM 21, 7 (July 1978), 558-565