A local-optimisation based strategy for cost-effective datasets storage of scientific applications in the cloud

Proceedings - 2011 IEEE 4th International Conference on Cloud Computing, CLOUD 2011

Volumn , Issue , 2011, Pages 179-186

  Yuan, Dong ^a   Yang, Yun ^a   Liu, Xiao ^a   Chen, Jinjun ^a  

^a SWINBURNE UNIVERSITY OF TECHNOLOGY   (Australia)

Author keywords

Cloud computing; Computation storage trade off; Datasets storage; Scientific applications

Indexed keywords

APPLICATION COST; COMPUTATION POWER; COMPUTATION-STORAGE TRADE-OFF; COMPUTING SYSTEM; COST MODELS; DATA ACCESSING; DATA DEPENDENCY GRAPHS; DATA PROVENANCE; DATA SETS; DATA-INTENSIVE APPLICATION; INFRASTRUCTURE INVESTMENT; PAY-AS-YOU-GO; SCIENTIFIC APPLICATIONS; STORAGE CAPACITY;

ASTROPHYSICS; COMPUTER SIMULATION; COMPUTER SYSTEMS; COST EFFECTIVENESS; COST REDUCTION; INVESTMENTS; OPTIMIZATION;

CLOUD COMPUTING;

EID: 80053163068     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CLOUD.2011.13     Document Type: Conference Paper

References (30)

1. "Amazon", http://aws.amazon.com/.
2. "Eucalyptus", http://open.eucalyptus.com/.
3. "Nimbus", http://www.nimbusproject.org/.
4. "OpenNebula", http://www.opennebula.org/.
5. I. Adams, D. D. E. Long, E. L. Miller, S. Pasupathy, and M. W. Storer, "Maximizing Efficiency by Trading Storage for Computation," in Workshop on Hot Topics in Cloud Computing, pp. 1-5, 2009.
6. M. Armbrust, A. Fox, R. Griffith, A. D. Joseph, R. Katz, A. Konwinski, G. Lee, D. Patterson, A. Rabkin, I. Stoica, and M. Zaharia, "A View of Cloud Computing," Commun. ACM, vol. 53, pp. 50-58, 2010.
7. M. D. d. Assuncao, A. d. Costanzo, and R. Buyya, "Evaluating the Cost-Benefit of Using Cloud Computing to Extend the Capacity of Clusters," in 18th ACM International Symposium on High Performance Distributed Computing, pp. 1-10, 2009.
8. Z. Bao, S. Cohen-Boulakia, S. B. Davidson, A. Eyal, and S. Khanna, "Differencing Provenance in Scientific Workflows," in 25th IEEE International Conference on Data Engineering, pp. 808-819, 2009.
9. R. Bose and J. Frew, "Lineage Retrieval for Scientific Data Processing: A Survey," ACM Comput. Surv., vol. 37, pp. 1-28, 2005. (Pubitemid 41246320)
10. th IEEE International Conference on e-Science, pp. 164-170, 2009.
11. R. Buyya, C. S. Yeo, S. Venugopal, J. Broberg, and I. Brandic, "Cloud Computing and Emerging IT Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility," Future Generation Computer Systems, vol. 25, pp. 599-616, 2009.
12. E. Deelman, D. Gannon, M. Shields, and I. Taylor, "Workflows and e-Science: An Overview of Workflow System Features and Capabilities," Future Generation Computer Systems, vol. 25, pp. 528-540, 2009.
13. E. Deelman, G. Singh, M. Livny, B. Berriman, and J. Good, "The Cost of Doing Science on the Cloud: the Montage Example," in ACM/IEEE Conference on Supercomputing, pp. 1-12, 2008.
14. I. Foster, J. Vockler, M. Wilde, and Z. Yong, "Chimera: A Virtual Data System for Representing, Querying, and Automating Data Derivation," in 14th International Conference on Scientific and Statistical Database Management, pp. 37-46, 2002.
15. I. Foster, Z. Yong, I. Raicu, and S. Lu, "Cloud Computing and Grid Computing 360-Degree Compared," in Grid Computing Environments Workshop, pp. 1-10, 2008.
16. S. K. Garg, R. Buyya, and H. J. Siegel, "Time and Cost Trade-Off Management for Scheduling Parallel Applications on Utility Grids," Future Generation Computer Systems, vol. 26, pp. 1344-1355, 2010.
17. P. Groth and L. Moreau, "Recording Process Documentation for Provenance," IEEE Transactions on Parallel and Distributed Systems, vol. 20, pp. 1246-1259, 2009.
18. P. K. Gunda, L. Ravindranath, C. A. Thekkath, Y. Yu, and L. Zhuang, "Nectar: Automatic Management of Data and Computation in Datacenters," in 9th Symposium on Operating Systems Design and Implementation, pp. 1-14, 2010.
19. G. Juve, E. Deelman, K. Vahi, and G. Mehta, "Data Sharing Options for Scientific Workflows on Amazon EC2," in ACM/IEEE Conference on Supercomputing, pp. 1-9, 2010.
20. D. Kondo, B. Javadi, P. Malecot, F. Cappello, and D. P. Anderson, "Cost-Benefit Analysis of Cloud Computing versus Desktop Grids," in 23th IEEE International Parallel & Distributed Processing Symposium pp. 1-12, 2009.
21. J. Li, M. Humphrey, D. Agarwal, K. Jackson, C. v. Ingen, and Y. Ryu, "eScience in the Cloud: A MODIS Satellite Data Reprojection and Reduction Pipeline in the Windows Azure Platform," in 24th IEEE International Parallel & Distributed Processing Symposium, pp. 1-12, 2010.
22. X. Liu, D. Yuan, G. Zhang, J. Chen, and Y. Yang, "SwinDeW-C: A Peer-to-Peer Based Cloud Workflow System for Managing Instance Intensive Applications," in Handbook of Cloud Computing: Springer, pp. 309-332, 2010.
23. B. Ludascher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger, M. Jones, and E. A. Lee, "Scientific Workflow Management and the Kepler System," Concurrency and Computation: Practice and Experience, pp. 1039-1065, 2005.
24. K.-K. Muniswamy-Reddy, P. Macko, and M. Seltzer, "Provenance for the Cloud," in 8th USENIX Conference on File and Storage Technology, pp. 197-210, 2010.
25. L. J. Osterweil, L. A. Clarke, A. M. Ellison, R. Podorozhny, A. Wise, E. Boose, and J. Hadley, "Experience in Using A Process Language to Define Scientific Workflow and Generate Dataset Provenance," in ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 319-329, 2008.
26. A. S. Szalay and J. Gray, "Science in an Exponential World," Nature, vol. 440, pp. 23-24, 2006.
27. D. Yuan, Y. Yang, X. Liu, and J. Chen, "A Cost-Effective Strategy for Intermediate Data Storage in Scientific Cloud Workflows," in 24th IEEE International Parallel & Distributed Processing Symposium, pp. 1-12, 2010.
28. D. Yuan, Y. Yang, X. Liu, G. Zhang, and J. Chen, "A Data Dependency Based Strategy for Intermediate Data Storage in Scientific Cloud Workflow Systems," Concurrency and Computation: Practice and Experience, 2010. (http://dx.doi.org/10.1002/cpe.1636)
29. D. Yuan, Y. Yun, X. Liu, and J. Chen, "On-demand Minimum Cost Benchmarking for Intermediate Datasets Storage in Scientific Cloud Workflow Systems," Journal of Parallel and Distributed Computing, vol. 72, pp. 316-332, 2011.
30. M. Zaharia, A. Konwinski, A. D. Joseph, R. Katz, and I. Stoica, "Improving MapReduce Performance in Heterogeneous Environments," in 8th USENIX Symposium on Operating Systems Design and Implementation, pp. 29-42, 2008.