Pareto frontier for job execution and data transfer time in hybrid clouds

Future Generation Computer Systems

Volumn 37, Issue , 2014, Pages 321-334

  Taheri, Javid ^a   Zomaya, Albert Y ^a   Siegel, Howard Jay ^b   Tari, Zahir ^c  

^a UNIVERSITY OF SYDNEY   (Australia)

^b Veterinary Dentistry and Oral Surgery   (United States)

^c RMIT UNIVERSITY   (Australia)

Author keywords

Big data; Cloud bursting; Pareto frontier; Particle swarm optimization; Private data

Indexed keywords

DATA TRANSFER; DNA SEQUENCES; PARTICLE SWARM OPTIMIZATION (PSO); PROBES; VIRUSES;

BI-OBJECTIVE OPTIMIZATION; CLOUD APPLICATIONS; COMPLEX PROBLEMS; COMPUTATIONAL TASK; NORMAL SEQUENCES; PARETO FRONTIERS; PRIVATE DATA; SENSITIVE INFORMATIONS;

BIG DATA;

EID: 84901606996     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2013.12.020     Document Type: Article

References (43)

1. D. Agrawal, S. Das, A.E. Abbadi, Big data and cloud computing: current state and future opportunities, in: Proceedings of the 14th International Conference on Extending Database Technology, Uppsala, Sweden, 2011, pp. 530-533.
2. Microsoft. (Visited 2013). http://www.windowsazure.com/.
3. Amazon. (Visited 2013). www.aws.amazon.com/ec2/.
4. myNoSQL. (Visited 2013). http://nosql.mypopescu.com/post/6361838342/ bigdata-volume-velocity-variability-variety.
5. A. Delgado Peris, J. Hernandez, E. Huedo, and I.M. Llorente Data location-aware job scheduling in the grid. Application to the GridWay metascheduler J. Phys. Conf. Ser. 219 2010 062043
6. J. Kołodziej, and S. Khan Data scheduling in data grids and data centers: a short taxonomy of problems and intelligent resolution techniques N.-T. Nguyen, J. Kołodziej, T. Burczyński, M. Biba, Transactions on Computational Collective Intelligence X. vol. 7776 2013 Springer Berlin, Heidelberg 103 119
7. C. Augonnet, J. Clet-Ortega, S. Thibault, R. Namyst, Data-aware task scheduling on multi-accelerator based platforms, in: 2010 IEEE 16th International Conference on Parallel and Distributed Systems, ICPADS, 2010, pp. 291-298.
8. G. Lee, B.-G. Chun, H. Katz, Heterogeneity-aware resource allocation and scheduling in the cloud, in: Presented at the Proceedings of the 3rd USENIX Conference on Hot Topics in Cloud Computing, Portland, OR, 2011.
9. T. Kosar Data-aware distributed batch scheduling Handbook of Research on Grid Technologies and Utility Computing: Concepts for Managing Large-Scale Applications 2009 IGI Global 41 48
10. K. Hasham, A.D. Peris, A. Anjum, D. Evans, S. Gowdy, J.M. Hernandez, E. Huedo, D. Hufnagel, F. Van Lingen, R. McClatchey, and S. Metson CMS workflow execution using intelligent job scheduling and data access strategies IEEE Trans. Nucl. Sci. 58 2011 1221 1232
11. J. Kolodziej, M. Szmajduch, S.U. Khan, L. Wang, and D. Chen Genetic-based solutions for independent batch scheduling in data grids European Council for Modeling and Simulation, ECMS 2013 2013
12. J. Taheri, A.Y. Zomaya, P. Bouvry, and S.U. Khan Hopfield neural network for simultaneous job scheduling and data replication in grids Future Gener. Comput. Syst. 29 2013 1885 1900
13. J. Taheri, Y. Choon Lee, A.Y. Zomaya, and H.J. Siegel A bee colony based optimization approach for simultaneous job scheduling and data replication in grid environments Comput. Oper. Res. 40 2013 1564 1578
14. J. Taheri, A. Zomaya, and S.U. Khan Grid simulation tools for job scheduling and datafile replication Samee U. Khan, A.Y. Zomaya, L. Wang, Scalable Computing and Communications: Theory and Practice 2013 John Wiley & Sons, Inc. Hoboken, New Jersey
15. J. Taheri, A.Y. Zomaya, A Pareto frontier for optimizing data transfer and job execution in grids, in: Presented at the 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops & Ph.D. Forum, IPDPSW, 2012.
16. J. Taheri, A.Y. Zomaya, and S.U. Khan Genetic algorithm in finding Pareto frontier of optimizing data transfer versus job execution in grids Concurrency and Computation: Practice and Experience 2012 pp. n/a-n/a
17. J. Taheri, Y.C. Lee, A.Y. Zomaya, Simultaneous job and data allocation in grid environments, The University of Sydney, Sydney, Australia, TR 6712011.
18. Montage. (visited 2013). http://montage.ipac.caltech.edu/.
19. G. Juve, E. Deelman, K. Vahi, G. Mehta, B. Berriman, B.P. Berman, P. Maechling, Scientific workflow applications on Amazon EC2, in: 2009 5th IEEE International Conference on E-Science Workshops, 2009, pp. 59-66.
20. V. Nefedova, R. Jacob, I. Foster, Z. Liu, Y. Liu, E. Deelman, G. Mehta, M.-H. Su, K. Vahi, Automating climate science: large ensemble simulations on the teragrid with the GriPhyN virtual data system, in: Presented at the Proceedings of the Second IEEE International Conference on E-Science and Grid Computing, 2006.
21. MIKE2.0. (Visited 2013). Big Data Definition (http://mike2. openmethodology.org/wiki/Big-Data-Definition).
22. T. White Hadoop: The Definitive Guide Original edition 2009 O'Reilly Media
23. ZDNET. (Visited 2013). http://www.zdnet.com/blog/virtualization/what-is- big-data/1708.
24. The-Economist. (Visited 2013). http://www.economist.com/node/15557443? story-id=15557443.
25. T. Segaran, and J. Hammerbacher Beautiful Data 2009 O'Reilly Media, Inc.
26. IBM. (Visited 2013). What is big data? http://www-01.ibm.com/software/ data/bigdata/.
27. GREENPLUM. (Visited 2013). http://www.greenplum.com/.
28. Wikibon. (Visited 2013). http://wikibon.org/wiki/v/Enterprise-Big-data.
29. FUJITSU, Personal data on the cloud: a global survey of customer attitudes (http://www.fujitsu.com/global/news/publications/dataprivacy.html), Visited 2013.
30. W.K. Hon, C. Millard, and I. Walden The problem of 'personal data' in cloud computing: what information is regulated? - the cloud of unknowing Int. Data Priv. Law 1 2011 211 228
31. R.-S. Chang, J.-S. Chang, and S.-Y. Lin Job scheduling and data replication on data grids Future Gener. Comput. Syst. 23 2007 846 860 (Pubitemid 46755405)
32. A. Anjum, R. McClatchey, A. Ali, and I. Willers Bulk scheduling with the DIANA scheduler IEEE Trans. Nucl. Sci. 53 2006 3818 3829 (Pubitemid 46113397)
33. R. McClatchey, A. Anjum, H. Stockinger, A. Ali, I. Willers, and M. Thomas Data Intensive and Network Aware (DIANA) grid scheduling J. Grid Comput. 5 2007 43 64 (Pubitemid 46909012)
34. GILDA. (Visited 2013). https://gilda.ct.infn.it/.
35. CERN. (Visited 2013). Compact Muon Solenoid (CMS) http://public.web.cern. ch/public/en/lhc/CMS-en.html.
36. P. Sang-Min, K. Jair-Hoom, Chameleon: a resource scheduler in a data grid environment, in: Proceedings of 3rd IEEE International Symposium on Cluster Computing and the Grid, CCGRID'03, 2003, pp. 258-265.
37. W. Hoschek, J. Jaen-Martinez, A. Samar, H. Stockinger, and K. Stockinger Data management in an international data grid project R. Buyya, M. Baker, Grid Computing, vol. 1971 2000 Springer-Verlag New York 77 90 (Pubitemid 33206211)
38. S. Hongzhang, L. Oliker, R. Biswas, Job superscheduler architecture and performance in computational grid environments, in: Proceedings of the ACM/IEEE SC2003 Conference, SC'03, 2003, pp. 44-58.
39. J. Blondin, Particle swarm optimization: a tutorial, (www.cs.armstrong. edu/saad/csci8100/pso-tutorial.pdf), 2009.
40. X.C. Liu, X.G. Qiu, B. Chen, Q. He, and K.D. Huang Scheduling parallel discrete event simulation jobs in the cloud IET Conference Publications vol. 2012 2012 pp. 72-72
41. K. Ranganathan, I. Foster, Decoupling computation and data scheduling in distributed data-intensive applications, in: Proceedings of 11th IEEE International Symposium on High Performance Distributed Computing, HPDC'02, 2002, pp. 352-358.
42. M. Tang, B.-S. Lee, X. Tang, and C.-K. Yeo The impact of data replication on job scheduling performance in the data grid Future Gener. Comput. Syst. 22 2006 254 268 (Pubitemid 41689807)
43. S. Abdi, and S. Mohamadi Two level job scheduling and data replication in data grid Int. J. Grid Comput. Appl. 1 2010 23 37