Controlling fairness and task granularity in distributed, online, non-clairvoyant workflow executions

Concurrency and Computation: Practice and Experience

Volumn 26, Issue 14, 2014, Pages 2347-2366

  Da Silva, Rafael Ferreira ^a,b   Glatard, Tristan ^a,c   Desprez, Frédéric ^d  

^a UNIVERSITÉ DE LYON   (France)

^b UNIVERSITY OF SOUTHERN CALIFORNIA   (United States)

^c MCGILL UNIVERSITY   (Canada)

^d UNIV LYON   (France)

Author keywords

Distributed computing infrastructures; Fairness; Non clairvoyant conditions; Online conditions; Scientific workflows; Task granularity

Indexed keywords

COMPUTING INFRASTRUCTURES; FAIRNESS; NON-CLAIRVOYANT CONDITIONS; ON-LINE CONDITIONS; SCIENTIFIC WORKFLOWS; TASK GRANULARITY;

MIDDLEWARE;

EID: 84923329002     PISSN: 15320626     EISSN: 15320634     Source Type: Journal    
DOI: 10.1002/cpe.3303     Document Type: Article

References (37)

1. Kiss T. Science gateways for the broader take-up of distributed computing infrastructures. Journal of Grid Computing 2012; 10(4):599-600.
2. Kephart JO, Chess DM. The vision of autonomic computing. Computer 2003; 36(1):41-50.
3. Taylor I, Deelman E, Gannon D. Workflows for e-Science: Scientific Workflows for Grids. Springer: New York, NJ, USA, 2006.
4. Muthuvelu N, Liu J, Soe N.L., Venugopal S., Sulistio A, Buyya R. A dynamic job grouping-based scheduling for deploying applications with fine-grained tasks on global grids. Proceedings of the 2005 Australasian Workshop on Grid Computing and e-Research - Volume 44, ACSW Frontiers '05, Australian Computer Society, Inc., Darlinghurst, Australia, Australia, 2005; 41-48.
5. Ng WK, Ang TF, Ling T.C., Liew CS. Scheduling framework for bandwidth-aware job grouping-based scheduling in grid computing. Malaysian Journal of Computer Science 2006; 19.
6. Ang TF, Ng WK, Ling T.C., Por LY, Liew CS. A bandwidth-aware job grouping-based scheduling on grid environment. Information Technology Journal 2009; 8:372-377.
7. Muthuvelu N, Chai I, Eswaran C. An adaptive and parameterized job grouping algorithm for scheduling grid jobs. 10th International Conference on Advanced Communication Technology, 2008. ICACT 2008, Vol. 2, Gangwon-Do, South Korea, 2008; 975-980.
8. Liu Q, Liao Y. Grouping-based fine-grained job scheduling in grid computing. ETCS '09, Vol. 1, Wuhan, Hubei, 2009; 556-559.
9. Soni VK, Sharma R, Mishra MK. Grouping-based job scheduling model in grid computing. World Academy of Science, Engineering and Technology 2010; 41:781-784.
10. Zomaya AY, Chan G. Efficient clustering for parallel tasks execution in distributed systems. 18th IPDPS, Santa Fe, New Mexico, USA, 2004; 167-174.
11. Muthuvelu N, Chai I, Chikkannan E., Buyya R. On-line task granularity adaptation for dynamic grid applications. Algorithms and Architectures for Parallel Processing, LNCS, Vol. 6081, Springer, Busan, Korea, 2010; 266-277.
12. Muthuvelu N, Vecchiola C, Chai I., Chikkannan E, Buyya R. Task granularity policies for deploying bag-of-task applications on global grids. Future Generation Computer Systems 2013; 29(1):170-181. Including Special section: AIRCC-NetCoM 2009 and Special section: Clouds and Service-Oriented Architectures.
13. Chen W, Ferreira da Silva R, Deelman E, Sakellariou R. Balanced task clustering in scientific workflows. 2013 IEEE 9th International Conference on eScience (eScience), Beijing, China, 2013; 188-195.
14. Zhao H, Sakellariou R. Scheduling multiple DAGs onto heterogeneous systems. 20th International Parallel and Distributed Processing Symposium, IPDPS 2006, Rhodes Island, Greece, 2006; 1-14.
15. Casanova H, Desprez F, Suter F. On cluster resource allocation for multiple parallel task graphs. Journal of Parallel and Distributed Computing 2010; 70(12):1193-1203.
16. Kleban SD, Clearwater SH. Fair share on high performance computing systems: what does fair really mean. 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, 2003. Proceedings. CCGRID 2003, Tokyo, Japan, 2003; 146-153.
17. Ferreira da Silva R, Glatard T, Desprez F. Workflow fairness control on online and non-clairvoyant distributed computing platforms. In Euro-Par 2013 Parallel Processing, Vol. 8097, Wolf F, Mohr B, Mey D (eds)., Lecture Notes in Computer Science. Springer Berlin Heidelberg: Aachen, Germany, 2013; 102-113.
18. Ferreira da Silva R, Glatard T, Desprez F. On-line, non-clairvoyant optimization of workflow activity granularity on grids. In Euro-Par 2013 Parallel Processing, Vol. 8097, Wolf F, Mohr B, Mey D (eds)., Lecture Notes in Computer Science. Springer Berlin Heidelberg: Aachen, Germany, 2013; 255-266.
19. Glatard T, Lartizien C, Gibaud B., Ferreira da Silva R, Forestier G, Cervenansky F, Alessandrini M, Benoit-Cattin H, Bernard O, Camarasu-Pop S, Cerezo N, Clarysse P, Gaignard A, Hugonnard P, Liebgott H, Marache S, Marion A, Montagnat J, Tabary J, Friboulet D. A virtual imaging platform for multi-modality medical image simulation. IEEE Transactions on Medical Imaging 2013; 32(1):110-118.
20. N'Takpe T., Suter F. Concurrent scheduling of parallel task graphs on multi-clusters using constrained resource allocations. Proceedings of the 2009 IEEE International Symposium on Parallel & Distributed Processing, IPDPS '09, Rome, Italy, 2009; 1-8.
21. Hsu CC, Huang KC, Wang FJ. Online scheduling of workflow applications in grid environments. Future Generation Computer Systems 2011; 27(6):860-870.
22. Sommerfeld D, Richter H. Efficient grid workflow scheduling using a two-tier approach. Proceedings of Healthgrid 2011, Bristol, UK, 2011; 1-12.
23. Hirales-Carbajal A, Tchernykh A, Yahyapour R., González-García J, Röblitz T, Ramírez-Alcaraz JM. Multiple workflow scheduling strategies with user run time estimates on a grid. Journal of Grid Computing 2012; 10:325-346.
24. Arabnejad H, Barbosa J. Fairness resource sharing for dynamic workflow scheduling on heterogeneous systems. 2012 IEEE 10th International Symposium on Parallel and Distributed Processing with Applications (ISPA), Leganes, Spain, 2012; 633-639.
25. Sabin G, Kochhar G, Sadayappan P. Job fairness in non-preemptive job scheduling. Proceedings of the 2004 International Conference on Parallel Processing, ICPP '04, Montreal, Quebec, Canada, 2004; 186-194.
26. Skowron P, Rzadca K. Non-monetary fair scheduling: a cooperative game theory approach. Proceedings of the Twenty-Fifth Annual ACM Symposium on Parallelism in Algorithms and Architectures, SPAA '13, ACM, New York, NY, USA, 2013; 288-297.
27. Montagnat J, Isnard B, Glatard T., Maheshwari K, Fornarino MB. A data-driven workflow language for grids based on array programming principles. Proceedings of the 4th Workshop on Workflows in Support of Large-Scale Science, WORKS '09, ACM, New York, NY, USA, 2009; 7:1-7:10.
28. Ferreira da Silva R, Glatard T. A science-gateway workload archive to study pilot jobs, user activity, bag of tasks, task sub-steps, and workflow executions. In Euro-Par 2012: Parallel Processing Workshops (CGWS-2012), Vol. 7640, Caragiannis I, Alexander M, Badia R, Cannataro M, Costan A, Danelutto M, Desprez F, Krammer B, Sahuquillo J, Scott S, Weidendorfer J (eds)., Lecture Notes in Computer Science. Springer Berlin Heidelberg: Rhodes Island, Greece, 2013; 79-88.
29. Glatard T, Montagnat J, Lingrand D., Pennec X. Flexible and efficient workflow deployment of data-intensive applications on grids with MOTEUR. International Journal of High Performance Computing Applications (IJHPCA) 2008; 22(3):347-360.
30. Tsaregorodtsev A, Brook N, Ramo A.C., Charpentier P., Closier J, Cowan G, Diaz R.G., Lanciotti E., Mathe Z, Nandakumar R, Paterson S, Romanovsky V, Santinelli R, Sapunov M, Smith AC, Miguelez MS, Zhelezov A. DIRAC3. The new generation of the LHCb grid software. Journal of Physics: Conference Series 2009; 219(6): 062029.
31. Ferreira da Silva R. A science-gateway for workflow executions: online and non-clairvoyant self-healing of workflow executions on grids. Ph.D. Thesis, Institut National des Sciences Appliquées de Lyon, 2013.
32. Ferreira da Silva R, Glatard T, Desprez F. Self-healing of workflow activity incidents on distributed computing infrastructures. Future Generation Computer Systems 2013; 29(8):2284-2294.
33. Jan S, Benoit D, Becheva E., Carlier T, Cassol F, Descourt P., Frisson T, Grevillot L, Guigues L, Maigne L, Morel C, Perrot Y, Rehfeld N, Sarrut D, Schaart DR, Stute S, Pietrzyk U, Visvikis D, Zahra N, Buvat I. Gate v6: a major enhancement of the gate simulation platform enabling modelling of CT and radiotherapy. Physics in Medicine and Biology 2011; 56(4):881-901.
34. Cao F, Commowick O, Bannier E., Ferré JC, Edan G, Barillot C. MRI estimation of T1 relaxation time using a constrained optimization algorithm. In Mbia'12 Proceedings of the Second International Conference on Multimodal Brain Image Analysis, Vol. 7509, Yap PT, Liu T, Shen D, Westin CF, Shen L (eds)., Lecture Notes in Computer Science. Springer Berlin Heidelberg: Berlin, Heidelberg, 2012; 203-214.
35. Jensen JA, Svendsen NB. Calculation of pressure fields from arbitrarily shaped, apodized, and excited ultrasound transducers. IEEE Transactions on Ultrasonics, Ferroelectrics and Frequency Control 1992; 39(2):262-267.
36. Reilhac A, Batan G, Michel C., Grova C, Tohka J, Collins D.L., Costes N., Evans AC. PET-SORTEO: validation and development of database of simulated PET volumes. IEEE Transactions on Nuclear Science 2005; 52(5):1321-1328.
37. Henry GJ. The UNIX system: the fair share scheduler. AT&T Bell Laboratories Technical Journal 1984; 63(8): 1845-1857.