The Grid Workloads Archive

Future Generation Computer Systems

Volumn 24, Issue 7, 2008, Pages 672-686

  Iosup, Alexandru ^a   Li, Hui ^b   Jan, Mathieu ^a   Anoep, Shanny ^a   Dumitrescu, Catalin ^a   Wolters, Lex ^b   Epema, Dick H J ^a  

^a Delft University of Technology ^*  (Netherlands)

^b LEIDEN UNIVERSITY   (Netherlands)

Author keywords

Grid computing; Grid traces; Grid workloads

Indexed keywords

COMPUTER AIDED SOFTWARE ENGINEERING; ELECTRONIC DATA INTERCHANGE; INFORMATION ANALYSIS; RESOURCE ALLOCATION; USER INTERFACES;

GRID TRACES; GRID WORKLOADS ARCHIVE (GWA); ORGANIZATIONAL PERMISSIONS; WORKLOAD DATA EXCHANGE;

GRID COMPUTING;

EID: 44249107046     PISSN: 0167739X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.future.2008.02.003     Document Type: Article

References (67)

1. EGEE Team, LCG (2004). lcg.web.cern.ch/LCG
2. Eerola P., Kónya B., Smirnova O., Ekelöf T., Ellert M., Hansen J.R., Nielsen J.L., Wäänänen A., Konstantinov A., Herrala J., Tuisku M., Myklebust T., Ould-Saada F., and Vinter B. The NorduGrid production grid infrastructure, status and plans. Int'l. Conf. on Grid Computing (GRID). IEEE Computer Society (2003) 158-165
3. The TeraGrid Project, Npaci, March 2006. www.teragrid.org
4. The Open Science Grid Project, OSG, Jul 2007.www.opensciencegrid.org/
5. Iosup A., and Epema D.H.J. GrenchMark: A framework for analyzing, testing, and comparing grids. IEEE/ACM Intl. Symp. on Cluster Computing and the Grid (CCGrid) (2006), IEEE Computer Society 313-320
6. Khalili O., He J., Olschanowsky C., Snavely A., and Casanova H. Measuring the performance and reliability of production computational grids. Int'l. Conf. on Grid Computing (GRID) (2006), IEEE Computer Society
7. Li H., Groep D., Wolters L., and Templon J. Job failure analysis and its implications in a large-scale production grid. IEEE Int'l. Conf. on e-Science and Grid Computing (e-Science) (2006), IEEE Computer Society pp. 27-27
8. Iosup A., Epema D., Couvares P., Karp A., and Livny M. Build-and-test workloads for grid middleware: Problem, analysis, and applications. IEEE/ACM Intl. Symp. on Cluster Computing and the Grid (CCGrid) (2007), IEEE Computer Society 205-213
9. P. Danzig, J. Mogul, V. Paxson, M. Schwartz, The Internet Traffic Archive, Aug 2007. ita.ee.lbl.gov/
10. CAIDA Team, The Cooperative Association for Internet Data Analysis, Aug 2007. www.caida.org/data/
11. Yeo J., Kotz D., and Henderson T. CRAWDAD: A community resource for archiving wireless data at dartmouth. SIGCOMM Comput Commun Rev 36 2 (2006) 21-22
12. The Parallel Workloads Archive Team, The parallel workloads archive logs, June 2006. www.cs.huji.ac.il/labs/parallel/workload/logs.html
13. B. Schroeder, G. Gibson, The computer failure data repository (CFDR), Aug 2007. cfdr.usenix.org/
14. Lublin U., and Feitelson D.G. The workload on parallel supercomputers: Modeling the characteristics of rigid jobs. Journal of Paralleland Distributed Computing 63 11 (2003) 1105-1122
15. Czajkowski K., Foster I.T., and Kesselman C. Resource co-allocation in computational grids. IEEE Int'l. Symp. on High Performance Distributed Computing (HPDC) (1999), IEEE Computer Society
16. Smith W., Foster I.T., and Taylor V.E. Scheduling with advanced reservations. Int'l. Parallel & Distributed Processing Symposium (IPDPS) (2000), IEEE Computer Society 127-132
17. A. Iosup, H. Li, C. Dumitrescu, L. Wolters, D.H.J.E pema, The Grid Workload Format, Nov. 2006. gwa.ewi.tudelft.nl/TheGridWorkloadFormat_v001.pdf
18. Iosup A., Epema D.H.J., Franke C., Papaspyrou A., Schley L., Song B., and Yahyapour R. On grid performance evaluation using synthetic workloads. In: Frachtenberg E., and Schwiegelshohn U. (Eds). Int'l. Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), Revised Selected Papers. Lecture Notes in Computer Science vol. 4376 (2007), Springer 232-255
19. Iosup A., Jan M., Sonmez O., and Epema D.H. On the dynamic resource availability in grids. Int'l. Conf. on Grid Computing (GRID) (2007), IEEE Computer Society
20. Bal H., et al. The Distributed ASCI Supercomputer project. Operating Systems Review 34 4 (2000) 76-96
21. Mohamed H., and Epema D. The design and implementation of the KOALA co-allocating grid scheduler. In: Sloot P.M.A., Hoekstra A.G., Priol T., Reinefeld A., and Bubak M. (Eds). European Grid Conference (EGC). Lecture Notes in Computer Science vol. 3470 (2005), Springer 640-650
22. Wrzesinska G., van Nieuwpoort R., Maassen J., and Bal H.E. Fault-tolerance, malleability and migration for divide-and-conquer applications on the grid. Int'l. Parallel & Distributed Processing Symposium (IPDPS) (2005), IEEE Computer Society
23. Bolze R., Cappello F., Caron E., Dayd M., Desprez F., Jeannot E., Jgou Y., Lanteri S., Leduc J., Melab N., Mornet G., Namyst R., Primet P., Quetier B., Richard O., Talbi E.-G., and Irena T. Grid'5000: A large scale and highly reconfigurable experimental grid testbed. International Journal of High Performance Computing Applications 20 4 (2006) 481-494
24. Capit N., Costa G.D., Georgiou Y., Huard G., Martin C., Mouni G., Neyron P., and Richard O. A batch scheduler with high level components. IEEE/ACM Intl. Symp. on Cluster Computing and the Grid (CCGrid) (2005), IEEE Computer Society 776-783
25. Ellert M., et al. Advanced Resource Connector middleware for lightweight computational grids. Future Generation Computer Systems 23 2 (2007) 219-240
26. Thain D., Tannenbaum T., and Livny M. Distributed computing in practice: The Condor experience. Concurrency - Practice and Experience 17 2-4 (2005) 323-356
27. Foster I., et al. The Grid2003 production grid: Principles and practice. IEEE Int'l. Symp. on High Performance Distributed Computing (HPDC) (2004), IEEE Computer Society 236-245
28. M. Mambelli, al, Atlas data challenge production on Grid3, 2005. griddev.uchicago.edu/download/atgce/doc.pkg/presentations/chep04-503-usa tlas-dc2.pdf
29. Iosup A., Dumitrescu C., Epema D.H., Li H., and Wolters L. How are real grids used? The analysis of four grid traces and its implications. Int'l. Conf. on Grid Computing (GRID) (2006), IEEE Computer Society 262-269
30. A. Iosup, M. Jan, O. Sonmez, D. Epema, The characteristics and performance of groups of jobs in grids, in: Int'l. European Conference on Parallel and Distributed Computing (Euro-Par), in: Lecture Notes in Computer Science, Springer, 2007, pp. 382-393
31. A. Iosup, D. Epema, T. Tannenbaum, M. Farrellee, M. Livny, Inter-operating grids through delegated matchmaking, in: Proc. of the ACM/IEEE Conference on High Performance Networking and Computing (SC), ACM Press, 2007 (in press)
32. Li H., Heusdens R., Muskulus M., and Wolters L. Analysis and synthesis of pseudo-periodic job arrivals in grids: A matching pursuit approach. IEEE/ACM Intl. Symp. on Cluster Computing and the Grid (CCGrid) (2007), IEEE Computer Society 183-196
33. Li H., Muskulus M., and Wolters L. Modeling job arrivals in a data-intensive grid. In: Frachtenberg E., and Schwiegelshohn U. (Eds). Int'l. Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), Revised Selected Papers. Lecture Notes in Computer Science vol. 4376 (2007), Springer 210-231
34. Li H. Long range dependent job arrival process and its implications in grid environments. Proc. of MetroGrid Workshop, Int'l. Conference on Networks for Grid Applications (GridNets07) (2007), ACM Press (in press)
35. Iosup A., Garbacki P., and Epema D.H. Provisioning and scheduling resources for world-wide data-sharing services. IEEE Int'l. Conf. on e-Science and Grid Computing (e-Science) (2006), IEEE Computer Society pp. 84-84
36. Li H., Groep D.L., Templon J., and Wolters L. Predicting job start times on clusters. IEEE/ACM Intl. Symp. on Cluster Computing and the Grid (CCGrid) (2004), IEEE Computer Society 301-308
37. Li H., and Wolters L. Towards a better understanding of workload dynamics on data-intensive clusters and grids. Int'l. Parallel & Distributed Processing Symposium (IPDPS) (2007), IEEE Computer Society 1-10
38. Li H., Groep D.L., and Wolters L. Mining performance data for metascheduling decision support in the Grid. Future Generation Computer Systems 23 1 (2007) 92-99
39. H.B. Newman, I.C. Legrand, P. Galvez, R. Voicu, C. Cirstoiu, Monalisa: A distributed monitoring service architecture, in: Computing in High Energy and Nuclear Physics (CHEP03), 2003
40. Z. Balaton, G. Gombas, GridLab Monitoring: Detailed Architecture Specification, EU Information Society Technologies Programme (IST) TR GridLab-11-D11.2-01-v1.2, http://www.gridlab.org/Resources/Deliverables/D11.2.pdf, 2001
41. C. Stratan, C. Cirstoiu, A. Iosup, On the accuracy of off-line monitoring information in grids, in: Proc. of the 17th Intl. Conference on Control Systems and Computer Science (CSCS-17), 2007, May, Bucharest, Romania
42. Feitelson D.G., and Nitzberg B. Job characteristics of a production parallel scientific workload on the NASA Ames iPSC/860. In: Feitelson D.G., and Rudolph L. (Eds). Int'l. Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), Revised Selected Papers. Lecture Notes in Computer Science vol. 949 (2007), Springer 337-360
43. Feitelson D.G. Memory usage in the LANL CM-5 workload. In: Feitelson D.G., and Rudolph L. (Eds). Int'l. Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), Revised Selected Papers. Lecture Notes in Computer Science vol. 1291 (1997), Springer 78-94
44. Windisch K., Lo V., Moore R., Feitelson D., and Nitzberg B. A comparison of workload traces from two production parallel machines. 6th Symp. Frontiers Massively Parallel Comput. (1996) 319-326
45. Hotovy S. Workload evolution on the Cornell Theory Center IBM SP2. In: Feitelson D.G., and Rudolph L. (Eds). Int'l. Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), Revised Selected Papers. Lecture Notes in Computer Science vol. 1162 (1996), Springer 27-40
46. Feitelson D.G., and Mu'alem Weil A. Utilization and predictability in scheduling the IBM SP2 with backfilling. 12th Intl. Parallel Processing Symp. (IPPS) (1998) 542-546
47. Mu'alem A.W., and Feitelson D.G. Utilization, predictability, workloads, and user runtime estimates in scheduling the IBM SP2 with backfilling. IEEE Transactions Parallel & Distributed Systems 12 6 (2001) 529-543
48. Cho K., Mitsuya K., and Kato A. Traffic data repository at the wide project. ATEC'00: Proceedings of the Annual Technical Conference on 2000 USENIX Annual Technical Conference (2000), USENIX Association, Berkeley, CA, USA 263-270
49. NTTT Team, Network Tools and Traffic Traces, Aug 2007. www.grid.unina.it/Traffic/
50. Performance Evaluation Laboratory, BYU trace distribution center, Aug 2007. tds.cs.byu.edu/tds/
51. Performance and Architecture Research Lab, NMSU trace database, Aug 2007. tracebase.nmsu.edu/
52. Pablo Research Group, CADRE: A national facility for I/O characterization and optimization, Aug 2007. www.renci.org/pablo/Project/CADRE/
53. Center for HPC Cluster Resource Management and Scheduling, Maui HPC workload/resource trace repository, Aug 2007. www.supercluster.org/research/traces/
54. D. Kondo, G. Fedak, F. Cappello, A.A. Chien, H. Casanova, The desktop grid availability traces archive, Aug 2007. vs25.lri.fr:4320/dg/
55. Kondo D., Fedak G., Cappello F., Chien A.A., and Casanova H. Characterizing resource availability in enterprise desktop grids. Future Generation Computer Systems 23 7 (2007) 888-903
56. B. Godfrey, I. Stoica, Repository of availability traces (RAT), Aug 2007. www.cs.berkeley.edu/~pbg/availability/
57. Ferrari D. Workload characterization and selection in computer performance measurement. IEEE Computer 5 4 (1972) 18-24
58. R. Jain, The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling, 1991, Winner of 1991 Best Advanced How-To Book, Systems award from the Computer Press Association
59. Calzarossa M., and Serazzi G. Workload characterization: A survey. Proceedings of the IEEE 81 8 (1993) 1136-1150
60. Berners-Lee T., Cailliau R., Groff J.-F., and Pollermann B. World-wide web: The information universe. Electronic Networking: Research, Applications and Policy 1 2 (1992) 74-82
61. DatCat Team, The DatCat Internet Measurement Data Catalog, Aug 2007. imdc.datcat.org/
62. MOAT Team, The NLANR Measurement and Network Analysis Group, Aug 2007. mna.nlanr.net/
63. Chapin S.J., Cirne W., Feitelson D.G., Jones J.P., Leutenegger S.T., Schwiegelshohn U., Smith W., and Talby D. Benchmarks and standards for the evaluation of parallel job schedulers. In: Feitelson D.G., and Rudolph L. (Eds). Int'l. Workshop on Job Scheduling Strategies for Parallel Processing (JSSPP), Revised Selected Papers. Lecture Notes in Computer Science vol. 1659 (1999), Springer 67-90
64. M. Andreica, N. Tapus, A. Iosup, D. Epema, C. Dumitrescu, I. Raicu, I. Foster, M. Ripeanu, Towards servmark, an architecture for testing grids, Tech. Rep. TR-0062, Institute on Resource Management and Scheduling, CoreGRID - Network of Excellence, November 2006
65. Buyya R., and Murshed M.M. GridSim: A toolkit for the modeling and simulation of distributed resource management and scheduling for grid computing. Concurrency and Computation: Practice and Experience 14 13-15 (2002) 1175-1220
66. Legrand A., Marchal L., and Casanova H. Scheduling distributed applications: the simgrid simulation framework. IEEE/ACM Intl. Symp. on Cluster Computing and the Grid (CCGrid) (2003), IEEE Computer Society 138-145
67. Dumitrescu C., and Foster I.T. GangSim: A simulator for grid scheduling studies. IEEE/ACM Intl. Symp. on Cluster Computing and the Grid (CCGrid) (2005), IEEE Computer Society 1151-1158