Modeling job arrivals in a data-intensive grid

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 4376 LNCS, Issue , 2007, Pages 210-231

  Li, Hui ^a   Muskulus, Michael ^a   Wolters, Lex ^a  

^a LEIDEN UNIVERSITY   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

CORRELATION METHODS; MARKOV PROCESSES; STOCHASTIC MODELS; TELECOMMUNICATION TRAFFIC;

AUTOCORRELATION STRUCTURES; DATA INTENSIVE GRID; TRAFFIC MODELS; VIRTUAL ORGANIZATION (VO);

GRID COMPUTING;

EID: 38049186468     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-540-71035-6_11     Document Type: Conference Paper

References (44)

1. S. Asmussen, O. Nerman and M. Olsson. Fitting phase-type distribution via the EM algorithm. Scand. J. Statist. 23:419-441, 1996.
2. A-L. Barabasi. The origin of bursts and heavy tails in human dynamics. Nature, 435:207-211, 2005.
3. S. Basu and E. Foufoula-Georgiou. Detection of nonlinearity and chaoticky in time series using the transportation distance function. Physics Letters A, 301:413-423, 2002.
4. J. Beran. Statistics for Long Memory Processes. Chapman and Hall, New York, 1994.
5. P. Brémaud. Markov Chains. Gibbs Fields, Monte Carlo Simulation, and Queues. Springer, New York, 2001.
6. S. J. Chapin, W. Cirne, D. G. Feitelson, J. P. Jones, S. T. Leutenegger, U, Schwiegelshohn, W. Smith, and D. Talby. Benchmarks and standards for the evaluation of parallel job schedulers. LNCS 1659:67-90. Springer-Verlag, 1999.
7. W. Cirne and F. Berman. A comprehensive model of the supercomputer workload. In IEEE 4th Annual Workshop on Workload Characterization, 2001.
8. A. C. Davison and D. V. Hinkley. Bootstrap Methods and Their Applications. Cambridge University Press, 1997.
9. A. B. Downey and D. G. Feitelson. The elusive goal of workload characterization. Performance Evaluation Review, 26(4): 14-29, 1999.
10. C. Dumitrescu, I. Raicu, and I. Foster. DI-GRUBER: A Distributed Approach to Grid Resource Brokering. In proceedings of Supercomputing '05, ACM, 2005.
11. Workload Management in EGEE and gLite. http://lxmi.mi.infn.it/egee-jra1- wm/.
12. EMpht program, http://home.imf.au.dk/asmus/.
13. D. G. Feitelson. Workload modeling for performance evaluation. LNCS 2459:114-141, 2002.
14. W. Fischer, and K. Meier-Hellstern. The Markov-modulated Poisson process (MMPP) cook-book. Performance Evaluation, 18(2):149-171, 1993.
15. Hui Li. Tools for Workload Modeling in the Grid, http://www.liacs.nl/ home/hli/gwm/.
16. H. Heffes and D. M. Lucantoni. A Markov modulated characterization of packetized voice and data traffic and related statistical multiplexer performance. IEEE J. on Sci. Areas in Comm., SAC-4(6):856-868, 1986.
17. D. L. Jagerman, B. Melamed, and W. Willinger. Stochastic modeling of traffic processes. Frontiers in Queueing: Models, Methods and Problems, CRC Press, 1996.
18. H. Kantz and T. Schreiber. Nonlinear Time Series Analysis. Cambridge University Press, 2003.
19. T. Karagiannis and M. Faloutsos. SELFIS: A Tool For Self-Similarity and Long-Range Dependence Analysis. In 1st Workshop on Fractals and Self-Similarity in Data Mining: Issues and Approaches, Canada, 2002.
20. H. R. Künsch. The jackknife and bootstrap for general stationary observations. The Annals of Statistics 17, 1217-1241, 1989.
21. The Worldwide LHC Computing Grid project. http://lcg.web.cern.ch/LCG/.
22. W. Leland, M. Taqqu, W. Willinger, and D. Wilson. On the self-similar nature of ethernet traffic (extended version). IEEE/ACM Trans. on Networking, 2(1):1-15, 1994.
23. H. Li, D. Groep, and L. Wolters. Workload Characteristics of a Multi-cluster Supercomputer. LNCS 3277:176-193, Springer-Verlag, 2004.
24. lp_solve5.5.0.7. http://lpsolve.sourceforge.net/5.5/.
25. U. Lublin and D. G. Feitelson. The workload on parallel supercomputers: modeling the characteristics of rigid jobs. J. Para, and Dist. Comput., 63(11): 1105-1122, 2003.
26. A. Löbel. Solving large-scale real-world minimum-cost flow problems by a network simplex method. Technical Report SC 96-7, Konrad-Zuse-Zentrum für Informationstechnik Berlin (ZIB), February 1996. Software available at http://www.zib.de/Optimization/Software/Mcf/.
27. E. Medernach. Workload analysis of a cluster in a Grid environment. In proceedings of 11th workshop on Job Scheduling Strategies for Parallel processing, 2005.
28. R. Moeckel and B. Murray. Measuring the distance between timeseries. Physica D, 102:187-194, 1997.
29. M. Muskulus et. al. Estimating differences between probability densities and time series. In preparation. Software available at http://www.math. leidenuniv.nl/~muskulus/.
30. M. F. Neuts. Structured Stochastic Matrices of M/G/1-type and their Applications. Marcel Dekker, NY, 1989.
31. J. Nabrzyski, J. M. Schopf, and J. Weglarz (Editors). Grid Resource Management: State of the Art and Future Trends. ISBN: 1402075758, Springer, 2003.
32. D. N. Politis. The Impact of Bootstrap Methods on Time Series Analysis Statistical Science 18(2):219-230, 2003.
33. Parallel Workload Archive, http://www.cs.huji.ac.il/labs/parallel/ workload/.
34. A. Riska. Aggregate Matrix-analytic Techniques and their Applications. PhD thesis, Department of Computer Science, College of William and Mary, 2002.
35. W. J. J. Roberts, Y. Ephraim, and E. Dieguez. On Ryden's EM algorithm for estimating MMPP's. IEEE Sig. Proc. Let., to appear.
36. The LCG Real Time Monitor, http://gridportal.hep.ph.ic.ac.uk/rtm/.
37. T. Ryden. Parameter estimation for Markov modulated Poisson processes. Communications in Statistics - Stochastic Models, 10(4):795-829, 1994.
38. T. Ryden. An EM algorithm for estimation in Markov-modulated Poisson processes. Comp. Stat. and Data Analysis, 21:431-447, 1996.
39. A. Schrijver. Theory of Linear and Integer Programming. Wiley, Chichester, 1998.
40. S. L. Scott. Bayesian Methods for Hidden Markov Models: Recursive Computing in the 21st Century. J. Am. Stat. Assoc., 97(457):337-351.
41. B. Song, C. Ernemann, and R. Yahyapour. Parallel Computer Workload Modeling with Markov Chains. LNCS 3277:47-62, Springer-Verlag, 2004.
42. M. S. Squillante, D. D. Yao, and L. Zhang. The impact of job arrival patterns on parallel scheduling. ACM SIGMETRICS Performance Evaluation Review, 26(4):52-59, 1999.
43. J. I. Takeuchi and K. Yamanishi. A Unified Framework for Detecting Outliers and Change Points from Time Series. IEEE Transactions on Knowledge and Data Engineering, 18(4):482-492, 2006.
44. W Willinger, M. S. Taqqu, and A. Erramilli. A Bibliographical Guide to Self-Similar Traffic and Performance Modeling for Modern High-Speed Networks. In Stochastic Networks: Theory and Applications: 339-366, Oxford University Press, 1996.