Priority-based resource scheduling in distributed stream processing systems for big data applications

Proceedings - 2014 IEEE/ACM 7th International Conference on Utility and Cloud Computing, UCC 2014

Volumn , Issue , 2014, Pages 363-370

  Bellavista, Paolo ^a   Corradi, Antonio ^a   Reale, Andrea ^a   Ticca, Nicola ^a  

^a UNIVERSITY OF BOLOGNA   (Italy)

Author keywords

Application level and Application specific Scheduling; Big Data; Cloud Computing Optimization; Distributed Stream Processing; Priority based Resource Scheduling; Vehicular Traffic Analysis

Indexed keywords

CLOUD COMPUTING; COST EFFECTIVENESS; DATA COMMUNICATION SYSTEMS; DATA HANDLING; DISTRIBUTED PARAMETER CONTROL SYSTEMS; FLOW GRAPHS; GRAPHIC METHODS; INPUT OUTPUT PROGRAMS; SCHEDULING;

APPLICATION DEVELOPERS; APPLICATION-SPECIFIC SCHEDULING; DATA STREAM PROCESSING; DESIGN AND IMPLEMENTS; DISTRIBUTED COMPUTING RESOURCES; DISTRIBUTED STREAM PROCESSING; EXPERIMENTAL EVALUATION; RESOURCE-SCHEDULING;

BIG DATA;

EID: 84946692108     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/UCC.2014.46     Document Type: Conference Paper

References (36)

1. H. Farhangi, "The path of the smart grid," Power and Energy Magazine, IEEE, vol. 8, no. 1, pp. 18-28, 2010
2. S. Djahel, M. Salehie, I. Tal, and P. Jamshidi, "Adaptive traffic management for secure and efficient emergency services in smart cities," in Proc. of the IEEE 2013 PerCom Conference. San Diego, CA, USA: IEEE, 2013, pp. 340-343
3. D. Barbieri, D. Braga, S. Ceri, E. Valle, Y. Huang, V. Tresp, A. Rettinger, and H. Wermser, "Deductive and inductive stream reasoning for semantic social media analytics," Intelligent Systems, IEEE, vol. 25, no. 6, pp. 32-41, 2010
4. Bloomberg L. P., "Bloomberg professional," http://www. bloomberg. com/ professional/, Web Page, last visited in Feb. 2014
5. Innovative Market Analysis, "MetaStock," http://www. metastock. com, Web Page, last visited in Feb. 2014
6. G. Cugola and A. Margara, "Processing flows of information: From data stream to complex event processing," ACM Comput. Surv., vol. 44, no. 3, pp. 15:1-15:62, 2012
7. S. Geisler, "Data stream management systems," in Data Exchange, Integration, and Streams, P. G. Kolaitis, M. Lenzerini, and N. Schweikardt, Eds. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, 2013, vol. 5, ch. 5, pp. 275-304
8. M. Stonebraker, U. Çetintemel, and S. Zdonik, "The 8 requirements of real-time stream processing," ACM SIGMOD Rec., vol. 34, no. 4, pp. 42-47, 2005
9. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly, "Dryad: distributed data-parallel programs from sequential building blocks," in Proc. of the 2nd ACM SIGOPS/EuroSys Conference, vol. 41, no. 3. ACM, 2007, pp. 59-72
10. J. Dean and S. Ghemawat, "Mapreduce: Simplified data processing on large clusters," Commun. ACM, vol. 51, no. 1, pp. 107-113, 2008
11. L. Barroso, J. Dean, and U. Holzle, "Web search for a planet: the google cluster architecture," IEEE Micro, vol. 23, no. 2, pp. 22-28, 2003
12. V. Rialle, F. Duchene, N. Noury, L. Bajolle, and J. Demongeot, "Health smart home: information technology for patients at home," Telemedicine Journal and E-Health, vol. 8, no. 4, pp. 395-409, 2002
13. T. Akidau, A. Balikov, K. Bekiroglu, S. Chernyak, J. Haberman, R. Lax, S. McVeety, D. Mills, P. Nordstrom, and S. Whittle, "Millwheel: Faulttolerant stream processing at internet scale," in Proc. of the 39th VLDB Conference, no. 11. VLDB Endowment, 2013, pp. 1-12
14. P. Bellavista, A. Corradi, and A. Reale, "The quasit model and framework for scalable data stream processing with quality of service," in In Proc. of the 5th MOBILWARE Conference. Springer Berlin Heidelberg, 2013, pp. 92-107
15. B. Gedik and H. Andrade, "A model-based framework for building extensible, high performance stream processing middleware and programming language for ibm infosphere streams," Soft. Pract. Exper., vol. 42, no. 11, pp. 1363-1391, 2012
16. B. Lohrmann, D. Warneke, and O. Kao, "Nephele streaming: stream processing under qos constraints at scale," Cluster Computing, pp. 1-18, Jul. 2013
17. L. Neumeyer, B. Robbins, A. Nair, and A. Kesari, "S4: Distributed stream computing platform," in Proc. of the 10th IEEE ICDMW Conference. Sydney, Australia: IEEE, 2010, pp. 170-177
18. N. Marz, "Storm project," http://storm-project. net/, last visited in Feb. 2014
19. J. Wolf, N. Bansal, K. Hildrum, S. Parekh, and D. Rajan, "Soda: An optimizing scheduler for large-scale stream-based distributed computer systems," in Proc. of the 9th Middleware Conference. Leuven, Belgium: Springer Berlin Heidelberg, 2008, pp. 306-325
20. L. Amini, N. Jain, A. Sehgal, J. Silber, and O. Verscheure, "Adaptive Control of Extreme-scale Stream Processing Systems," in Proc. of the 26th IEEE ICDCS Conference. Lisbon, Portugal: Ieee, 2006, pp. 71-71
21. Y. Zhou, C. Ooi, K.-l. Tan, and J. Wu, "Efficient Dynamic Operator Placement in a Locally Distributed Continuous Query System," in Proc. of the 2006 OTM Confederated Conferences CoopIS, DOA, GADA, and ODBASE. Springer, 2006, pp. 54-71
22. J. Kay and P. Lauder, "A fair share scheduler," Communications of the ACM, vol. 31, no. 1, pp. 44-55, 1988
23. "Quasit Project Web Site," http://www-lia. deis. unibo. it/Research/quasit/, Web Page, last visited in Feb. 2014
24. M. Odersky, P. Altherr, V. Cremet, B. Emir, S. Maneth, S. Micheloud, N. Mihaylov, M. Schinz, E. Stenman, and M. Zenger, "An overview of the scala programming language," Ècole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland, Tech. Rep., 2004
25. Typesafe Inc., "Akka Toolkit," http://akka. io, Web Page, last visited in Feb. 2014
26. C. A. Waldspurger and W. E. Weihl, "Lottery scheduling: Flexible proportional-share resource management," in Proceedings of the 1st USENIX OSDI Conference. USENIX Association, 1994
27. M. Behrisch, L. Bieker, J. Erdmann, and D. Krajzewicz, "Sumosimulation of urban mobility-an overview," in SIMUL 2011, The Third International Conference on Advances in System Simulation, Barcelona, Spain, 2011, pp. 55-60
28. D. Krajzewicz, M. Heinrich, M. Milano, P. Bellavista, T. Stützle, J. Harri, T. Spyropoulos, R. Blokpoel, S. Hausberger, and M. Fellendorf, "Colombo: Investigating the potential of v2x for traffic management purposes assuming low penetration rates," in 9th ITS European Congress, Dublin, Ireland, 2013
29. Y. Xing, S. Zdonik, and J.-H. Hwang, "Dynamic load distribution in the borealis stream processor," in Proc. of the 21st ICDE Conference. Tokyo, Japan: IEEE, 2005, pp. 791-802
30. Y. Xing, J.-h. Hwang, and S. Zdonik, "Providing resiliency to load variations in distributed stream processing," in Proc. of the 32nd VLDB Conference. Seoul, Korea: VLDB Endowment, 2006, pp. 775-786
31. V. Kumar, B. Cooper, and K. Schwan, "Distributed stream management using utility-driven self-adaptive middleware," in Proc. of the 2nd ICAC Conference. IEEE, 2005, pp. 3-14
32. I. Boutsis and V. Kalogeraki, "Radar: Adaptive rate allocation in distributed stream processing systems under bursty workloads," in Proc. of the 31st IEEE SRDS Symposium. IEEE, 2012, pp. 285-290
33. L. Eggert and J. Heidemann, "Application-level differentiated services for web servers," World Wide Web, vol. 2, no. 3, pp. 133-142, 1999
34. "Await: Efficient overload management for busy multi-tier web services under bursty workloads," in Proc. of the 10th ICWE Conference. Vienna, Austria: Springer Berlin Heidelberg, 2010, pp. 81-97
35. T. Sandholm and K. Lai, "Mapreduce optimization using regulated dynamic prioritization," in Proc. of the 11th ACM SIGMETRICS Conference. Seattle, WA, USA: ACM, 2009, pp. 299-310
36. -, "Dynamic proportional share scheduling in hadoop," in Job Scheduling Strategies for Parallel Processing, ser. Lecture Notes in Computer Science, E. Frachtenberg and U. Schwiegelshohn, Eds. Springer Berlin Heidelberg, 2010, vol. 6253, pp. 110-131.