Adaptive online scheduling in storm

DEBS 2013 - Proceedings of the 7th ACM International Conference on Distributed Event-Based Systems

Volumn , Issue , 2013, Pages 207-218

  Aniello, Leonardo ^a   Baldoni, Roberto ^a   Querzoni, Leonardo ^a  

^a SAPIENZA UNIVERSITY OF ROME   (Italy)

Author keywords

CEP; Distributed event processing; Scheduling; Storm

Indexed keywords

CATASTROPHIC FAILURES; CEP; COMPUTING INFRASTRUCTURES; CONTINUOUSLY MONITORING SYSTEM; DISRUPTIVE TECHNOLOGY; DISTRIBUTED EVENT PROCESSING; REAL-TIME COMPUTATIONS; TOPOLOGY STRUCTURE;

DATA FLOW ANALYSIS; DISTRIBUTED COMPUTER SYSTEMS; OPEN SYSTEMS; SCHEDULING; SOFTWARE ARCHITECTURE; STORMS;

TOPOLOGY;

EID: 84881177989     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2488222.2488267     Document Type: Conference Paper

References (30)

1. Gartner says big data creates big jobs: 4.4 million it jobs globally to support big data by 2015. http://www.gartner.com/newsroom/id/2207915.
2. Storm. http://storm-project.net/.
3. Streammine3g. https://streammine3g.inf.tu-dresden.de/trac.
4. Data never sleeps infographic. http://www.domo.com/learn/7/236#videos- and-infographics, 2012.
5. D. J. Abadi, Y. Ahmad, M. Balazinska, M. Cherniack, J. hyon Hwang, W. Lindner, A. S. Maskey, E. Rasin, E. Ryvkina, N. Tatbul, Y. Xing, and S. Zdonik. The design of the borealis stream processing engine. In Proceedings of the 2nd Biennial Conference on Innovative Data Systems Research, 2005.
6. L. Amini, H. Andrade, R. Bhagwan, F. Eskesen, R. King, P. Selo, Y. Park, and C. Venkatramani. Spc: a distributed, scalable platform for data mining. In Proceedings of the 4th international workshop on Data mining standards, services and platforms, 2006.
7. L. Aniello, L. Querzoni, and R. Baldoni. Input data organization for batch processing in time window based computations. In Proceedings of the 28th Symposium On Applied Computing, 2013.
8. R. Baldoni, G. A. Di Luna, D. Firmani, and G. Lodi. A model for continuous query latencies in data streams. In Proceedings of the 1st International Workshop on Algorithms and Models for Distributed Event Processing, 2011.
9. S. Bansal, P. Kumar, and K. Singh. An improved duplication strategy for scheduling precedence constrained graphs in multiprocessor systems. IEEE Transactions on Parallel and Distributed Systems, 2003.
10. Y. Baram, R. El-Yaniv, and K. Luz. Online choice of active learning algorithms. The Journal of Machine Learning Research, 2004.
11. A. Brito, A. Martin, T. Knauth, S. Creutz, D. Becker, S. Weigert, and C. Fetzer. Scalable and low-latency data processing with stream mapreduce. In Proceedings of the 2011 IEEE Third International Conference on Cloud Computing Technology and Science, 2011.
12. M. Cammert, C. Heinz, J. Kramer, B. Seeger, S. Vaupel, and U. Wolske. Flexible multi-threaded scheduling for continuous queries over data streams. In Proceedings of the 2007 IEEE 23rd International Conference on Data Engineering Workshop, 2007.
13. T. Condie, N. Conway, P. Alvaro, J. M. Hellerstein, K. Elmeleegy, and R. Sears. Mapreduce online. In Proceedings of the 7th USENIX conference on Networked systems design and implementation, 2010.
14. J. Dean and S. Ghemawat. Mapreduce: simplified data processing on large clusters. Communications of the ACM, 2008.
15. C. Eaton, D. Deroos, T. Deutsch, G. Lapis, and P. Zikopoulos. Understanding Big Data. McGraw Hill, 2012.
16. A. H. Hormati, Y. Choi, M. Kudlur, R. Rabbah, T. Mudge, and S. Mahlke. Flextream: Adaptive compilation of streaming applications for heterogeneous architectures. In Proceedings of the 2009 18th International Conference on Parallel Architectures and Compilation Techniques, 2009.
17. F. P. Junqueira and B. C. Reed. The life and times of a zookeeper. In Proceedings of the 21st annual symposium on Parallelism in algorithms and architectures, 2009.
18. Y.-K. Kwok and I. Ahmad. Dynamic critical-path scheduling: An effective technique for allocating task graphs to multiprocessors. IEEE Transactions on Parallel and Distributed Systems, 1996.
19. G. T. Lakshmanan, Y. G. Rabinovich, and O. Etzion. A stratified approach for supporting high throughput event processing applications. In Proceedings of the 3rd ACM International Conference on Distributed Event-Based Systems, 2009.
20. A. Martin, C. Fetzer, and A. Brito. Active replication at (almost) no cost. In Proceedings of the 2011 IEEE 30th International Symposium on Reliable Distributed Systems, 2011.
21. A. Martin, T. Knauth, S. Creutz, D. Becker, S. Weigert, C. Fetzer, and A. Brito. Low-overhead fault tolerance for high-throughput data processing systems. In Proceedings of the 2011 31st International Conference on Distributed Computing Systems, 2011.
22. L. A. Moakar, A. Labrinidis, and P. K. Chrysanthis. Adaptive class-based scheduling of continuous queries. In Proceedings of the 2012 IEEE 28th International Conference on Data Engineering Workshops, 2012.
23. L. Neumeyer, B. Robbins, A. Nair, and A. Kesari. S4: Distributed stream computing platform. In Proceedings of the 2010 IEEE International Conference on Data Mining Workshops, 2010.
24. P. Pietzuch, J. Ledlie, J. Shneidman, M. Roussopoulos, M. Welsh, and M. Seltzer. Network-aware operator placement for stream-processing systems. In Proceedings of the 22nd International Conference on Data Engineering, 2006.
25. M. A. Sharaf, P. K. Chrysanthis, and A. Labrinidis. Preemptive rate-based operator scheduling in a data stream management system. In Proceedings of the ACS/IEEE 2005 International Conference on Computer Systems and Applications, 2005.
26. M. A. Suleman, M. K. Qureshi, Khubaib, and Y. N. Patt. Feedback-directed pipeline parallelism. In Proceedings of the 19th international conference on Parallel architectures and compilation techniques, 2010.
27. T. White. Hadoop: The Definitive Guide. O'Reilly Media, Inc., 2012.
28. J. Wolf, N. Bansal, K. Hildrum, S. Parekh, D. Rajan, R. Wagle, K.-L. Wu, and L. Fleischer. Soda: an optimizing scheduler for large-scale stream-based distributed computer systems. In Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware, 2008.
29. Y. Xing, J.-H. Hwang, U. Çetintemel, and S. Zdonik. Providing resiliency to load variations in distributed stream processing. In Proceedings of the 32nd international conference on Very large data bases, 2006.
30. Y. Xing, S. Zdonik, and J.-H. Hwang. Dynamic load distribution in the borealis stream processor. In Proceedings of the 21st International Conference on Data Engineering, 2005.