Uncertainty-aware elastic virtual machine scheduling for stream processing systems

Proceedings - 18th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, CCGRID 2018

Volumn , Issue , 2018, Pages 62-71

  Imai, Shigeru ^a   Patterson, Stacy ^a   Varela, Carlos A ^a  

^a RENSSELAER POLYTECHNIC INSTITUTE   (United States)

Author keywords

Quality of service; Resource scheduling; Stream processing

Indexed keywords

BENCHMARKING; CLUSTER COMPUTING; GRID COMPUTING; ONLINE SYSTEMS; SCHEDULING; VIRTUAL MACHINE;

BENCHMARK APPLICATIONS; RESOURCE-SCHEDULING; SCHEDULING FRAMEWORKS; STATISTICAL TECHNIQUES; STREAM PROCESSING; STREAM PROCESSING SYSTEMS; UNCERTAINTY QUANTIFICATIONS; VIRTUAL MACHINE SCHEDULING;

QUALITY OF SERVICE;

EID: 85050956627     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CCGRID.2018.00021     Document Type: Conference Paper

References (20)

1. Amazon Web Services, "Amazon Auto Scaling," http://aws. Amazon.com/autoscaling/.
2. Y. Kwon, M. Balazinska, B. Howe, and J. Rolia, "SkewTune: Mitigating Skew in Mapreduce Applications," in Proc. ACM SIGMOD Conf. on Manage. of Data, 2012, pp. 25-36.
3. C. Delimitrou and C. Kozyrakis, "HCloud: Resource-efficient provisioning in shared cloud systems," in ACM SIGOPS Operating Systems Review, vol. 50, no. 2, 2016, pp. 473-488.
4. N. Rameshan, Y. Liu, L. Navarro, and V. Vlassov, "Augmenting elasticity controllers for improved accuracy," in Proc. IEEE Conf. on Autonomic Computing, 2016, pp. 117-126.
5. M. A. Rodriguez and R. Buyya, "Deadline Based Resource Provisioning and Scheduling Algorithm for Scientific Workflows on Clouds," IEEE Trans. on Cloud Computing, vol. 2, no. 2, pp. 222-235, 2014.
6. R. Hu, J. Jiang, G. Liu, and L. Wang, "KSwSVR: A new load forecasting method for efficient resources provisioning in cloud," in Proc. IEEE Conf. on Services Computing, 2013, pp. 120-127.
7. J. Jiang, J. Lu, G. Zhang, and G. Long, "Optimal cloud resource auto-scaling for web applications," in in Proc. IEEE Symp. on Cluster, Cloud and Grid Computing, 2013, pp. 58-65.
8. S. Imai, S. Patterson, and C. A. Varela, "Maximum Sustainable Throughput Prediction for Data Stream Processing over Public Clouds," in Proc. IEEE Symp. on Cluster, Cloud and Grid Computing, May 2017.
9. G. E. Box, G. M. Jenkins, G. C. Reinsel, and G. M. Ljung, Time series analysis: forecasting and control. John Wiley & Sons, 2015.
10. N. Roy, A. Dubey, and A. Gokhale, "Efficient autoscaling in the cloud using predictive models for workload forecasting," in Proc. IEEE Conf. on Cloud Computing, 2011, pp. 500-507.
11. Y. Liu, N. Rameshan, E. Monte, V. Vlassov, and L. Navarro, "ProRenaTa: Proactive and reactive tuning to scale a distributed storage system," in Proc. IEEE Symp. on Cluster, Cloud and Grid Computing, 2015, pp. 453-464.
12. J. Kreps and L. Corp, "Kafka : A distributed messaging system for log processing," ACM SIGMOD Workshop on Networking Meets Databases, p. 6, 2011.
13. A. Toshniwal, S. Taneja, A. Shukla, K. Ramasamy, J. M. Patel, S. Kulkarni, J. Jackson, K. Gade, M. Fu, J. Donham, N. Bhagat, S. Mittal, and D. Ryaboy, "Storm@twitter," in Proc. of SIGMOD Conf. on Manage. of Data, 2014, pp. 147-156.
14. P. Carbone, S. Ewen, S. Haridi, A. Katsifodimos, V. Markl, and K. Tzoumas, "Apache Flink: Stream and batch processing in a single engine," IEEE Data Eng. Bulletin, vol. 38, no. 4, Dec 2015.
15. S. Imai, S. Patterson, and C. A. Varela, "Maximum sustainable throughput prediction for large-scale data streaming systems," Department of Computer Science, Rensselaer Polytechnic Institute, Tech. Rep., November 2017, (Extended journal version of [8] in review).
16. Intel Corporation, "Storm benchmark," https://github.com/intel-hadoop/storm-benchmark.
17. G. D. F. Morales and A. Bifet, "SAMOA: Scalable Advanced Massive Online Analysis," Journal of Machine Learning Research, vol. 16, pp. 149-153, Jan 2015.
18. M. Arlitt and T. Jin, "A workload characterization study of the 1998 world cup web site," IEEE Network, vol. 14, no. 3, pp. 30-37, May 2000.
19. Twitter, "Twitter API reference," https://dev.Twitter.com/docs.
20. ADS-B Exchange, "ADS-B Exchange-World's largest co-op of unfiltered filght data," https://www.Adsbexchange.com/.