Treadmill: Attributing the Source of Tail Latency through Precise Load Testing and Statistical Inference

Proceedings - 2016 43rd International Symposium on Computer Architecture, ISCA 2016

Volumn , Issue , 2016, Pages 456-468

  Zhang, Yunqi ^a   Meisner, David ^b   Mars, Jason ^a   Tang, Lingjia ^a  

^a UNIVERSITY OF MICHIGAN   (United States)

^b FACEBOOK   (United States)

Author keywords

data center; load testing; tail latency

Indexed keywords

COMPUTER ARCHITECTURE; COMPUTER HARDWARE; HARDWARE; INTERNET; QUALITY CONTROL; RECONFIGURABLE HARDWARE; SPORTING GOODS;

DATA CENTERS; EVALUATION METHODOLOGIES; HARDWARE CONFIGURATIONS; INHERENT VARIABILITY; MEASUREMENT AND ANALYSIS; PRODUCTION WORKLOADS; STATISTICAL INFERENCE; TAIL LATENCY;

LOAD TESTING;

EID: 84988376949     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ISCA.2016.47     Document Type: Conference Paper

References (55)

1. D. Meisner, C. M. Sadler, L. A. Barroso, W.-D. Weber, and T. F. Wenisch, "Power management of online data-intensive services," in Proceedings of the International Symposium on Computer Architecture (ISCA), 2011.
2. J. Dean and L. A. Barroso, "The tail at scale," Commun. ACM, 2013.
3. J. Li, N. K. Sharma, D. R. K. Ports, and S. D. Gribble, "Tales of the tail: Hardware, os, and application-level sources of tail latency," in Proceedings of the Symposium on Cloud Computing (SoCC), 2014.
4. B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan, and R. Sears, "Benchmarking cloud serving systems with ycsb," in Proceedings of the Symposium on Cloud Computing (SoCC), 2010.
5. X. Fan, W.-D. Weber, and L. A. Barroso, "Power provisioning for a warehouse-sized computer," in Proceedings of the International Symposium on Computer Architecture (ISCA), 2007.
6. M. Ferdman, A. Adileh, O. Kocberber, S. Volos, M. Alisafaee, D. Jevdjic, C. Kaynak, A. D. Popescu, A. Ailamaki, and B. Falsafi, "Clearing the clouds: A study of emerging scale-out workloads on modern hardware," in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2012.
7. K. Lim, P. Ranganathan, J. Chang, C. Patel, T. Mudge, and S. Reinhardt, "Understanding and designing new server architectures for emerging warehouse-computing environments," in Proceedings of the International Symposium on Computer Architecture (ISCA), 2008.
8. L. Wang, J. Zhan, C. Luo, Y. Zhu, Q. Yang, Y. He, W. Gao, Z. Jia, Y. Shi, S. Zhang, C. Zheng, G. Lu, K. Zhan, X. Li, and B. Qiu, "Bigdatabench: A big data benchmark suite from internet services," in Proceedings of the International Symposium on High Performance Computer Architecture (HPCA), 2014.
9. J. Hauswald, M. A. Laurenzano, Y. Zhang, C. Li, A. Rovinski, A. Khurana, R. G. Dreslinski, T. Mudge, V. Petrucci, L. Tang, and J. Mars, "Sirius: An open end-to-end voice and vision personal assistant and its implications for future warehouse scale computers," in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2015.
10. J. Hauswald, Y. Kang, M. A. Laurenzano, Q. Chen, C. Li, T. Mudge, R. G. Dreslinski, J. Mars, and L. Tang, "Djinn and tonic: Dnn as a service and its implications for future warehouse scale computers," in Proceedings of the International Symposium on Computer Architecture (ISCA), 2015.
11. L. A. Barroso, J. Clidaras, and U. Hölzle, "The datacenter as a computer: An introduction to the design of warehousescale machines," Synthesis lectures on computer architecture, 2013.
12. D. G. Feitelson, E. Frachtenberg, and K. L. Beck, "Development and deployment at facebook," IEEE Internet Computing, 2013.
13. C. Delimitrou and C. Kozyrakis, "Paragon: Qos-aware scheduling for heterogeneous datacenters," in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2013.
14. -, "Quasar: Resource-efficient and qos-aware cluster management," in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2014.
15. S. Kanev, K. Hazelwood, G.-Y. Wei, and D. Brooks, "Tradeoffs between power management and tail latency in warehouse-scale applications," in Proceedings of the International Symposium on Workload Characterization (IISWC), 2014.
16. H. Kasture and D. Sanchez, "Ubik: Efficient cache sharing with strict qos for latency-critical workloads," in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2014.
17. D. Lo and C. Kozyrakis, "Dynamic management of turbomode in modern multi-core chips," in Proceedings of the International Symposium on High Performance Computer Architecture (HPCA), 2014.
18. D. Lo, L. Cheng, R. Govindaraju, L. A. Barroso, and C. Kozyrakis, "Towards energy proportionality for large-scale latency-critical workloads," in Proceedings of the International Symposium on Computer Architecuture (ISCA), 2014.
19. S. M. Zahedi and B. C. Lee, "Ref: Resource elasticity fairness with sharing incentives for multiprocessors," in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2014.
20. Y. Zhang, M. A. Laurenzano, J. Mars, and L. Tang, "Smite: Precise qos prediction on real-system smt processors to improve utilization in warehouse scale computers," in Proceedings of the International Symposium on Microarchitecture (MICRO), 2014.
21. C. H. Hsu, Y. Zhang, M. A. Laurenzano, D. Meisner, T. Wenisch, J. Mars, L. Tang, and R. G. Dreslinski, "Adrenaline: Pinpointing and reining in tail queries with quick voltage boosting," in Proceedings of the International Symposium on High Performance Computer Architecture (HPCA), 2015.
22. H. Yang, A. Breslow, J. Mars, and L. Tang, "Bubble-flux: Precise online qos management for increased utilization in warehouse scale computers," in Proceedings of the International Symposium on Computer Architecture (ISCA), 2013.
23. L. Tang, J. Mars, X. Zhang, R. Hagmann, R. Hundt, and E. Tune, "Optimizing google's warehouse scale computers: The numa experience," in Proceedings of the International Symposium on High Performance Computer Architecture (HPCA), 2013.
24. J. Mars, L. Tang, R. Hundt, K. Skadron, and M. L. Soffa, "Bubble-up: Increasing utilization in modern warehouse scale computers via sensible co-locations," in Proceedings of the International Symposium on Microarchitecture (MICRO), 2011.
25. L. Tang, J. Mars, N. Vachharajani, R. Hundt, and M. L. Soffa, "The impact of memory subsystem resource sharing on datacenter applications," in Proceedings of the International Symposium on Computer Architecture (ISCA), 2011.
26. R. Koenker, Quantile regression, 2005.
27. A. Likhtarov, R. Nishtala, R. McElroy, H. Fugal, A. Grynenko, and V. Venkataramani, "Introducing mcrouter: A memcached protocol router for scaling memcached deployments," 2014.
28. "Faban performance workload creation and execution framework," http://faban.org/.
29. J. Leverich and C. Kozyrakis, "Reconciling high server utilization and sub-millisecond quality-of-service," in Proceedings of the European Conference on Computer Systems (EuroSys), 2014.
30. "Spec jenterprise," https://www.spec.org/jEnterprise2010/.
31. "Spec sip-infrastructure 2011," https://www.spec.org/specsip/.
32. B. Schroeder, A. Wierman, and M. Harchol-Balter, "Open versus closed: A cautionary tale." in Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (NSDI), 2006.
33. C. Curtsinger and E. D. Berger, "Stabilizer: Statistically sound performance evaluation," in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2013.
34. H. Fugal, "Wangle," https://github.com/facebook/wangle.
35. R. Nishtala, H. Fugal, S. Grimm, M. Kwiatkowski, H. Lee, H. C. Li, R. McElroy, M. Paleczny, D. Peek, P. Saab, D. Stafford, T. Tung, and V. Venkataramani, "Scaling memcache at facebook," in Proceedings of the USENIX Conference on Networked Systems Design and Implementation (NSDI), 2013.
36. B. Atikoglu, Y. Xu, E. Frachtenberg, S. Jiang, and M. Paleczny, "Workload analysis of a large-scale key-value store," in Proceedings of the International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS), 2012.
37. B. Fitzpatrick, "Distributed caching with memcached," Linux J., 2004.
38. C. Aniszczyk, "Caching with twemcache," https://blog.twitter.com/2012/caching-with-twemcache, 2012.
39. B. Fan, D. G. Andersen, and M. Kaminsky, "Memc3: Compact and concurrent memcache with dumber caching and smarter hashing." in Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (NSDI), 2013.
40. K. Lim, D. Meisner, A. G. Saidi, P. Ranganathan, and T. F. Wenisch, "Thin servers with smart pipes: Designing soc accelerators for memcached," in Proceedings of the International Symposium on Computer Architecture (ISCA), 2013.
41. T. H. Hetherington, M. O'Connor, and T. M. Aamodt, "MemcachedGPU: Scaling-up scale-out key-value stores," in Proceedings of the Symposium on Cloud Computing (SoCC), 2015.
42. T. Hetherington, T. Rogers, L. Hsu, M. O'Connor, and T. Aamodt, "Characterizing and evaluating a key-value store application on heterogeneous cpu-GPU systems," in Proceedings of the International Symposium on Performance Analysis of Systems and Software (ISPASS), 2012.
43. S. Li, H. Lim, V. W. Lee, J. H. Ahn, A. Kalia, M. Kaminsky, D. G. Andersen, O. Seongil, S. Lee, and P. Dubey, "Architecting to achieve a billion requests per second throughput on a single key-value store server platform," in Proceedings of the International Symposium on Computer Architecture (ISCA), 2015.
44. A. Gutierrez, M. Cieslak, B. Giridhar, R. G. Dreslinski, L. Ceze, and T. Mudge, "Integrated 3d-stacked server designs for increasing physical density of key-value stores," in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2014.
45. A. B. de Oliveira, S. Fischmeister, A. Diwan, M. Hauswirth, and P. F. Sweeney, "Why you should care about quantile regression," in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2013.
46. M. Harchol-Balter, Performance Modeling and Design of Computer Systems: Queueing Theory in Action, 2013.
47. R. W. Keener, "Theoretical statistics: Topics for a core course," International Statistical Review, 2012.
48. T. Mytkowicz, A. Diwan, M. Hauswirth, and P. F. Sweeney, "Producing wrong data without doing anything obviously wrong!" in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2009.
49. A. Alameldeen and D. Wood, "Variability in architectural simulations of multi-threaded workloads," in High-Performance Computer Architecture (HPCA). Proceedings. The Ninth International Symposium on, 2003.
50. D. Tsafrir and D. Feitelson, "Instability in parallel job scheduling simulation: the role of workload flurries," in Parallel and Distributed Processing Symposium (IPDPS). 20th International, 2006.
51. D. Tsafrir, K. Ouaknine, and D. Feitelson, "Reducing performance evaluation sensitivity and variability by input shaking," in Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS). 15th International Symposium on, 2007.
52. A. Georges, D. Buytaert, and L. Eeckhout, "Statistically rigorous java performance evaluation," in Proceedings of the Annual Conference on Object-oriented Programming Systems and Applications (OOPSLA), 2007.
53. M. B. Breughe and L. Eeckhout, "Selecting representative benchmark inputs for exploring microprocessor design spaces," ACM Trans. Archit. Code Optim.
54. K. Shen, "Request behavior variations," in Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2010.
55. D. Meisner, J. Wu, and T. F. Wenisch, "Bighouse: A simulation infrastructure for data center systems," in Proceedings of the International Symposium on Performance Analysis of Systems and Software (ISPASS), 2012.