Fingerprinting the datacenter: Automated classification of performance crises

EuroSys'10 - Proceedings of the EuroSys 2010 Conference

Volumn , Issue , 2010, Pages 111-124

  Bodík, Peter ^a   Goldszmidt, Moises ^b   Fox, Armando ^a   Woodard, Dawn B ^c   Andersen, Hans ^d  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b MICROSOFT RESEARCH   (United States)

^c Department of Molecular Biology and Genetics   (United States)

^d MICROSOFT   (United States)

Author keywords

datacenters; performance; web applications

Indexed keywords

AUTOMATED CLASSIFICATION; AUTOMATIC CLASSIFICATION; DATA CENTERS; END-TO-END PERFORMANCE; HIGH AVAILABILITY; LARGE-SCALE PRODUCTION; PERFORMANCE METRICS; RECOVERY ACTIONS; RIGOROUS EVALUATION; RUNNING APPLICATIONS; STATISTICAL SELECTION; WEB APPLICATION;

AUTOMATIC INDEXING; BENCHMARKING; WORLD WIDE WEB;

METAL RECOVERY;

EID: 77954599934     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1755913.1755926     Document Type: Conference Paper

References (25)

1. HP OpenView, welcome.hp.com/country/us/en/prodserv/software.html.
2. P. Barham, A. Donnelly, R. Isaacs, and R. Mortier. Using magpie for request extraction and workload modelling. In OSDI'04: Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, pages 18-18, Berkeley, CA, USA, 2004. USENIX Association.
3. P. Bodík, A. Fox, M. I. Jordan, D. Patterson, A. Banerjee, R. Jagannathan, T. Su, S. Tenginakai, B. Turner, and J. Ingalls. Advanced tools for operators at Amazon.com. In Hot Topics in Autonomic Computing (HotAC), 2006.
4. P. Bodík, M. Goldszmidt, and A. Fox. Hilighter: Automatically building robust signatures of performance behavior for small- and large-scale systems. In A. Fox and S. Basu, editors, SysML. USENIX Association, 2008.
5. M. Y. Chen, E. Kiciman, A. Accardi, E. A. Brewer, D. Patterson, and A. Fox. Path-based failure and evolution management. In Proc. 1st USENIX/ACM Symposium on Networked Systems Design and Implementation (NSDI'04), San Francisco, CA, March 2004.
6. I. Cohen, M. Goldszmidt, T. Kelly, J. Symons, and J. S. Chase. Correlating instrumentation data to system states: A building block for automated diagnosis and control. In Proc. 6th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2004), San Francisco, CA, Dec 2004.
7. I. Cohen, S. Zhang, M. Goldszmidt, J. Symons, T. Kelly, and A. Fox. Capturing, indexing, clustering, and retrieving system history. In A. Herbert and K. P. Birman, editors, SOSP, pages 105-118. ACM, 2005.
8. B. Cook, S. Babu, G. Candea, and S. Duan. Toward Self-Healing Multitier Services. 2007.
9. S. Duan and S. Babu. Guided problem diagnosis through active learning. In ICAC 2008, pages 45-54,Washington, DC, USA, 2008. IEEE Computer Society.
10. K. Glerum, K. Kinshumann, S. Greenberg, G. Aul, V. Orgovan, G. Nichols, D. Grant, G. Loihle, and G. Hunt. Debugging in the (very) large: Ten years of implementation and experience. In 22nd ACM Symposium on Operating Systems Principles (SOSP 2009), Big Sky, Montana, Oct 2009.
11. M. Goldszmidt, I. Cohen, S. Zhang, and A. Fox. Three research challenges at the intersection of machine learning, statistical inference, and systems. In Proc. Tenth Workshop on Hot Topics in Operating Systems (HotOS-X), Santa Fe, NM, June 2005.
12. S. Guha and A. McGregor. Stream order and order statistics: Quantile estimation in random-order streams. SIAM Journal on Computing, 38(5):2044-2059, 2009.
13. K. Koh, S.-J. Kim, and S. Boyd. An interior-point method for large-scale L1-regularized logistic regression. Journal of Machine Learning Research, 8:1519-1555, 2007. (Pubitemid 47210273)
14. N. Lachiche and P. Flach. Improving accuracy and cost of two-class and multi-class probabilistic classi.ers using ROC curves. In 20th International Conference on Machine Learning (ICML03), 2003.
15. M. Massie. The ganglia distributed monitoring system: design, implementation, and experience. Parallel Computing, 30(7):817-840, July 2004.
16. D. Patterson, A. Brown, P. Broadwell, G. Candea, M. Chen, J. Cutler, P. Enriquez, A. Fox, E. Kiciman, M. Merzbacher, D. Oppenheimer, N. Sastry, W. Tetzlaff, J. Traupamn, and N. Treuhaft. Recovery oriented computing (ROC): Motivation, definition, techniques, and case studies. Technical report, UC Berkeley, March 2002.
17. S. Pertet, R. Gandhi, and P. Narasimhan. Fingerpointing correlated failures in replicated systems. In SYSML'07: Proceedings of the 2nd USENIX workshop on Tackling computer systems problems with machine learning techniques, pages 1-6, Berkeley, CA, USA, 2007. USENIX Association.
18. J. A. Redstone, M. M. Swift, , and B. N. Bershad. Using computers to diagnose computer problems. In 9th Workshop on Hot Topics in Operating Systems (HotOS-IX), Elmau, Germany, 2003.
19. P. Reynolds, J. L.Wiener, J. C. Mogul, M. A. Shah, C. Killian, and A. Vahdat. Experiences with Pip: .nding unexpected behavior in distributed systems. In SOSP '05: Proceedings of the twentieth ACM symposium on Operating systems principles, pages 1-2, New York, NY, USA, 2005. ACM.
20. D. Woodard and M. Goldszmidt. Model-based clustering for online crisis identification in distributed computing. Technical report, Microsoft Research, 2009.
21. S. A. Yemini, S. Kliger, E. Mozes, Y. Yemini, and D. Ohsie. High speed and robust event correlation. Communications Magazine, IEEE, 34(5):82-90, 1996.
22. M. Young and P. T. Hastie. L1 regularization path algorithm for generalized linear models, 2006.
23. C. Yuan, N. L. J.-R. Wen, J. Li, Z. Zhang, Y.-M. Wang, and W.-Y. Ma. Automated known problem diagnosis with event traces. In EuroSys 2006, Leuven, Belgium, April 2006.
24. J. Zhang, Z. Ghahramani, and Y. Yang. A probabilistic model for online document clustering with application to novelty detection. In L. K. Saul, Y. Weiss, and L. Bottou, editors, Advances in Neural Information Processing Systems 17, pages 1617-1624. MIT Press, Cambridge, MA, 2005.
25. S. Zhang, I. Cohen, M. Goldszmidt, J. Symons, and A. Fox. Ensembles of models for automated diagnosis of system performance problems. In 2005 Intl. Conf. on Dependable Systems and Networks (DSN 2005), Yokohama, Japan, June 2005.