Holistic configuration management at Facebook

SOSP 2015 - Proceedings of the 25th ACM Symposium on Operating Systems Principles

Volumn , Issue , 2015, Pages 328-343

  Tang, Chunqiang ^a   Kooburat, Thawan ^a   Venkatachalam, Pradeep ^a   Chander, Akshay ^a   Wen, Zhe ^a   Narayanan, Aravind ^a   Dowell, Patrick ^a   Karl, Robert ^a  

^a FACEBOOK   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTIFICIAL INTELLIGENCE; INTERNET TELEPHONY; LEARNING SYSTEMS; LOAD TESTING; MOBILE DEVICES; MOBILE TELECOMMUNICATION SYSTEMS; SOCIAL NETWORKING (ONLINE);

BACKEND SYSTEM; CONFIGURATION MANAGEMENT; GLOBAL REGIONS; MACHINE LEARNING MODELS; MOBILE APPS; ONLINE-CONFIGURATIONS; PRODUCT FEATURE; USAGE STATISTICS;

PRODUCT DESIGN;

EID: 84957974988     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2815400.2815401     Document Type: Conference Paper

References (37)

1. ADYA, A., COOPER, G., MYERS, D., and PIATEK, M. Thialfi: a client notification service for internet-scale applications. In Proceedings of the 23rd ACM Symposium on Operating Systems Principles (2011), pp. 129-142. SOSP'11.
2. APACHE THRIFT. http://thrift.apache.org/.
3. ATTARIYAN, M., and FLINN, J. Automating configuration troubleshooting with dynamic information flow analysis. In Proceedings of 9th USENIX Symposium on Operating Systems Design and Implementation (2010), pp. 237-250. OSDI'10.
4. AWS ELASTIC BEANSTALK. http://aws.amazon.com/elasticbeanstalk/.
5. BRONSON, N., AMSDEN, Z., CABRERA, G., CHAKKA, P., DIMOV, P., DING, H., FERRIS, J., GIARDULLO, A., KULKARNI, S., LI, H., ET AL. TAO: Facebook's distributed data store for the social graph. In Proceedings of the 2013 USENIX Annual Technical Conference (2013). USENIX ATC'13.
6. BURROWS, M. The Chubby lock service for loosely-coupled distributed systems. In Proceedings of 7th USENIX Symposium on Operating Systems Design and Implementation (2006), pp. 335-350. OSDI'06.
7. CHEF. http://www.opscode.com/chef/.
8. COHEN, B. Incentives build robustness in bittorrent. In Proceedings of the 1st Workshop on Economics of Peer-to-Peer systems (2003), pp. 68-72.
9. CONRADI, R., and WESTFECHTEL, B. Version models for software configuration management. ACM Computing Surveys 30, 2 (1998), 232-282.
10. DELAET, T., JOOSEN, W., and VAN BRABANT, B. A survey of system configuration tools. In Proceedings of the 24th Large Installation System Administration Conference (2010). LISA'10.
11. DIBOWITZ, P. Really large scale systems configuration: config management @ Facebook, 2013. https://www.socallinuxexpo.org/scale11x-supporting/default/files/presentations/cfgmgmt.pdf.
12. DUVALL, P. M., MATYAS, S., and GLOVER, A. Continuous integration: improving software quality and reducing risk. Pearson Education, 2007.
13. FOWLER, M., and HIGHSMITH, J. The agile manifesto. Software Development 9, 8 (2001), 28-35.
14. GIT. http://git-scm.com/.
15. GOOGLE APP ENGINE. https://appengine.google.com/.
16. HEROKU. https://www.heroku.com/.
17. HUANG, P., and BOLOSKY, W. J., SINGH, A., and ZHOU, Y. ConfValley: A systematic configuration validation framework for cloud services. In Proceedings of the 10th European Conference on Computer Systems (2015), p. 19. EuroSys'15.
18. HUNT, P., KONAR, M., JUNQUEIRA, F. P., and REED, B. ZooKeeper: wait-free coordination for internet-scale systems. In Proceedings of the 2010 USENIX Annual Technical Conference (2010), pp. 11-11. USENIX ATC'10.
19. ISARD, M. Autopilot: automatic data center management. ACM SIGOPS Operating Systems Review 41, 2 (2007), 60-67.
20. JAVA SCRIPT OBJECT NOTATION (JSON). http://www.json.org/.
21. MAHAJAN, R., WETHERALL, D., and ANDERSON, T. Understanding BGP misconfiguration. In Proceedings of the ACM SIGCOMM 2002 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (2002), pp. 3-16. SIGCOMM'02.
22. MOGUL, J. C., CHAN, Y.-M., and KELLY, T. Design, implementation, and evaluation of duplicate transfer detection in HTTP. In Proceedings of the 1st Symposium on Networked Systems Design and Implementation (2004), pp. 43-56. NSDI'04.
23. MUTHITACHAROEN, A., CHEN, B., and MAZIERES, D. A low-bandwidth network file system. In Proceedings of the 18th ACM Symposium on Operating Systems principles (2001), pp. 174-187. SOSP'01.
24. OPPENHEIMER, D., GANAPATHI, A., and PATTERSON, D. A. Why do Internet services fail, and what can be done about it? In Proceedings of the 4th USENIX Symposium on Internet Technologies and Systems (2003). USITS'03.
25. PAPPAS, V., XU, Z., LU, S., MASSEY, D., TERZIS, A., and ZHANG, L. Impact of configuration errors on DNS robustness. In Proceedings of the ACM SIGCOMM 2004 Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication (2004), pp. 319-330. SIGCOMM'04.
26. PHABRICATOR. http://phabricator.org/.
27. POWER, A., 2011. Making Facebook Self-Healing, https://www.facebook.com/notes/facebook-engineering/making-facebook-self-healing/10150275248698920.
28. PUPPET. https://puppetlabs.com/.
29. ROSSI, C. Ship early and ship twice as often. https://www.facebook.com/notes/facebook-engineering/ship-early-and-ship-twice-as-often/10150985860363920.
30. SHERMAN, A., and LISIECKI, P. A., BERKHEIMER, A., and WEIN, J. ACMS: the Akamai configuration management system. In Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation (2005), pp. 245-258. NSDI'05.
31. SU, Y.-Y., ATTARIYAN, M., and FLINN, J. AutoBash: improving configuration management with operating system causality analysis. In Proceedings of the 21st ACM Symposium on Operating Systems Principles (2007), pp. 237-250. SOSP'07.
32. WHITAKER, A., and COX, R. S., and GRIBBLE, S. D. Configuration debugging as search: Finding the needle in the haystack. In Proceedings of 6th USENIX Symposium on Operating Systems Design and Implementation (2004), pp. 77-90. OSDI'04.
33. WOOL, A. A quantitative study of firewall configuration errors. IEEE Computer 37, 6 (2004), 62-67.
34. XU, T., ZHANG, J., HUANG, P., ZHENG, J., SHENG, T., YUAN, D., ZHOU, Y., and PASUPATHY, S. Do not blame users for misconfigurations. In Proceedings of the 24th ACM Symposium on Operating Systems Principles (2013), pp. 244-259. SOSP'13.
35. XU, T., and ZHOU, Y. Systems approaches to tackling configuration errors: A survey. ACM Computing Surveys 47, 4 (2015), 70.
36. YIN, Z., MA, X., ZHENG, J., ZHOU, Y., and BAIRAVASUNDARAM, L. N., and PASUPATHY, S. An empirical study on configuration errors in commercial and open source systems. In Proceedings of the 23rd ACM Symposium on Operating Systems Principles (2011), pp. 159-172. SOSP'11.
37. ZHANG, J., RENGANARAYANA, L., ZHANG, X., GE, N., BALA, V., XU, T., and ZHOU, Y. EnCore: Exploiting system environment and correlation information for misconfiguration detection. In Proceedings of the 19th Architectural Support for Programming Languages and Operating Systems (2014), pp. 687-700. ASPLOS'14.