Understanding and detecting real-world performance bugs

Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI)

Volumn , Issue , 2012, Pages 77-87

  Jin, Guoliang ^a   Song, Linhai ^a   Shi, Xiaoming ^a   Scherpelz, Joel ^a   Lu, Shan ^a  

^a UNIVERSITY OF WISCONSIN MADISON   (United States)

Author keywords

Characteristics study; Performance bugs; Rule based bug detection

Indexed keywords

BUG DETECTION; CHARACTERISTICS STUDY; CODE SEQUENCES; COMPREHENSIVE STUDIES; MOZILLA; MULTI CORE; PERFORMANCE BUGS; PERFORMANCE DEGRADATION; PERFORMANCE PROBLEMS; REAL-WORLD PERFORMANCE; RESOURCE WASTES; SINGLE-THREADED; SOFTWARE SUITE;

COMPUTER PROGRAMMING LANGUAGES; DEGRADATION; ENERGY EFFICIENCY;

PROGRAM DEBUGGING;

EID: 84863459107     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2254064.2254075     Document Type: Conference Paper

References (60)

1. M. K. Aguilera, J. C. Mogul, J. L.Wiener, P. Reynolds, and A. Muthitacharoen. Performance debugging for distributed systems of black boxes. In SOSP, 2003.
2. M. Attariyan and J. Flinn. Automating configuration troubleshooting with dynamic information flow analysis. In OSDI, 2010.
3. W. Baek and T. M. Chilimbi. Green: a framework for supporting energy-conscious programming using controlled approximation. In PLDI, 2010.
4. C. Cadar, D. Dunbar, and D. Engler. Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs. In OSDI, 2008.
5. V. Chipounov, V. Kuznetsov, and G. Candea. S2E: a platform for invivo multi-path analysis of software systems. In ASPLOS, 2011.
6. A. Chou, B. Chelf, D. Engler, and M. Heinrich. Using meta-level compilation to check FLASH protocol code. In ASPLOS, 2000.
7. A. Chou, J. Yang, B. Chelf, S. Hallem, and D. R. Engler. An empirical study of operating system errors. In SOSP, 2001.
8. I. Cohen, M. Goldszmidt, T. Kelly, J. Symons, and J. S. Chase. Correlating instrumentation data to system states: a building block for automated diagnosis and control. In OSDI, 2004.
9. M. Corporation. Performance Testing Guidance for Web Applications. Microsoft Press, 2007.
10. A. Diwan, M. Hauswirth, T. Mytkowicz, and P. F. Sweeney. Traceanalyzer: a system for processing performance traces. Softw. Pract. Exper., March 2011.
11. B. Dufour, B. G. Ryder, and G. Sevitsky. A scalable technique for characterizing the usage of temporaries in framework-intensive java applications. In FSE, 2008. 86.
12. R. F. Dugan. Performance lies my professor told me: the case for teaching software performance engineering to undergraduates. In WOSP, 2004.
13. D. Engler, D. Y. Chen, S. Hallem, A. Chou, and B. Chelf. Bugs as deviant behavior: A general approach to inferring errors in systems code. In SOSP, 2001.
14. M. Ernst, A. Czeisler, W. G. Griswold, and D. Notkin. Quickly detecting relevant program invariants. In ICSE, 2000.
15. Fortify. HP Fortify Static Code Analyzer (SCA). https://www.fortify.com/ products/hpfssc/source-code-analyzer.html.
16. M. Gabel, J. Yang, Y. Yu, M. Goldszmidt, and Z. Su. Scalable and systematic detection of buggy inconsistencies in source code. In OOPSLA, 2010.
17. P. Godefroid, N. Klarlund, and K. Sen. Dart: directed automated random testing. In PLDI, 2005.
18. S. Gulwani, K. K. Mehra, and T. Chilimbi. Speed: precise and efficient static estimation of program computational complexity. In POPL, 2009.
19. M. Hauswirth, A. Diwan, P. F. Sweeney, and M. C. Mozer. Automating vertical profiling. In OOPSLA, 2005.
20. J. K. Hollingsworth, R. B. Irvin, and B. P. Miller. The integration of application and system based metrics in a parallel program performance tool. In PPOPP, 1991.
21. D. Hovemeyer and W. Pugh. Finding bugs is easy. In OOPSLA, 2004.
22. InfoWorld. Top 10 open source hall of famers. http://www.infoworld.com/d/ open-source/top-10-open-sourcehall-famers-848.
23. R. B. Irvin and B. P. Miller. Mapping performance data for high-level and data views of parallel program performance. In ICS, 1996.
24. J Meter. Java desktop application designed for load tests. http://jakarta.apache.org/jmeter.
25. M. Jovic, A. Adamoli, and M. Hauswirth. Catch me if you can: performance bug detection in the wild. In OOPSLA, 2011.
26. C. Killian, K. Nagaraj, S. Pervez, R. Braud, J. W. Anderson, and R. Jhala. Finding latent performance bugs in systems implementations. In FSE, 2010.
27. C. Lattner and V. Adve. LLVM: A compilation framework for lifelong program analysis & transformation. In CGO, 2004.
28. Z. Li, S. Lu, S. Myagmar, and Y. Zhou. CP-Miner: A Tool for Finding Copy-paste and Related Bugs in Operating System Code. In OSDI, 2004.
29. Z. Li, L. Tan, X. Wang, Y. Zhou, and C. Zhai. Have things changed now?: an empirical study of bug characteristics in modern open source software. In ASID, 2006.
30. Z. Li and Y. Zhou. PR-Miner: Automatically extracting implicit programming rules and detecting violations in large software code. In FSE, Sept 2005.
31. S. Liu, K. Pattabiraman, T. Moscibroda, and B. G. Zorn. Flikker: saving DRAM refresh-power through critical data partitioning. In ASPLOS, 2011.
32. T. Liu and E. D. Berger. Precise detection and automatic mitigation of false sharing. In OOPSLA, 2011.
33. V. B. Livshits and T. Zimmermann. Dynamine: Finding common error patterns by mining software revision histories. In FSE, 2005.
34. S. Lu, S. Park, E. Seo, and Y. Zhou. Learning from mistakes - a comprehensive study of real world concurrency bug characteristics. In ASPLOS, 2008.
35. J. Mellor-Crummey, R. J. Fowler, G. Marin, and N. Tallent. Hpcview: A tool for top-down analysis of node performance. J. Supercomput., 23(1):81-104, 2002.
36. N. Meng, M. Kim, and K. S. McKinley. Systematic editing: generating program transformations from an example. In PLDI, 2011.
37. Microsoft. MSDN SAL annotations. http://msdn2.microsoft.com/enus/library/ ms235402.aspx.
38. B. P. Miller, M. D. Callaghan, J. M. Cargille, J. K. Hollingsworth, R. B. Irvin, K. L. Karavanic, K. Kunchithapadam, and T. Newhall. The paradyn parallel performance measurement tool. Computer, 28(11), 1995.
39. N. Mitchell. The diary of a datum: an approach to modeling runtime complexity in framework-based applications. In ECOOP, 2006.
40. I. Molyneaux. The Art of Application Performance Testing: Help for Programmers and Quality Assurance. O'Reilly Media, 2009.
41. G. E. Morris. Lessons from the colorado benefits management system disaster. www.ad-mkt-review.com/publichtml/air/ai200411.html, 2004.
42. G. Muller, Y. Padioleau, J. L. Lawall, and R. R. Hansen. Semantic patches considered helpful. Operating Systems Review, 40(3):90-92, 2006. (Pubitemid 44299052)
43. N. Palix, G. Thomas, S. Saha, C. Calvès, J. Lawall, and G. Muller. Faults in linux: ten years later. In ASPLOS, 2011.
44. S. E. Perl and W. E.Weihl. Performance assertion checking. In SOSP, 1993.
45. T. Richardson. 1901 census site still down after six months. http://www.theregister.co.uk/2002/07/03/1901censussitestilldown/.
46. C. J. Rossbach, O. S. Hofmann, and E. Witchel. Is transactional programming actually easier? In PPoPP, 2010.
47. A. Shankar, M. Arnold, and R. Bodik. Jolt: lightweight dynamic analysis and removal of object churn. In OOPSLA, 2008.
48. K. Shen, C. Stewart, C. Li, and X. Li. Reference-driven performance anomaly identification. In SIGMETRICS, 2009.
49. R. L. Sites. Identifying dark latency.
50. C. U. Smith and L. G. Williams. Software performance antipatterns. In Proceedings of the 2nd international workshop on Software and performance, 2000.
51. Stefan Bodewig. Bug 45396: There is no hint in the javadocs. https://issues.apache.org/bugzilla/showbug.cgi?id=45396#c4.
52. C. Stewart, K. Shen, A. Iyengar, and J. Yin. Entomomodel: Understanding and avoiding performance anomaly manifestations. In MASCOTS, 2010.
53. M. Sullivan and R. Chillarege. A comparison of software defects in database management systems and operating systems. In FTCS, 1992.
54. L. Torvalds. Sparse - a semantic parser for c. http://www.kernel.org/pub/ software/devel/sparse/.
55. J. S. Vetter and P. H. Worley. Asserting performance expectations. In Supercomputing, 2002.
56. wikipedia. Chi-squared test. http://en.wikipedia.org/wiki/Chisquaredtest.
57. G. Xu, M. Arnold, N. Mitchell, A. Rountev, and G. Sevitsky. Go with the flow: profiling copies to find runtime bloat. In PLDI'09, 2009.
58. G. Xu, N. Mitchell, M. Arnold, A. Rountev, E. Schonberg, and G. Sevitsky. Finding low-utility data structures. In PLDI, 2010.
59. W. Xu, L. Huang, A. Fox, D. Patterson, and M. I. Jordan. Detecting large-scale system problems by mining console logs. In SOSP, 2009.
60. A. Zeller. Isolating cause-effect chains from computer programs. In FSE, 2002.