Automated detection of performance regressions using statistical process control techniques

ICPE'12 - Proceedings of the 3rd Joint WOSP/SIPEW International Conference on Performance Engineering

Volumn , Issue , 2012, Pages 299-310

  Nguyen, Thanh H D ^a   Adams, Bram ^a   Jiang, Zhen Ming ^a   Hassan, Ahmed E ^a   Nasser, Mohamed ^b   Flora, Parminder ^b  

^a QUEEN S UNIVERSITY   (Canada)

^b RESEARCH IN MOTION   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATED DETECTION; CONTROL CHARTS; HISTORICAL DATA; OPEN-SOURCE SOFTWARES; PERFORMANCE COUNTERS; REGRESSION TESTING; SOFTWARE DEVELOPMENT PROCESS; SOFTWARE SYSTEMS; SOFTWARE TEAMS; TEST RUNS;

SOFTWARE ENGINEERING; SOFTWARE TESTING; STATISTICAL PROCESS CONTROL;

REGRESSION ANALYSIS;

EID: 84861086105     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2188286.2188344     Document Type: Conference Paper

References (19)

1. A. Avritzer and E. R. Weyuker. The automatic generation of load test suites and the assessment of the resulting software. IEEE Transactions on Software Engineering (TSE), 21(9):705-716, 1995.
2. M. Y. Chen, E. Kiciman, E. Fratkin, A. Fox, and E. Brewer. Pinpoint: Problem determination in large, dynamic internet services. In International Conference on Dependable Systems and Networks (DSN), pages 595-604, 2002.
3. L. Cherkasova, K. Ozonat, M. Ningfang, J. Symons, and E. Smirni. Anomaly? application change? or workload change? towards automated detection of application performance anomaly and change. In International Conference on Dependable Systems and Networks (DSN), pages 452-461, 2008.
4. I. Cohen, M. Goldszmidt, T. Kelly, J. Symons, and J. S. Chase. Correlating instrumentation data to system states: a building block for automated diagnosis and control. In Symposium on Opearting Systems Design Implementation, pages 231-244, San Francisco, CA, 2004. USENIX Association.
5. Dell Inc. DVD Store Test Application, 2010. Ver. 2.1.
6. K. C. Foo. Automated discovery of performance regressions in enterprise applications. Master's thesis, 2011.
7. K. C. Foo, J. Zhen Ming, B. Adams, A. E. Hassan, Z. Ying, and P. Flora. Mining performance regression testing repositories for automated performance analysis. In International Conference on Quality Software (QSIC), pages 32-41, 2010.
8. H. W. Gunther. Websphere application server development best practices for performance and scalability. IBM WebSphere Application Server Standard and Advanced Editions - White paper, 2000.
9. Hewlett Packard. Loadrunner, 2010.
10. M. Jiang, M. A. Munawar, T. Reidemeister, and P. A. S. Ward. Automatic fault detection and diagnosis in complex software systems by information-theoretic monitoring. In International Conference on Dependable Systems Networks (DSN), pages 285-294, 2009.
11. Z. M. Jiang, A. E. Hassan, G. Hamann, and P. Flora. Automatic identi-cation of load testing problems. In International Conference on Software Maintenance (ICSM), pages 307-316, 2008.
12. Z. M. Jiang, A. E. Hassan, G. Hamann, and P. Flora. Automatic performance analysis of load tests. In International Conference in Software Maintenance (ICSM), pages 125-134, Edmonton, 2009.
13. H. Malik. A methodology to support load test analysis. In International Conference on Software Engineering (ICSE), pages 421-424, Cape Town, South Africa, 2010. ACM.
14. Microsoft Corp. Windows reliability and performance monitor, 2011.
15. MySQL AB. Mysql community server, 2011. Ver. 5.5.
16. W. Shewhart. Economic Control of Quality of Manufactured Product. American Society for Quality Control, 1931.
17. The Apache Software Foundation. Tomcat, 2010. Ver. 5.5.
18. I. Trubin. Capturing workload pathology by statistical exception detection system. In Computer Measurement Group (CMG), 2005.
19. E. J. Weyuker and F. I. Vokolos. Experience with performance testing of software systems: Issues, an approach, and case study. IEEE Transactions on Software Engineering (TSE), 26(12):1147-1156, 2000. (Pubitemid 32130689)