Automated verification of load tests using control charts

Proceedings - Asia-Pacific Software Engineering Conference, APSEC

Volumn , Issue , 2011, Pages 282-289

  Nguyen, Thanh H D ^a   Adams, Bram ^a   Jiang, Zhen Ming ^a   Hassan, Ahmed E ^a   Nasser, Mohamed ^b   Flora, Parminder ^b  

^a QUEEN S UNIVERSITY   (Canada)

^b RESEARCH IN MOTION   (Canada)

Author keywords

Load Testing; Mining software repository; Performance Testing

Indexed keywords

AUTOMATED VERIFICATION; CONTROL CHARTS; FURTHER DEVELOPMENT; LOAD TEST; MINING SOFTWARE REPOSITORY; OPEN SOURCE SOFTWARE SYSTEMS; PERFORMANCE COUNTERS; PERFORMANCE PROBLEMS; PERFORMANCE TESTING; PERFORMANCE VERIFICATION; SOFTWARE COMPANY; SOFTWARE DEVELOPMENT PROCESS; STATISTICAL QUALITY CONTROL; SYSTEM UNDER TEST; TACIT KNOWLEDGE; TELECOMMUNICATION SOFTWARE; TESTING PROCEDURE;

FLOWCHARTING; LOAD TESTING; SOFTWARE ENGINEERING; TESTING;

SOFTWARE TESTING;

EID: 84856615181     PISSN: 15301362     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/APSEC.2011.59     Document Type: Conference Paper

References (24)

1. J. Palmer, "Designing for web site usability," Computer, vol. 35, no. 7, pp. 102-103, 2002. (Pubitemid 34740992)
2. E. J. Weyuker and F. I. Vokolos, "Experience with performance testing of software systems: issues, an approach, and case study," IEEE Transactions on Software Engineering (TSE), vol. 26, no. 12, pp. 1147-1156, 2000. (Pubitemid 32130689)
3. K. C. Foo, J. Zhen Ming, B. Adams, A. E. Hassan, Z. Ying, and P. Flora, "Mining performance regression testing repositories for automated performance analysis," in International Conference on Quality Software (QSIC), 2010, pp. 32-41.
4. D. Montgomery, Introduction to statistical quality control. John Wiley, 2005.
5. Dell Inc., "Dvd store test application," 2010, ver. 2.1.
6. B. Beizer, Software system testing and quality assurance. Van Nostrand Reinhold, 1984.
7. A. Avritzer and E. R. Weyuker, "The automatic generation of load test suites and the assessment of the resulting software," IEEE Transactions on Software Engineering (TSE), vol. 21, no. 9, pp. 705-716, 1995.
8. Mozilla Project, "Talos," 2010.
9. Hewlett Packard, "Loadrunner," 2010.
10. Microsoft Corp., "Visual studio," 2011.
11. -, "Windows reliability and performance monitor," 2011.
12. Linux Kernel Organization Inc., "Linux kernel," 2011.
13. W. Shewhart, Economic control of quality of manufactured product. American Society for Quality Control, 1931.
14. The Apache Software Foundation, "Tomcat," 2010, ver. 5.5.
15. MySQL AB, "Mysql community server," 2011, ver. 5.5.
16. K. C. Foo, "Automated discovery of performance regressions in enterprise applications," Master's thesis, 2011.
17. H. Malik, "A methodology to support load test analysis," in International Conference on Software Engineering (ICSE). Cape Town, South Africa: ACM, 2010, pp. 421-424.
18. Z. M. Jiang, A. E. Hassan, G. Hamann, and P. Flora, "Automatic identification of load testing problems," in International Conference on Software Maintenance (ICSM), 2008, pp. 307-316.
19. -, "Automatic performance analysis of load tests," in International Conference in Software Maintenance (ICSM), Edmonton, 2009, pp. 125-134.
20. I. Cohen, M. Goldszmidt, T. Kelly, J. Symons, and J. S. Chase, "Correlating instrumentation data to system states: a building block for automated diagnosis and control," in Symposium on Opearting Systems Design Implementation. San Francisco, CA: USENIX Association, 2004, pp. 231-244.
21. M. Jiang, M. A. Munawar, T. Reidemeister, and P. A. S. Ward, "Automatic fault detection and diagnosis in complex software systems by information-theoretic monitoring," in International Conference on Dependable Systems Networks (DSN), 2009, pp. 285-294.
22. M. Y. Chen, E. Kiciman, E. Fratkin, A. Fox, and E. Brewer, "Pinpoint: problem determination in large, dynamic internet services," in International Conference on Dependable Systems and Networks (DSN), 2002, pp. 595-604.
23. L. Cherkasova, K. Ozonat, M. Ningfang, J. Symons, and E. Smirni, "Anomaly? application change? or workload change? towards automated detection of application performance anomaly and change," in International Conference on Dependable Systems and Networks (DSN), 2008, pp. 452-461.
24. I. Trubin, "Capturing workload pathology by statistical exception detection system," in Computer Measurement Group (CMG), 2005.