Using load tests to automatically compare the subsystems of a large enterprise system

Proceedings - International Computer Software and Applications Conference

Volumn , Issue , 2010, Pages 117-126

  Malik, Haroon ^a   Adams, Bram ^a   Hassan, Ahmed E ^a   Flora, Parminder ^b   Hamann, Gilbert ^b  

^a QUEEN S UNIVERSITY   (Canada)

^b RESEARCH IN MOTION   (Canada)

Author keywords

Load test; PCA; Performance; Signatures

Indexed keywords

APPLICATION PROGRAMS; ENTERPRISE RESOURCE PLANNING; LOAD TESTING;

AUTOMATED APPROACH; INDUSTRIAL SYSTEMS; OPERATIONAL KNOWLEDGE; PERFORMANCE; PERFORMANCE COUNTERS; PERIODIC MAINTENANCE; SIGNATURES; STATISTICAL TECHNIQUES;

SOFTWARE TESTING;

EID: 78751683297     PISSN: 07303157     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/COMPSAC.2010.18     Document Type: Conference Paper

References (33)

1. Beizer. B., "Software System Testing and Quality Assurance"' Van Nostrand Reinhold, March 1984.
2. Avritzer. A., Larson. B., "Load testing software using deterministic state testing", In Proceedings of the ACM SIGSOFT international symposium on Software, 1993.
3. Jiang, Z. M., Hassan, A. E., Hamann, G., Flora, P., Automated Performance Analysis of Load Tests. In Proceedings of the 25th IEEE International Conference on Software Maintenance (ICSM) 2009, Edmonton, Canada, September 20-26, 2009.
4. Reynolds, P., Killian, C., Wiener, J. L., Mogul, J. C., Shah, M. A., and Vahdat, A. 2006. Pip: detecting the unexpected in distributed systems. In Proceedings of the 3rd Conference on Networked Systems Design & Implementation - Volume 3 (San Jose, CA, May 08-10, 2006). USENIX Association, Berkeley, CA, 9-9
5. M, Jiang., Munawar, M.A., Reidemeister, T.; Ward, P.A.S., "Automatic fault detection and diagnosis in complex software systems by information-theoretic monitoring," Dependable Systems & Networks, 2009. DSN '09. IEEE/IFIP International Conference on , vol., no., pp.285-294, June 29 2009-July 2 2009
6. Z. Guo, G. Jiang, H. Chen, and K. Yoshihira. Tracking probabilistic correlation of monitoring data for fault detection in complex systems. In DSN, pages 259-268, 2006.
7. G. Jiang, H. Chen, and K. Yoshihira. Modeling and tracking of transaction flow dynamics for fault detection in complex systems. IEEE Trans. on Dependable and Secure Computing, 3(4):312-326, 2006. (Pubitemid 44837034)
8. M. A. Munawar and P. A. Ward. Adaptive monitoring in enterprise software systems. In SysML, June 2006.
9. Jiang, M., Munawar, M. A., Reidemeister, T., and Ward, P. A. 2008. Information-theoretic modeling for tracking the health of complex software systems. In Proceedings of the 2008 Conference of the Center For Advanced Studies on Collaborative Research: Meeting of Minds (Ontario, Canada, October 27-30, 2008). CASCON '08. ACM, New York, NY, 236-247
10. Thakkar, D., Hassan, A. E., Hamann, G., and Flora, P. 2008. A framework for measurement based performance modeling. In Proceedings of the 7th international Workshop on Software and Performance (Princeton, NJ, USA, June 23-26, 2008). WOSP '08. ACM, New York, NY, 55-66.
11. Malik, H.; Chowdhury, I.; Hsiao-Ming Tsou; Zhen Ming Jiang; Hassan, A.E., "Understanding the rationale for updating a function's comment," Software Maintenance, 2008. ICSM 2008. IEEE International Conference on , vol., no., pp.167-176, Sept. 28 2008-Oct. 4 2008
12. Jolliffe IT., "Principal Component Analysis", Second Edition. New York, Springer-Verlag; (Springer Series in Statistics), 2002.
13. Ringberg, H., Soule, A., Rexford, J., Diot, C., "Sensitivity of PCA for traffic anomaly detection", In ACM SIGMETRICS,San Diego, CA, USA, 2007.
14. Malik, H., Jiang, Z. M., Bram, A, Hassan, A. E., "Automatic Comparison of Load Tests to Support the Performance Analysis of Large Enterpirse Systems", The European Conference on Software Maintenance and Reengineering, CSMR 2010, Mar 15-18, In press.
15. Sandeep, S. Ratna; Swapna, M.; Niranjan, Thirumale; Susarla, Sai & Nandi, Siddhartha: CLUEBOX: A Performance Log Analyzer for Automated Troubleshooting. USENIX Association (2008) .
16. Xie, T., Marinov, D., and Notkin, D. 2004. Rostra: A Framework for Detecting Redundant Object-Oriented Unit Tests. In Proceedings of the 19th IEEE international Conference on Automated Software Engineering (September 20-24, 2004). Automated Software Engineering. IEEE Computer Society, Washington, DC, 196-205. (Pubitemid 40423544)
17. Rothermel, G. and Harrold, M. J. 1997. A safe, efficient regression test selection technique. ACM Trans. Softw. Eng. Methodol. 6, 2 (Apr. 1997), 173-210. (Pubitemid 127684922)
18. Jain R. 1992. The art of computer systems performance analysis. John Wiley.
19. Jolliffe IT., "Principal Component Analysis", Second Edition. New York, Springer-Verlag; (Springer Series in Statistics), 2002.
20. M.W. Knop, J.M. Schopf and P.A. Dinda, "Windows performance monitoring and data reduction using watch tower", Workshop on Self-Healing, Adaptive and self-MANaged Systems (SHAMAN), 2002.
21. I. Cohen, M. Goldszmidt, T. Kelly, J. Symons, and J. S. Chase. Correlating instrumentation data to system states: A building block for automated diagnosis and control. In Proc. 6th USENIX OSDI,San Francisco, CA, Dec. 2004.
22. Cohen, I., Zhang, S., Goldszmidt, M., Symons, J., Kelly, T., and Fox, A. 2005. Capturing, indexing, clustering, and retrieving system history. In Proceedings of the Twentieth ACM Symposium on Operating Systems Principles (Brighton, United Kingdom, October 23-26, 2005). SOSP '05. ACM, New York, NY, 105-118 (Pubitemid 44892208)
23. Rosner B., Fundamentals of Biostatistics, 4th Edition, Duxbury Press, Belmont, California, USA, 1995.
24. Huck, K.A. and Malony, A.D., "PerfExplorer: A Performance Data Mining Framework For Large-Scale Parallel Computing," Supercomputing, 2005. Proceedings of the ACM/IEEE SC 2005 Conference, vol., no., pp. 41-41, 2005.
25. A. Brown. and D. Patterson., An Active Approach to Characterizing Dynamic Dependencies for Problem Determination in a Distributed Environment. In Seventh IFIP/IEEE International Symposium on Integrated Network Management, Seattle, WA, May 2001.
26. J. Choi, M. Choi, and S. Lee. An Alarm Correlation and Fault Identification Scheme Based on OSI Managed Object Classes. In IEEE International Conference on Communications, Vancouver, BC, Canada, 1999.
27. B. Gruschke. A New Approach for Event Correlation basedon Dependency Graphs. In 5th Workshop of the OpenView University Association: OVUA'98, Rennes, France, April 1998.
28. A. Yemini and S. Kliger. High Speed and Robust Event Correlation. IEEE Communication Magazine, 34(5):82-90, May 1996.
29. P. Barham, A. Donnelly, R. Isaacs, and R. Mortier. Using Magpie for request extraction and workload modelling. In OSDI, 2004.
30. Chen, M.Y.; Kiciman, E.; Fratkin, E.; Fox, A.; Brewer, E., "Pinpoint: problem determination in large, dynamic Internet services," Dependable Systems and Networks, 2002. DSN 2002. Proceedings. International Conference on , vol., no., pp. 595-604, 2002
31. Stress Tool: http://www.oldskool.org/pc/throttle, Downloaded: December 2009.
32. Malik, H. Hassan, A.E., "Supporting software evolution using adaptive change propagation heuristics," Software Maintenance, 2008. ICSM 2008. IEEE International Conference on , vol., no., pp.177-186, Sept. 28 2008-Oct. 4 2008
33. Hassan, A. E. and Holt, R. C. 2004. Predicting Change Propagation in Software Systems. In Proceedings of the 20th IEEE international Conference on Software Maintenance (September 11-14, 2004). ICSM. IEEE Computer Society, Washington, DC, 284-293.