Web workload generation challenges - An empirical investigation

Software - Practice and Experience

Volumn 42, Issue 5, 2012, Pages 629-647

  Hashemian, Raoufehsadat ^a   Krishnamurthy, Diwakar ^a   Arlitt, Martin ^a,b  

^a UNIVERSITY OF CALGARY   (Canada)

^b HEWLETT PACKARD LABORATORIES   (United States)

Author keywords

benchmarking tools; performance testing; workload generator

Indexed keywords

BENCHMARKING TOOLS; EMPIRICAL INVESTIGATION; END USERS; JAVA NETWORK; JAVA VIRTUAL MACHINES; PERFORMANCE TESTING; ROOT CAUSE; WEB-BASED SYSTEM; WORKLOAD GENERATION; WORKLOAD GENERATOR;

SOFTWARE ENGINEERING;

BENCHMARKING;

EID: 84859442632     PISSN: 00380644     EISSN: 1097024X     Source Type: Journal    
DOI: 10.1002/spe.1093     Document Type: Article

References (53)

1. RUBiS-Homepage. [01 October 2010].
2. TPC-W- Homepage. [01 October 2010].
3. SPEC-Benchmarks. [01 October 2010].
4. PHP: Hypertext Processor. [01 October 2010].
5. Enterprise Java Bean Technology. [01 October 2010].
6. Mosberger D, Jin T,. httperf: A tool for measuring web server performance. ACM SIGMETRICS Performance Evaluation Review 1998; 26 (3): 31-37. (Pubitemid 128605767)
7. Banga G, Druschel P,. Measuring the capacity of a web server under realistic loads. World Wide Web 1999; 2 (1): 69-83.
8. Apache JMeter. [01 October 2010].
9. Feitelson D,. The forgotten factor: facts on performance evaluation and its dependence on workloads. Proceeding of Int. Euro-Par Conference 2002; 2400: 49-60.
10. Paxson V, Floyd S,. Wide area traffic: the failure of Poisson modeling. IEEE/ACM Transactions on Networking 1995; 3 (3): 226-244.
11. Ferrari D,. On the foundations of artificial workload design. ACM SIGMETRICS Performance Evaluation Review 1984; 12 (3): 8-14.
12. Barford P, Crovella M,. The surge traffic generator: generating representative web workloads for network and server performance evaluation. Proceedings of the ACM SIGMETRICS 1998: 151-160. (Pubitemid 128593231)
13. Schroeder B, Wierman A, Harchol-Balter M,. Open versus closed: a cautionary tale. Proceedings of the 3rd Conference on Networked Systems Design & Implementation 2006; 3: 18-18.
14. Kant K, Tewari V, Iyer R,. GEIST: Generator of ecommerce and internet server traffic. Proceedings of Int. Symposium on Performance Analysis of Systems and Software 2001: 49-56.
15. Liu Z, Niclausse N, Jalpa-Villanueva C,. Traffic model and performance evaluation of web servers. Performance Evaluation 2001; 46 (2-3): 77-100. (Pubitemid 32768291)
16. Krishnamurthy D, Rolia J, Majumdar S,. A Synthetic Workload Generation Technique for Stress Testing Session-Based Systems. Proceedings of the IEEE Transactions on Software Engineering 2006; 32 (11): 868-882. (Pubitemid 46437707)
17. Guitart J, Carrera D, Torres J, Ayguadé E, Labarta J,. Tuning dynamic web applications using fine-grain analysis. Proceedings of the 13th Euromicro Conference on Parallel, Distributed and Network-Based Processing 2005: 84-91. (Pubitemid 43727659)
18. Padala P, Zhu X, Wang Z, Singhal S, Shin K,. Performance evaluation of virtualization technologies for server consolidation. Technical Report HPL-2007-59 2007.
19. Malkowski S, Hedwig M, Pu C,. Experimental evaluation of N-tier systems: observation and analysis of multi-bottlenecks. Proceedings of the IEEE International Symposium on Workload Characterization 2009: 118-127.
20. Sicard S, Boyer F, De Palma N,. Using components for architecture-based management: the self-repair case. Proceedings of the 30th International Conference on Software Engineering 2008: 101-110.
21. Guitart J, Carrera D, Beltran V, Torres J, Ayguadé E,. Dynamic CPU provisioning for self-managed secure web applications in SMP hosting platforms. Computer Networks 2008; 52 (7): 1390-1409.
22. Wood T, Cherkasova L, Ozonat K, Shenoy P,. Profiling and modeling resource usage of virtualized applications. Proceedings of the 9th ACM/IFIP/USENIX International Conference on Middleware 2008: 366-387, DOI: 10.1007/978-3-540-89856-6-19.
23. Rolia J, Krishnamurthy D, Casale G, Dawson S,. BAP: a benchmark-driven algebraic method for the performance engineering of customized services. Proceedings of the First Joint WOSP/SIPEW International Conference on Performance Engineering 2010: 3-14.
24. Kusic D, Kephart JO, Kandasamy N, Jiang G,. Power and performance management of virtualized computing environments via lookahead control. Cluster Computing 2009; 12 (1): 1-15.
25. Ramamurthy P, Sekar V, Akella A, Krishnamurthy B, Shaikh A,. Remote profiling of resource constraints of web servers using mini-flash crowds. USENIX Annual Technical Conference on Annual Technical Conference 2008: 185-198, DOI: 10.1.1.145.4897.
26. Nahum E,. Deconstructing SPECweb99. 7th International Workshop on Web Content Caching and Distribution (WCW) 2002.
27. Nahum E, Rosu M, Seshan S, Almeida J,. The effects of wide-area conditions on WWW server performance. Proceedings of the ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems 2001: 257-267. (Pubitemid 32865941)
28. Lighttpd fly light. [01 October 2010].
29. The Apache HTTP Server Project. [01 October 2010].
30. Guitart J, Carrera D, Torres J, Ayguadé E, Labarta J,. Successful experiences tuning dynamic web applications using fine-grain analysis. Research report number: UPC-DAC-2004-3 / UPC-CEPBA-2004-2, 2004.
31. Brecht T,. Linux: increasing the number of open file descriptors. Online tutorial available at: [01 October 2010].
32. TCPDUMP/LIBPCAP public repository. [01 October 2010].
33. Paxson V,. Bro: a system for detecting network intruders in real-time. Computer Networks 1999; 31 (23-24): 2435-2463. DOI: 10.1016/S1389-1286(99)00112- 7.
34. Arlitt M, Krishnamurthy B, Mogul JC,. Predicting short-transfer latency from TCP arcana: a trace-based validation. Proceedings of the 5th ACM SIGCOMM Conference on Internet Measurement 2005: 19-19.
35. Hypertext Transfer Protocol-HTTP/1.1. [01 October 2010].
36. SYSSTAT. [01 October 2010].
37. Java Documentation Link. [01 October 2010].
38. Java Documentation Link. [01 October 2010].
39. httperf Manual. [01 October 2010].
40. Jain R,. The Art of Computer Systems Performance Analysis. John Wiley & Sons: New York, 1991.
41. Tong L, Lau FCM,. Exploiting memory usage patterns to improve garbage collections in Java. Proceedings of the 8th International Conference on the Principles and Practice of Programming in Java (PPPJ '10). ACM, New York, NY, USA, 2010; 39-48, DOI:.
42. Georges A, Buytaert D, Eeckhout L,. Statistically rigorous Java performance evaluation. Proceedings of the 22nd Annual ACM SIGPLAN Conference on Object-Oriented Programming Systems and Applications. ACM, New York, NY, USA, 2007; 57-76, DOI:.
43. Brecht T, Arjomandi E, Li C, Pham H,. Controlling garbage collection and heap growth to reduce the execution time of Java applications. ACM Transactions on Programming Languages and Systems 2006; 28 (5): 908-941. (Pubitemid 44371445)
44. Blackburn SM, Cheng P, McKinley KS,. Myths and realities: the performance impact of garbage collection. Proceedings of the Joint International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS '04/Performance '04). ACM, New York, NY, USA, 2004; 25-36.
45. GCJ: The GNU Compiler for Java-GNU Project-Free Software Foundation (FSF). [01 October 2010].
46. Oracle (Sun) Java HotSpot Technology. [01 October 2010].
47. Amza C, Chanda A, Cox A, Elnikety S, Gil R, Rajamani K, Cecchet E, Marguerite J,. Specification and implementation of dynamic Web site benchmarks. Proceedings of WWC-5: IEEE 5th Annual Workshop on Workload Characterization 2002: 3-13.
48. Java Documentation Link. [01 October 2010].
49. Java Documentation Link. [01 October 2010].
50. Prefork-Apache HTTP Server. [01 October 2010].
51. Lighttpd-Server.max-workerDetails-lighty labs. [01 October 2010].
52. Souders S,. Roundup on Parallel Connections. Available at: [01 October 2010].
53. Cormode G, Krishnamurthy B,. Key differences between Web1.0 and Web2.0. First Monday 2008; 13 (6).