Cutting corners: Workbench automation for server benchmarking

Proceedings of the 2008 USENIX Annual Technical Conference, USENIX 2008

Volumn , Issue , 2008, Pages 241-254

  Shivam, Piyush ^a   Marupadi, Varun ^b   Chase, Jeff ^b   Subramaniam, Thileepan ^b   Babu, Shivnath ^b  

^a SUN MICROSYSTEMS   (United States)

^b DUKE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATION;

AUTOMATION POLICY; BENCHMARKING ACTIVITY; COST TRADE-OFF; CUTTING CORNERS; FILE SERVERS; HARDWARE RESOURCES; STORAGE SERVERS; WORKLOAD GENERATORS;

BENCHMARKING;

EID: 85067359901     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (27)

1. D. C. Anderson and J. S. Chase. Fstress: A flexible network file service benchmark. Technical Report CS-2002-01, Duke University, Department of Computer Science, January 2002.
2. M. Arlitt and C. Williamson. Web server workload characterization: The search for invariants. In Proceedings of the ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, April 1996.
3. T. Bray. Bonnie file system benchmark, 1996. http://www.textuality.com/bonnie.
4. A. B. Brown, A. Chanda, R. Farrow, A. Fedorova, P. Maniatis, and M. L. Scott. The many faces of systems research: And how to evaluate them. In Proceedings of the 10th conference on Hot Topics in Operating Systems, June 2005.
5. P. Chen and D. Patterson. A new approach to I/O performance evaluation—self-scaling I/O benchmarks, predicted I/O performence. In Proceedings of the ACM SIGMETRICS Conference on Measurement and Modeling of Computer Systems, May 1993.
6. S. P. E. Corporation. SPEC SFS release 3.0 run and report rules, 2001.
7. T. P. P. Council. TPC benchmark C standard specification, August 1992. Edited by François Raab.
8. M. Crovella, M. Taqqu, and A. Bestavros. In A Practical Guide To Heavy Tails, chapter 1 (Heavy-Tailed Probability Distributions in the World Wide Web). Chapman & Hall, 1998.
9. R. Doyle, J. Chase, S. Gadde, and A. Vahdat. The trickle-down effect: Web caching and server request distribution. In Proceedings of the Sixth International Workshop on Web Caching and Content Delivery, June 2001.
10. D. Ellard and M. Seltzer. NFS tricks and benchmarking traps. In Proceedings of the FREENIX 2003 Technical Conference, June 2003.
11. S. Gold. Defects in SFS 2.0 which affect the working-set, July 2001. http://www.spec.org/osg/sfs97/sfs97_defects.html.
12. D. Irwin, J. S. Chase, L. Grit, A. Yumerefendi, D. Becker, and K. G. Yocum. Sharing Networked Resources with Brokered Leases. In Proc. of the USENIX Annual Technical Conf., Jun 2006.
13. R. Jain. The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling. John Wiley & Sons, May 1991.
14. J. Katcher. Postmark: A new file system benchmark. Technical Report 3022, Network Appliance, October 1997.
15. B. Keith and M. Wittle. LADDIS: The next generation in NFS file server benchmarking. In Proceedings of the USENIX Annual Technical Conference, June 1993.
16. J. C. Mogul. Brittle metrics in operating systems research. In Proceedings of the the 7th Workshop on Hot Topics in Operating Systems, March 1999.
17. R. H. Myers and D. C. Montgomery. Response Surface Methodology: Process and Product in Optimization Using Designed Experiments. John Wiley & Sons, Inc., New York, NY, USA, 1995.
18. National laboratory for applied network research (NLANR). http://moat.nlanr.net.
19. C. Roadknight, I. Marshall, and D. Vearer. File popularity characterisation. In Proceedings of the 2nd Workshop on Internet Server Performance, May 1999.
20. Y. Saito, B. Bershad, and H. Levy. Manageability, availability and performance in Porcupine: A highly scalable, cluster-based mail service. In Proceedings of the 17th ACM Symposium on Operating System Principles, December 1999.
21. B. Schroeder, A. Wierman, and M. Harchol-Balter. Open versus closed: A cautionary tale. In Proeceedings of the 3rd Symposium on Networked Systems Design and Implementation, April 2006.
22. C. Small, N. Ghosh, H. Saleed, M. Seltzer, and K. Smith. Does systems research measure up. Technical Report TR-16-97, Harvard University, Department of Computer Science, November 1997.
23. K. A. Smith. Workload-Specific File System Benchmarks. PhD thesis, Harvard University, Cambridge, MA, January 2001.
24. D. Tang and M. Seltzer. Lies, Damned Lies, and File System Benchmarks. In VINO: The 1994 Fall Harvest. Harvard Division of Applied Sciences Technical Report TR-34-94, December 1994.
25. A. Traeger, N. Joukov, C. P. Wright, and E. Zadok. A nine year study of file system and storage benchmarking. Technical Report FSL-07-01, Computer Science Department, Stony Brook University, May 2007.
26. C. P. Wright, N. Joukov, D. Kulkarni, Y. Miretskiy, and E. Zadok. Auto-pilot: A platform for system software benchmarking. In Proceedings of the USENIX Annual Technical Conference, FREENIX Track, April 2005.
27. A. Yumerefendi, P. Shivam, D. Irwin, P. Gunda, L. Grit, A. Demberel, J. Chase, and S. Babu. Towards an autonomic computing testbed. In Proceedings of the Workshop on Hot Topics in Autonomic Computing, June 2007.