Distilling the essence of proprietary workloads into miniature benchmarks

Transactions on Architecture and Code Optimization

Volumn 5, Issue 2, 2008, Pages

  Joshi, Ajay ^a   Eeckhout, Lieven ^b   Bell Jr , Robert H ^c   John, Lizy K ^a  

^a UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^b GHENT UNIVERSITY   (Belgium)

^c IBM   (United States)

Author keywords

Benchmark cloning; Benchmarks; Workload characterization

Indexed keywords

BENCHMARK CLONING; BENCHMARKS; SYNTHETIC BENCHMARK; WORKLOAD CHARACTERIZATION;

CLONING; COMPUTER ARCHITECTURE; COMPUTER NETWORKS; COMPUTER SCIENCE; COMPUTER SYSTEMS; PRODUCT DEVELOPMENT; PROJECT MANAGEMENT; STANDARDIZATION;

BENCHMARKING;

EID: 51149119096     PISSN: 15443566     EISSN: 15443973     Source Type: Journal    
DOI: 10.1145/1400112.1400115     Document Type: Article

References (44)

1. BELL, R. AND JOHN, L. 2005. Improved automatic test case synthesis for performance model validation. In Proceedings of the International Conference on Supercomputing. 111-120.
2. BELL, R. AND JOHN, L. 2006. Efficient power analysis using synthetic testcases. In Proceedings of the IEEE International Symposium on Workload Characterization. 110-118.
3. BROOKS, D., TIWARI, V., AND MARTONOSI, M. 2000. Wattch: A framework for architectural-level power analysis and optimizations. In Proceedings of the Annual International Symposium on Computer Architecture. 83-94.
4. BURGER, D. AND AUSTIN, T. 1997. The SimpleScalar toolset, version 2.0, University of Wisconsin-Madison, Computer Sciences Department Tech. Rep. #1342.
5. CHANDRA, D., GUO, F., KIM, S., AND SOLIHIN, Y. 2005. Predicting inter-thread cache contention on a chip multiprocessor architecture. In Proceedings of the International Symposium on High Performance Computer Architecture. 1-12.
6. CHEN, P. AND PATTERSON, D. 1993. Anew approach toI/O performance evaluation - Self-scaling I/O benchmarks. In Proceedings of the ACM SIGMETRICS Conference on Measuring and Modeling of Computer Systems. 340-351.
7. COLEMAN, C. 1998. Using Inline Assembly with Gcc. See www.cs.virginia.edu/∼clc5q/gcc-inline-asm.pdf.
8. COLLINS, J., WANG, J., CHRISTOPHER, H., TULLSEN, D., HUGHES, C., LEE, Y., LAVERY. D., AND SHEN, J. 2001. Speculative precomputation: Long-range prefetching of delinquent loads. In Proceedings of the Annual International Symposium on Computer Architecture. 14-25.
9. CONTE, T. AND HWU, W.-M. 1990. Benchmark characterization for experimental system evaluation. In Proceedings of the 1990 Hawaii International Conference on System Sciences (HICSS). Architecture Track, vol. I. 6-16.
10. CONTE, T., HIRSCH, M., AND MENEZES, K. 1996. Reducing state loss for effective trace sampling of superscalar processors. In Proceedings of the International Conference on Computer Design. 468-477.
11. CURNOW, H. AND WICHMAN, B. 1976. A synthetic benchmark. Comput. J. 19, 1, 43-49.
12. DUBEY, P., ADAMS, G., III, AND FLYNN, M. 1994. Instruction window trade-offs and characterization of program parallelism. IEEE Trans. Comput. 431-442.
13. EECKHOUT, L. AND DE BOSSCHERE, K. 2001. Hybrid analytical-statistical modeling for efficiently exploring architecture and workload design spaces. In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques. 25-34.
14. EECKHOUT, L., NUSSBAUM, S., SMITH, J., AND DE BOSSCHERE, K. 2003. Statistical simulation: Adding efficiency to the computer designer's toolbox. IEEE Micro. 23, 5, 26-38.
15. EECKHOUT, L., BELL, R., STOUGIE, B., DE BOSSCHERE, K., AND JOHN, L. 2004. Control flow modeling in statistical simulation for accurate and efficient processor design studies. In Proceedings of the Annual International Symposium on Computer Architecture. 350-361.
16. EYERMAN, S., EECKHOUT, L., AND DE BOSSCHERE, K. 2006. Efficient design space exploration of high performance embedded out-of-order processors. In Proceedings of the design, Automation and Test in Europe. 351-356.
17. FERRARI, D. 1984. On the foundations of artificial workload design. In Proceedings of ACM SIGMETRICS Conference on Modeling and Measurement of Computer Systems. 8-14.
18. GENBRUGGE, D. AND EECKHOUT, L. 2008. Memory data flow modeling in statistical simulation for the efficient eploration of microprocessor design spaces. IEEE Trans. Comput. 57, 1, 41-54.
19. HAUNGS, M., SALLEE, P., AND FARRENS, M. 2000. Branch transition rate: A new metric for improved branch classification analysis. In Proceedings of the International Symposium on High Performance Computer Architecture. 241-250.
20. HENNING, J. 2000. SPEC CPU2000: Measuring CPU performance in the new millennium. IEEE Comput. 33, 7, 28-35.
21. HSIEH, C. AND PEDRAM, M. 1998. Microprocessor power estimation using profile-driven program synthesis. IEEE Trans. Comput. Aided Design Integrated Circ. Sys. 17, 11, 1080-1089.
22. IYENGAR, V. AND TREVILLYAN, L. 1996. Evaluation and generation of reduced traces for benchmarks. Tech. Rep. RC20610. IBM Research Division. T. J. Watson Research Center.
23. IYENGAR, V., TREVILLYAN, L., AND BOSE, P. 1996. Representative traces for processor models with infinite cache. In Proceedings of the International Symposium on High Performance Computer Architecture. 62-73.
24. JOSHI, A., EECKHOUT, L., BELL, R., AND JOHN, L. 2006a. Performance cloning: A technique for disseminating proprietary applications as benchmarks. In Proceedings of the IEEE International Symposium on Workload Characterization. 105-115.
25. JOSHI, A., YI, J., BELL, R., EECKHOUT, L., JOHN, L., AND LILJA, D. 2006b. Evaluating the efficacy of statistical simulation for design space exploration. In Proceedings of the IEEE International Symposium on Performance Analysis of Systems and Software. 70-79.
26. LAFAGE, T. AND SEZNEC, A. 2001. Choosing representative slices of program execution for microarchitecture simulations: A preliminary approach to the data stream. Kluwer International Series in Engineering and Computer Science Series, Workload Characterization of Emerging Computer Applications. 145-163.
27. LUK, C., COHN, R., MUTH, R., PATIL, H., KLAUSER, A., LOWNEY, G., WALLACE, S., REDDI, V., AND HAZELWOOD, K. 2005. PIN: Building customized program analysis tools with dynamic instrumentation. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation. 190-200.
28. NOONBURG, D. AND SHEN, J. 1994. Theoretical modeling of superscalar processor performance. In Proceedings of the International Symposium on Microarchitecture. 52-62.
29. NUSSBAUM, S. AND SMITH, J. 2001. Modeling superscalar processors via statistical simulation. In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques. 15-24.
30. OSKIN, M., CHONG, F., AND FARRENS, M. 2000. HLS: Combining statistical and symbolic simulation to guide microprocessor design. In Proceedings of the Annual International Symposium on Computer Architecture. 71-82.
31. RINGENBERG, J., PELOSKI, C., OEHMKE, D., AND MUDGE, T. 2005. Intrinsic checkpointing: A methodology for decreasing simulation time through binary modification. In Proceedings of the International Symposium on Performance Analysis of System and Software. 78-88.
32. SAVEEDRA, R. AND SMITH, A. 1996. Analysis of benchmark characteristics and benchmark performance prediction. ACM Trans. Comput. Syst. 14, 4, 344-384.
33. SHERWOOD, T., PERELMAN, E., HAMERLY, G., AND CALDER, B. 2002. Automatically characterizing program behavior. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems. 1-13.
34. SimPoint Website. http://www-cse.ucsd.edu/∼calder/simpoint/
35. SKADRON, K., MARTONOSI, M., AUGUST, D., HILL, M., LILJA, D., AND PAI, V. 2003. Challenges in computer architecture evaluation. IEEE Comput. 36, 8, 30-36.
36. SORENSON, E. AND FLANAGAN, J. 2002. Evaluating synthetic trace models using locality surfaces. In Proceedings of the IEEE International Workshop on Workload Characterization. 23-33.
37. SREENIVASAN, K. AND KLEINMAN, A. 1974. On the construction of a representative synthetic workload. Commun. ACM. 127-133.
38. SRIVASTAVA, A. AND EUSTACE, A. 1994. ATOM: A system for building customized program analysis tools. Tech. Rep. 94/2, Western Research Lab, Compaq (Mar.).
39. STOUTCHININ, A., AMARAL, J., GAO, G., DEHNERT, J., JAIN, S., AND DOUILLET, A. 2001. Speculative prefetching of induction pointers. In Proceedings of Compiler Construction 2001, European Joint Conferences on Theory and Practice of Software. 289-303.
40. WEICKER, R. 1990. An overview of common benchmarks. IEEE Comput. 23, 12, 65-75.
41. WEICKER, R. 1984. Dhrystone: A synthetic systems programming benchmark. Communi. ACM. 1013-1030.
42. WUNDERLICH, R., WENISCH, T., FALSAFI, B., AND HOE, J. 2003. SMARTS: Accelerating microarchitecture simulation via rigorous statistical sampling. In Proceedings of the Annual International Symposium on Computer Architecture. 84-95.
43. WONG, W. AND MORRIS, R. 1998. Benchmark synthesis using the LRU cache hit function. IEEE Trans. Comput. 37, 6, 637-645.
44. WU, Y. 2002. Efficient discovery of regular stride patterns in irregular programs and its use in compiler prefetching. In Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation. 210-221.