The exigency of benchmark and compiler drift: Designing tomorrow's processors with yesterday's tools

Proceedings of the International Conference on Supercomputing

Volumn , Issue , 2006, Pages 75-86

  Yi, Joshua J ^a   Vandierendonck, Hans ^b   Eeckhout, Lieven ^b   Lilja, David J ^c  

^a FREESCALE SEMICONDUCTOR INC   (United States)

^b GHENT UNIVERSITY   (Belgium)

^c UNIVERSITY OF MINNESOTA   (United States)

Author keywords

Benchmark drift; Compiler drift; Microprocessor design

Indexed keywords

BENCHMARK DRIFT; COMPILER DRIFT; MICROPROCESSOR DESIGN;

ALGORITHMS; BENCHMARKING; DATA STORAGE EQUIPMENT; OPTIMIZATION; PROGRAM PROCESSORS;

PROGRAM COMPILERS;

EID: 34547429489     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1183401.1183414     Document Type: Conference Paper

References (27)

1. Bell Jr., R. and John, L., "The Case for Automatic Synthesis of Miniature Benchmarks," In Proceedings of the Workshop on Modeling, Benchmarking, and Simulation (MoBS'05) (Madison, WI, USA, June 4-8, 2005), 88-97.
2. Bell Jr., R. and John, L., "Improved Automatic Testcase Synthesis for Performance Model Validation," In Proceedings of the International Conference on Supercomputing (ICS'05) (Cambridge, MA, USA, June 20-22, 2005), 111-120.
3. Brooks, D., Tiwari, V., and Martonosi, M., "Wattch: A Framework, for Architectural-Level Power Analysis and Optimizations," In Proceedings of the International Symposium on Computer Architecture (ISCA'00) (Vancouver, Canada, June 10-14, 2000), 83-94.
4. Brooks, D., Bose, P., Schuster, S., Jacobson, H., Kudva, P., Buyuktosunoglu, A., Wellman, J., Zyuban, V., Gupta, M., and Cook, P., "Power-Aware Microarchitecture: Design and Modeling Challenges for Next-Generation Microprocessors," IEEE Micro, 20, 6, (Nov./Dec. 2000), 26-44.
5. Burger, D. and Austin, T. "The SimpleScalar Tool Set, Version 2.0," ACM Computer Architecture News, (June 1997), 13-25.
6. Calder, B., Grunwald, D., and Zorn, B., "Quantifying Behavioral Differences Between C and C++ Programs," Journal of Programming Languages, 2, 4, (1994), 313-351.
7. Carlton, A., "Lessons Learned from 072.sc", SPEC Newsletter, (Mar. 1995).
8. Choi, Y., Knies, A., Gerke, L., and Ngai, T., "The Impact of If-Conversion on Branch Prediction and Program Execution on. the Intel Itanium Processor," In Proceedings of the International Symposium on Microarchitecture (Micro'01) (Austin, TX, December 2-5, 2001), 182-191.
9. Dujmovic, J. and Dujmovic, I., "Evolution and Evaluation of SPEC Benchmarks," ACM SIGMETRICS Performance Evaluation Review, 26, 3, (Dec. 1998), 2-9.
10. Eeckhout, L., Vandierendonck, H., and De Bosschere, K., "Workload Design: Selecting Representative Program-Input Pairs," In Proceedings of the International Conference on Parallel Architectures and Compilation Techniques (PACT •02) (Charlottesville, VA, USA, September 22-25, 2002), 83-94.
11. Hamerly, G., Perelman, E., and Calder, B., "How to Use SimPoint to Pick Simulation Points," ACM SIGMETRICS Performance Evaluation Review, 31, 4, (Mar. 2004), 25-30.
12. Hennessy, J. and Patterson, D., "Computer architecture: A Quantitative Approach," Morgan-Kauffman, San. Francisco, CA, 2003.
13. Henning, J., "SPEC CPU2000: Measuring CPU Performance in the New Millennium," IEEE Computer, 33, 1, (Jul. 2000), 28-35.
14. Henning, J., "SPEC CPU2000 Memory Footprint," http://www.spec.org/cpu2000/analysis/memory
15. Lilja, D., "Measuring Computer Performance," Cambridge University Press, New York, NY, 2000.
16. Phansalkar, A., Joshi, A., Eeckhout, L., and John, L., "Measuring Program Similarity: Experiments with SPEC CPU Benchmark Suites," Proceedings of the International Symposium on Performance Analysis of Systems and Software (ISPASS '05 ) (Austin, TX, March 20-22, 2005), 10-20.
17. Plackett, R. and Burman, J. "The Design of Optimum Multifactorial Experiments," Biometrika, 33, 4, (June 1946), 305-325.
18. Sherwood, T., Perelman, E., Hamerly, G., and Calder, B., "Automatically Characterizing Large Scale Program Behavior," In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'02) (San Jose, CA, USA, October 5-9, 2002), 45-57.
19. [ 19] http://www.cs.ucsd.edu/~calder/simpoint
20. Skadron, K., Martonosi, M., August, D., Hill, M., Lilja, D., and Pai, V., "Challenges in Computer Architecture Evaluation," IEEE Computer, 36, 8, (Aug. 2003), 30-36.
21. http://www.spec.org
22. Vandierendonck, H. and De Bosschere, K., "Eccentric and Fragile Benchmarks," In Proceedings of the International Symposium on Performance Analysis of Systems and Software (ISPASS '04) (Austin, TX, March 10-12, 2004), 2-11.
23. http://www.eecs.um.ich.edu/~chriswea/benchmarks/SPEC2000.html
24. Weicker, R., "An Example of Benchmark Obsolescence: 023.eqntott," SPEC Newsletter, (Dec. 1995).
25. Wulf, W. and McKee, S., "Hitting the Memory Wall: Implications of the Obvious," ACM Computer Architecture News, 23, 1, (Mar. 1995), 20-24.
26. Yi, J., Lilja, D., and Hawkins, D., "A Statistically Rigorous Approach for Improving Simulation Methodology," In Proceedings of the International Symposium on High-Performance Computer Architecture (HPCA '03) (Anaheim, CA, USA, February 8-12, 2003), 281-291.
27. Yi, J., Kodakara, S., Sendag, R., Lilja, D., and Hawkins, D., "Characterizing and Comparing Prevailing Simulation Techniques," In Proceedings of the International Symposium on High-Performance Computer Architecture (HPCA '05) (San Francisco, CA, USA, February 12-16, 2005), 266-277.