Hierarchical means: Single number benchmarking with workload cluster analysis

Proceedings of the 2007 IEEE International Symposium on Workload Characterization, IISWC

Volumn , Issue , 2007, Pages 204-213

  Yoo, Richard M ^a   Lee, Hsien Hsin S ^a   Lee, Han ^b   Chow, Kingsum ^b  

^a GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

^b INTEL CORPORATION   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CLUSTER ANALYSIS; QUALITY ASSURANCE; RANDOM VARIABLES; REDUNDANCY; RELIABILITY; STANDARDS; TECHNICAL PRESENTATIONS;

CALCULATION METHODS; CASE STUDIES; CLUSTER CHARACTERISTICS; CONSORTIUM MEMBERS; CURRENT STANDARD; GEOMETRIC MEAN (MG); INDIVIDUAL (PSS 544-7); INHERENT REDUNDANCY; INTERNATIONAL SYMPOSIUM; WORKLOAD CHARACTERIZATION;

BENCHMARKING;

EID: 47349111047     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IISWC.2007.4362196     Document Type: Conference Paper

References (21)

1. R. Narayanan, B. Ozisikyilmaz, J. Zambreno, G. Memik, and A. Choudhary, "MineBench: A benchmark suite for data mining workloads," in Proceedings of the 2006 IEEE International Symposium on Workload Characterization, October 2006, pp. 182-188.
2. O. Altun, N. Dursunoglu, and M. Amasyali, "Clustering application benchmark," in Proceedings of the 2006 IEEE International Symposium on Workload Characterization, October 2006, pp. 178-181.
3. R. Pozo and B. Miller, "SciMark, a web-based benchmark for numerical computing in Java," In Summary of Activities for Fiscal Year 2005, Mathematical and Computational Sciences Division, NIST Information Technology Laboratory, pp. 46-47, 2005.
4. LAPACK/ScaLAPACK Development Forum, http://icl.cs.utk.edu/lapack- forum/archives/lapack/msg00190.html.
5. L. Eeckhout, J. Sampson, and B. Calder, "Exploiting program microarchitecture independent characteristics and phase behavior for reduced benchmark suite simulation," in Proceedings of the 2005 IEEE International Symposium on Workload Characterization, 2005, pp. 2-12.
6. K. Hoste and L. Eeckhout, "Comparing benchmarks using key microarchitecture-independent characteristics," in Proceedings of the 2006 IEEE International Symposium on Workload Characterization, October 2006, pp. 83-92.
7. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd Ed. Prentice Hall, July 1998.
8. T. Kohonen, Self-Organizing Maps, 3rd Ed. Springer, May 2006.
9. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification, 2nd Ed. Wiley-Interscience, October 2000.
10. H. Vandierendonck and K. D. Bosschere, "Experiments with subsetting benchmark suites," in Proceedings of the IEEE 7th Annual Workshop on Workload Characterization, October 2004, pp. 55-62.
11. J. J. Yi, R. Sendag, L. Eeckhout, A. Joshi, D. J. Lilja, and L. K. John, "Evaluating benchmark subsetting approaches," in Proceedings of the 2006 IEEE International Symposium on Workload Characterization, October 2006, pp. 93-104.
12. A. Phansalkar, A. Joshi, and L. K. John, "Analysis of redundancy and application balance in the SPEC CPU2006 benchmark suite," in ISCA '07: Proceedings of the 34th annual international symposium on Computer architecture. New York, NY, USA: ACM Press, 2007, pp. 412-423.
13. "SPEC JVM98 benchmarks." Standard Performance Evaluation Corporation, http://www.spec.org/jvm98/.
14. S. M. Blackburn, R. Garner, C. Hoffman, A. M. Khan, K. S. McKinley, R. Bentzur, A. Diwan, D. Feinberg, D. Frampton, S. Z. Guyer, M. Hirzel, A. Hosking, M. Jump, H. Lee, J. E. B. Moss, A. Phansalkar, D. Stefanovíc, T. VanDrunen, D. von Dincklage, and B. Wiedermann, "The DaCapo benchmarks: Java benchmarking development and analysis," in OOPSLA '06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-Oriented Programming, Systems, Languages, and Applications. New York, NY, USA: ACM Press, Oct. 2006.
15. M. Hauswirth, P. F. Sweeney, A. Diwan, and M. Hind, "Vertical profiling: Understanding the behavior of object-oriented applications," in Proceedings of the 19th Annual ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications, 2004, pp. 251-269.
16. K. Chow, R. Morin, and K. Shiv, "Enterprise Java performance: Best practices," in Intel Technology Journal, volume 7, issue 1, February 2003, pp. 32-46.
17. G. Chen, M. Kandemir, N. Vijaykrishnan, and M. J. Irwin, "PennBench: A benchmark suite for embedded Java," in the IEEE 5th Annual Workshop on Workload Characterization, November 2005.
18. M. T. Conte, A. R. Trick, J. C. Gyllenhaal, and W.-M. W. Hwu, "A study of code reuse and sharing characteristics of Java applications," in the IEEE 1st Annual Workshop on Workload Characterization, November 1998.
19. J. E. Smith, "Characterizing computer performance with a single number," Communications of the ACM, vol. 31, no. 10, pp. 1202-1206, 1988.
20. J. R. Mashey, "War of the benchmark means: time for a truce," SIGARCH Computer Architecture News, vol. 32, no. 4, pp. 1-14, 2004.
21. L. K. John, "More on finding a single number to indicate overall performance of a benchmark suite," SIGARCH Computer Architecture News, vol. 32, no. 1, pp. 3-8, 2004.