SubsetTrio: An evolutionary, geometric, and statistical benchmark subsetting framework

ACM Transactions on Modeling and Computer Simulation

Volumn 21, Issue 3, 2011, Pages

  Jin, Zhanpeng ^a   Cheng, Allen C ^b  

^a University of Illinois at Urbana Champaign   (United States)

^b NOKIA RESEARCH CENTER   (United States)

Author keywords

Benchmark; Convex hull; Evolutionary process; Fitness; Principal component analysis; Subsetting

Indexed keywords

BENCHMARK; CONVEX HULL; EVOLUTIONARY PROCESS; FITNESS; SUBSETTING;

COMPUTATIONAL GEOMETRY; EVOLUTIONARY ALGORITHMS; MULTIVARIANT ANALYSIS;

PRINCIPAL COMPONENT ANALYSIS;

EID: 79952550796     PISSN: 10493301     EISSN: 15581195     Source Type: Journal    
DOI: 10.1145/1921598.1921605     Document Type: Article

References (42)

1. Barber, C.B., Dobkin, D.P., and Huhdanpaa, H. 1996. The Quickhull algorithm for convex hulls. ACM Trans. Math. Softw. 22, 4, 469-483. (Pubitemid 126417394)
2. Berg, M.D., Cheong, O., van Kreveld, M., and Overmars, M. 2000. Computational Geometry Algorithms and Applications. Springer, Berlin.
3. Bryan, P. D., Rosier, M. C., and Conte, T. M. 2007. Reverse state reconstruction for sampled microarchitectural simulation. In Proceedings of the International Symposium on Performance Analysis of Systems and Software. IEEE, Los Alamitos, CA, 190-199. (Pubitemid 350237499)
4. Citron, D. 2003. MisSPECulation: Partial and misleading use of SPEC CPU2000 in computer architecture conferences. Panel discussion. In Proceedings of the 30th International Symposium on Computer Architecture. ACM, New York, 52-61.
5. Citron, D., Hennessy, J., Patterson, D., and Sohi, G. 2003. The use and abuse of SPEC: An ISCA panel. IEEE Micro 23, 4, 73-77.
6. Citron, D., Hurani, A., and Gnadrey, A. 2006. The harmonic or geometric mean: Does it really matter? ACM SIGARCH Comput. Architect. News 34, 4, 18-25.
7. Coello Coello, C.A., Lamont, G.B., and Van Veldhuizen, D. A. 2007. Evolutionary Algorithms for Solving Multi-objective Problems (Genetic and Evolutionary Computation) 2nd Ed. Springer, Berlin.
8. Collette, Y. and Siarry, P. 2003. Multi-objective Optimization: Principles and Case Studies 1st Ed. Springer, Berlin.
9. Deb, K. 2001. Multi-Objective Optimization Using Evolutionary Algorithms. Wiley, New York.
10. Eeckhout, L., Sampson, J., and Calder, B. 2005. Exploiting program microarchitecture independent characteristics and phase behavior for reduced benchmark suite simulation. In Proceedings of the IEEE International Symposium on Workload Characterization. IEEE, Los Alamitos, CA, 2-12. (Pubitemid 44460130)
11. Eeckhout, L., Vandierendonck, H., and Bosschere, K. D. 2002. Workload design: Selecting representative program-input pairs. In Proceedings of the 11th International Conference on Parallel Architectures and Compilation Techniques. IEEE, Los Alamitos, CA, 83-94.
12. Eeckhout, L., Vandierendonck, H., and Bosschere, K. D. 2003a. Designing computer architecture research workloads. IEEE Computer 36, 2, 65-71.
13. Eeckhout, L., Vandierendonck, H., and Bosschere, K. D. 2003b. Quantifying the impact of input data sets on program behavior and its applications. J. Instrument-Level Parall. 5, 1, 33.
14. Fernandez, V., Martinez, R. G., Gonzalez, R., and Rodriguze, L. 1997. Genetic algorithms appliedto clustering. in Proceedings of the Winter Simulation Conference. 1307-1314.
15. Fraser, A. and Burnell, D. 1970. Computer Models in Genetics. McGraw-Hill, New York.
16. Goldberg, D. E. 1989. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Reading, MA.
17. Goodman, J.E. and O'Rourke, J. 2004. Handbook of Discrete and Computational Geometry 2nd Ed. CRC Press, Boca Raton, FL.
18. Henning, J. L. 2000. SPEC CPU2000: Measuring CPU performance in the new millennium. IEEE Computer 33, 7, 28-35.
19. Holland, J. H. 1975. Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor, MI.
20. Holland, J. H. 1992. Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. MIT Press, Cambridge, MA.
21. Hoste, K. and Eeckhout, L. 2006. Comparing benchmarks using key microarchitecture-independent characteristics. In Proceedings of the IEEE International Symposium on Workload Characterization. IEEE, Los Alamitos, CA, 83-92.
22. Jin, Z. and Cheng, A. C. 2008a. Evolutionary benchmark subsetting. IEEE Micro 28, 6, 20-36.
23. Jin, Z. and Cheng, A. C. 2008b. Improve simulation efficiency using statistical benchmark subsetting: An ImplantBench case study. In Proceedings of the 45th Design Automation Conference. ACM/EDAC/IEEE, 970-973.
24. Joshi, A., Phansalkar, A., Eeckhout, L., and John, L. K. 2006. Measuring benchmark similarity using inherent program characteristics. IEEE Trans. Comput. 55, 6, 769-782.
25. Kleinosowski, A. and Lilja, D. 2002. MinneSPEC: A new SPEC benchmark workload for simulation-based computer architecture research. IEEE Comput. Architect. Lett. 1, 1, 7-10.
26. Mashey, J. R. 2004. War of the benchmark means: Time for a truce. ACM SIGARCH Comput. Architect. News 32, 4, 1-14.
27. Onwubolu, G.C. and Babu, B. V. 2004. New Optimization Techniques in Engineering. Springer, Berlin.
28. Phansalkar, A., Joshi, A., Eeckhout, L., and John, L. K. 2004. Measuring program similarity. Lab for Computer Architecture Tech. rep. TR-050127-01, University of Texas at Austin.
29. Phansalkar, A., Joshi, A., Eeckhout, L., and John, L. K. 2005. Measuring program similarity: Experiments with SPEC CPU benchmark suites. in Proceedings of the International Symposium on Performance Analysis of Systems and Software. IEEE, Los Alamitos, CA, 10-20. (Pubitemid 43804299)
30. Phansalkar, A., Joshi, A., and John, L. K. 2007a. Analysis of redundancy and application balance in the SPEC CPU2006 benchmark suite. in Proceedings of the 34th International Symposium on Computer Architecture. ACM, New York, 412-423. (Pubitemid 47582121)
31. Phansalkar, A., Joshi, A., and John, L. K. 2007b. Subsetting the SPEC CPU2006 benchmark suite. ACM SIGARCH Comput. Architect. News 35, 1, 69-76.
32. Preparata, F.P. and Hong, S. J. 1977. Convex hulls of -finite sets of points in two and three dimensions. Comm. ACM 20, 2, 87-93.
33. Sherwood, T., Perelman, E., Hamerly, G., and Calder, B. 2002. Automatically characterizing large scale program behavior. In Proceedings of the 10th International Conference on Architectural Support for Programming Languages and Operating Systems. ACM, New York, 45-57. (Pubitemid 44892222)
34. Standard Performance Evaluation Corp. (SPEC). 2008. All SPEC CPU2000 results published by SPEC. http://www.spec.org/cpu2000/results/cpu2000.html
35. Vandeputte, F. and Eeckhout, L. 2008. Phase complexity surfaces: Characterizing time-varying program behavior. In Proceedings of the 3rd International Conference on High Performance Embedded Architectures and Compilers. Springer, Berlin, 320-334.
36. Wenisch, T.F., Wunderlich, R.E., Falsafi, B., and Hoe, J. C. 2003. Applying SMARTs to SPEC CPU2000. CALCM Tech. rep. 2003-1, Carnegie Mellon University, Pittsburgh, PA.
37. Wenisch, T. F., Wunderlich, R. E., Falsafi, B., and Hoe, J. C. 2005. TurboSMARTS: Accurate microarchitecture simulation sampling in minutes. In Proceedings of the International Conference on Measurement and Modeling of Computer Systems. ACM, New York, 408-409. (Pubitemid 43275453)
38. Wunderlich, R. E., Wenisch, T. F., Falsafi, B., and Hoe, J. C. 2003. SMARTS: Accelerating microarchitectural simulation via rigorous statistical sampling. In Proceedings of the 30th International Symposium on Computer Architecture. ACM, New York, 84-97.
39. Yi, J.J., Lilja, D.J., and Hawkins, D. M. 2003. A statistically rigorous approach for improving simulation methodology. In Proceedings of the 9th International Symposium on High Performance Computer Architecture. IEEE Computer Society, Los Alamitos, CA, 281-291.
40. Yi, J.J., Lilja, D.J., and Hawkins, D. M. 2005. Improving computer architecture simulation methodology by adding statistical rigor. IEEE Trans. Comput. 54, 11, 1360-1373. (Pubitemid 41709697)
41. Yi, J.J., Sendag, R., Eeckhout, L., Joshi, A., Lilja, D.J., and John, J. K. 2006a. Evaluating benchmark subsetting approaches. In Proceedings of the International Symposium on Workload Characterization. IEEE, Los Alamitos, CA, 93-104.
42. Yi, J.J., Vandierendonck, H., Eeckhout, L., and Lilja, D. J. 2006b. The exigency of benchmark and compiler drift: Designing tomorrow's processors with yesterday's tools. In Proceedings of the 20th International Conference on Supercomputing. ACM, New York, 75-86. (Pubitemid 47168494)