The DaCapo benchmarks: Java benchmarking development and analysis

ACM SIGPLAN Notices

Volumn 41, Issue 10, 2006, Pages 169-190

  Blackburn, Stephen M ^a,b   Garner, Robin ^b   Hoffmann, Chris ^c   Khan, Asjad M ^c   McKinley, Kathryn S ^d   Bentzur, Rotem ^e   Diwan, Amer ^f   Feinberg, Daniel ^e   Frampton, Daniel ^b   Guyer, Samuel Z ^g   Hirzel, Martin ^h   Hosking, Antony ⁱ   Jump, Maria ^d   Lee, Han ^a   Moss, J Eliot B ^c   Phansalkar, Aashish ^d   Stefanović, Darko ^e   VanDrunen, Thomas ^j   Von Dincklage, Daniel ^f   Wiedermann, Ben ^d  

^a INTEL CORPORATION   (United States)

^b AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

^c UNIVERSITY OF MASSACHUSETTS   (United States)

^d UNIVERSITY OF TEXAS AT AUSTIN   (United States)

^e UNIVERSITY OF NEW MEXICO   (United States)

^f UNIVERSITY OF COLORADO   (United States)

^g Tufts ^*  (United States)

^h IBM T J WATSON RESEARCH CENTER   (United States)

ⁱ PURDUE UNIVERSITY   (United States)

^j WHEATON COLLEGE   (United States)

more..

Author keywords

Benchmark; DaCapo; Java; Methodology; SPEC

Indexed keywords

BENCHMARKING SELECTION; GARBAGE COLLECTION; MEMORY MANAGEMENT; METHODOLOGY;

BENCHMARKING; COMPUTATIONAL COMPLEXITY; COMPUTER PROGRAMMING; COMPUTER SOFTWARE SELECTION AND EVALUATION; PROGRAM COMPILERS; TIME SERIES ANALYSIS;

JAVA PROGRAMMING LANGUAGE;

EID: 33750426693     PISSN: 03621340     EISSN: 03621340     Source Type: Journal    
DOI: 10.1145/1167515.1167488     Document Type: Article

References (42)

1. B. Alpern, D. Attanasio, J. J. Barton, A. Cocchi, S. F. Hummel, D. Lieber, M. Margen, T. Ngo, J. Shepherd, and S. Smith. Implementing Jalapeño in Java. In ACM Conference on Object - Oriented Programming Systems, Languages, and Applications, Denver, CO, Nov. 1999.
2. Apache Software Foundation. Apache Software License, 2000. http://www.opensource.org/licenses/apachepf.php.
3. C. Attanasio, D. Bacon, A. Cocchi, and S. Smith. A comparative evaluation of parallel garbage collectors. In Proceedings of the Fourteenth Workshop on Languages and Compilers for Parallel Computing, Cumberland Falls, KY, Aug. 2001.
4. S. M. Blackburn, P. Cheng, and K. S. McKinley. Myths and realities: The performance impact of garbage collection. In Proceedings of the ACM Conference on Measurement & Modeling Computer Systems, pages 25-36, NY, NY, June 2004.
5. S. M. Blackburn, P. Cheng, and K. S. McKinley. Oil and water? High performance garbage collection in Java with MMTk. In Proceedings of the 26th International Conference on Software Engineering, pages 137-146, Scotland, UK, May 2004.
6. S. M. Blackburn, R. Garner, C. Hoffman, A. M. Khan, K. S. McKinley, R. Bentzur, A. Diwan, D. Feinberg, D. Frampton, S. Z. Guyer, M. Hirzel, A. Hosking, M. Jump, H. Lee, J. E. B. Moss, A. Phansalkar, D. Stefanovid, T. VanDrunen, D. von Dincklage, and B. Wiedermann. The DaCapo Benchmarks: Java benchmarking development and analysis (extended version). Technical Report TR-CS-06-01, Dept. of Computer Science, Australian National University, 2006. http://www. dacapobench.org.
7. S. M. Blackburn and A. Hosking. Barriers: Friend or foe? In The International Symposium on Memory Management, pages 143-151, Oct. 2004.
8. S. M. Blackburn and K. S. McKinley. In or out? Putting write barriers in their place. In The International Symposium on Memory Management, pages 175-184, June 2002.
9. S. M. Blackburn, S. Singhai, M. Hertz, K. S. McKinley, and J. E. B. Moss. Pretenuring for Java. In ACM Conference on Object - Oriented Programming Systems, Languages, and Applications, pages 342-352, Tampa, FL, Oct. 2001. ACM.
10. T. Brecht, E. Ariomandi, C. Li, and H. Pham. Controlling garbage collection and heap growth to reduce the execution time of Java applications. In ACM Conference on Object - Oriented Programming Systems, Languages, and Applications, pages 353-366, Tampa, FL, 2001.
11. B. Cahoon and K. S. McKinley. Data flow analysis for software prefetching linked data structures in java controller. In The International Conference on Parallel Architectures and Compilation Techniques, pages 280-291, Barcelona, Spain, Sept. 2001.
12. S. R. Chidamber and C. F. Kemerer. A metrics suite for object-oriented design. IEEE Transactions on Software Engineering, 20(6):476-493, June 1994.
13. F. Chow, A. Wright, and K. Lai. Characterization of java workloads by principal components analysis and indert branches. In Proceedings of the Workshop on Workload Characterization, pages 11-19, Dallas, TX, Nov. 1998.
14. DaCapo Project. The DaCapo Benchmarks, beta-2006-08, 2006. http://www.dacapobench.org.
15. S. Dieckmann and U. Hölzle. A study of the allocation behavior of the SPECjvm98 Java benchmarks. In European Conference on Object-Oriented Programming, June 1999.
16. A. Diwan, D. Tarditi, and J. E. B. Moss. Memory subsystem performance of programs using copying garbage collection. In Conference Record of the Twenty-First ACM Symposium on Principles of Programming Languages, pages 1-14, Portland, OR, Jan. 1994.
17. B. Dufour, K. Driesen, L. Hendren, and C. Verbrugge. Dynamic metrics for Java. In ACM Conference on Object - Oriented Programming Systems, Languages, and Applications, pages 149-168, Anaheim, CA, Oct. 2003.
18. G. H. Dunteman. Principal Components Analysis. Sage Publications, 1989.
19. L. Eeckhout, A. Georges, and K. De Bosschere. How Java programs interact with virtual machines at the microarchitecture level. In ACM Conference on Object - Oriented Programming Systems, Languages, and Applications, pages 169-186, Anaheim, CA, October 2003.
20. R. Fitzgerald and D. Tarditi. The case for profile-directed selection of garbage collectors. In The International Symposium on Memory Management, pages 111-120, Minneapolis, MN, Oct. 2000.
21. M. Hauswirth, A. Diwan, P. Sweeney, and M. Mozer. Automating vertical profiling. In ACM Conference, on Object - Oriented Programming Systems, Languages, and Applications, pages 281-296, San Diego, CA, October 2005.
22. M. W. Hicks, J. T. Moore, and S. Nettles. The measured cost of copying garbage collection mechanisms. In ACM International Conference on Functional Programming, pages 292-305, 1997.
23. U. Hölzle and D. Ungar. Do object-oriented languages need special hardware support? In European Conference on Object-Oriented Programming, pages 283-302, London, UK, 1995.
24. A. L. Hosking, J. E. B. Moss, and D. Stefanović. A comparative performance evaluation of write barrier implementations. In ACM Conference on Object - Oriented Programming Systems, Languages, and Applications, pages 92-109, Vancouver, BC, Oct. 1992.
25. X. Huang, Z. Wang, S. M. Blackburn, K. S. McKinley, J. E. B. Moss, and P. Cheng. The garbage collection advantage: Improving mutator locality. In ACM Conference on Object - Oriented Programming Systems, Languages, and Applications, pages 69-80, Vancouver, BC, 2004.
26. Java Grande Forum. The Java Grande Benchmark Suite, 2006. http://www.epcc.ed.ac.uk/javagrande/.
27. A. Joshi, A. Phansalkar, L. Eeckhout, and L. John. Measuring benchmark similarity using inherent program characteristics. IEEE Transactions on Computers, 55(6):769-782, June 2006.
28. J. Kim and Y. Hsu. Memory system behavior of Java programs: methodology and analysis. In Proceedings of the ACM Conference on Measurement & Modeling Computer Systems, pages 264-274, Santa Clara, California, June 2000.
29. T. Li, L. John, V. Narayanan, A. Sivasubramaniam, J. Sabarinathan, and A. Murthy. Using complete system simulation to characterize SPECjvm98 benchmarks. In Proceedings of the 2000 ACM International Conference on Supercomputing, pages 22-33, Santa Fe, NM, 2000.
30. Y. Luo and L. John. Workload characterization of multithreaded Java servers. In IEEE International Symposium on Performance Analysis of Systems and Software, pages 128-136, 2001.
31. M. Marden, S. Lu, K. Lai, and M. Lipasti. Comparison of memory system behavior in Java and non-Java commercial workloads. In Proceedings of the Workshop on Computer Architecture Evaluation using Commercial Workloads, Boston, MA, Feb. 2002.
32. R. Radhakrishnan, N. Vijaykrishnan, L. K., A. Sivasubramaniam, J. Rubio, and J. Sabarinathan. Java runtime systems: Characterization and architectural implications. IEEE Transactions on Computers, 50(2):131-146, Feb. 2001.
33. A. Rajan, S. Hu, and J. Rubio. Cache performance in Java virtual machines: A study of constituent phases. In IEEE International Workshop on Workload Characterization, Nov. 2002.
34. A. Rogers, M. Carlisle, J. H. Reppy, and L. J. Hendren. Supporting dynamic data structures on distributed-memory machines. ACM Transactions on Programming Languages and Systems, 17(2):233-263, Mar. 1995.
35. Y. Shuf, M. J. Serran, M. Gupta, and J. P. Singh. Characterizing the memory behavior of Java workloads: A structured view and opportunities for optimizations. In Proceedings of the ACM Conference on Measurement & Modeling Computer Systems, pages 194-205, Cambridge, MA, June 2001.
36. D. D. Spinellis. ckjm Chidamber and Kemerer metrics Software, v 1.6. Technical report, Athens University of Economics and Business, 2005. http://www.spinellis.gr/sw/ckjm.
37. Standard Performance Evaluation Corporation. SPECjvm98 Documentation, release 1.03 edition, March 1999.
38. Standard Performance Evaluation Corporation. SPECjbb2000 (Java Business Benchmark) Documentation, release 1.01 edition, 2001.
39. D. Stefanović. Properties of Age-Based Automatic Memory Reclamation Algorithms. PhD thesis, Department of Computer Science, University of Massachusetts, Amherst, Massachusetts, Dec. 1998.
40. D. Stefanović, M. Hertz, S. M. Blackburn, K. McKinley, and J. Moss. Olderfirst garbage collection in practice: Evaluation in a Java virtual machine. In Memory System Performance, Berlin, Germany, June 2002.
41. D. Tarditi and A. Diwan. Measuring the cost of memory management. Lisp and Symbolic Computation, 9(4), Dec. 1996.
42. Virtutec, Inc. Virtutech Simics, 2006. http://www.simics.net.