Statistically rigorous Java performance evaluation

Proceedings of the Conference on Object-Oriented Programming Systems, Languages, and Applications, OOPSLA

Volumn , Issue , 2007, Pages 57-76

  Georges, Andy ^a   Buytaert, Dries ^a   Eeckhout, Lieven ^a  

^a GHENT UNIVERSITY   (Belgium)

Author keywords

Benchmarking; Data analysis; Java; Methodology; Statistics

Indexed keywords

ADDITION REACTIONS; BENCHMARKING; COMPUTER PROGRAMMING LANGUAGES; COMPUTER SOFTWARE; COMPUTER SOFTWARE SELECTION AND EVALUATION; COMPUTER SYSTEMS PROGRAMMING; LEAD; LINGUISTICS; NEODYMIUM; OBJECT ORIENTED PROGRAMMING; REFUSE COLLECTION; REFUSE DISPOSAL; STATISTICAL METHODS; WASTE DISPOSAL;

AVERAGE PERFORMANCE; DATA ANALYSIS; EXECUTION TIME; GARBAGE COLLECTION (GC); GARBAGE COLLECTOR (GC); INTERNATIONAL CONFERENCES; JAVA PROGRAMS; JUST IN TIME (JIT) COMPILATION; LANGUAGES (TRADITIONAL); LANGUAGES AND SYSTEMS; NON-DETERMINISM; OBJECT-ORIENTED PROGRAMMING; PERFORMANCE EVALUATION (PE); RUN TIME; RUN TIME SYSTEM (RTS); RUN-TO-RUN (R2R); STEADY-STATE PERFORMANCE; SYSTEM EFFECTS; THREAD SCHEDULING; TIMER-BASED; VIRTUAL MACHINE (VM);

JAVA PROGRAMMING LANGUAGE;

EID: 42149194967     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1297027.1297033     Document Type: Conference Paper

References (25)

1. M. Arnold, S. Fink, D. Grove, M. Hind, and P. F. Sweeney. Adaptive optimization in the Jalapeño JVM. In OOPSLA, pages 47-65, Oct. 2000.
2. M. Arnold, M. Hind, and B. G. Ryder. Online feedback-directed optimization of Java. In OOPSLA, pages 111-129, Nov. 2002.
3. K. Barabash, Y. Ossia, and E. Petrank. Mostly concurrent garbage collection revisited. In OOPSLA, pages 255-268, Nov. 2003.
4. O. Ben-Yitzhak, I. Goft, E. K. Kolodner, K. Kuiper, and V. Leikehman. An algorithm for parallel incremental compaction. In ISMM, pages 207-212, Feb. 2003.
5. S. Blackburn, P. Cheng, and K. McKinley. Myths and reality: The performance impact of garbage collection. In SIGMETRICS, pages 25-36, June 2004.
6. S. Blackburn, P. Cheng, and K. McKinley. Oil and water? High performance garbage collection in Java with JMTk. In ICSE, pages 137-146, May 2004.
7. S. M. Blackburn, R. Garner, C. Hoffmann, A. M. Khang, K. S. McKinley, R. Bentzur, A. Diwan, D. Feinberg, D. Frampton, S. Z. Guyer, M. Hirzel, A. Hosking, M. Jump, H. Lee, J. E. B. Moss, A. Phansalkar, D. Stefanovic, T. VanDrunen, D. von Dincklage, and B. Wiedermann. The DaCapo benchmarks: Java benchmarking development and analysis. In OOPSLA, pages 169-190, Oct. 2006.
8. S. M. Blackburn and K. S. McKinley. In or out?: Putting write barriers in their place. In ISMM, pages 281-290, June 2002.
9. S. M. Blackburn and K. S. McKinley. Ulterior reference counting: Fast garbage collection without a long wait. In OOPSLA, pages 344-358, Oct. 2003.
10. L. Eeckhout, A. Georges, and K. De Bosschere. How Java programs interact with virtual machines at the microarchitectural level. In OOPSLA, pages 169-186, Oct. 2003.
11. D. Gu, C. Verbrugge, and E. M. Gagnon. Relative factors in performance analysis of Java virtual machines. In VEE, pages 111-121, June 2006.
12. M. Hauswirth, P. F. Sweeney, A. Diwan, and M. Hind. Vertical profiling: Understanding the behavior of object-priented applications. In OOPSLA, pages 251-269, Oct. 2004.
13. J.L. Hintze, and R.D. Nelson. Violin Plots: A Box Plot-Density Trace Synergism In The American Statistician, Volume 52(2), pages 181-184, May 1998.
14. X. Huang, S. M. Blackburn, K. S. McKinley, J. E. B. Moss, Z. Wang, and P. Cheng. The garbage collection advantage: Improving program locality. In OOPSLA, pages 69-80, Oct. 2004.
15. R.A. Johnson and D.W. Wichern Applied Multivariate Statistical Analysis Prentice Hall, 2002
16. D. J. Lilja. Measuring Computer Performance: A Practitioner's Guide. Cambridge University Press, 2000.
17. J. Maebe, D. Buytaert, L. Eeckhout, and K. De Bosschere. Javana: A system for building customized Java program analysis tools. In OOPSLA, pages 153-168, Oct. 2006.
18. P. McGachey and A. L. Hosking. Reducing generational copy reserve overhead with fallback compaction. In ISMM, pages 17-28, June 2006.
19. J. Neter, M. H. Kutner, W Wasserman, and C. J. Nachtsheim Applied Linear Statistical Models WCB/McGraw-Hill, 1996.
20. N. Sachindran and J. E. B. Moss. Mark-copy: Fast copying GC with less space overhead. In OOPSLA, pages 326-343, Oct. 2003.
21. K. Sagonas and J. Wilhelmsson. Mark and split. In ISMM, pages 29-39, June 2006.
22. D. Siegwart and M. Hirzel. Improving locality with parallel hierarchical copying GC. In ISMM, pages 52-63, June 2006.
23. Standard Performance Evaluation Corporation. SPECjvm98 Benchmarks, http://www.spec.org/jvm98.
24. P. F. Sweeney, M. Hauswirth, B. Cahoon, P. Cheng, A. Diwan, D. Grove, and M. Hind. Using hardware performance monitors to understand the behavior of Java applications. In VM, pages 57-72, May 2004.
25. C. Zhang, K. Kelsey, X. Shen, C. Ding, M. Hertz, and M. Ogihara. Program-level adaptive memory management. In ISMM, pages 174-183, June 2006.