Precise regression benchmarking with random effects: Improving mono benchmark results

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 4054 LNCS, Issue , 2006, Pages 63-77

  Kalibera, Tomas ^a   Tuma, Petr ^a  

^a CHARLES UNIVERSITY   (Czech Republic)

Author keywords

Benchmark precision; Performance evaluation; Random effects; Regression benchmarking

Indexed keywords

BENCHMARKING; COMPUTER SCIENCE; MATHEMATICAL MODELS; PROGRAM COMPILERS; RANDOM PROCESSES; STATISTICAL METHODS;

BENCHMARK PRECISION; PERFORMANCE EVALUATION; RANDOM EFFECTS; REGRESSION BENCHMARKING;

REGRESSION ANALYSIS;

EID: 33746347072     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/11777830_5     Document Type: Conference Paper

References (16)

1. Smith, C.U., Williams, L.G.: Performance Solutions: A Practical Guide to Creating Responsive, Scalable Software. Addison-Wesley, Reading, MA, USA (2001)
2. Kalibera, T., Bulej, L., Tuma, P.: Benchmark precision and random initial state. In: Proceedings of SPECTS 2005, SCS (2005) 853-862
3. Kalibera, T., Bulej, L., Tuma, P.: Automated detection of performance regressions: The Mono experience. In: MASCOTS, IEEE Computer Society (2005) 183-190
4. Bulej, L., Kalibera, T., Tuma, P.: Repeated results analysis for middleware regression benchmarking. Performance Evaluation 60 (2005) 345-358
5. Lo, S.L., Grisby, D., Riddoch, D., Weatherall, J., Scott, D., Richardson, T., Carroll, E., Evers, D., Meerwald, C.: Free high performance orb, http://omniorb.sourceforge.net (2006)
6. Novell, Inc.: The Mono Project, http://www.mono-project.com (2006)
7. ECMA: ECMA-335: Common Language Infrastructure (CLI). ECMA (2002)
8. Distributed Systems Research Group: Mono regression benchmarking. http://nenya.ms.mff.cuni.cz/projects/mono (2005)
9. Free Software Foundation: The gnu compiler collection, http://gcc.gnu.org (2006)
10. Gu, D., Verbrugge, C., Gagnon, E.: Code layout as a source of noise in JVM performance. In: Component And Middleware Performance Workshop, OOPSLA 2004. (2004)
11. Wasserman, L.: All of Statistics: A Concise Course in Statistical Inference. Springer, New York, NY, USA (2004)
12. Jain, R.: The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling. Wiley-Interscience, New York, NY, USA (1991)
13. Buble, A., Bulej, L., Tuma, P.: CORBA benchmarking: A course with hidden obstacles. In: IPDPS, IEEE Computer Society (2003) 279
14. DOC Group: TAO performance scoreboard. http://www.dre.vanderbilt.edu/ stats/performance-shtml (2006)
15. Prochazka, M., Madan, A., Vitek, J., Liu, W.: RTJBench: A Real-Time Java Benchmarking Framework. In: Component And Middleware Performance Workshop, OOPSLA 2004. (2004)
16. Weisstein, E.W.: Mathworld-a wolfram web resource. http://mathworld. wolfram.com (2006)