Using benchmarking to advance research: A challenge to software engineering

Proceedings - International Conference on Software Engineering

Volumn , Issue , 2003, Pages 74-83

  Sim, Susan Elliott ^a   Easterbrook, Steve ^a   Holt, Richard C ^b  

^a UNIVERSITY OF TORONTO   (Canada)

^b UNIVERSITY OF WATERLOO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; BENCHMARKING; DATABASE SYSTEMS; ENGINEERING RESEARCH; INFORMATION RETRIEVAL SYSTEMS; REQUIREMENTS ENGINEERING; REVERSE ENGINEERING;

ADVANCE RESEARCH; INFORMATION RETRIEVAL ALGORITHMS;

SOFTWARE ENGINEERING;

EID: 0037925631     PISSN: 02705257     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/icse.2003.1201189     Document Type: Conference Paper

References (22)

1. M. N. Armstrong and C. Trudeau, "Evaluating Architectural Extractors," presented at Working Conference on Reverse Engineering, Honolulu, HI, pp. 30-39, October 12-14, 1998.
2. Victor R. Basili, Gianluigi Caldiera, and H. Dieter Rombach, "The Goal Question Metric Approach," in Encyclopedia of Software Engineering, Two Volume Set, Gianluigi Caldiera and Dieter H. Rombach, Eds. New York City: John Wiley and Sons, Inc., pp. 528-532, 1994.
3. Berndt Bellay and Harald Gall, "An Evaluation of Reverse Engineering Tool Capabilities," Software Maintenance: Research and Practice, vol. 10, pp. 305-331, 1998.
4. Ivan T. Bowman, Michael W. Godfrey, and Ric Holt, "Connecting Architecture Reconstruction Frameworks," Journal of Information and Software Technology, vol. 42, no. 2, pp. 93-104, 1999.
5. Serge Demeyer, Tom Mens, and Michel Wermelinger, "Towards a Software Evolution Benchmark," presented at Proceedings of the International Workshop on Principles of Software Evolution, Vienna, Austria, pp. 172-175, 10-11 September 2002.
6. Martin S. Feather, Stephen Fickas, Anthony Finkelstein, and Axel van Lamsweerde, "Requirements and Specification Exemplars," Automated Software Engineering, vol. 4, pp. 419-438, 1997.
7. Rudolf Ferenc, Susan Elliot Sim, Richard C. Holt, Rainer Koschke, and Tibor Gyimóthy, "Towards a Standard Schema for C/C++," presented at Eighth Working Conference on Reverse Engineering, Stuttgart, Germany, pp. 49-58, 2-5 October 2001.
8. Jim Gray, "The Benchmark Handbook: For Database and Transaction Processing Systems," San Mateo, CA: Morgan Kaufman Publishers, Inc, 1991.
9. John L. Henning, "SPEC CPU2000: Measuring CPU Performance in the New Millennium," IEEE Computer, pp. 28-35, July, 2000.
10. Richard C. Holt, Andreas Winter, and Andy Schürr, "GXL: Toward a Standard Exchange Format.," presented at Seventh Working Conference on Reverse Engineering, Brisbane, Queensland, Australia, pp. 162-171, 23-25 November 2000.
11. Barbara Ann Kitchenham, "Evaluating software engineering methods and tools. Part 1 to 12.," ACM SIGSOFT Software Engineering Notes, vol. 21-23, 1996-1998.
12. Thomas S. Kuhn, The Structure of Scientific Revolutions, Third Edition. Chicago: The University of Chicago Press, 1996.
13. William E. McUmber and Betty H. C. Cheng, "A Generic Framework for Formalizing UML," presented at Proceedings of the Twenty-third International Conference on Software Engineering, Toronto, Canada, pp. 433-442, 12-19 May 2001.
14. Gail C. Murphy, David Notkin, William G. Griswold, and Erica S. Lan, "An Empirical Study of Static Call Graph Extractors," ACM Transactions on Software Engineering and Methodology, vol. 7, no. 2, pp. 158-191, 1998.
15. Shari Lawrence Pfleeger, "Experimental Design and Analysis in Software Engineering, Parts 1-6," ACM SIGSOFT Software Engineering Notes, vol. 19-20, 1994-1995.
16. M Plath and M. Ryan, "SFI: A Feature Integration Tool," in Advances in Computer Science, R. Berghammer and Y. Lakhnech, Eds. Heidelberg, Germany: Springer Verlag, pp. 201-216, 1999.
17. Susan Elliott Sim, Richard C. Holt, and Steve Easterbrook, "On Using a Benchmark to Evaluate C++ Extractors," presented at Tenth International Workshop on Program Comprehension, Paris, France, pp. 114-123.
18. Susan Elliott Sim and Margaret-Anne D. Storey, "A Structured Demonstration of Program Comprehension Tools," presented at Seventh Working Conference on Reverse Engineering, Brisbane, Queensland, Australia, pp. 184-193, 23-25 November 2000.
19. Susan Elliott Sim, Margaret-Anne D. Storey, and Andreas Winter, "A Structured Demonstration of Five Program Comprehension Tools: Lessons Learnt," presented at Seventh Working Conference on Reverse Engineering, Brisbane, Queensland, Australia, pp. 210-212, 23-25 November 2000.
20. Tirumale Sreemani and Joanne M. Atlee, "Feasibility of Model Checking Software Requirements: A Case Study," presented at Proceedings of COMPAS'96, Gaithersburg, Maryland, pp. 77-88, 17-21 June 1996.
21. Walter F. Tichy, "Should Computer Scientists Experiment More?" IEEE Computer, pp. 32-40, May 1998.
22. Ellen M. Voorhees and Donna Harman, "Overview of the Eighth Text REtrieval Conference (TREC-8)," presented at Text REtrieval Conference (TREC-8), Gaithersburg, Maryland, pp. 1-24, November 17-19, 2000.