An experimental evaluation of continuous testing during development

ISSTA 2004 - Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis

Volumn , Issue , 2004, Pages 76-85

  Saff, David ^a   Ernst, Michael D ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Continuous compilation; Continuous testing; Test first development; Unit testing

Indexed keywords

BIT ERROR RATE; COMPUTER PROGRAMMING; COMPUTER SOFTWARE SELECTION AND EVALUATION; COMPUTERIZED TOMOGRAPHY; INTEROPERABILITY; JUST IN TIME PRODUCTION; OPTIMIZATION; PROGRAM COMPILERS; PROGRAM DEBUGGING; REGRESSION ANALYSIS;

CONTINUOUS COMPILATION; CONTINUOUS TESTING; PERFORMING MAINTENANCE; TEST-FIRST DEVELOPMENT;

COMPUTER SCIENCE;

EID: 23744460210     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1013886.1007523     Document Type: Conference Paper

References (39)

1. V.R. Basili. The role of experimentation in software engineering: past, current, and future. In Proceedings of the 18th International Conference on Software Engineering, pages 442-449, Berlin, Germany, Mar. 25-29, 1996.
2. K. Beck. Extreme Programming Explained: Embrace Change. Addison-Wesley, 1999.
3. K. Beck. Test-Driven Development: By Example. Addison-Wesley, Boston, 2002.
4. M. Burnett, C. Cook, O. Pendse, G. Rothermel, J. Summet, and C. Wallace. End-user software engineering with assertions in the spreadsheet paradigm. In ICSE'03, Proceedings of the 25th International Conference on Software Engineering, pages 93-103, Portland, Oregon, May 6-8, 2003.
5. B. Childers, J. W. Davidson, and M. L. Soffa. Continuous compilation: A new approach to aggressive and adaptive code transformation. In International Parallel and Distributed Processing Symposium (IPDPS'03), pages 205-214, Nice, France, Apr. 22-26, 2003.
6. A. Cypher, D. C. Haibert, D. Kurlander, H. Lieberman, D. Maulsby, B. A. Myers, and A. Turransky, editors. Watch What I Do: Programming by Demonstration. MIT Press, Cambridge, MA, 1993.
7. A. Dunsmore, M. Roper, and M. Wood. Further investigations into the development and evaluation of reading techniques for object-oriented code inspection. In ICSE'02, Proceedings of the 24th International Conference on Software Engineering, pages 47-57, Orlando, Florida, May 22-24, 2002.
8. Eclipse. http://www.eel ipse.org.
9. Emacs. http://www.emacs.org.
10. M. J. Harrold, R. Gupta, and M. L. Soffa. A methodology for controlling the size of a test suite. ACM Transactions on Software Engineering and Methodology, 2(3):270-285, July 1993.
11. P. Henderson and M. Weiser. Continuous execution: The VisiProg environment. In Proceedings of the 8rd International Conference on Software Engineering, pages 68-74, London, Aug. 28-30, 1985.
12. JetBrains IntelliJ IDEA. http://www.intellij.com/idea/.
13. JUnit.http://www.junit.org.
14. M. Karasick. The architecture of Montana: an open and extensible programming environment with an incremental C++ compiler. In FSE '98, Proceedings of the 6th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 131-142, Lake Buena Vista, FL, USA, Nov. 3-5, 1998.
15. T. Lau, P. Domingos, and D. S. Weld. Version space algebra and its application to programming by demonstration. In International Conference on Machine Learning, pages 527-534, Stanford, CA, June 2000.
16. H. K. N. Leung and L. White. Insights into regression testing. In Proceedings of the Conference on Software Maintenance, pages 60-69, Miami, FL, Oct. 16-19, 1989.
17. R. C. Miller. Lightweight Structure in Text. PhD thesis, Computer Science Department, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, May 2002.
18. Also available as CMU Computer Science technical report CMU-CS-02-134 and CMU Human-Computer Interaction Institute technical report CMU-HCII-02-103.
19. D. L. Moody, G. Sindre, T. Brasethvik, and A. Sølvberg. Evaluating the quality of information models: empirical testing of a conceptual model quality framework. In ICSE'03, Proceedings of the 25th International Conference on Software Engineering, pages 295-305, Portland, Oregon, May 6-8, 2003.
20. J. W. Nimmer and M. D. Ernst. Invariant inference for static checking: An empirical evaluation. In Proceedings of the ACM SIGSOFT 10th International Symposium on the Foundations of Software Engineering (FSE 2002), pages 11-20, Charleston, SC, Nov. 20-22, 2002.
21. R. P. Nix. Editing by example. ACM Trans. Prog. Lang. Syst., 7(4):600-621, Oct. 1985.
22. A. Orso, T. Apiwattanapong, and M. J. Harrold. Leveraging field data for impact analysis and regression testing. In Proceedings of the 10th European Software Engineering Conference and the 11th ACM SIGSOFT Symposium on the Foundations of Software Engineering, Helsinki, Finland, Sept. 3-5, 2003.
23. A. Orso, D. Liang, M. J. Harrold, and R. Lipton. Gamma system: Continuous evolution of software after deployment. In ISSTA 2002, Proceedings of the 2002 International Symposium on Software Testing and Analysis, pages 65-69, Rome, Italy, July 22-24, 2002.
24. C. Pavlopoulou and M. Young. Residual test coverage monitoring. In ICSE '99, Proceedings of the 21st International Conference on Software Engineering, pages 277-284, Los Angeles, CA, USA, May 19-21, 1999.
25. M. P. Plezbert and R. K. Cytron. Does "just in time" = "better late than never"? In Proceedings of the 24th Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pages 120-131, Paris, France, Jan. 15-17, 1997.
26. G. Rothermel and M. J. Harrold. Analyzing regression test selection techniques. IEEE Transactions on Software Engineering, 22(8):529-551, Aug. 1996.
27. G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold. Prioritizing test cases for regression testing. IEEE Transactions on Software Engineering, 27(10):929-948, Oct. 2001.
28. D. Saff. Automated continuous testing to speed software development. Master's thesis, MIT Department of Electrical Engineering and Computer Science, Cambridge, MA, Feb. 3, 2004.
29. D. Saff and M. D. Ernst. Reducing wasted development time via continuous testing. In Fourteenth International Symposium on Software Reliability Engineering, pages 281-292, Denver, CO, Nov. 17-20, 2003.
30. D. Saff and M. D. Ernst. Automatic mock object creation for test factoring. In ACM SIGPLAN/SIGSOFT Workshop on Program Analysis for Software Tools and Engineering (PASTE'04), Washington, DC, USA, June 7-8, 2004.
31. D. Saff and M. D. Ernst. Continuous testing in Eclipse. In 2nd Eclipse Technology Exchange Workshop (eTX), Barcelona, Spain, Mar. 30, 2004.
32. M. D. Schwartz, N. M. Delisle, and V. S. Begwani. Incremental compilation in Magpie. In Proceedings of the ACM SIGPLAN '84 Symposium on Compiler Construction, pages 122-131, Montreal, Canada, June 17-22, 1984.
33. S. Siegel. Object-Oriented Software Testing: A Hierarchical Approach. John Wiley & Sons, 1996.
34. M. L. Soffa. Continuous testing. Personal communication, Feb. 2003.
35. A. Srivastava and J. Thiagarajan. Effectively prioritizing tests in development environment. In ISSTA 2002, Proceedings of the 2002 International Symposium on Software Testing and Analysis, pages 97-106, Rome, Italy, July 22-24, 2002.
36. Tacoma community college elementary algebra syllabus. http://www.tacoma.etc.edu/home/jkellerrn/MATH090/default.htm.
37. W. E. Wong, J. R. Morgan, S. London, and H. Agrawal. A study of effective regression testing in practice. In Eighth International Symposium on Software Reliability Engineering, pages 264-274, Albuquerque, NM, Nov. 2-5, 1997.
38. A. Zeller. Yesterday, my program worked. Today, it does not. Why? In Proceedings of the 7th European Software Engineering Conference and the 7th ACM SIGSOFT Symposium on the Foundations of Software Engineering, pages 253-267, Toulouse, France, Sept. 6-9, 1999.
39. M. K. Zimmerman, K. Lundqvist, and N. Leveson. Investigating the readability of state-based formal requirements specification languages. In ICSE'02, Proceedings of the 24th International Conference on Software Engineering, pages 33-43, Orlando, Florida, May 22-24, 2002.