Examining random and designed tests to detect code mistakes in scientific software

Journal of Computational Science

Volumn 2, Issue 1, 2011, Pages 47-56

  Kelly, Diane ^a   Gray, Robert ^a   Shao, Yizhen ^b  

^a ROYAL MILITARY COLLEGE OF CANADA   (Canada)

^b UNIVERSITY OF OTTAWA   (Canada)

Author keywords

Computational software; Random testing; Software testing; Tolerance problem

Indexed keywords

CODE STRUCTURE; COMPUTATIONAL SOFTWARE; MULTIPLE FACTORS; ONE-FACTOR; PREDICTIVE POWER; RANDOM TESTING; SCIENTIFIC SOFTWARES; STRUCTURAL METRICS; TEST INPUTS; TOLERANCE PROBLEM;

COMPUTER SOFTWARE SELECTION AND EVALUATION; SOFTWARE TESTING;

TESTING;

EID: 79953109971     PISSN: 18777503     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jocs.2010.12.002     Document Type: Article

References (24)

1. Andrews J.H., Briande L.C., Labiche Y., Namin A.S. Using mutation analysis for assessing and comparing testing coverage criteria. IEEE Transactions on Software Engineering 2006, 32(8):608-624.
2. Beizer B. Software Testing Techniques 1990, Van Nostrand Reinhold, New York, USA.
3. The Architecture of Scientific Software 2001, Kluwer Academic Publishers, Boston, USA. R.F. Boisvert, P.T.P. Tang (Eds.).
4. N. Cote, An exploration of a testing strategy to support refactoring, Master's thesis, Royal Military College of Canada, Kingston, Canada, 2005.
5. Dubois P.F. Object Technology for Scientific Computing: Object-oriented Numerical Software in Eiffel and C 1997, Prentice-Hall, Inc., Upper Saddle River, NJ, USA.
6. R. Gray, Investigating test selection techniques for scientific software, Master's thesis, Royal Military College of Canada, Kingston, Canada, 2010.
7. Hamlet D. When only random testing will do. Proceedings First International Workshop on Random Testing 2006, 1-9. ACM Press.
8. Hatton L., Roberts A. How accurate is scientific software?. IEEE Transactions on Software Engineering 1994, 20(10):785-797.
9. D. Hook, Using code mutation to study code faults in scientific software, Master's thesis, Queen's University, Kingston, Canada, 2009; available at http://qspace.library.queensu.ca/handle/1974/1765.
10. Hook D., Kelly D. Testing for trustworthiness in scientific software. 2nd International Workshop on Software Engineering for Computational Science and Engineering, Vancouver, BC, Proceedings International Conference on Software Engineering 2009.
11. Hook D., Kelly D. Mutation sensitivity testing. IEEE Computing in Science and Engineering 2009, (November/December):2-9.
12. D. Kelly, S. Thorsteinson, D. Hook, Scientific software testing: analysis in four dimensions, IEEE software, preprint available online in IEEE Computer Society Digital Library, January 2010.
13. Kelly D., Hook D., Sanders R. Five recommended practices for scientists who write computational software. IEEE Computing in Science and Engineering 2009, (September/October):48-53.
14. Kelly D., Cote N., Shepard T. Software engineers and nuclear scientists: teaming up to do testing. Proceedings Canadian Nuclear Society Conference 2007, Canadian Nuclear Society.
15. Knupp P., Salari K. Verification of Computer Codes in Computational Science and Engineering 2003, Chapman & Hall (CRC Press), Boca Raton, USA.
16. MATMute: mutator tool, available at http://matmute.sourceforge.net/.
17. McCabe T.J. A complexity measure. IEEE Transactions on Software Engineering 1976, SE-2(4):308-320.
18. Miller G. Scientific publishing: a scientist's nightmare: software problem leads to five retractions. Science 2006, 314(5807):1856-1857.
19. Musa J.D., Iannino A., Okumoto K. Software Reliability 1990, McGraw Hill, New York, USA.
20. Myers G.J. The Art of Software Testing 1979, John Wiley & Sons, New York, USA.
21. Offutt A.J., Untch R.H. Uniting the Orthogonal, Mutation 2000 2000, Kluwer Academic, pp. 34-44.
22. Roach P.J. Verification and Validation in Computational Science and Engineering 1998, Hermosa Publishers, New Mexico, USA.
23. Stevenson D.E. A critical look at quality in large-scale simulations. IEEE Computing in Science and Engineering 1999, (May/June):53-63.
24. Weyuker E. On testing non-testable programs. The Computer Journal 1982, 25(4):65-70.