Assessing climate model software quality: A defect density analysis of three models

Geoscientific Model Development

Volumn 5, Issue 4, 2012, Pages 1009-1022

  Pipitone, J ^a   Easterbrook, S ^a  

^a UNIVERSITY OF TORONTO   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

CLIMATE MODELING; DATA ACQUISITION; ERROR ANALYSIS; ERROR CORRECTION; GLOBAL CLIMATE; MODEL TEST; QUALITATIVE ANALYSIS; RELIABILITY ANALYSIS; SOFTWARE;

EID: 84865182146     PISSN: 1991959X     EISSN: 19919603     Source Type: Journal    
DOI: 10.5194/gmd-5-1009-2012     Document Type: Article

References (30)

1. Aranda, J. and Venolia, G.: The secret life of bugs: Going past the errors and omissions in software repositories, in: ICSE '09: Proceedings of the 31st International Conference on Software Engineering, 298-308, IEEE Computer Society, Washington, DC, USA, doi:10.1109/ICSE.2009.5070530, 2009.
2. Basili, V. R., Carver, J. C., Cruzes, D., Hochstein, L. M., Hollingsworth, J. K., Shull, F., and Zelkowitz, M. V.: Understanding the High-Performance-Computing Community: A Software Engineer's Perspective, Software, IEEE, 25, 29-36, doi:10.1109/MS.2008.103, 2008.
3. Brown, N., Ozkaya, I., Sangwan, R., Seaman, C., Sullivan, K., Zazworka, N., Cai, Y., Guo, Y., Kazman, R., Kim, M., Kruchten, P., Lim, E., MacCormack, A., and Nord, R.: Managing technical debt in software-reliant systems, Proceedings of the FSE/SDP workshop on Future of software engineering research - FoSER '10, 47 pp., doi:10.1145/1882362.1882373, 2010.
4. Carver, J. C., Kendall, R. P., Squires, S. E., and Post, D. E.: Software Development Environments for Scientific and Engineering Software: A Series of Case Studies, in: ICSE '07: Proceedings of the 29th International Conference on Software Engineering, 550-559, IEEE Computer Society, Washington, DC, USA, doi:10.1109/ICSE.2007.77, 2007. (Pubitemid 47430689)
5. Chulani, S. and Boehm, B.: Modeling Software Defect Introduction and Removal: COQUALMO (COnstructive QUALity MOdel), Tech. rep., Center for Software Engineering, USC-CSE-99-510, 1999.
6. Easterbrook, S. M. and Johns, T. C.: Engineering the Software for Understanding Climate Change, Comput. Sci. Eng., 11, 65-74, doi:10.1109/MCSE. 2009.193, 2009.
7. Fink, A.: How to Conduct Surveys: A Step-by-Step Guide, Sage Publications, Inc, fourth edition edn., 2008.
8. Hatton, L.: Safer C: Developing Software for in High-Integrity and Safety-Critical Systems, McGraw-Hill, Inc., New York, NY, USA, 1995.
9. Hatton, L.: N-version design versus one good version, IEEE Software, 14, 71-76, doi:10.1109/52.636672, 1997a. (Pubitemid 127576294)
10. Hatton, L.: The T experiments: errors in scientific software, IEEE Comput. Sci. Eng., 4, 27-38, doi:10.1109/99.609829, 1997b.
11. Hook, D.: Using Code Mutation to Study Code Faults in Scientific Software, Master's thesis, Queen's University, 2009.
12. Hook, D. and Kelly, D.: Testing for trustworthiness in scientific software, in: SECSE '09: Proceedings of the 2009 ICSE Workshop on Software Engineering for Computational Science and Engineering, pp. 59-64, IEEE Computer Society, Washington, DC, USA, doi:10.1109/SECSE.2009.5069163, 2009.
13. IEEE: IEEE Standard Glossary of Software Engineering Terminology, Tech. rep., doi:10.1109/IEEESTD.1990.101064, 1990.
14. IEEE: IEEE standard for a software quality metrics methodology, Tech. rep., doi:10.1109/IEEESTD.1998.243394, 1998.
15. ISO: ISO/IEC, 9126-1:2001(E) Software engineering - Product quality, Tech. rep., http://www.iso.org/iso/iso-catalogue/catalogue-tc/catalogue-detail. htm?csnumber=22749, Last accessed Feb, 2012, 2001.
16. Jones, C.: Applied Software Measurement: Global Analysis of Productivity and Quality, McGraw-Hill Osborne Media, 3 edn., 2008.
17. Kelly, D. and Sanders, R.: Assessing the quality of scientific software, in: First International Workshop on Software Engineering for Computational Science & Engineering, http://cs.ua.edu/~{}SECSE08/Papers/Kelly.pdf, Last accessed Feb, 2012, 2008.
18. Kelly, D. F.: A Software Chasm: Software Engineering and Scientific Computing, Software, IEEE, 24, 120-119, doi:10.1109/MS.2007.155, 2007. (Pubitemid 350153722)
19. Kitchenham, B. and Pfleeger, S. L.: Software quality: the elusive target special issues section, Software, IEEE, 13, 12-21, doi:10.1109/52.476281, 1996.
20. Li, M. N., Malaiya, Y. K., and Denton, J.: Estimating the Number of Defects: A Simple and Intuitive Approach, in: Proc. 7th Int'l Symposium on Software Reliability Engineering, ISSRE, 307- 315, available at: http://citeseer.ist.psu.edu/viewdoc/summary? doi=10.1.1.37.5000 (last access: 27 July 2012), 1998.
21. McGuffie, K. and Henderson-Sellers, A.: A Climate Modelling Primer (Research & Developments in Climate & Climatology), John Wiley & Sons, 2nd Edn., 2005.
22. Park, R. E.: Software Size Measurement: A Framework for Counting Source Statements, Tech. rep., Software Engineering Institute, 1992.
23. Pfleeger, S. L. and Hatton, L.: Investigating the influence of formal methods, Computer, 30, 33-43, doi:10.1109/2.566148, 1997. (Pubitemid 127556434)
24. Segal, J.: Models of Scientific Software Development, in: Proc. 2008 Workshop Software Eng. in Computational Science and Eng. (SECSE 08), http://oro.open.ac.uk/17673/, 2008.
25. Segal, J. and Morris, C.: Developing Scientific Software, IEEE Software, 25, 18-20, doi:10.1109/MS.2008.85, 2008.
26. Shackley, S., Young, P., Parkinson, S., and Wynne, B.: Uncertainty, Complexity and Concepts of Good Science in Climate Change Modelling: Are GCMs the Best Tools?, Climatic Change, 38, 159-205, doi:10.1023/A:1005310109968, 1998. (Pubitemid 28139946)
27. Solomon, S., Qin, D., Manning, M., Chen, Z., Marquis, M., Averyt, K. B., Tignor, M., and Miller, H. L. (Eds.): Climate Change 2007 - The Physical Science Basis: Working Group I Contribution to the Fourth Assessment Report of the IPCC, Cambridge University Press, Cambridge, UK and New York, NY, USA, 2007.
28. Stevenson, D. E.: A critical look at quality in largescale simulations, Comput. Science Eng., 1, 53-63, doi:10.1109/5992.764216, 1999. (Pubitemid 129600439)
29. van Vliet, H.: Software Engineering: Principles and Practice, 2nd Edn., Wiley, 2000.
30. Zimmermann, T., Premraj, R., and Zeller, A.: Predicting Defects for Eclipse, in: PROMISE '07: Proceedings of the Third International Workshop on Predictor Models in Software Engineering, pp. 9+, IEEE Computer Society, Washington, DC, USA, doi:10.1109/PROMISE.2007.10, 2007. (Pubitemid 350240055)