Software quality evaluation for security cots products

International Journal of Software Engineering and Knowledge Engineering

Volumn 20, Issue 1, 2010, Pages 27-48

  Villalba, M Teresa ^a   Fernández Sanz, Luis ^b   Cuadrado Gallego, Juan J ^b   Martínez, Jose J ^b  

^a UNIVERSIDAD EUROPEA DE MADRID   (Spain)

^b UNIVERSITY OF ALCALÁ   (Spain)

Author keywords

Commercial products; COTS; Security software; Software quality

Indexed keywords

COMMERCIAL PRODUCTS; COMMERCIAL SOFTWARE PRODUCTS; QUALITY CHARACTERISTIC; SECURITY EXPERTS; SECURITY SOFTWARE; SOFTWARE QUALITY; SOFTWARE QUALITY EVALUATION; SOFTWARE QUALITY MODELS; STATISTICAL ANALYSIS; SYSTEMATIC REVIEW;

QUALITY CONTROL;

COMPUTER SOFTWARE SELECTION AND EVALUATION;

EID: 77952265953     PISSN: 02181940     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0218194010004633     Document Type: Article

References (61)

1. ISO 15408-1, ISO/IEC 15408-15411:2005. Information technology - Security techniques - Evaluation criteria for IT security - Part 1: Introduction and general model, International Standards Organization, Geneva, 2005.
2. Common Criteria, Common Methodology for Information Technology Security Evaluation (version 2.2, revision 311), NOR-004v1, 2004. Available from: http://www.commoncriteriaportal.org.
3. ISO 9126-9131, ISO/IEC 9126-1 Software engineering - Product quality. Part 1: Quality model, International Standards Organization, 2001.
4. B. Kitchenham and S. L. Pfleeger, Software quality: The elusive target, IEEE Softw. 13 (1996) 12-21 .
5. R. G. Dromey, Cornering the Chimera, IEEE Softw. 13 (1996) 33-43.
6. F. Ye and T. Kelly, COTS product selection for safety-critical systems, in COTS-Based Software Systems, California, USA, 2004, pp. 53-62.
7. European Cooperation for Space Standarization (ECSS), Space product assurance. Methods and techniques to support the assessment of software dependability and safety, R. A. S. Division, ESA Publications Division, 2006, p. 122.
8. S. Antonia and X. Michalis, E-commerce system quality assessment using a model based on ISO 9126 and Belief Networks, Software Quality Control 16 (2008) 107-129.
9. L. Olsina and G. A. Rossi, Quantitative method for quality evaluation of web sites and applications, IEEE Multimedia 9 (2002) 20-29.
10. J. P. Carvallo, X. Franch and C. Quer, Defining a quality model for mail servers, in Proceedings of the Second International Conference on COTS-Based Software Systems (Springer-Verlag, 2003).
11. X. B. Illa, X. Franch and J. A. Pastor, Formalising ERP selection criteria, in Proceedings of the 10th International Workshop on Software Specification and Design, IEEE Computer Society, 2000.
12. S. E. Y. Zhang, L. Cranor and J. Hong, Evaluating anti-phishing tools, in Proceedings of the 14th Annual Network & Distributed System Security Symposium, San Diego, CA, 28th Feb.-2 Mar. 2007.
13. R. Chiong and S. Dhakal, On the insecurity of personal firewall, in Proceedings of the 3rd International Symposium on Information Technology, Kuala Lumpur, Malaysia, 2008, pp. 2301-2310.
14. H. Olivier and J. Nathalie, A feature selection and evaluation scheme for computer virus detection, in Proceedings of the Sixth International Conference on Data Mining, 2006, pp. 891-895.
15. R. Durst, T. Champion, B.Witten, E. Miller and L. Spagnuolo, Testing and evaluating computer intrusion detection systems, Communications of ACM 42 (1999) 53-61.
16. A. S. Jadhav and R. M. Sonar, Evaluating and selecting software packages: A review, Information and Software Technology 51 (2009) 555-563.
17. X. Franch and J. P. Carvallo, Using quality models in software package selection, IEEE Software 20 (2003) 34-41.
18. G. Malak, L. Badri, M. Badri and H. Sahraoui, Towards a multidimensional model for web-based applications quality assessment, in E-Commerce and Web Technologies 3182 (2004) Computer Science Journal, 316-327.
19. B. Behkamal, M. Kahani and M. K. Akbari, Customizing ISO 9126 quality model for evaluation of B2B applications, Elsevier Information and Software Technology Journal, in Press.
20. F. Losavio, L. Chirinos, A. Matteo, N. Ĺevy and A. Ramdane-Cherif, ISO quality standards for measuring architectures, Journal of Systems and Software 72 (2004) 209-223.
21. L. Olsina and G. Rossi, Measuring web application quality with WebQEM, IEEE MultiMedia 9 (2002) 20-29.
22. M. T. Villalba and L. Ferńandez-Sanz, Technical report - Evaluation report of Cryptosec 2048, e.Security, European Security 13 (2007) 78-81.
23. M. T. Villalba and L. Ferńandez-Sanz, Technical report - Evaluation report of Internet Security and Acceleration (ISA) Server 2006, e.Security, European Security 14 (2007) 78-81
24. D. Kunda, STACE: Social technical approach to COTS software evaluation, in Component-Based Software Quality, LNCS 2693. Vol.2693, A. Cechich, M. Piattini and A. Vallecillo, Eds.: Springer-Verlag, 2003, pp. 64-84.
25. T. Marco, J. Letizia, S. Carl-Fredrik Rensen and W. Alf Inge, COTS products characterization, in Proceedings of the 14th International Conference on Software Engineering and Knowledge Engineering, Ischia, Italy, 2002, pp. 335-338.
26. D. Kunda and L. Brooks, Identifying and classifying processes (traditional and soft factors) that suuport COTS component selection: A case study, European Journal of Information Systems 9 (2000) 226-234.
27. ISO 15408-2, ISO/IEC 15408-15412:2005. Information technology - Security techniques - Evaluation criteria for IT security - Part 2: Security functional requirements, first ed. International Standards Organization, Geneva, 2005.
28. J. Kontio, G. Caldiera and V. R. Basili, Defining factors, goals and criteria for reusable component evaluation, in Proceedings of the 1996 conference of the Centre for Advanced Studies on Collaborative Research, Toronto, Ontario, Canada, 1996, pp. 21-32.
29. M. Ochs, D. Pfahl, G. Chrobok-Diening and B. Nothhelfer-Kolb, A method for efficient measurement-based COTS assessment and selection - Method description and evaluation results, in Proceedings of the 7th IEEE International Software Metrics Symposium, 2001, pp. 285-296.
30. S. Comella-Dorda, J. C. Dean, E. Morris and P. Oberndorf, A process for COTS software product evaluation, in Proceedings of the First International Conference on COTS-Based Software Systems, LNCS 2255, Orlando, USA, 2002, pp. 86-96.
31. P. K. Lawlis, K. E. Mark, D. A. Thomas and T. Courtheyn, A formal process for evaluating COTS software products, IEEE Computer 34 (2001) 58-63.
32. J. P. Carvallo, X. Franch and C. Quer, Managing non-technical requirements in COTS components selection, in Proceedings of the 14th IEEE International Requirements Engineering Conference, 2006, pp. 316-321.
33. N. Fenton and S. L. Pfleeger, Software Metrics: A Rigorous and Practical Approach, 2nd ed. (PWS Publishing Co., USA, Boston, MA, 1997).
34. P. A. Oberndorf, L. Brownsword, E. Morris and C. Sledge, Workshop on COTS-Based Systems, Software Engineering Institute, Carnegie Mellon University, CMU/SEI-97-SR-019, Nov. 1997.
35. S. Comella-Dorda, J. Dean, G. Lewis, E. Morris, P. Oberndorf and E. Harper, A Process for COTS Software Product Evaluation, Carnegie Mellon, Software Engineering Institute, Pittsburgh, CMU/SEI-2003-TR-017 ESC-TR-2003-2017, 2004.
36. S. J. Koyanl, R.W. N. Balley, A. J. R., S., B. C. K. Mulligan and M. Tolson, Research-Based Web Design & Usability Guidelines, U.S. Dept. of Health and Human Services. http://www.usability.gov/pdfs/guidelines.html, 2006.
37. J. Kirakowski and M. Corbett, SUMI: The software usability measurement inventory, British Journal of Educational Technology 24 (1993) 210-212.
38. G. Gediga, K.-C. Hamborg and I. Düntsch, The isometrics usability inventory. An operationalisation of ISO 9241-10 supporting summative and formative evaluation of software systems, Behaviour and Information Technology 18 (1999) 151-164.
39. ISO 9126-2, ISO/IEC 9126-9132:2003 - Software engineering - Product quality - Part 2: External metrics, Geneva: International Standards Organization, 2003.
40. ISO 9241-110, ISO/IEC 9241-10110 Ergonomics of human-system interaction, Part 110: Dialogue principles, International Standards Organization, Geneva, 2006.
41. ISO 9241-11, ISO 9241-9311 Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11: Guidance on usability, International Standards Organization, Geneva, 1998.
42. ISO 25051, ISO/IEC 25051 Software engineering-Software product Quality Requirements and Evaluation (SQuaRE) - Requirements for quality of Commercial Off-The-Shelf (COTS) software product and instructions for testing, First ed International Standards Organization, Geneva, 2006.
43. IEEE, IEEE Std. 1061 - 1998. Standard for a software quality metrics methodology, IEEE Computer Society, 1998.
44. M. E. A. Obeso, Metodoloǵia de Medicíony Evaluacíon de la Usabilidad en Sitios Web Educativos, in Departamento de Inforḿatica, Oviedo, Universidad de Oviedo, 2005.
45. L. S. V. Ṕerez and A. F. G. Torńes, MECHDAV - A quality model for the technical evaluation of applications development tools in visual enviroments, in Software Measurement European Forum, Rome, Italy, 2005, pp. 155-163.
46. A. Avizienis, J.-C. Laprie, B. Randell and C. Landwehr, Basic concepts and taxonomy of dependable and secure computing, IEEE Transactions on Dependable and Secure Computing 1 (2004) 11-33.
47. M. Bertoa and A. Vallecillo, Atributos de calidad para componentes COTS, in Proceedings of IDEAS'02, La Habana, Cuba, 2002, pp. 352-363.
48. M. Morisio and M. Torchiano, Definition and classification of COTS: A proposal, in ICCBSS '02: Proceedings of the First International Conference on COTS-Based Software Systems, London, UK, 2002, pp. 165-175.
49. J. P. Carvallo and X. Franch, Extending the ISO/IEC 9126-1 quality model with nontechnical factors for COTS components selection, in Proceedings of the 2006 International Workshop on Software Quality, Shanghai, China, 2006, pp. 9-14.
50. N. I. o. S. a. T. C. S. Division., Security Requirements for Cryptographic modules, Gaithersburg, MD, NIST, 2001.
51. ISO 14598-1, ISO/IEC 14598-14601 Information Technology - Software Product Evaluation - Part 1: General Overview, International Standards Organization, Geneva, 1999.
52. ISO, ISO/IEC 15939 Systems and software engineering-Measurement process, International Standards Organization, Geneva, 2007.
53. T. M. Khoshgoftaar, E. Allen, A. Naik, W. D. Jones and J. P. Hudepohl, Using classification trees for software quality models: Lessons learned, International Journal of Software Engineering and Knowledge Engineering 9 (1999) 217-231. (Pubitemid 129667602)
54. C. M. Lott, Measurement support in software engineering environments, International Journal of Software Engineering and Knowledge Engineering 4 (1994) 409-426.
55. M. J. J. Hall, R. Hall and J. Zeleznikow, A process for evaluating legal knowledgebased systems based upon the context criteria contingency-guidelines framework, in Proceedings of the 9th International Conference on Artificial Intelligence and Law ACM, UK, 2003.
56. W. Stefan and D. Florian, An integrated approach to quality modeling, in Proceedings of the 5th International Workshop on Software Quality, 2007, pp. 1-6.
57. J. F. Hair, R. L. Tatham, R. E. Anderson and W. Black, Multivariate Data Analysis, 5th ed. (Prentice Hall, 1998).
58. B. G. Tabachnick and L. S. Fidell, Using Multivariate Statistics, 5th ed. (Pearson Education, 2006).
59. L. Hu and P. M. Bentler, Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives, Structural Equation Modeling 6 (1999) 1-55.
60. M. T. Villalba and L. Ferńandez-Sanz, Technical report - Evaluation report of Internet Application Gateway (IAG) 2007, e.Security, European Security 18 (2008) 53-57.
61. M. T. Villalba and L. Ferńandez-Sanz, Technical report - Evaluation report of SonicWall Firewall, e.Security, European Security 24 (2009) 77-81.