Evaluating software product quality: A systematic mapping study

Proceedings - 2014 Joint Conference of the International Workshop on Software Measurement, IWSM 2014 and the International Conference on Software Process and Product Measurement, Mensura 2014

Volumn , Issue , 2014, Pages 141-151

  Ouhbi, Sofia ^a   Idri, Ali ^a   Aleman, Jose Luis Fernandez ^b   Toval, Ambrosio ^b  

^a Mohammed V University in Rabat   (Morocco)

^b UNIVERSITY OF MURCIA   (Spain)

Author keywords

[No Author keywords available]

Indexed keywords

CLASSIFICATION (OF INFORMATION); DATA MINING; MAPPING; OPEN SOURCE SOFTWARE; QUALITY CONTROL; SOFTWARE QUALITY;

CLASSIFICATION CRITERION; DOMAIN SPECIFIC; EMPIRICAL EVALUATIONS; EVALUATING SOFTWARE; OPEN SOURCE SOFTWARE PROJECTS; QUALITY OF SOFTWARES; SOURCE CODES; SYSTEMATIC MAPPING STUDIES;

OPEN SYSTEMS;

EID: 84924054129     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IWSM.Mensura.2014.30     Document Type: Conference Paper

References (109)

1. B. Kitchenham and S. L. Pfleeger, "Software quality: The elusive target," IEEE Softw., vol. 13, no. 1, pp. 12-21, Jan. 1996.
2. ISO/IEC Standard. ISO-9126 Software Product Evaluation -Quality Characteristics and Guidelines for Their Use, ISO/IEC Std., 1999.
3. (2009) Standish-Group, CHAOS summary. [Online]. Available: http://blog.standishgroup.com/pmresearch
4. J. A. McCall, Quality Factors. John Wiley &Sons, Inc., 2002.
5. B. W. Boehm, J. R. Brown, H. Kaspar, and M. Lipow, Characteristics of software quality, ser. TRW Softw. Technol. Amsterdam: North-Holland, 1978.
6. R. G. Dromey, "Cornering the Chimera," IEEE Softw., vol. 13, no. 1, pp. 33-43, 1996.
7. ISO/IEC 25010 standard. Systems and software engineering -Systems and software Quality Requirements and Evaluation (SQuaRE) -System and software quality models., ISO/IEC Std., 2011.
8. M. Ortega, M. Pérez, and T. Rojas, "Construction of a systemic quality model for evaluating a software product," Software Quality Control, vol. 11, no. 3, pp. 219-242, Jul. 2003.
9. W. Leopairote, A. Surarerks, and N. Prompoon, "Evaluating software quality in use using user reviews mining," in Computer Science and Software Engineering (JCSSE), 2013 10th International Joint Conference on. IEEE, 2013, pp. 257-262.
10. A. Ampatzoglou, G. Frantzeskou, and I. Stamelos, "A methodology to assess the impact of design patterns on software quality," Inf. Softw. Technol., vol. 54, no. 4, pp. 331-346, Apr. 2012.
11. S. Zhong, T. M. Khoshgoftaar, and N. Seliya, "Analyzing software measurement data with clustering techniques," IEEE Intelligent Systems, vol. 19, no. 2, pp. 20-27, Mar. 2004.
12. P. Brereton, B. A. Kitchenham, D. Budgen, M. Turner, and M. Khalil, "Lessons from applying the systematic literature review process within the software engineering domain," J. Syst. Softw., vol. 80, no. 4, pp. 571-583, Apr. 2007.
13. K. Petersen, R. Feldt, S. Mujtaba, and M. Mattsson, "Systematic mapping studies in software engineering," in 12th International Conference on Evaluation and Assessment in Software Engineering. Bari, Italy: Blekinge Institute of Technology, 2008, pp. 71-80.
14. L. M. Karg, M. Grottke, and A. Beckhaus, "A systematic literature review of software quality cost research," Journal of Systems and Software, vol. 84, no. 3, pp. 415-427, Mar. 2011.
15. F. Elberzhager, J. Münch, and V. T. N. Nha, "A systematic mapping study on the combination of static and dynamic quality assurance techniques," Information and Software Technology, vol. 54, no. 1, pp. 1-15, Jan. 2012.
16. J. Landis and G. Koch, "The measurement of observer agreement for categorical data," Biometrics, vol. 33, pp. 159-174, 1977.
17. J. Wen, S. Li, Z. Lin, Y. Hu, and C. Huang, "Systematic literature review of machine learning based software development effort estimation models," Inf. Softw. Technol., vol. 54, no. 1, pp. 41-59, Jan. 2012.
18. M. Jorgensen and M. Shepperd, "A systematic review of software development cost estimation studies," IEEE Trans. Softw. Eng., vol. 33, no. 1, pp. 33-53, Jan. 2007.
19. S. Ouhbi, A. Idri, J. L. Fernández-Alemán, and A. Toval, "Requirements engineering education: A systematic mapping study," Requirements Engineering, pp. 1-20, 2013.
20. (2013) NASA Metrics Data Program (MDP) Repository. [Online]. Available: http://mdp.ivv.nasa.gov
21. ISBSG, "International Software Benchmarking Standards Group," 2013. [Online]. Available: http://www.isbsg.org
22. T. Menzies, B. Caglayan, E. Kocaguneli, J. Krall, F. Peters, and B. Turhan, "The PROMISE Repository of empirical software engineering data," 2012. [Online]. Available: http://promisedata.googlecode.com
23. "Guide to the Software Quality Body of Knowledge (SQuBOK)," JUSE: The Union of Japanese Scientists and Engineers, Tech. Rep., 2007.
24. A. Abran and J. W. a. Moore, Guide to the software engineering body of knowledge (SWEBOK), C. Los Alamitos, Ed. IEEE Computer Society, 2004. [Online]. Available: http://www.swebok.org
25. X. GENG, L. YANG, and X. HUANG, "A software quality evaluation method based on the principle of maximum coordination and subordination." Journal of Theoretical &Applied Information Technology, vol. 47, no. 1, 2013.
26. V. Pasrija and P. R. Srivastava, "Evaluation of software quality using choquet integral approach," International Journal of Fuzzy System Applications (IJFSA), vol. 3, no. 1, pp. 51-81, 2013.
27. O. Pomorova and T. Hovorushchenko, "Artificial neural network for software quality evaluation based on the metric analysis," in East-West Design &Test Symposium, 2013. IEEE, 2013, pp. 1-4.
28. M. HAJDU-MACELARU, "Software quality evaluation using fuzzy theory1," Scientific Studies &Research. Series Mathematics &Informatics, vol. 23, no. 1, 2013.
29. X. Wang, M. Ceberio, S. Virani, A. Garcia, and J. Cummins, "A hybrid algorithm to extract fuzzy measures for software quality assessment," Journal of Uncertain Systems, vol. 7, no. 3, pp. 219-237, 2013.
30. L. Yu and S. Ramaswamy, "Using bug report as a software quality measure: An empirical study," International Journal of Information Quality, vol. 3, no. 2, pp. 164-183, 2013.
31. P. Tomas, M. Escalona, and M. Mejias, "Open source tools for measuring the internal quality of java software products. a survey," Computer Standards &Interfaces, vol. 36, no. 1, pp. 244-255, 2013.
32. G. Singh, "Metrics for measuring the quality of objectoriented software," SIGSOFT Softw. Eng. Notes, vol. 38, no. 5, pp. 1-5, Aug. 2013. [Online]. Available: http://doi.acm.org/10.1145/2507288.2507311
33. M.-C. Lee and T. Chang, "Software measurement and software metrics in software quality," International Journal of Software Engineering &Its Applications, vol. 7, no. 4, 2013.
34. C. Otieno, W. Mwangi, and S. Kimani, "Framework to assess software quality in erp systems," in Scientific Conference Proceedings, 2013.
35. Y. Zou, C. Liu, Y. Jin, and B. Xie, "Assessing software quality through web comment search and analysis," in Safe and Secure Software Reuse. Springer, 2013, pp. 208-223.
36. N. Baliyan and S. Kumar, "Quality assessment of software as a service on cloud using fuzzy logic," in IEEE International Conference on Cloud Computing in Emerging Markets (CCEM). IEEE, 2013, pp. 1-6.
37. N. R. Carvalho, A. Simões, and J. J. Almeida, "Open source software documentation mining for quality assessment," in Advances in Information Systems and Technologies. Springer, 2013, pp. 785-794.
38. S. Rehman and S. U. Khan, "Software quality assessment model for global software development," in e-Proceeding of Software Engineering Postgraduates Workshop (SEPoW), 2013, p. 104.
39. K. Lochmann, J. Ramadani, and S. Wagner, "Are comprehensive quality models necessary for evaluating software quality?" in Proceedings of the 9th International Conference on Predictive Models in Software Engineering. ACM, 2013, p. 3.
40. T. A. Alrawashdeh, M. Muhairat, and A. Althunibat, "Evaluating the quality of software in erp systems using the iso 9126 model," International Journal of Ambient Systems and Applications (IJASA), 2013.
41. P. Kumar, R. Yadav, and D. Bansal, "Usability and evaluation of software quality using software metrics," International Journal of Computer Applications, vol. 71, no. 3, pp. 9-14, 2013.
42. D. Gade, "The evaluation of software quality," Master's thesis, University of Nebraska -Lincoln, 2013.
43. N. F. Duarte Filho, C. I. da Silva e Pádua, P. H. de Souza Bermejo, A. L. Zambalde, U. S. de Barros et al., "Saasquality-A method for quality evaluation of software as a service (saas)." International Journal of Computer Science &Information Technology, vol. 5, no. 3, 2013.
44. B. Jayasingh, S. D. Virivinti, and N. Pranav, "Expert rating based software quality evaluation," International Journal of Engineering, vol. 2, no. 10, 2013.
45. P. Dhiman, Manish, and R. Chawla, "A clustered approach to analyze the software quality using software defects," Advanced Computing &Communication Technologies, International Conference on, pp. 36-40, 2012.
46. S. HABIBI, "Evaluating the quality of a software application for web services integration," Master's thesis, University of Gothenburg, 2012.
47. J. G and C. V. Babu, "An integrated approach for measuring software quality and code readability," vol. 1, no. 2, 2011.
48. S. Wagner, "The use of application scanners in software product quality assessment," in Proceedings of the 8th international workshop on Software quality, ser. WoSQ '11. New York, NY, USA: ACM, 2011, pp. 42-49.
49. F. A. Fontana and S. Maggioni, "Metrics and antipatterns for software quality evaluation," in 34th IEEE Software Engineering Workshop, 2011, pp. 48-56.
50. B. Vanderose and N. Habra, "Tool-support for a model-centric quality assessment: QuaTALOG," Joint Conference of the International Workshop on Software Measurement and the International Conference on Software Process and Product Measurement, pp. 263-268, 2011.
51. I. Turnu, G. Concas, M. Marchesi, and R. Tonelli, "The fractal dimension metric and its use to assess object-oriented software quality," in Proceedings of the 2nd International Workshop on Emerging Trends in Software Metrics, ser. WETSoM '11. New York, NY, USA: ACM, 2011, pp. 69-74.
52. Y. Liu, T. M. Khoshgoftaar, and N. Seliya, "Evolutionary optimization of software quality modeling with multiple repositories," IEEE Trans. Softw. Eng., vol. 36, no. 6, pp. 852-864, Nov. 2010.
53. M. V. V. Saradhi and B. R. Sastry, "ESPQ: A new object oriented design metric for software quality measurement," International Journal of Engineering Science and Technology, 2010.
54. A. K. Groven, K. Haaland, R. Glott, and A. Tannenberg, "Security measurements within the framework of quality assessment models for free/libre open source software," in Proceedings of the Fourth European Conference on Software Architecture: Companion Volume, ser. ECSA '10. New York, NY, USA: ACM, 2010, pp. 229-235.
55. T. M. Khoshgoftaar, Y. Xiao, and K. Gao, "Software quality assessment using a multi-strategy classifier," Information Sciences, p. in press, 2010.
56. S. Wagner, "A bayesian network approach to assess and predict software quality using activity-based quality models," Information and Software Technology, vol. 52, no. 11, pp. 1230-1241, 2010.
57. G. Gousios and D. Spinellis, "Alitheia core: An extensible software quality monitoring platform," in Proceedings of the 31st International Conference on Software Engineering, ser. ICSE '09. Washington, DC, USA: IEEE Computer Society, 2009, pp. 579-582.
58. D. Spinellis, G. Gousios, V. Karakoidas, P. Louridas, P. J. Adams, I. Samoladas, and I. Stamelos, "Evaluating the quality of open source software," Electron. Notes Theor. Comput. Sci., vol. 233, pp. 5-28, Mar. 2009.
59. M. R. Marri, S. Thummalapenta, and T. Xie, "Improving software quality via code searching and mining," in Proceedings of the 2009 ICSE Workshop on Search-Driven Development-Users, Infrastructure, Tools and Evaluation, ser. SUITE '09. Washington, DC, USA: IEEE Computer Society, 2009, pp. 33-36.
60. A. Kumar, P. S. Grover, and R. Kumar, "A quantitative evaluation of aspect-oriented software quality model (aosquamo)," SIGSOFT Softw. Eng. Notes, vol. 34, no. 5, pp. 1-9, Oct. 2009.
61. C.-W. Chang, C.-R. Wu, and H.-L. Lin, "Integrating fuzzy theory and hierarchy concepts to evaluate software quality," Software Quality Journal, vol. 16, no. 2, pp. 263-276, 2008.
62. N. Seliya and T. M. Khoshgoftaar, "Software quality analysis of unlabeled program modules with semisupervised clustering," Trans. Sys. Man Cyber. Part A, vol. 37, no. 2, pp. 201-211, Mar. 2007.
63. S. Roubtsov, A. Telea, and D. Holten, "Squavisit: A software quality assessment and visualisation toolset," in Proceedings of the Seventh IEEE International Working Conference on Source Code Analysis and Manipulation, ser. SCAM '07. Washington, DC, USA: IEEE Computer Society, 2007, pp. 155-156.
64. M. Shin, S. Ratanothayanon, A. L. Goel, and R. A. Paul, "Parsimonious classifiers for software quality assessment," in Proceedings of the 10th IEEE High Assurance Systems Engineering Symposium, ser. HASE '07. Washington, DC, USA: IEEE Computer Society, 2007, pp. 411-412.
65. L. Lin and H.-M. Lee, "A fuzzy software quality assessment model to evaluate user satisfaction," in Proceedings of the Second International Conference on Innovative Computing, Informatio and Control, ser. ICICIC '07. Washington, DC, USA: IEEE Computer Society, 2007, p. 438.
66. K. Stroggylos and D. Spinellis, "Refactoring-does it improve software quality?" in Proceedings of the 5th International Workshop on Software Quality, ser. WoSQ '07. Washington, DC, USA: IEEE Computer Society, 2007, p. 10.
67. C.-B. Chen, C.-T. Lin, C.-H. Wang, and C.-W. Chang, "Model for measuring quality of software in DVRS using the gap concept and fuzzy schemes with ga," Inf. Softw. Technol., vol. 48, no. 3, pp. 187-203, Mar. 2006.
68. K. Lee and S. J. Lee, "A quantitative software quality evaluation model for the artifacts of component based development," in Proceedings of the Sixth International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing and First ACIS International Workshop on Self-Assembling Wireless Networks, ser. SNPD-SAWN '05. Washington, DC, USA: IEEE Computer Society, 2005, pp. 20-25.
69. H.-W. Jung, S.-G. Kim, and C.-S. Chung, "Measuring software product quality: A survey of ISO/IEC 9126," IEEE Softw., vol. 21, no. 5, pp. 88-92, Sep. 2004.
70. N. F. Schneidewind, "Body of knowledge for software quality measurement," Computer, vol. 35, no. 2, pp. 77-83, Feb. 2002.
71. S. S. So, S. D. Cha, and Y. R. Kwon, "Empirical evaluation of a fuzzy logic-based software quality prediction model," Fuzzy Sets Syst., vol. 127, no. 2, pp. 199-208, Apr. 2002.
72. L. C. Briand, J. Wüst, J. W. Daly, and D. V. Porter, "Exploring the relationship between design measures and software quality in objectoriented systems," J. Syst. Softw., vol. 51, no. 3, pp. 245-273, May 2000.
73. E. J. Weyuker, "Evaluation techniques for improving the quality of very large software systems in a cost-effective way," J. Syst. Softw., vol. 47, no. 2-3, pp. 97-103, Jul. 1999.
74. X. F. Liu, "A quantitative approach for assessing the priorities of software quality requirements," J. Syst. Softw., vol. 42, no. 2, pp. 105-113, Aug. 1998.
75. F. Lanubile and G. Visaggio, "Evaluating predictive quality models derived from software measures: lessons learned," J. Syst. Softw., vol. 38, no. 3, pp. 225-234, Sep. 1997.
76. R. Takahashi, "Software quality classification model based on Mc-Cabe's complexity measures," J. Syst. Softw., vol. 38, no. 1, pp. 61-69, Jul. 1997.
77. R. Gulezian, "Software quality measurement and modeling, maturity, control and improvement," in Proceedings of the 2nd IEEE Software Engineering Standards Symposium, ser. ISESS '95. Washington, DC, USA: IEEE Computer Society, 1995, pp. 52-59.
78. C. Maroto and P. Tormos, "Project management: An evaluation of software quality," vol. 1, no. 2, pp. 209-221, 1994.
79. L. Radlinski, "An expert-driven bayesian network model for simulating and predicting software quality," in eKNOW 2013, The Fifth International Conference on Information, Process, and Knowledge Management, 2013, pp. 26-31.
80. T. Xie, S. Thummalapenta, D. Lo, and C. Liu, "Data mining for software engineering," Computer, vol. 42, no. 8, pp. 55-62, Aug. 2009.
81. B. SOLEY, Richard Mark et CURTIS, "The Consortium for IT Software Quality (CISQ)," Software Quality. Increasing Value in Software and Systems Development, pp. 3-9, 2013.
82. M. R. Lyu, "Software reliability engineering: A roadmap," in 2007 Future of Software Engineering, ser. FOSE '07. Washington, DC, USA: IEEE Computer Society, 2007, pp. 153-170.
83. C. Seiffert, T. M. Khoshgoftaar, and J. Van Hulse, "Improving software-quality predictions with data sampling and boosting," Trans. Sys. Man Cyber. Part A, vol. 39, no. 6, pp. 1283-1294, Nov. 2009.
84. K. Gao, T. M. Khoshgoftaar, and H. Wang, "An empirical investigation of filter attribute selection techniques for software quality classification," in Proceedings of the 10th IEEE international conference on Information Reuse &Integration, ser. IRI'09. Piscataway, NJ, USA: IEEE Press, 2009, pp. 272-277.
85. S. Zhong, T. M. Khoshgoftaar, and N. Seliya, "Unsupervised learning for expert-based software quality estimation," in Proceedings of the Eighth IEEE international conference on High assurance systems engineering, ser. HASE'04. Washington, DC, USA: IEEE Computer Society, 2004, pp. 149-155.
86. J. HOWISON, M. CONKLIN, and K. CROWSTON, "FLOSSmole: A collaborative repository for FLOSS research data and analyses," vol. 1, no. 3, pp. 17-26, 2006. [Online]. Available: http://flossmole.org
87. I. Herraiz, D. Izquierdo-Cortazar, F. Rivas-Hernández, J. Gonzalez-Barahona, G. Robles, S. Duenas-Dominguez, C. Garcia-Campos, J. F. Gato, and L. Tovar, "Flossmetrics: Free/libre/open source software metrics," in 13th European Conference on Software Maintenance and Reengineering, 2009. CSMR'09. IEEE, 2009, pp. 281-284. [Online]. Available: http://flossmetrics.org
88. I. Keivanloo, C. Forbes, J. Rilling, and P. Charland, "Towards sharing source code facts using linked data," in Proceedings of the 3rd International Workshop on Search-Driven Development: Users, Infrastructure, Tools, and Evaluation, ser. SUITE '11. New York, NY, USA: ACM, 2011, pp. 25-28. [Online]. Available: http://www.secold.org
89. I. Keivanloo, C. Forbes, A. Hmood, M. Erfani, C. Neal, G. Peristerakis, and J. Rilling, "A linked data platform for mining software repositories," in 9th IEEE Working Conference on Mining Software Repositories (MSR). IEEE, 2012, pp. 32-35.
90. A. Ampatzoglou, S. Charalampidou, and I. Stamelos, "Research state of the art on GoF design patterns: A mapping study," Journal of Systems and Software, 2013.
91. B. A. Kitchenham, D. Budgen, and O. Pearl Brereton, "Using mapping studies as the basis for further research-A participantobserver case study," Information and Software Technology, vol. 53, no. 6, pp. 638-651, 2011.
92. V. Garousi, A. Mesbah, A. Betin-Can, and S. Mirshokraie, "A systematic mapping study of web application testing," Information and Software Technology, vol. 55, no. 8, pp. 1374-1396, 2013.
93. S. Easterbrook, J. Singer, M.-A. Storey, and D. Damian, "Selecting empirical methods for software engineering research," in Guide to advanced empirical software engineering. Springer, 2008, pp. 285-311.
94. J. Portillo-Rodríguez, A. Vizcaíno, M. Piattini, and S. Beecham, "Tools used in global software engineering: A systematic mapping review," Inf. Softw. Technol., vol. 54, no. 7, pp. 663-685, Jul. 2012.
95. C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, and A. Wesslén, Experimentation in software engineering: An introduction. Norwell, MA, USA: Kluwer Academic Publishers, 2000.
96. P. R. Mateo, M. P. Usaola, and J. L. Fernández Alemán, "Validating 2nd-order mutation at system level," IEEE Transactions on Software Engineering, vol. 39, pp. 570-587, 2012.
97. N. R. Draper and H. Smith, Applied Regression Analysis. New York, USA : Wiley, 1998.
98. J. A. Hartigan, Clustering Algorithms, 99th ed. New York, NY, USA: John Wiley &Sons, Inc., 1975.
99. Y. Wang and J. Vassileva, "Bayesian network-based trust model," in Proceedings of the 2003 IEEE/WIC International Conference on Web Intelligence, ser. WI '03. Washington, DC, USA: IEEE Computer Society, 2003, p. 372.
100. D. Heckerman, "A tutorial on learning with bayesian networks," in Innovations in Bayesian Networks, ser. Studies in Computational Intelligence, D. Holmes and L. Jain, Eds. Springer Berlin Heidelberg, 2008, vol. 156, pp. 33-82.
101. B. Yegnanarayana, Artificial Neural Networks. Prentice-Hall of India Pvt.Ltd, 2004.
102. J. Yen and R. Langari, Fuzzy logic: intelligence, control, and information. Upper Saddle River, NJ, USA: Prentice-Hall, Inc., 1999.
103. V. N. Vapnik, "An overview of statistical learning theory," Trans. Neur. Netw., vol. 10, no. 5, pp. 988-999, Sep. 1999.
104. D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning, 1st ed. Addison-Wesley Professional, 1989.
105. W. Banzhaf, P. Nordin, R. E. Keller, and F. D. Francone, Genetic programming: An introduction: on the automatic evolution of computer programs and its applications. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1998.
106. D. B. Leake, "Case-based reasoning," in Encyclopedia of Computer Science. Chichester, UK: John Wiley and Sons Ltd., 2003, pp. 196-197.
107. Z. Pawlak, "Rough sets," in Proceedings of the 1995 ACM 23rd annual conference on Computer science, ser. CSC '95. New York, NY, USA: ACM, 1995, pp. 262-264.
108. J. R. Quinlan, C4.5: programs for machine learning. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1993.
109. H. D. Margaret, "Data mining introductory and advanced topics," Pearsons Education Inc, 2003.