Assessing the effect of screen mockups on the comprehension of functional requirements

ACM Transactions on Software Engineering and Methodology

Volumn 24, Issue 1, 2014, Pages

  Ricca, Filippo ^a   Scanniello, Giuseppe ^b   Torchiano, Marco ^c   Reggio, Gianna ^a   Astesiano, Egidio ^a  

^a UNIVERSITY OF GENOA   (Italy)

^b UNIVERSITY OF BASILICATA   (Italy)

^c POLITECNICO DI TORINO   (Italy)

Author keywords

analysis models; controlled experiment; family of experiments; replicated experiments; Screen mockups; use cases

Indexed keywords

EFFICIENCY; SOFTWARE ENGINEERING; USER INTERFACES;

ANALYSIS MODELS; COMPREHENSION TASKS; CONTROLLED EXPERIMENT; DESKTOP APPLICATIONS; ENGINEERING COMMUNITY; FUNCTIONAL REQUIREMENT; REPLICATED EXPERIMENT; REQUIREMENTS DOCUMENT;

MOCKUPS;

EID: 84907980696     PISSN: 1049331X     EISSN: 15577392     Source Type: Journal    
DOI: 10.1145/2629457     Document Type: Article

References (71)

1. S. Abraho, C. Gravino, E. Insfran, G. Scanniello, and G. Tortora. 2013. Assessing the effectiveness of sequence diagrams in the comprehension of functional requirements: Results from a family of five experiments. IEEE Trans. Softw. Eng. 29, 3, 327-342.
2. S. Adolph, A. Cockburn, and P. Bramble. 2002. Patterns for Effective Use Cases. Addison-Wesley Longman Publishing Co., Inc., Boston, MA.
3. A. Agresti. 2007. An Introduction to Categorical Data Analysis. Wiley-Interscience.
4. B. Anda, D. I. K. Sjoberg, and M. Jorgensen. 2001. Quality and understandability of use case models. In Proceedings of the European Conference on Object-Oriented Programming. Berlin, Springer-Verlag, 402-428.
5. G. Andrew and P. Drew. 2009. Use case diagrams in support of use case modeling: Deriving understanding from the picture. J. Datab. Manage. 20, 1, 1-24.
6. J. Aranda, N. Ernst, J. Horkoff, and S. Easterbrook. 2007. A framework for empirical evaluation of model comprehensibility. In Proceedings of the ICSE Workshop on Modeling in Software Engineering. IEEE, 7-13.
7. E. Astesiano, M. Cerioli, G. Reggio, and F. Ricca. 2007. Phased highly-interactive approach to teaching UML-based software development. In Proceedings of the Symposium at MODELS 2007. 9-19.
8. E. Astesiano and G. Reggio. 2012. A disciplined style for use case based requirement specification. Tech. Rep. DISI-TR-12-04. DISI University of Genova, Italy. http://softeng.disi.unige.it/TR/Disciplined Requirements.pdf.
9. V. Basili, G. Caldiera, and D. H. Rombach. 1994. The Goal Question Metric Paradigm, Encyclopedia of Software Engineering. Wiley.
10. V. Basili, F. Shull, and F. Lanubile. 1999. Building knowledge through families of experiments. IEEE Trans. Softw. Eng. IEEE, 456-473.
11. L. Bratthall and C. Wohlin. 2002. Is it possible to decorate graphical software design and architecture models with qualitative information - An Experiment. IEEE Trans. Softw. Eng. 28, 12, 1181-1193.
12. B. Bruegge and A. H. Dutoit. 2003. Object-Oriented Software Engineering: Using UML, Patterns and Java, 2nd Ed. Prentice-Hall.
13. J. Carver, L. Jaccheri, S. Morasca, and F. Shull. 2003. Issues in using students in empirical studies in software engineering education. In Proceedings of the International Symposium on Software Metrics. IEEE Computer Society, Los Alamitos, CA, 239-250.
14. M. Ceccato, P. Di Penta, P. Falcarin, F. Ricca, M. Torchiano, and P. Tonella. 2014. A family of experiments to assess the effectiveness and efficiency of source code obfuscation techniques. Empir. Softw. Eng. 19, 4, 1040-1074.
15. M. Ciolkowski, D. Muthig, and J. Rech. 2004. Using academic courses for empirical validation of software development processes. In Proceedings of the EUROMICRO Conference. IEEE Computer Society, Los Alamitos, CA, 354-361.
16. N. Cliff. 1993. Dominance statistics: Ordinal analyses to answer ordinal questions. Psychol. Bull. 114, 3, 494-509.
17. A. Cockburn. 2000. Writing Effective Use Cases. Addison-Wesley, Reading, MA.
18. J. Conallen. 2002. Building Web Applications with UML, Second Edition. Addison-Wesley, Reading, MA.
19. N. Condori-Fernandez, M. Daneva, K. Sikkel, R. Wieringa, O. Dieste, and O. Pastor. 2009. Research findings on empirical evaluation of requirements specifications approaches. In Proceedings of the Workshop on Requirements Engineering. Valparaiso University Press, 121-128.
20. O. J. Dunn. 1961. Multiple comparisons among means. J. ASA, 56, 52-64.
21. J. Ferreira, J. Noble, and R. Biddle. 2007. Agile Development Iterations and UI Design. In Proceedings of the Agile Conference (AGILE). 50-58.
22. M. Fowler. 2003. UML Distilled: A Brief Guide to the Standard Object Modeling Language, 3rd Ed. Addison-Wesley Longman Publishing Co., Inc., Boston, MA.
23. C. Gravino, G. Scanniello, and G. Tortora. 2008. An empirical investigation on dynamic modeling in requirements engineering. In Proceedings of the Conference on Model Driven Engineering Languages and Systems. IEEE Computer Society, Los Alamitos, CA, 615-629.
24. J. Hannay and M. Jorgensen. 2008. The role of deliberate artificial design elements in software engineering experiments. IEEE Trans. Softw. Eng. 34, 2, 242-259.
25. H. R. Hartson and E. C. Smith. 1991. Rapid prototyping in human-computer interface development. Interacting with Computers 3, 1, 51-91.
26. D. Hendrix, J. Cross, and S. Maghsoodloo. 2002. The effectiveness of control structure diagrams in source code comprehension activities. IEEE Trans. Softw. Eng. 28, 5, 463-477.
27. J. J. Higgins. 2004. An Introduction to Modern Nonparametric Statistics. Brooks/Cole.
28. L. Hochstein, V. R. Basili, M. V. Zelkowitz, J. K. Hollingsworth, and J. Carver. 2005. Combining self-reported and automatic data to improve programming effort measurement. ACM SIGSOFT Softw. Eng. Notes 30, 5, 356-365.
29. K. Hogarty and J. Kromrey. 2001. We've been reporting some effect sizes: Can you guess what they mean In Proceedings of the Annual Meeting of the American Educational Research Association.
30. L. Huang and M. Holcombe. 2009. Empirical investigation towards the effectiveness of test first programming. Inf. Softw. Tech. 51, 1, 182-194.
31. W. S. Humphrey. 1995. A Discipline for Software Engineering. Addison-Wesley Longman Publishing Co., Inc., Boston, MA.
32. ISO. 1991. Information technology-software product evaluation: Quality characteristics and guidelines for their use, ISO/IEC IS 9126, ISO, Geneva.
33. ISO. 2000. Ergonomic requirements for office work with visual display terminals (VDTs) - Part 9: Requirements for non-keyboard input devices. ISO 9241-11, ISO, Geneva, Switzerland.
34. ISO. 2011. Systems and software engineering-Systems and software quality requirements and Evaluation (SQuaRE)-System and software quality models. ISO/IEC 25010, ISO, Geneva, Switzerland.
35. I. Jacobson. 2004. Object-Oriented Software Engineering: A Use Case Driven Approach. Addison-Wesley Longman Publishing Co., Inc., Redwood City, CA.
36. N. Juristo and A. M. Moreno. 2001. Basics of Software Engineering Experimentation. Kluwer Academic Publishers, Englewood Cliffs, NJ.
37. R. I. Kabacoff. 2011. R in Action - Data Analysis and Graphics with R. Manning Publications Co., Shelter Island, NY.
38. E. Kamsties, A. von Knethen, and R. Reussner. 2003. A controlled experiment to evaluate how styles affect the understandability of requirements specifications. Inf. Softw. Tech. 45, 14, 955-965.
39. B. Kitchenham, H. Al-Khilidar, M. Babar, M. Berry, K. Cox, J. Keung, F. Kurniawati, M. Staples, H. Zhang, and L. Zhu. 2008. Evaluating guidelines for reporting empirical software engineering studies. Empir. Softw. Eng. 13, 97-121.
40. B. Kitchenham, S. Pfleeger, L. Pickard, P. Jones, D. Hoaglin, K. El Emam, and J. Rosenberg. 2002. Preliminary guidelines for empirical research in software engineering. IEEE Trans. Softw. Eng. 28, 8, 721-734.
41. J. A. Landay and B. A. Myers. 1995. Interactive sketching for the early stages of user interface design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press/Addison-Wesley Publishing Co., New York, NY, 43-50.
42. S. Lauesen. 2002. Software Requirements: Styles and Techniques. Addison-Wesley.
43. M. Lindvall, I. Rus, F. Shull, M. V. Zelkowitz, P. Donzelli, A. M. Memon, V. R. Basili, P. Costa, R. T. Tvedt, L. Hochstein, S. Asgari, C. Ackermann, and D. Pech. 2005. An evolutionary testbed for software technology evaluation. Innov. Syst. Softw. Eng. 1, 1, 3-11.
44. th IEEE International Conference on Engineering of Complex Computer Systems (ICECCS'08). IEEE Computer Society, Los Alamitos, CA, 203-212.
45. B. Meyer. 1985. On formalism in specification. IEEE Softw. 3, 1, 6-25.
46. H. Motulsky. 2010. Intuitive Biostatistics: A Non-Mathematical Guide to Statistical Thinking. Oxford University Press.
47. R. Mugridge and W. Cunningham. 2005. Fit for Developing Software: Framework for Integrated Tests. Prentice-Hall.
48. M. O'Docherty. 2005. Object-Oriented Analysis and Design: Understanding System Development with UML 2.0 1st Ed. Wiley.
49. A. N. Oppenheim. 1992. Questionnaire Design, Interviewing and Attitude Measurement. Pinter, London, UK.
50. S. L. Pfleeger and W. Menezes. 2000. Marketing technology to software practitioners. IEEE Softw. 17, 1, 27-33.
51. K. T. Phalp, J. Vincent, and K. Cox. 2007. Improving the quality of use case descriptions: Empirical assessment of writing guidelines. Softw. Qual. Cont. 15, 4, 383-399.
52. R Core Team. 2013. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria.
53. F. Ricca, M. Di Penta, M. Torchiano, P. Tonella, and M. Ceccato. 2010a. How developers' experience and ability influence web application comprehension tasks supported by UML stereotypes: A Series of Four Experiments. IEEE Trans. Softw. Eng. 36, 96-118.
54. F. Ricca, G. Scanniello, M. Torchiano, G. Reggio, and E. Astesiano. 2010b. On the effectiveness of screen mockups in requirements engineering: results from an internal replication. In Proceedings of the International Symposium on Empirical Software Engineering and Measurement. ACM, New York, NY.
55. F. Ricca, G. Scanniello, M. Torchiano, G. Reggio, and E. Astesiano. 2010c. On the effort of augmenting use cases with screen mockups: results from a preliminary empirical study. In Proceedings of the International Symposium on Empirical Software Engineering and Measurement. ACM, New York.
56. F. Ricca, M. Torchiano, M. Di Penta, M. Ceccato, and P. Tonella. 2009. Using acceptance tests as a support for clarifying requirements: A series of experiments. Inf. Softw. Tech. 51, 2, 270-283.
57. J. Romano, J. D. Kromrey, J. Coraggio, and J. Skowronek. 2006. Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen's d for evaluating group differences on the NSSE and other surveys In Proceedings of the Annual Meeting of the Florida Association of Institutional Research.
58. G. Scanniello, C. Gravino, M. Genero, J. A. Cruz-Lemus, and G. Tortora. 2014. On the impact of UML analysis models on source code comprehensibility and modifiability. Trans. Softw. Eng. Meth. 23, 2, 13:1-13:26.
59. G. Scanniello, C. Gravino, and G. Tortora. 2010. Investigating the role of UML in the software modeling and maintenance - A preliminary industrial survey. In Proceedings of ICEIS. 141-148.
60. G. Scanniello, F. Ricca, M. Torchiano, C. Gravino, and G. Reggio. 2013. Estimating the effort to develop screen mockups. In Proceedings of the EUROMICRO Conference on Software Engineering and Advanced Applications. 341-348.
61. S. Shapiro and M. Wilk. 1965. An analysis of variance test for normality. Biometrika 52, 3-4, 591-611.
62. F. Shull, J. C. Carver, S. Vegas, and N. J. Juzgado. 2008. The role of replications in empirical software engineering. Empir. Softw. Eng. 13, 2, 211-218.
63. M. Staron, L. Kuzniarz, and C. Wohlin. 2006. Empirical assessment of using stereotypes to improve comprehension of UML models: A set of experiments. J. Syst. Softw. 79, 5, 727-742.
64. M. Torchiano. 2014. Effsize: Efficientc Effects Size Computation. R package version 1.2-2.
65. M. Torchiano, F. C. A. Tomassetti, F. Ricca, A. Tiso, and G. Reggio. 2013. Relevance, benefits, and problems of software modelling and model driven techniques-A survey in the Italian industry. J. Syst. Softw. 86, 8, 2110-2126.
66. H. van der Linden, G. Boers, H. Tange, J. Talmon, and A. Hasman. 2003. Proper: A multi disciplinary EPR system. Int. J. Med. Inf. 70, 2-3, 149-160.
67. B. Wheeler. 2010. lmPerm: Permutation Tests for Linear Models. R package version 1.1-2.
68. C. Wohlin, P. Runeson, M. Host, M. Ohlsson, B. Regnell, and A. Wessln. 2012. Experimentation in Software Engineering. Springer.
69. R. Young. 2001. Effective Requirements Practice. Addison-Wesley, Boston, MA.
70. T. Yue, L. C. Briand, and Y. Labiche. 2009. A use case modeling approach to facilitate the transition towards analysis models: Concepts and empirical evaluation. In Proceedings of the 12th International Conference on Model Driven Engineering Languages and Systems (MODELS'09). Springer-Verlag, Berlin, 484-498.
71. M. K. Zimmerman, K. Lundqvist, and N. Leveson. 2002. Investigating the readability of state-based formal requirements specification languages. In Proceedings of the International Conference on Software Engineering. ACM, New York, 33-43.