Graphical user interface (GUI) testing: Systematic mapping and repository

Information and Software Technology

Volumn 55, Issue 10, 2013, Pages 1679-1694

  Banerjee, Ishan ^a   Nguyen, Bao ^a   Garousi, Vahid ^b,c   Memon, Atif ^a  

^a Institute of Advanced Computer Science   (United States)

^b UNIVERSITY OF CALGARY   (Canada)

^c MIDDLE EAST TECHNICAL UNIVERSITY   (Turkey)

Author keywords

Bibliometrics; GUI application; Paper repository; Systematic mapping; Testing

Indexed keywords

BIBLIOMETRICS; CLASSIFICATION SCHEME; GRAPHICAL USER INTERFACES (GUI); GUI APPLICATIONS; RESEARCH COMMUNITIES; RESEARCH QUESTIONS; SOURCES OF INFORMATIONS; SYSTEMATIC MAPPING;

INFORMATION SCIENCE; LAKES; MAPPING; RESEARCH; SOFTWARE TESTING; SYSTEM THEORY; TESTING; TOOLS;

GRAPHICAL USER INTERFACES;

EID: 84880786120     PISSN: 09505849     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.infsof.2013.03.004     Document Type: Review

References (109)

1. K. Petersen, R. Feldt, S. Mujtaba, M. Mattsson, Systematic mapping studies in software engineering, in: 12th International Conference on Evaluation and Assessment in Software Engineering, vol. 17, issue 1, 2007, pp. 1-10.
2. W. Afzal, R. Torkar, R. Feldt. A systematic mapping study on non-functional search-based software testing, in: 20th International Conference on Software Engineering and Knowledge Engineering (SEKE 2008), 2008.
3. W. Afzal, R. Torkar, and R. Feldt A systematic review of search-based testing for non-functional system properties Information and Software Technology 51 2009 957 976
4. R.V. Binder, Testing object-oriented software: a survey, in: Proceedings of the Tools-23: Technology of Object-Oriented Languages and Systems, 1997, pp. 374-.
5. Y. Jia, and M. Harman An analysis and survey of the development of mutation testing IEEE Transactions on Software Engineering 2008 2010 1 32
6. D. Budgen, M. Turner, P. Brereton, and B. Kitchenham Using mapping studies in software engineering Proceedings of PPIG 2008 2008 Lancaster University 195 204
7. Z.A. Barmi, A.H. Ebrahimi, R. Feldt, Alignment of requirements specification and testing: a systematic mapping study, in: Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, ICSTW '11, 2011, pp. 476-485.
8. P.A. da Mota Silveira Neto, I.d. Carmo Machado, J.D. McGregor, E.S. de Almeida, and S.R. de Lemos Meira A systematic mapping study of software product lines testing Information and Software Technology 53 2011 407 423
9. A. Fernandez, E. Insfran, and S. Abrahlo Usability evaluation methods for the web: a systematic mapping study Information and Software Technology 53 8 2011 789 817
10. B.A. Kitchenham, D. Budgen, and O.P. Brereton Using mapping studies as the basis for further research ¢ a participant-observer case study Information and Software Technology 53 6 2011 638 651
11. M. Palacios, J. GarcÍa-Fanjul, and J. Tuya Testing in service oriented architectures with dynamic binding: a mapping study Information and Software Technology 53 2011 171 189
12. J. Portillo-Rodriguez, A. Vizcaino, M. Piattini, and S. Beecham Tools used in global software engineering: a systematic mapping review Information and Software Technology 2012
13. B. Kitchenham, S. Charters, Guidelines for Performing Systematic Literature Reviews in Software Engineering, Version, 2(EBSE 2007-001):2007-001, 2007.
14. O.E. Ariss, D. Xu, S. Dandey, B. Vender, P. McClean, B. Slator. A systematic capture and replay strategy for testing complex GUI based java applications, in: Conference on Information Technology, 2010, pp. 1038-1043.
15. A.C.R. Paiva, N. Tillmann, J.C.P. Faria, R.F.A.M. Vidal, Modeling and testing hierarchical GUIs, in: Workshop on Abstract State Machines, 2005.
16. A.M. Memon, M.E. Pollack, and M.L. Soffa Hierarchical GUI test case generation using automated planning IEEE Transactions on Software Engineering 27 2 2001 144 155 (Pubitemid 32254261)
17. L. White, H. Almezen, Generating test cases for GUI responsibilities using complete interaction sequences, in: Symposium on Software Reliability Engineering, 2000, p. 110.
18. L. Baresi, M. Young, Test Oracles. Technical Report CIS-TR-01-02, University of Oregon, Dept. of Computer and Information Science, Eugene, Oregon, USA, August 2001.
19. J. Takahashi An automated oracle for verifying GUI objects ACM SIGSOFT Software Engineering Notes 26 4 2001 83 88
20. Q. Xie, and A.M. Memon Designing and comparing automated test oracles for GUI-based software applications ACM Transactions on Software Engineering and Methodology 16 1 2007 1 36
21. A.M. Memon, M.L. Soffa, M.E. Pollack, Coverage criteria for GUI testing, in: Software Engineering Conference held Jointly with ACM SIGSOFT Symposium on Foundations of Software Engineering, 2001, pp. 256-267. (Pubitemid 35466930)
22. X. Yuan, M.B. Cohen, A.M. Memon, GUI interaction testing: incorporating event context, in: IEEE Transactions on Software Engineering, 2010.
23. L. Zhao, K.-Y. Cai, Event handler-based coverage for GUI testing, in: Conference on Quality Software, 2010, pp. 326-331.
24. Z. Hui, R. Chen, S. Huang, B. Hu, Gui regression testing based on function-diagram, in: Intelligent Computing and Intelligent Systems (ICIS), 2010 IEEE International Conference on, vol. 2, October 2010, pp. 559-563.
25. A.M. Memon Automatically repairing event sequence-based GUI test suites for regression testing ACM Transactions on Software Engineering and Methodology 18 2 2008 1 36
26. L.J. White, Regression testing of GUI event interactions, in: Conference on Software Maintenance, 1996, pp. 350-358.
27. http://www.cs.umd.edu/atif/testbeds/testbeds2009.htm.
28. http://www.cs.umd.edu/atif/testbeds/testbeds2010.htm.
29. http://www.cs.umd.edu/atif/testbeds/testbeds2011.htm.
30. http://www.cs.umd.edu/atif/testbeds/testbeds2013.htm.
31. F.M. Theodore, D. Hellmann, and Ali. Hosseini-Khayat Agile Interaction Design and Test-Driven Development of User Interfaces - A Literature Review vol. 9 2010 Springer
32. A.M. Memon, and B.N. Nguyen Advances in automated model-based system testing of software applications with a GUI front-end M.V. Zelkowitz, Advances in Computers vol. 80 2010 Academic Press pp. nnn-nnn
33. http://www.softqual.ucalgary.ca/projects/2012/GUI-SM/.
34. M.B. Dwyer, V. Carr, L. Hines, Model checking graphical user interfaces using abstractions, in: ESEC/SIGSOFT FSE, 1997, pp. 244-261. (Pubitemid 127132031)
35. X. Yuan, and A.M. Memon Generating event sequence-based test cases using GUI runtime state feedback IEEE Transactions on Software Engineering 36 2010 81 95
36. M.J. Harrold Testing: a roadmap Proceedings of the Conference on The Future of Software Engineering, ICSE '00 2000 ACM New York, NY, USA 61 72
37. S. McConnell Daily build and smoke test IEEE Software 13 4 1996 143 144
38. X. Yuan, and A.M. Memon Using GUI run-time state as feedback to generate test cases Proceedings of the 29th International Conference on Software Engineering, ICSE '07 2007 IEEE Computer Society Washington, DC, USA 396 405 (Pubitemid 47430674)
39. V. Basili, G. Caldiera, and H. Rombach Encyclopedia of Software Engineering 1994 John Wiley & Sons Inc. Chapter Goal Question Metric Approach, pp. 528-532
40. H. Zhang, M.A. Babar, and P. Tell Identifying relevant studies in software engineering Information and Software Technology 53 2011 625 637
41. N. Abdallah, S. Ramakrishnan, Automated stress testing of windows mobile GUI applications, in: International Symposium on Software Reliability Engineering, 2009.
42. H. Okada, and T. Asahi GUITESTER: a log-based usability testing tool for graphical user interfaces IEICE Transactions on Information and Systems 82 1999 1030 1041 (Pubitemid 30510863)
43. M. Safoutin, C. Atman, R. Adams, T. Rutar, J. Kramlich, and J. Fridley A design attribute framework for course planning and learning assessment IEEE Transactions on Education 43 2 2000 188 199
44. F. Belli, Finite-state testing and analysis of graphical user interfaces, in: Symposium on Software Reliability Engineering, 2001, p. 34.
45. R.K. Shehady, D.P. Siewiorek, A method to automate user interface testing using variable finite state machines, in: Symposium on Fault-Tolerant Computing, 1997, p. 80.
46. C. Bertolini, A. Mota, Using probabilistic model checking to evaluate GUI testing techniques, in: Conference on Software Engineering and Formal Methods, 2009, pp. 115-124.
47. K. Magel, I. Alsmadi, GUI structural metrics and testability testing, in: Conference on Software Engineering and Applications, 2007, pp. 91-95.
48. S. Ganov, C. Killmar, S. Khurshid, and D.E. Perry Event listener analysis and symbolic execution for testing GUI applications Formal Methods and Software Engineering 5885 1 2009 69 87
49. Y. Tsujino A verification method for some GUI dialogue properties Systems and Computers in Japan 2000 38 46 (Pubitemid 32090376)
50. J. Strecker, A. Memon, Relationships between test suites, faults, and fault detection in GUI testing, in: Conference on Software Testing, Verification, and Validation, 2008, pp. 12-21.
51. D. Amalfitano, A.R. Fasolino, P. Tramontana, Rich internet application testing using execution trace data, in: Conference on Software Testing, Verification, and Validation Workshops, 2010, pp. 274-283.
52. A.M. Memon, M.E. Pollack, M.L. Soffa, Plan generation for GUI testing, in: Conference on Artificial Intelligence Planning and Scheduling, 2000, pp. 226-235.
53. P. Li, T. Huynh, M. Reformat, and J. Miller A practical approach to testing GUI systems Empirical Software Engineering 12 4 2007 331 357 (Pubitemid 47115400)
54. B. Robinson, P. Brooks, An initial study of customer-reported GUI defects, in: Conference on Software Testing, Verification, and Validation Workshops, 2009, pp. 267-274.
55. GUITAR - A GUI Testing frAmewoRk. < http://guitar.sourceforge.net >.
56. A. Derezinska, T. Malek, Unified automatic testing of a GUI applications' family on an example of RSS aggregators, in: Multiconference on Computer Science and Information Technology, 2006, pp. 549-559.
57. A. Derezinska, and T. Malek Experiences in testing automation of a family of functional-and GUI-similar programs Journal of Computer Science and Applications 4 1 2007 13 26
58. C. Hu, I. Neamtiu, Automating gui testing for android applications, in: Proceedings of the 6th International Workshop on Automation of Software Test, 2011, pp. 77-83.
59. C. Hu, and I. Neamtiu A gui bug finding framework for android applications Proceedings of the 2011 ACM Symposium on Applied Computing, SAC '11 2011 ACM 1490 1491
60. A. Holmes, M. Kellogg, Automating functional tests using selenium, in: agile Conference, 2006, pp. 270-275. (Pubitemid 46679616)
61. L. Feng, S. Zhuang, Action-driven automation test framework for graphical user interface (GUI) software testing, in: Autotestcon, 2007, pp. 22-27.
62. Y. Hou, R. Chen, Z. Du, Automated GUI testing for J2ME software based on FSM, in: Conference on Scalable Computing and Communications, 2009, pp. 341-346.
63. K. Conroy, M. Grechanik, M. Hellige, E. Liongosari, Q. Xie, Automatic test generation from GUI applications for testing web services, in: Conference on Software Maintenance, 2007, pp. 345-354.
64. V. Chinnapongsea, I. Lee, O. Sokolsky, S. Wang, P.L. Jones, Model-based testing of GUI-driven applications, in: Workshop on Software Technologies for Embedded and Ubiquitous Systems, 2009, pp. 203-214.
65. M. Alles, D. Crosby, C. Erickson, B. Harleton, M. Marsiglia, G. Pattison, C. Stienstra, Presenter First: Organizing Complex GUI Applications for Test-Driven Development, in: Proceedings of the conference on AGILE 2006, 2006, pp. 276-288.
66. T. Daboczi, I. Kollar, G. Simon, T. Megyeri, Automatic testing of graphical user interfaces, in: Instrumentation and Measurement Technology Conference, 2003, pp. 441-445.
67. R. Lo, R. Webby, R. Jeffery, Sizing and estimating the coding and unit testing effort for GUI systems, in: Software Metrics Symposium, 1996, p. 166.
68. T.-H. Chang, T. Yeh, R.C. Miller, GUI testing using computer vision, in: Conference on Human factors in computing systems, 2010, pp. 1535-1544.
69. S. Arlt, C. Bertolini, and M. Schäf Behind the scenes: an approach to incorporate context in GUI test case generation Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, ICSTW '11 2011 IEEE Computer Society 222 231
70. S. Herbold, J. Grabowski, S. Waack, U. Bünting, Improved bug reporting and reproduction through non-intrusive gui usage monitoring and automated replaying, in: Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, ICSTW '11, 2011, pp. 232-241.
71. X. Yuan, and A.M. Memon Iterative execution-feedback model-directed GUI testing Information and Software Technology 52 5 2010 559 575
72. A.M. Memon, M.E. Pollack, and M.L. Soffa Automated test oracles for GUIs ACM SIGSOFT Software Engineering Notes 25 6 2000 30 39 (Pubitemid 32399969)
73. A. Jaaskelainen, A. Kervinen, M. Katara, Creating a test model library for GUI testing of Smartphone applications, in: Conference on Quality Software, 2008, pp. 276-282.
74. A. Kervinen, M. Maunumaa, T. Paakkonen, and M. Katara Model-based testing through a GUI Formal Approaches to Software Testing 3997 1 2006 16 31 (Pubitemid 44165374)
75. M. Navarro, P. Luis, S. Ruiz, D.M. Perez, and Gregorio A proposal for automatic testing of GUIs based on annotated use cases Advances in Software Engineering 2010 1 2010 1 8
76. W.-K. Chen, T.-H. Tsai, H.-H. Chao, Integration of specification-based and CR-based approaches for GUI testing, in: Conference on Advanced Information Networking and Applications, 2005, pp. 967-972. (Pubitemid 43803954)
77. S. Ganov, C. Kilmar, S. Khurshid, D. Perry, Test generation for graphical user interfaces based on symbolic execution, in: Proceedings of the International Workshop on Automation of Software Test, 2008.
78. K.-Y. Cai, L. Zhao, H. Hu, C.-H. Jiang, On the test case definition for GUI testing, in: Conference on Quality Software, 2005, pp. 19-28.
79. Q. Xie, A. Memon, Rapid "Crash Testing" for continuously evolving GUI-based software applications, in: Conference on Software Maintenance, 2005, pp. 473-482. (Pubitemid 43790960)
80. Q. Xie, and A.M. Memon Using a pilot study to derive a GUI model for automated testing ACM Transactions on Software Engineering and Methodology 18 2 2008 1 33
81. M. Grechanik, Q. Xie, C. Fu, Experimental assessment of manual versus tool-based maintenance of GUI-directed test scripts, Conference on Software Maintenance, 2009, pp. 9-18.
82. C. McMahon, History of a large test automation project using selenium, in: Proceedings of the 2009 Agile Conference, 2009, pp. 363-368.
83. R.M. Patton, G.H. Walton, An automated testing perspective of graphical user interfaces, in: The Interservice/Industry Training, Simulation and Education Conference, 2003.
84. S. Yip, D. Robson, Applying formal specification and functional testing to graphical user interfaces, in: Advanced Computer Technology, Reliable Systems and Applications European Computer Conference, 1991, pp. 557-561.
85. S. Yip, D. Robson, Graphical user interfaces validation: a problem analysis and a strategy to solution, in: Conference on System Sciences, 1991.
86. D.H. Nguyen, P. Strooper, J.G. Suess, Model-based testing of multiple GUI variants using the GUI test generator, in: Workshop on Automation of Software Test, 2010, pp. 24-30.
87. M. Grechanik, Q. Xie, C. Fu, Maintaining and evolving GUI-directed test scripts, in: Conference on Software Engineering, 2009, pp. 408-418.
88. D. Amalfitano, A.R. Fasolino, and P. Tramontana A GUI crawling-based technique for android mobile application testing Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, ICSTW '11 2011 IEEE Computer Society 252 261
89. C. Bertolini, A. Mota, E. Aranha, C. Ferraz, GUI testing techniques evaluation by designed experiments, in: Conference on Software Testing, Verification and Validation, 2010, pp. 235-244.
90. C. Lowell, and J. Stell-Smith Successful automation of GUI driven acceptance testing Extreme Programming and Agile Processes in Software Engineering 2675 1 2003 1011 1012
91. M. Ye, B. Feng, Y. Lin, L. Zhu, Neural networks based test cases selection strategy for GUI testing, in: Congress on Intelligent Control and Automation, 2006, pp. 5773-5776.
92. B. Daniel, Q. Luo, M. Mirzaaghaei, D. Dig, D. Marinov, and M. Pezzè Automated GUI refactoring and test script repair Proceedings of the First International Workshop on End-to-End Test Script Engineering, ETSE '11 2011 ACM New York, NY, USA 38 41
93. Y. Shewchuk, V. Garousi, Experience with maintenance of a functional GUI test suite using IBM rational functional tester, in: Proceedings of the International Conference on Software Engineering and Knowledge Engineering, 2010, pp. 489-494.
94. C. Bertolini, G. Peres, M. Amorim, A. Mota, An empirical evaluation of automated black-box testing techniques for crashing GUIs, in: Software Testing Verification and Validation, 2009, pp. 21-30.
95. C. Bertolini, A. Mota, A framework for GUI testing based on use case design, in: Conference on Software Testing, Verification, and Validation Workshops, 2010, pp. 252-259.
96. A. Jaaskelainen, M. Katara, A. Kervinen, M. Maunumaa, T. Paakkonen, T. Takala, H. Virtanen, Automatic GUI test generation for smartphone applications - an evaluation, in: Conference on Software Engineering, 2009, pp. 112-122.
97. O.-H. Kwon, S.-M. Hwang, Mobile GUI testing tool based on image flow, in: Conference on Computer and Information Science, 2008, pp. 508-512.
98. J. Chen, and S. Subramaniam Specification-based testing for GUI-based applications Software Quality Journal 10 2 2002 205 224
99. A.C. Paiva, J.C. Faria, N. Tillmann, and R.A. Vidal A model-to-implementation mapping tool for automated model-based GUI testing Formal Methods and Software Engineering 3785 1 2005 450 464 (Pubitemid 43763461)
100. R. Gove, J. Faytong, Identifying infeasible GUI test cases using support vector machines and induced grammars, in: Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, 2011, pp. 202-211.
101. A.C.R. Paiva, J.C.P. Faria, and P.M.C. Mendes Reverse engineered formal models for GUI testing Formal Methods for Industrial Critical Systems 4916 1 2008 218 233 (Pubitemid 351775476)
102. M. Cunha, A. Paiva, H. Ferreira, R. Abreu, PETTool: a pattern-based GUI testing tool, in: International Conference on Software Technology and Engineering, 2010, pp. 202-206.
103. S. Ali, L.C. Briand, H. Hemmati, and R.K. Panesar-Walawege A systematic review of the application and empirical investigation of search-based test case generation IEEE Transactions on Software Engineering 36 2010 742 762
104. N. Juristo, A.M. Moreno, and S. Vegas Reviewing 25 years of testing technique experiments Empirical Software Engineering 9 2004 7 44
105. P. McMinn Search-based software test data generation: a survey: research articles Software Testing, Verification and Reliability 14 2004 105 156
106. M. Grindal, J. Offutt, and S.F. Andler Combination testing strategies: a survey Software Testing, Verification, and Reliability 15 2005 167 199 (Pubitemid 41320808)
107. C.S. Pǎsǎreanu, and W. Visser A survey of new trends in symbolic execution for software testing and analysis International Journal on Software Tools for Technology Transfer 11 4 2009 339 353
108. S.A.M. Mark Harman, Y. Zhang, Search based Software Engineering: A Comprehensive Analysis and Review of Trends Techniques and Applications, Technical Report TR-09-03, Department of Computer Science, King's College London, April 2009.
109. http://crestweb.cs.ucl.ac.uk/resources/sbse-repository/.