A systematic mapping study of software product lines testing

Information and Software Technology

Volumn 53, Issue 5, 2011, Pages 407-423

  Da Mota Silveira Neto, Paulo Anselmo ^a,b   Carmo MacHado, Ivan Do ^a,b   McGregor, John D ^d   De Almeida, Eduardo Santana ^a,c   De Lemos Meira, Silvio Romero ^a,b  

^a RiSE Reuse in Software Engineering   (Brazil)

^b FEDERAL UNIVERSITY OF PERNAMBUCO   (Brazil)

^c FEDERAL UNIVERSITY OF BAHIA   (Brazil)

^d AMRL   (United States)

Author keywords

Mapping study; Software product lines; Software testing

Indexed keywords

FURTHER DEVELOPMENT; MAPPING STUDIES; RESEARCH QUESTIONS; SOFTWARE DEVELOPMENT; SOFTWARE PRODUCT LINE; SOFTWARE PRODUCT LINES; STATE OF RESEARCH; SYSTEM DEVELOPMENT; SYSTEMATIC MAPPING;

COMPUTER SOFTWARE SELECTION AND EVALUATION; MAPPING; RESEARCH; SOFTWARE TESTING;

SOFTWARE DESIGN;

EID: 79952451558     PISSN: 09505849     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.infsof.2010.12.003     Document Type: Conference Paper

References (84)

1. W. Afzal, R. Torkar, R. Feldt, A systematic mapping study on non-functional search-based software testing, in: SEKE'08: Proceedings of the 20th International Conference on Software Engineering and Knowledge Engineering, Redwood City, California, USA, 2008, pp. 488-493
2. W. Afzal, R. Torkar, and R. Feldt A systematic review of search-based testing for non-functional system properties Information and Software Technology 51 6 2009 957 976
3. J. Al-Dallal, and P.G. Sorenson Testing software assets of framework-based product families during application engineering stage Journal of Software 3 5 2008 11 25
4. P. Ammann, and J. Offutt Introduction to Software Testing 1st ed. 2008 Cambridge University Press
5. J. Bailey, D. Budgen, M. Turner, B. Kitchenham, P. Brereton, S. Linkman, Evidence relating to object-oriented software design: a survey, in: ESEM '07: Proceedings of the First International Symposium on Empirical Software Engineering and Measurement, Washington, DC, USA, 2007, pp. 482-484.
6. Beatriz Prez Lamancha, and M.P. Macario Polo Usaola Towards an automated testing framework to manage variability using the UML testing profile AST'09: Proceedings of the Workshop on Automation of Software Test (ICSE) 2009 Vancouver Canada 10 17
7. A. Bertolino, Software testing research: achievements, challenges, dreams, in: FOSE '07: Future of Software Engineering, Washington, DC, USA, 2007, pp. 85-103. (Pubitemid 47484919)
8. A. Bertolino, S. Gnesi, PLUTO: a test methodology for product families, in: Software Product-Family Engineering, 5th International Workshop, PFE, Siena, Italy, 2003a, pp. 181-197.
9. A. Bertolino, and S. Gnesi Use case-based testing of product lines ACM SIGSOFT Software Engineering Notes 28 5 2003 355 358
10. Y.M. Bezerra, T.A.B. Pereira, G.E. da Silveira, A systematic review of software product lines applied to mobile middleware, in: ITNG '09: Proceedings of the 2009 Sixth International Conference on Information Technology: New Generations, Washington, DC, USA, 2009, pp. 1024-1029.
11. P. Brereton, B.A. Kitchenham, D. Budgen, M. Turner, and M. Khalil Lessons from applying the systematic literature review process within the software engineering domain Journal of Systems and Software 80 4 2007 571 583 (Pubitemid 46329126)
12. D. Budgen, M. Turner, P. Brereton, and B. Kitchenham Using Mapping Studies in Software Engineering Proceedings of PPIG Psychology of Programming Interest Group 2008 2008 Lancaster University UK 195 204
13. L. Chen, M.A. Babar, N. Ali, Variability management in software product lines: a systematic review, in: SPLC'09: Proceedings of 13th Software Product Line Conference, San Francisco, CA, USA, 2009.
14. M.B. Cohen, M.B. Dwyer, and J. Shi Coverage and adequacy in software product line testing ROSATEA '06: Proceedings of the ISSTA 2006 Workshop on Role of Software Architecture for Testing and Analysis 2006 ACM New York, NY, USA 53 63 (Pubitemid 46645442)
15. N. Condori-Fernandez, M. Daneva, K. Sikkel, R. Wieringa, O. Dieste, O. Pastor, A systematic mapping study on empirical evaluation of software requirements specifications techniques, in: ESEM '09: Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement, Washington, DC, USA, 2009, pp. 502-505.
16. C. Condron, A domain approach to test automation of product lines, in: SPLiT'04: In International Workshop on Software Product Line Testing, Boston, MA, USA, 2004, pp. 27-35.
17. C. Denger, R. Kolb, Testing and inspecting reusable product line components: first empirical results, in: ISESE'06: Proceedings of the International Symposium on Empirical Software Engineering, New York, NY, USA, 2006, pp. 184-193 (Pubitemid 46634453)
18. T. Dyb, and T. Dingsøyr Empirical studies of agile software development: a systematic review Information and Software Technology 50 9-10 2008 833 859
19. T. Dyb, and T. Dingsøyr Strength of evidence in systematic reviews in software engineering ESEM '08: Proceedings of the Second ACM - IEEE International Symposium on Empirical Software Engineering and Measurement 2008 ACM New York, NY, USA 178 187
20. O.O. Edwin, Testing in Software Product Lines. Master's thesis, Department of Software Engineering and Computer Science, Blekinge Institute of Technology, Sweden, 2007.
21. E. Engström, M. Skoglund, P. Runeson, Empirical evaluations of regression test selection techniques: a systematic review, in: ESEM '08: Proceedings of the Second ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, New York, NY, USA, 2008, pp. 22-31
22. Y. Feng, X. Liu, J. Kerridge, A product line based aspect-oriented generative unit testing approach to building quality components, in: COMPSAC'07 - Proceedings of the 31st Annual International Computer Software and Applications Conference, Washington, DC, USA, 2007, pp. 403-408. (Pubitemid 350301823)
23. D. Ganesan, U. Maurer, M. Ochs, B. Snoek, M. Verlage, Towards testing response time of instances of a web-based product line, in: SPLIT'05: International Workshop on Software Product Line Testing, Rennes, France, 2005
24. B. Geppert, J.J. Li, F. Rler, D.M. Weiss, Towards generating acceptance tests for product lines, in: ICSR'04: Proceedings of 8th International Conference on Software Reuse, 2004, pp. 35-48.
25. R.F. Goldsmith, D. Graham, The forgotten phase, in: Software Development Magazine, 2002, pp. 45-47.
26. T.L. Graves, M.J. Harrold, J.-M. Kim, A. Porter, and G. Rothermel An empirical study of regression test selection techniques ACM Transaction on Software Engineering Methodology 10 2 2001 184 208 (Pubitemid 33602334)
27. M.J. Harrold, Architecture-based regression testing of evolving systems, in: ROSATEA'98: International Worshop on Role of Architecture in Testing and Analysis, Marsala, Sicily, Italy, 1998, pp. 73-77.
28. J. Hartmann, M. Vieira, A. Ruder, A UML-based approach for validating product lines, in: SPLIT '04: Proceedings of the International Workshop on Software Product Line Testing, Boston, MA, 2004, pp. 58-65.
29. M. Jaring, R.L. Krikhaar, J. Bosch, Modeling variability and testability interaction in software product line engineering, in: ICCBSS'08: 7th International Conference on Composition-Based Software Systems, 2008, pp. 120-129.
30. L. Jin-hua, L. Qiong, L. Jing, The w-model for testing software product lines, in: ISCSCT '08: Proceedings of the International Symposium on Computer Science and Computational Technology, Los Alamitos, CA, USA, 2008, pp. 690-693.
31. N. Juristo, A.M. Moreno, and S. Vegas Reviewing 25 years of testing technique experiments Empirical Software Engineering 9 1-2 2004 7 44
32. N. Juristo, A.M. Moreno, and W. Strigel Guest editors' introduction: software testing practices in industry IEEE Software 23 4 2006 19 21
33. N. Juristo, A.M. Moreno, S. Vegas, and M. Solari In search of what we experimentally know about unit testing IEEE Software 23 6 2006 72 80 (Pubitemid 44824160)
34. E. Kamsties, K. Pohl, S. Reis, A. Reuys, Testing variabilities in use case models, in: PFE'03: Proceedings of 5th International Workshop Software Product-Family Engineering, Siena, Italy, 2003, pp. 6-18
35. S. Kang, J. Lee, M. Kim, W. Lee, Towards a formal framework for product line test development, in: CIT '07: Proceedings of the 7th IEEE International Conference on Computer and Information Technology, Washington, DC, USA, 2007, pp. 921-926
36. R. Kauppinen, Testing framework-based software product lines, Master's thesis, University of Helsinki Department of Computer Science, 2003.
37. R. Kauppinen, J. Taina, Rita environment for testing framework-based software product lines, in: SPLST'03 - Proceedings of the 8th Symposium on Programming Languages and Software Tools, Kuopio, Finland, 2003, pp. 58-69
38. R. Kauppinen, J. Taina, A. Tevanlinna, Hook and template coverage criteria for testing framework-based software product families, in: SPLIT '04: Proceedings of the International Workshop on Software Product Line Testing, Boston, MA, USA, 2004, pp. 7-12
39. T. Kishi, and N. Noda Formal verification and software product lines Communications of the ACM 49 12 2006 73 77
40. B. Kitchenham What's up with software metrics? A preliminary mapping study Journal of Systems and Software 83 1 2010 37 51
41. B. Kitchenham, S. Charters, Guidelines for performing Systematic Literature Reviews in Software Engineering, Technical Report EBSE 2007-001, Keele University and Durham University Joint Report, 2007.
42. B.A. Kitchenham, S.L. Pfleeger, L.M. Pickard, P.W. Jones, D.C. Hoaglin, K.E. Emam, and J. Rosenberg Preliminary guidelines for empirical research in software engineering IEEE Transactions on Software Engineering 28 8 2002 721 734
43. B.A. Kitchenham, T. Dyba, M. Jorgensen, Evidence-based software engineering, in: ICSE'04 : Proceedings of the 26th International Conference on Software Engineering, Washington, DC, USA, 2004, pp. 273-281
44. B.A. Kitchenham, E. Mendes, and G.H. Travassos Cross versus within-company cost estimation studies: a systematic review IEEE Transactions on Software Engineering 33 5 2007 316 329 (Pubitemid 46681200)
45. R. Kolb, A risk-driven approach for efficiently testing software product lines, in: IESEF'03 - Fraunhofer Institute for Experimental Software Engineering, 2003.
46. R. Kolb, D. Muthig, Challenges in testing software product lines, in: CONQUEST'03 - Proceedings of 7th Conference on Quality Engineering in Software Technology, Nuremberg, Germany, 2003, pp. 81-95
47. R. Kolb, D. Muthig, Making testing product lines more efficient by improving the testability of product line architectures, in: ROSATEA '06: Proceedings of the ISSTA Workshop on Role of Software Architecture for Testing and Analysis, New York, NY, USA, 2006, pp. 22-27. (Pubitemid 46645437)
48. B.P. Lamancha, M.P. Usaola, and M.P. Velthius Software product line testing - a systematic review ICSOFT International Conference on Software and Data Technologies 2009 INSTICC Press 23 30
49. J.J. Li, D.M. Weiss, J.H. Slye, Automatic system test generation from unit tests of exvantage product family, in: SPLIT '07: Proceedings of the International Workshop on Software Product Line Testing, Kyoto, Japan, 2007a, pp. 73-80.
50. J.J. Li, B. Geppert, F. Roessler, D. Weiss, Reuse execution traces to reduce testing of product lines, in: SPLIT '07: Proceedings of the International Workshop on Software Product Line Testing, Kyoto, Japan, 2007b.
51. L.B. Lisboa, V.C. Garcia, D. Lucrédio, E.S. de Almeida, S.R. de Lemos Meira, and R.P. de Mattos Fortes A systematic review of domain analysis tools Information and Software Technology 52 1 2010 1 13
52. J. McGregor, P. Sodhani, S. Madhavapeddi, Testing variability in a software product line, in: SPLIT '04: Proceedings of the International Workshop on Software Product Line Testing, Boston, Massachusetts, USA, 2004, p. 45
53. J.D. McGregor, Structuring test assets in a product line effort, in: ICSE'01: In Proceedings of the 2nd International Workshop on Software Product Lines: Economics, Architectures, and Implications, Toronto, Ontario, Canada, 2001a, pp. 89-92
54. J.D. McGregor, Testing a software product line, Technical Report CMU/SEI-2001-TR-022, 2001b.
55. J.D. McGregor, Building reusable test assets for a product line, in: ICSR'02: Proceedings of 7th International Conference on Software Reuse, Austin, Texas, USA, 2002, pp. 345-346
56. M.B.S. Moraes, E.S. Almeida, S.R. de Lemos Meira, A systematic review on software product lines scoping, in: ESELAW'09: Proceedings of the VI Experimental Software Engineering Latin American Workshop, Sao Carlos-SP, Brazil, 2009
57. H. Muccini, and A. van der Hoek Towards testing product line architectures Electronic Notes in Theoretical Computer Science 82 6 2003
58. H. Muccini, M.S. Dias, D.J. Richardson, Towards software architecture-based regression testing, in: WADS '05: Proceedings of the Workshop on Architecting Dependable Systems, New York, NY, USA, 2005, pp. 1-7.
59. H. Muccini, M. Dias, and D.J. Richardson Software architecture-based regression testing Journal of Systems and Software 79 10 2006 1379 1396 (Pubitemid 44292340)
60. C. Nebut, F. Fleurey, Y.L. Traon, J.-M. JTzTquel, A requirement-based approach to test product families, in: PFE'03: Proceedings of 5th International Workshop Software Product-Family Engineering, Siena, Italy, 2003, pp. 198-210.
61. C. Nebut, Y.L. Traon, J.-M. Jézéquel, System testing of product lines: from requirements to test cases, in: [34], 2006, pp. 447-477.
62. D. Needham, S. Jones, A software fault tree metric, in: ICSM'06: Proceedings of the International Conference on Software Maintenance, Philadelphia, Pennsylvania, USA, 2006, pp. 401-410. (Pubitemid 47133374)
63. L.M. Northrop, P.C. Clements, A framework for software product line practice, version 5.0. Technical report, Software Engineering Institute, 2007.
64. E. Olimpiew, H. Gomaa, Reusable system tests for applications derived from software product lines, in: SPLIT '05: Proceedings of the International Workshop on Software Product Line Testing, Rennes, France, 2005a.
65. E.M. Olimpiew, H. Gomaa, Model-based testing for applications derived from software product lines, in: A-MOST'05: Proceedings of the 1st International Workshop on Advances in model-based testing, New York, NY, USA., 2005b, pp. 1-7.
66. E.M. Olimpiew, H. Gomaa, Reusable model-based testing, in: ICSR '09: Proceedings of the 11th International Conference on Software Reuse, Berlin, Heidelberg, 2009, pp. 76-85.
67. K. Petersen, R. Feldt, S. Mujtaba, and M. Mattsson Systematic mapping studies in software engineering EASE '08: Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering 2008 University of Bari Italy
68. K. Pohl, and A. Metzger Software product line testing Communications of the ACM 49 12 2006 78 81
69. K. Pohl, and E. Sikora Documenting variability in test artefacts Software Product Lines 2005 Springer 149 158
70. K. Pohl, G. Böckle, and F.J. van der Linden Software Product Line Engineering: Foundations, Principles and Techniques 2005 Springer
71. R. Pretorius, D. Budgen, A mapping study on empirical evidence related to the models and forms used in the UML, in: ESEM'08: Proceedings of Empirical Software Engineering and Measurement, Kaiserslautern, Germany, 2008, pp. 342-344.
72. S. Reis, A.P.K. Metzger, A reuse technique for performance testing of software product lines, in: SPLIT '05: Proceedings of the International Workshop on Software Product Line Testing, Baltimore, Maryland, USA, 2006.
73. S. Reis, A. Metzger, K. Pohl, Integration testing in software product line engineering: a model-based technique, in: FASE'07: Proceedings of the Fundamental Approaches to Software Engineering, Braga, Portugal, 2007, pp. 321-335. (Pubitemid 350259359)
74. A. Reuys, E. Kamsties, K. Pohl, S. Reis, Model-based system testing of software product families, in: CAiSE'05: Proceedings of International Conference on Advanced Information Systems Engineering, 2005, pp. 519-534. (Pubitemid 41336125)
75. A. Reuys, S. Reis, E. Kamsties, K. Pohl, The scented method for testing software product lines, in: [34], 2006, pp. 479-520.
76. G. Rothermel, and M.J. Harrold Analyzing regression test selection techniques IEEE Transactions on Software Engineering 22 8 1996 529 551 (Pubitemid 126771681)
77. J. Rumbaugh, I. Jacobson, G. Booch, Unified Modeling Language Reference Manual, second ed., Pearson Higher Education, 2004
78. E.D. Souza Filho, R. Oliveira Cavalcanti, D.F. Neiva, T.H. Oliveira, L.B. Lisboa, E.S. Almeida, S.R. Lemos Meira, Evaluating domain design approaches using systematic review, in: ECSA '08: Proceedings of the 2nd European conference on Software Architecture, Berlin, Heidelberg, 2008, pp. 50-65.
79. A. Tevanlinna, J. Taina, and R. Kauppinen Product family testing: a survey ACM SIGSOFT Software Engineering Notes 29 2 2004 12
80. D. Šmite, C. Wohlin, T. Gorschek, and R. Feldt Empirical evidence in global software engineering: a systematic review Empirical Software Engineering 15 1 2010 91 118
81. D.M. Weiss The product line hall of fame SPLC '08: Proceedings of the 2008 12th International Software Product Line Conference 2008 IEEE Computer Society Washington, DC, USA 39
82. R. Wieringa, N.A.M. Maiden, N.R. Mead, and C. Rolland Requirements engineering paper classification and evaluation criteria: a proposal and a discussion Requirements Engineering 11 1 2006 102 107 (Pubitemid 43118937)
83. A. Wübbeke, Towards an efficient reuse of test cases for software product lines, in: SPLC'08: Proceedings of Software Product Line Conference, 2008, pp. 361-368.
84. H. Zeng, W. Zhang, D. Rine, Analysis of testing effort by using core assets in software product line testing, in: SPLIT '04: Proceedings of the International Workshop on Software Product Line Testing, Boston, MA. 2004, pp. 1-6.