Achieving scalable model-based testing through test case diversity

ACM Transactions on Software Engineering and Methodology

Volumn 22, Issue 1, 2013, Pages

  Hemmati, Hadi ^a   Arcuri, Andrea ^b   Briand, Lionel ^c  

^a UNIVERSITY OF WATERLOO   (Canada)

^b SIMULA RESEARCH LABORATORY   (Norway)

^c UNIVERSITY OF LUXEMBOURG   (Luxembourg)

Author keywords

Verification

Indexed keywords

AUTOMATED TEST CASE GENERATION; INDUSTRIAL CASE STUDY; INDUSTRIAL CONTEXT; MODEL BASED TESTING; RESOURCE CONSTRAINT; SELECTION TECHNIQUES; SYSTEM LEVEL TESTING; TEST CASE SELECTION;

COMPLEX NETWORKS; FAULT DETECTION; INDUSTRY; SCALABILITY; VERIFICATION;

TESTING;

EID: 84874821350     PISSN: 1049331X     EISSN: 15577392     Source Type: Journal    
DOI: 10.1145/2430536.2430540     Document Type: Article

References (63)

1. AARTS, E. H. L. AND LENSTRA, J. K. 2003. Local Search in Combinatorial Optimization. Princeton University Press.
2. ALI, S., BRIAND, L. C., HEMMATI, H., AND PANESAR-WALAWEGE, R. K. 2010a. A systematic review of the application and empirical investigation of search-based test-case generation. IEEE Trans. Software Eng. 36, 742-762.
3. ALI, S., HEMMATI, H., HOLT, N. E., ARISHOLM, E., AND BRIAND, L. 2010b. Model transformations as a strategy to automate model-based testing: A tool and industrial case studies Simula Research Laboratory, Tech. Rep. 2010-01.
4. ARCURI, A. 2010. Longer is better: On the role of test sequence length in software testing. In Proceedings of the IEEE International Conference on Software Testing, Verification and Validation. 469-478.
5. ARCURI, A. AND BRIAND, L. 2011. A practical guide for using statistical tests to assess randomized algorithms in software engineering. In Proceedings of the ACM/IEEE International Conference on Software Engineering.
6. ARCURI, A. AND YAO, X. 2008. Search based software testing of object-oriented containers. Inf. Sci. 178, 3075-3095.
7. BINDER, R. V. 1999. Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison-Wesley Professional.
8. CARTAXO, E. G., MACHADO, P. D. L., AND NETO, F. G. O. 2011. On the use of a similarity function for test case selection in the context of model-based testing. Software Test. Verif. Reliab. 21, 2, 75-110.
9. CHEN, T. Y., KUOA, F. C., MERKELA, R. G., AND TSEB, T. H. 2010. Adaptive Random Testing: The ART of test case diversity. J. Syst. Software 83, 60-66.
10. CHEN, Y., PROBERT, R. L., AND URAL, H. 2009. Regression test suite reduction based on SDL models of system requirements. J. Software Maintenance Evol. Res. Pract. 21, 379-405.
11. CORMEN, T. H., LEISERSON, C. E., RIVEST, R. L., AND STEIN, C. 2001. Introduction to Algorithms. MIT Press.
12. DALAL, S., JAIN, A., AND POORE, J. 2005.Workshop on advances in model-based software testing. In Proceedings of the 27th International Conference on Software Engineering. 680.
13. DONG, G. AND PEI, J. 2007. Sequence Data Mining. Springer.
14. DROSTE, S., JANSEN, T., AND WEGENER, I. 2002. On the analysis of the (1+1) evolutionary algorithm. Theor. Comput. Sci. 276, 51-81.
15. DURBIN, R., EDDY, S. R., KROGH, A., AND MITCHISON, G. 1999. Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids. Cambridge University Press.
16. ELBAUM, S. G., MALISHEVSKY, A. G., AND ROTHERMEL, G. 2002. Test case prioritization: A family of empirical studies. IEEE Trans. Software Eng. 28, 159-182. (Pubitemid 34198321)
17. FELDT, R., TORKAR, R., GORSCHEK, T., AND AFZAL, W. 2008. Searching for cognitively diverse tests: Towards universal test diversity metrics. In Proceedings of the 1st Workshop on Search-Based Software Testing. 178-186.
18. FELLER, W. 1968. An Introduction to Probability Theory and Its Applications. Wiley.
19. GAMMA, E., HELM, R., JOHNSON, R., AND VLISSIDES, J. 1995. Design Patterns: Elements of Reusable Object- Oriented Software. Addison-Wesley.
20. GOLDBERG, D. E. 2001. Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley Professional.
21. GUSFIELD, D. 1997. Algorithms on Strings, Trees and Sequences: Computer Science and Computational Biology. Cambridge University Press.
22. HAMLET, R. 1994. Random testing. In Encyclopedia of Software Engineering. Wiley, 970-978.
23. HARMAN, M. 2007. The current state and future of search based software engineering. In Proceedings of the Conference on the Future of Software Engineering. IEEE Computer Society, 342-357. (Pubitemid 47484936)
24. HARMAN, M. AND JONES, B. F. 2001. Search-based software engineering. Inf. Software Tech. 43, 833-839. (Pubitemid 33050963)
25. HARMAN, M. AND MCMINN, P. 2010. A theoretical and empirical study of search based testing: Local, global and hybrid search. IEEE Trans. Software Eng. 36, 226-247.
26. HAUPT, R. L. AND HAUPT, S. E. 1998. Practical Genetic Algorithms. Wiley-Interscience.
27. HEMMATI, H., ARCURI, A., AND BRIAND, L. 2010b. Reducing the cost of model-based testing through test case diversity. In Proceedings of the 22nd IFIP International Conference on Testing Software and Systems. Lecture Notes in Computer Science, vol. 6435. 63-78.
28. HEMMATI, H., ARCURI, A., AND BRIAND, L. 2011. Empirical investigation of the effects of test suite properties on similarity-based test case selection. In Proceedings of the 4th International Conference on Software Testing, Verification and Validation.
29. HEMMATI, H. AND BRIAND, L. 2010. An industrial investigation of similarity measures for model-based test case selection. In Proceedings of the 21st IEEE International Symposium on Software Reliability Engineering. 141-150.
30. HEMMATI, H., BRIAND, L., ARCURI, A., AND ALI, S. 2010a. An enhanced test case selection approach for modelbased testing: An industrial case study. In Proceedings of the 18th ACM International Symposium on Foundations of Software Engineering. 267-276.
31. JAIN, A. K. 2010. Data clustering: 50 years beyond K-means. Pattern Recognit. Lett. 31, 8, 651-666.
32. JIANG, B., ZHANG, Z., CHAN, W. K., AND TSE, T. H. 2009. Adaptive random test case prioritization. In Proceedings of the 25th IEEE/ACM International Conference on Automated Software Engineering. 233-244.
33. JONES, J. A. AND HARROLD, M. J. 2003. Test-suite reduction and prioritization for modified condition/decision coverage. IEEE Trans. Software Eng. 29, 195-209.
34. JOURDAN, G. V., RITTHIRUANGDECH, P., AND URAL, H. 2006. Test suite reduction based on dependence analysis. In Proceedings of the 21th International Symposium on Computer and Information Sciences. Springer, 1021-1030. (Pubitemid 44859627)
35. KOREL, B., KOUTSOGIANNAKIS, G., AND TAHAT, L. H. 2007. Model-based test prioritization heuristic methods and their evaluation. In Proceedings of the 3rd Workshop on Advances in Model Based Testing. 34-43. (Pubitemid 351204015)
36. LAKHOTIA, K., HARMAN, M., AND MCMINN, P. 2007. A multi-objective approach to search-based test data generation. In Proceedings of the Genetic and Evolutionary Computation Conference. 1098-1105. (Pubitemid 47291668)
37. LEDRU, Y., PETRENKO, A., AND BORODAY, S. 2009. Using string distances for test case prioritisation. In Proceedings of the 24th IEEE/ACM International Conference on Automated Software Engineering. 510-514.
38. LEON, D. AND PODGURSKI, A. 2003. A comparison of coverage-based and distribution-based techniques for filtering and prioritizing test cases. In Proceedings of the 14th IEEE International Symposium on Software Reliability Engineering. 442-456.
39. LI, Z., HARMAN, M., AND HIERONS, R. M. 2007. Search algorithms for regression test case prioritization. IEEE Trans. Software Eng. 33, 225-237. (Pubitemid 46523213)
40. MA, X. Y., SHENG, B. K. AND YE, C. Q. 2005. Test-Suite Reduction Using Genetic Algorithm. In Advanced Parallel Processing Technologies. Springer, 253-262.
41. MANNING, C. D., RAGHAVAN, P., AND SCHÜTZE, H. 2008. Introduction to Information Retrieval. Cambridge University Press.
42. MASRI, W., PODGURSKI, A., AND LEON, D. 2007. An empirical study of test case filtering techniques based on exercising information flows. IEEE Trans. Software Eng. 33, 454-477. (Pubitemid 46965228)
43. MATHUR, A. P. 2008. Foundations of Software Testing. Addison-Wesley Professional.
44. MCMASTER, S. AND MEMON, A. 2008. Call-stack coverage for GUI test suite reduction. IEEE Trans. Software Eng. 34, 99-115. (Pubitemid 351350371)
45. MICHALEWICZ, Z. AND SCHOENAUER, M. 1996. Evolutionary Algorithms for Constrained Parameter Optimization Problems. Evol. Comput. 4, 1-32. (Pubitemid 126707158)
46. MOSCATO, P. AND COTTA, C. 2010. A modern introduction to memetic algorithms. In Handbook of Metaheuristics. Springer, 141-183.
47. ORSO, A., DO, H., ROTHERMEL, G., HARROLD, M. J., AND ROSENBLUM, D. S. 2007. Using component metadata to regression test component-based software. Software Test. Verif. Reliab. 17, 61-94. (Pubitemid 46865346)
48. PENDER, T. 2003. UML Bible. Wiley.
49. RAMANATHAN, M. K., KOYUT ̈URK, M., GRAMA, A., AND JAGANNATHAN, S. 2008. PHALANX: A graph-theoretic framework for test case prioritization. In Proceedings of the 23rd Annual ACM Symposium on Applied Computing. 667-673.
50. ROTHERMEL, G., HARROLD, M. J., RONNE, J. V., AND HONG, C. 2002. Empirical studies of test-suite reduction. Software Test. Verif. Reliab. 12, 219-249.
51. SIMÃO, A. D. S., MELLO, R. F. D., AND SENGER, L. J. 2006. A technique to reduce the test case suites for regression testing based on a self-organizing neural network architecture. In Proceedings of the 30th Annual International Computer Software and Applications Conference. 93-96. (Pubitemid 46661711)
52. TAN, P. N., STEINBACH, M., AND KUMAR, V. 2006. Introduction to Data Mining. Addison-Wesley.
53. UTTING, M. AND LEGEARD, B. 2006. Practical Model-Based Testing: A Tools Approach. Morgan-Kaufmann.
54. VAANDRAGER, F. W. 2006. Does it pay off? Model-based verification and validation of embedded systems. Tech.Rep. ICIS-R06019, ICIS Radboud University Nijmegen, 43-66.
55. VARGHA, A. AND DELANEY, H. D. 2000. A critique and improvement of the CL common language effect size statistics of McGraw and Wong. J. Educ. Behav. Stat. 25, 101-132.
56. VEGA, D., SCHIEFERDECKER, I., AND DIN, G. 2007. Test data variance as a test quality measure: Exemplified for TTCN-3. In Proceedings of the 19th IFIP International Conference on Testing of Communicating Systems and 7th International Workshop on Formal Approaches to Testing of Software. 351-364. (Pubitemid 47290171)
57. WHITE, L. J. AND COHEN, E. I. 1980. A domain strategy for computer program testing. IEEE Trans. Software Eng. 6, 247-257.
58. WHITLEY, D. 1989. The GENITOR algorithm and selection pressure: Why rank-based allocation of reproductive trials is best. In Proceedings of the 3rd International Conference on Genetic Algorithms. 116-121.
59. WOHLIN, C., RUNESON, P., HOST, M., OHLSSON, M. C., REGNELL, B., AND WESSLEN, A. 2000. Experimentation in Software Engineering: An Introduction. Kluwer Academic Publishers.
60. WOLPERT, D. AND MACREADY, W. G. 1997. No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1, 67-82. (Pubitemid 127771019)
61. XIAO, M., EL-ATTAR, M., REFORMAT, M., AND MILLER, J. 2007. Empirical evaluation of optimization algorithms when used in goal-oriented automated test data generation techniques. Empirical Soft. Eng. 12, 183-239. (Pubitemid 46375363)
62. XU, R. AND WUNSCH, D. 2005. Survey of clustering algorithms. IEEE Trans. Neural Networks 16, 645-678. (Pubitemid 40718010)
63. YOO, S., HARMAN, M., TONELLA, P., AND SUSI, A. 2009. Clustering test cases to achieve effective and scalable prioritisation incorporating expert knowledge. In Proceedings of the 18th ACM International Symposium on Software Testing and Analysis. 201-212.