A framework for observation-based modelling in model-based testing

VTT Publications

Volumn , Issue 727, 2010, Pages 1-211

  Kanstrén, Teemu ^a  

^a NONE

Author keywords

Model based testing; Observation based modelling; Test automation; Test generation

Indexed keywords

ABSTRACTION LEVEL; ARCHITECTURAL SOLUTIONS; AUTOMATED ASSISTANCE; AUTOMATED TOOL SUPPORT; CONSTRUCTIVE APPROACH; EXPENSIVE PARTS; MODEL BASED TESTING; MODEL CREATION; OBSERVATION-BASED MODELLING; PRACTICAL SYSTEMS; SOFTWARE LIFE CYCLES; SYSTEM UNDER TEST; TEST AUTOMATION; TEST COVERAGE; TEST GENERATIONS; TEST MODELS;

AUTOMATION; COMPUTER SOFTWARE SELECTION AND EVALUATION; PROGRAM DEBUGGING; RESEARCH; SOFTWARE ENGINEERING; SOFTWARE TESTING; SPECIFICATIONS; TESTING;

MATHEMATICAL MODELS;

EID: 78651264720     PISSN: 12350621     EISSN: 14550849     Source Type: Book Series    
DOI: None     Document Type: Article

References (122)

1. Bertolino, A. Software Testing Research: Achievements, Challenges, Dreams. In: Proceedings of Future of Software Engineering (FOSE07), 2007. Pp. 85-103.
2. Knuth, D. Knuth: Frequently Asked Questions. [Online]. HYPERLINK: http://www-csfaculty.stanford.edu/~knuth/faq.html.
3. Gelperin, D. and Hetzel, B. The Growth of Software Testing. Communications of the ACM, Vol. 31, No. 6, pp. 687-695, June 1988.
4. Utting, M. and Legeard, B. Practical Model-Based Testing: A Tools Approach. Morgan Kaufmann, 2007.
5. Bertolino, A., Polini, A., Inverardi, P. and Muccini, H. Towards Anti-Model-Based-Testing. In: Fast Abstracts in International Conference on Dependable Systems and Networks (DSN'04), Florence, 2004.
6. th ed. Addison Wesley, 2006.
7. Bertolino, A. and Marchetti, E. Software Testing. In: Guide to Software Engineering Body of Knowledge (SWEBOK). Bourque, P. and Dupuis, R. (eds.). IEEE Computer Society, 2004, ch. 5. Pp. 5-1-5-16.
8. st International Conference on Software & Systems Engineering and their Applications, Paris, France, 2008.
9. Kanstrén, T. Towards a Deeper Understanding of Test Coverage. Journal of Software Maintenance and Evolution: Research and Practice, Vol. 20, No. 1, pp. 59-76, 2008.
10. Glass, R. L., Vessey, I. and Ramesh, V. Research in Software Engineering: An Analysis of the Literature. Information and Software Technology, Vol. 44, pp. 491-506, 2002.
11. Runeson, P. and Höst, M. Guidelines for conducting and reporting case study research in software engineering. Empirical Software Engineering, Vol. 14, No. 2, pp. 131-164, April 2009.
12. Järvinen, P. On Research Methods. Tampere, Finland: Tampereen yliopistopaino Oy, 2004.
13. Nunamaker, J. F., Chem, M. and Purdin, T. D. M. Systems development in information systems research. Journal of Management Information Systems, Vol. 7, No. 3, pp. 89-106, 1991.
14. March, S. T. and. Smith, G. F. Design and Natural Science Research on Information Technology. Decision Support Systems, Vol. 15, pp. 251-266, 1995.
15. Hevner, A. R., March, S. T., Park, J. and Ram, S. Design Science in Information Systems Research. Management Information Systems Quarterly, Vol. 28, No. 1, 2004.
16. Ramesh, V., Glass, R. L. and Vessey, I. Research in Computer Science: An Empirical Study. Journal of Systems and Software, Vol. 70, pp. 165-176, 2004.
17. Bealey, M. Analysis. In: Stanford Encyclopedia of Philosophy, 2009. [Online]. HYPERLINK: http://plato.stanford.edu/entries/analysis/http://plato. stanford.edu/entries/analysis/.
18. Kitchenham, B., Pickard, L. and Pfleeger, S. L. Case Studies for Method and Tool Evaluation. IEEE Software, Vol. 12, No. 4, pp. 52-62, July 1995.
19. Harjumaa, L. Improving the software inspection process with patterns. Oulu, Finland: University of Oulu, 2005.
20. th International Conference on Software Engineering (ICSE05), St. Louis, Missouri, USA, 2005. Pp. 571-579.
21. th International Conference on Software Engineering Research, Management and Applications, Prague, Czech Republic, 2008. Pp. 31-38.
22. Binder, R. V. Design for Testability in Object-Oriented Systems. Communications of the ACM, Vol. 37, No. 9, pp. 87-101, September 1994.
23. th IEEE International Conference on Automated Software Engineering (ASE04), 2004. Pp. 112-121.
24. nd ed. Addison-Wesley, 2003.
25. th International Conference on Software Engineering (ICSE06), Shanghai, China, 2006. Pp. 92-101.
26. Rehman, M. J., Jabeen, F., Bertolino, A. and Polini, A. Testing Software Components for Integration: A Survey of Issues and Techniques. Journal of Software Testing, Verification and Reliability, Vol. 17, pp. 95-133, 2007.
27. Ernst, M. D., Perkins, J. H., Guo, P. J., McCamant, S., Pacheco, C., Tschantz, M. S., and Xiao, C. The Daikon System for Dynamic Detection of Likely Invariants. Science of Computer Programming, 2007.
28. th International Conference on Software Engineering (ICSE'05), 2005. Pp. 402-411.
29. Do, H., Elbaum, S. and Rothermel, G. Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact. Empirical Software Engineering, Vol. 10, No. 4, pp. 405-435, 2005.
30. DeMillo, R. A., Lipton, R. J. and Sayward, F. G. Hints on Test Data Selection: Help for the Practicing Programmer. IEEE Computer, Vol. 11, No. 4, 1978.
31. DeMillo, R. A. and Offutt, A. J. Constraint-Based Automatic Test Data Generation. IEEE Transactions on Software Engineering, Vol. 17, No. 9, September 1991.
32. nd IEEE International Conference on Software Maintenance (ICSM'06), 2006.
33. Santelices, R., Jones, J. A., Yu, Y. and Harrold, M. J. Lightweight Fault-Localization Using Multiple Coverage Types. In: International Conference on Software Engineering (ICSE'09), Vancouver, Canada, 2009.
34. Rothermel, G., Elbaum, S., Malishevsky, A. G., Kallakuri, P. and Qiu, X. On Test Suite Composition and Cost-Effective Regression Testing. ACM Transactions on Software Engineering, Vol. 13, No. 3, pp. 277-331, 2004.
35. Wilde, N. and Scully, M. C. Software Reconnaissance: Mapping Program Features to Code. Journal of Software Maintenance: Research and Practice, Vol. 7, No. 1, January/February 1995.
36. Eisenbarth, T., Koschke, R. and Simon, D. Locating Features in Source Code. IEEE Transactions on Software Engineering, Vol. 29, No. 3, pp. 210-214, March 2003.
37. Zeller, A. Why Programs Fail: A Guide to Systematic Debugging. Morgan Kaufmann, 2005.
38. Andrews, J. H., Briand, L. C., Labiche, Y. and Namin, A. S. Using Mutation Analysis for Assessing and Comparing Testing Coverage Criteria. IEEE Transactions on Software Engineering, Vol. 32, No. 8, pp. 608-624, August 2006.
39. Harder, M., Mellen, J. and Ernst, M. D. Improving Test Suites via Operational Abstraction. In: International Conference on Software Engineering, Portland, Oregon, 2003. Pp. 60-71.
40. Duran, J. and Ntafos, S. An Evaluation of Random Testing. IEEE Transactions on Software Engineering, Vol. 10, No. 4, pp. 438-444, 1984.
41. Gutjahr, W. Partition Testing Versus Random Testing: The Influence of Uncertainty. IEEE Transactions on Software Engineering, Vol. 25, No. 5, pp. 661-674, 1999.
42. Ciupa, I., Leitner, A., Oriol, M. and Meyer, B. ARTOO: Adaptive Random Testing for Object-Oriented Software, 2008, pp. 71-80.
43. Boshernitsan, M., Doong, R. and Savoia, A. From Daikon to Agitator: Lessons and Challenges in Building a Commercial Tool for Developer Testing. In: Proc. International Symposium on Software Testing and Analysis (ISSTA'06), Portland, Maine, 2006. Pp. 169-179.
44. st International Conference on Software Testing, Verification and Validation, Lillehammer, Norway, 2008. Pp. 3-11.
45. Clarke, L. A. A System to Generate Test Data and Symbolically Execute Programs. IEEE Transactions on Software Engineering, Vol. 2, No. 3, pp. 215-222, September 1976.
46. King, J. C. Symbolic Execution and Program Testing. Communications of the ACM, Vol. 19, No. 7, pp. 385-394, July 1976.
47. Gotlieb, A., Botella, B. and Rueher, M. Automatic Test Data Generation using Constraint Solving Techniques. In: Proc. International Symposium on Software Testing and Analysis (ISSTA'98), 1998. Pp. 53-62.
48. Tillmann, N. and Schulte, W. Unit Tests Reloaded: Parameterized Unit Testing with Symbolic Execution. IEEE Software, Vol. 23, No. 4, pp. 38-47, July/August 2006.
49. th International Conference on Software Engineering (ICSE'96), 1996. Pp. 71-80.
50. McMinn, P. Search-Based Software Test Data Generation: A Survey. Journal of Software Testing, Verification and Reliability, Vol. 14, pp. 105-156, 2004.
51. Korel, B. Automated Software Test Data Generation. IEEE Transactions on Software Engineering, Vol. 16, No. 8, pp. 870-879, 1990. (Pubitemid 20738724)
52. Grindal, M., Offutt, J. and Andler, S. F. Combination Testing Strategies: A Survey. Journal of Software Testing, Verification and Reliability, Vol. 15, pp. 167-199, 2005.
53. Ostrand, T. J. and Balcer, M. J. The Category-Partition Method for Specifying and Generating Functional Tests. Communications of the ACM, Vol. 31, No. 6, pp. 676-686, June 1988.
54. th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on Foundations of Software Engineering, Amsterdam, The Netherlands, 2009.
55. Harman, M. The Current State and Future of Search Based Software Engineering. In: Proc. Future of Software Engineering (FOSE'07), 2007.
56. th International Conference on Information Systems Analysis and Synthesis, Orlando, Florida, 2003.
57. th Annual Conference on Genetic and Evolutionary Computation (GECCO'07), 2007. Pp. 1098-1105.
58. Tonella, P. Evolutionary Testing of Classes. In: Proc. International Symposium of Software Testing and Analysis (ISSTA'04), Boston, Massachusetts, 2004. Pp. 119-128.
59. th Annual Conference on Genetic and Evolutionary Computation (GECCO'07), 2007. Pp. 1074-1081.
60. Baudry, B., Fleurey, F., Jézéqual, J.-M. and Traon, Y. L. From Genetic to Bacteriological Algorithms for Mutation-Based Testing. Journal of Software Testing, Verification and Reliability, Vol. 15, pp. 73-96, 2005.
61. nd International Workshop on Automation of Software Test (AST'07), 2007.
62. Sneed, H. and Huang, S. The Design and use of WSDL-Test: A Tool for Testing Web Services. Journal of Software Maintenance and Evolution: Research and Practice, Vol. 19, pp. 297-314, 2007.
63. th International Conference on Software Engineering (ICSE'07), 2007.
64. th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on Foundations of Software Engineering (ESEC/FSE'07), Dubrovnik, Croatia, 2007. Pp. 185-194.
65. th International Conference on Software Engineering (ICSE'07), 2007.
66. Pacheso, C. and Ernst, M. D. Eclat: Automatic Generation and Classification of Test Inputs. In: Proc. European Conference on Object-Oriented Programming (ECOOP'05), 2005. Pp. 504-527.
67. Elbaum, S. and Diep, M. Profiling Deployed Software: Assessing Strategies and Testing Opportunities. IEEE Transactions on Software Engineering, Vol. 31, No. 4, pp. 312-327, April 2005.
68. th International Conference on Software Engineering, Minneapolis, USA, 2007.
69. Pacheco, C., Lahiri, S. K. and Ball, T. Finding Errors in. NET with Feedback-Directed Random Testing. In: Proceedings of the International Symposium on Software Testing and Analysis (ISSTA08), Seattle, Washington, USA, 2008. Pp. 87-95.
70. th International Conference on Software Engineering, Melbourne, Australia, 1992. Pp. 105-118.
71. th Int'l. Conf. on Software. Eng. Advances (ICSEA2009), Porto, Portugal, 2009.
72. Antoy, S. and Hamlet, D. Automatically Checking an Implementation against Its Formal Specification. IEEE Transactions on Software Engineering, Vol. 26, No. 1, pp. 55-69, January 2000.
73. Andrews, J. H. and Zhang, Y. General Test Result Checking with Log File Analysis. IEEE Transaction on Software Eng., Vol. 29, No. 7, pp. 634-648, July 2003.
74. Ducasse, S., Gîrba, T. and Wuyts, R. Object-Oriented Legacy System Trace-Based Logic Testing. In: Proc. European Conference on Software Maintenance and Reengineering (CSMR'06), 2006.
75. th International Conference on Program Comprehension (ICPC'06), 2006, Behavioral Program Documentation Allowing Lightweight Verification.
76. Xie, T. and Notkin, D. Tool-Assisted Unit-Test Generation and Selection Based on Operational Abstractions. Journal of Automated Software Engineering, Vol. 13, No. 3, pp. 345-371, July 2006.
77. st International Conference on Software Engineering, Vancouver, Canada, 2009.
78. th International Conference on Automated Software Engineering, 2004.
79. Haran, M., Karr, A., Last, M., Orso, A., Porter, A. A., Sanil, A. and Fouché, S. Techniques for Classifying Executions of Deployed Software to Support Software Engineering Tasks. IEEE Transactions on Software Engineering, Vol. 33, No. 5, pp. 287-304, May 2007.
80. Jin, H., Wang, Y., Chen, N.-W., Gou, Z.-J. and Wang, S. Artificial Neural Network for Automatic Test Oracles Generation. In: International Conference on Computer Science and Software Engineering, Wuhan, Hubei, 2008. Pp. 727-730.
81. Bowring, J. F., Rehg, J. M. and Harrold, M. J. Active Learning for Automatic Classification of Software Behaviour. In: Proceedings of the International Symposium on Software Testing and Analysis, Boston, Massachusetts, USA, 2004. Pp. 195-205.
82. Elbaum, S., Chin, H. N., Dwyer, M. B. and Jorde, M. Carving and Replaying Differential Unit Test Cases from System Test Cases. IEEE Transactions on Software Engineering, Vol. 35, No. 1, pp. 29-45, January/February 2009.
83. Mackinnon, T., Freeman, S. and Craig, P. Endo-Testing: Unit Testing with Mock Objects. In: Proceedings of eXtreme Programming and Flexible Processes in Software Engineering - XP2000, Cagliari, Sardinia, Italy, 2000.
84. st IEEE/ACM International Conference on Automated Software Engineering, Tokyo, Japan, 2006. Pp. 365-368.
85. th Annual IEEE/ACM International Conference on Automated Software Engineering (ASE 2005), Long Beach, CA, USA, 2005, pp. 114-123.
86. Beyer, D., Chlipala, A. J., Henzinger, T. A., Jhala, R. and Majumdar, R. Generating Tests from Counterexamples. In: Proceedings of the 26th International Conference on Software Engineering, 2004. Pp. 326-335.
87. Pesonen, J. Extending Software Integration Testing Using Aspects in Symbian OS. In: Proceedings of Testing: Academic and Industrial Conference - Practice and Research Techniques (TAIC-PART 2006), 2006. Pp. 147-151.
88. th Euromicro Conference on Software Engineering and Advanced Applications, Parma, Italy, 2008. Pp. 321-328.
89. Binder, R. V. Testing Object-Oriented Systems - Models, Patterns and Tools. Addison-Wesley, 1999.
90. st Workshop on Model-Based Testing in Practice (MoTiP2008), Berlin, Germany, 2008.
91. Neto, A. D., Subramanyan, R., Vieira, M., Travassos, G. H. and Shull, F. Improving Evidence about Software Technologies: A Look at Model-Based Testing. IEEE Software, Vol. 25, No. 3, pp. 10-13, May/June 2008.
92. Ramamoorthy, C. V., Ho, S. F. and Chen, W. T. On the Generation of Program Test Data. IEEE Transaction on Software Engineering, Vol. 2, No. 4, pp. 293-300, 1976.
93. th IEEE International Conference on Engineering of Complex Computer Systems (ICECCS02), 2002. Pp. 163-169.
94. Bringmann, E. and Krämer, A. Model-Based Testing of Automotive Systems. In: Proceedings of the International Conference on Software Testing, Verification and Validation, 2008. Pp. 485-493.
95. th International Conference on Software Engineering, St. Louis, Missouri, USA, 2005. Pp. 392-401.
96. th International Conference on Software Engineering (ICSE08), Leipzig, Germany, 2008. Pp. 669-671.
97. Chikofsky, E. J. and Cross II, E. J. Reverse Engineering and Design Recovery: A Taxonomy. IEEE Software, Vol. 7, No. 1, pp. 13-17, 1990.
98. Storey, M.-A. Theories, Tools and Research Methods in Program Comprehension: Past, Present and Future. Software Quality Journal, Vol. 14, No. 3, pp. 183-208, September 2006.
99. Cornelissen, B., Zaidman, A., Deursen, A. van, Moonen, L. and Koschke, R. A Systematic Survey of Program Comprehension through Dynamic Analysis. IEEE Transaction on Software Eng., 2009.
100. th International Conference on Internet Monitoring and Protection (ICIMP 2009), Venice/Mestre, Italy, 2009. Pp. 109-115.
101. Kanstrén, T., Piel, E. and Gross, H.-G. Observation Based Modeling for Model-Based Testing. Submitted to Journal of Software Testing, Verification and Reliability, 2009.
102. Biggerstaff, T., Mitbander, B. G. and Webster, D. E. Program Understanding and the Concept Assignment Problem. Communications of the ACM, Vol. 37, No. 5, pp. 72-82, May 1994.
103. Antoniol, G. and Guéhéneuc, Y.-G. Feature Identification: An Epidemiological Metaphor. IEEE Transactions on Software Engineering, Vol. 32, No. 9, pp. 627-641, September 2006.
104. th IEEE International Conference on Program Comprehension, 2008. Pp. 236-241.
105. Robillard, M. P. and Murphy, G. C. Representing Concerns in Source Code. ACM Transactions on Software Engineering and Methodology, Vol. 16, No. 1, February 2007.
106. th International Conference on Software Engineering (ICSE'07), 2007.
107. th International Conference on Software Engineering (ICSE08), Leipzig, Germany, 2008. Pp. 501-510.
108. th International Conference on Formal Engineering Methods (ICFEM04), Seattle, Washington, USA, 2004. Pp. 290-305.
109. Cook, J. E. and Du, Z. Discovering Thread Interactions in a Concurrent System. Journal of Systems and Software, Vol. 77, No. 3, pp. 285-297, September 2005.
110. Cook, J. E. and Wolf, A. Discovering Models of Software Processes from Event-Based Data. ACM Transactions on Software Engineering and Methodology, Vol. 7, No. 3, pp. 215-249, 1998.
111. Walkinshaw, N., Bogdanov, K., Ali, S. and Holcombe, M. Automated Discovery of State Transitions and their Functions in Source Code. Software Testing, Verification and Reliability, Vol. 18, No. 2, pp. 99-121, June 2008.
112. Parsons, T., Mos, A., Trofin M., Gshwind, T. and Murphy, J. Extracting Interactions in Component-Based Systems. IEEE Transactions on Software Engineering, Vol. 34, No. 6, pp. 783-799, November/December 2008.
113. Briand, L. C., Labiche, Y. and Leduc, J. Towards the Reverse Engineering of UML Sequence Diagrams for Distributed Java Software. IEEE Transactions on Software Engineering, Vol. 32, No. 9, pp. 642-663, September 2006.
114. Bennett, C., Myers, D., Storey, M.-A., German, D. M., Ouellet, D., Salois, M. and Charland, P. A Survey and Evaluation of Tool Features for Understanding Reverse-Engineered Sequence Diagrams. Journal of Software Maintenance and Evolution: Research and Practice, Vol. 20, No. 4, pp. 291-315, July 2008.
115. th International Conference on Software Engineering, Portland, Oregon, USA, 2003. Pp. 430-441.
116. Schmerl, B., Aldrich, J., Garlan, D., Kazman, R. and Yan, H. Discovering Architectures from Running Systems. IEEE Transactions on Software Engineering, Vol. 32, No. 7, pp. 454-466, July 2006.
117. Aalst, W. M. P. van der, Rubin, V., Verbeek, H. M. W., Dongen, B. F. van, Kindler, E. and Günther, C. W. Process Mining: A Two-Step Approach to Balance Between Underfitting and Overfitting. Software and Systems Modeling (SoSyM), 2009.
118. th International Conference on Applications and Theory of Petri Nets and Other Models of Concurrency (ICATPN07), Siedlce, Poland, 2007.
119. Lo, D., Khoo, S.-C. and Liu, C. Mining Temporal Rules for Software Maintenance. Journal of Software Maintenance and Evolution: Research and Practice, Vol. 20, No. 4, pp. 227-247, 2008.
120. th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE 2009).
121. th International IEEE Conference on Program Comprehension (ICPC06). Pp. 212-224.
122. Jacky, J., Veanes, M., Campbell, C. and Schulte, W. Model-Based Software Testing and Analysis with C#. Cambridge University Press, 2008.