INTRODUCTION TO SOFTWARE TESTING

Introduction to Software Testing

Volumn , Issue , 2016, Pages 1-345

  Ammann, Paul ^a   Offutt, Jeff ^a  

^a GEORGE MASON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85123212227     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/9781316771273     Document Type: Book

References (325)

1. Abdurazik, A. and Offutt, J. (2000). Using UML collaboration diagrams for static checking and test generation. In Proceedings of the Third International Conference on the Unified Modeling Language (UML '00), pages 383-395, York, UK.
2. Abdurazik, A. and Offutt, J. (2006). Coupling-based class integration and test order. In Workshop on Automation of Software Test (AST 2006), pages 50-56, Shanghai, China.
3. Abdurazik, A. and Offutt, J. (2007). Using couplingbased weights for the class integration and test order problem. The Computer Journal, pages 1-14. DOI: 10.1093/comjnl/bxm054.
4. Acree, A. T., Budd, T. A., DeMillo, R. A., Lipton, R. J., and Sayward, F. G. (1979). Mutation analysis. Technical report GIT-ICS-79/08, School of Information and Computer Science, Georgia Institute of Technology, Atlanta, GA.
5. Akers, S. B. (1959). On a theory of boolean functions. Journal Society Industrial Applied Mathematics, 7(4):487-498.
6. Alexander, R. T. and Offutt, J. (1999). Analysis techniques for testing polymorphic relationships. In Proceedings of the Thirtieth IEEE International Conference on Technology of Object-Oriented Languages and Systems (TOOLS USA '99), pages 104-114, Santa Barbara, CA.
7. Alexander, R. T. and Offutt, J. (2000). Criteria for testing polymorphic relationships. In Proceedings of the 11th IEEE International Symposium on Software Reliability Engineering, pages 15-23, San Jose, CA.
8. Alexander, R. T. and Offutt, J. (2004). Coupling-based testing of O-O programs. Journal of Universal Computer Science, 10(4):391-427.
9. Allen, F. E. and Cocke, J. (1976). Aprogram data flow analysis procedure. Communications of the ACM, 19(3):137-146.
10. Ambler, S. and Associates (2004). Examining the agile cost of change curve. Agile modeling online blog. www.agilemodeling.com/essays/costOfChange.htm, last access: February 2016.
11. Ammann, P. and Black, P. E. (2000). A specification-based coverage metric to evaluate test sets. International Journal of Quality, Reliability, and Safety Engineering, 8(4):1-26.
12. Ammann, P. E. and Knight, J. C. (1988). Data diversity:An approach to software fault tolerance. IEEE Transactions on Computers, 37(4):418-425.
13. Ammann, P. and Offutt, J. (1994). Using formal methods to derive test frames in category-partition testing. In Proceedings of the Ninth Annual Conference on Computer Assurance (COMPASS 94), pages 69-80, Gaithersburg, MD.
14. Ammann, P. E., Black, P. E., and Majurski, W. (1998). Using model checking to generate tests from specifications. In Second IEEE International Conference on Formal Engineering Methods (ICFEM’98), pages 46-54, Brisbane, Australia.
15. Ammann, P., Offutt, J., and Huang, H. (2003). Coverage criteria for logical expressions. In Proceedings of the 14th IEEE International Symposium on Software Reliability Engineering, pages 99-107, Denver, CO.
16. Ammann, P., Frazer, G., and Franz Wotawa, e. (2012a). Special issue on model-based testing volume 1: Foundations and applications of modelbased testing. Software Testing, Verification, and Reliability, 22(5).
17. Ammann, P., Frazer, G., and Franz Wotawa, e. (2012b). Special issue onmodel-based testing volume 2: Formal approaches to model-based testing. Software Testing, Verification, and Reliability, 22(6).
18. Ammann, P., Frazer, G., and Franz Wotawa, e. (2012c). Special issue on model-based testing volume 3: Beyond conformance testing. Software Testing, Verification, and Reliability, 22(7).
19. Anand, S., Burke, E. K., Chen, T. Y., Clark, J., Cohen, M. B., Grieskamp, W., Harman, M., Harrold, M. J., and McMinn, P. (2013). An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software, 86(8):1978-2001.
20. Andre, D. M. S. (1979). Pilot mutation system (PIMS) user’s manual. Technical report GIT-ICS-79/04, Georgia Institute of Technology.
21. Andrews, J. H., Briand, L. C., Labiche, Y., and Namin, A. S. (2006). Using mutation analysis for assessing and comparing testing coverage criteria. IEEE Transactions on Software Engineering, 32(8):608.
22. Ardis, M., Budgen, D., Hislop, G. W., Offutt, J., Sebern, M., and Visser, W. (2015). SE2014: Curriculum guidelines for undergraduate degree programs in software engineering. IEEE Computer, 48(11):106-109. Full report:www.acm.org/education/se2014.pdf, last access: July 2016.
23. Atlee, J. M. (1994). Native model-checking of SCR requirements. In Fourth International SCR Workshop.
24. Atlee, J. M. and Gannon, J. (1993). State-based model checking of event-driven system requirements. IEEE Transactions on Software Engineering, 19(1):24-40.
25. Avizienis, A. (1985). The N-version approach to fault-tolerant software. IEEE Transactions on Software Engineering, SE11(12):1491-1501.
26. Balcer, M., Hasling, W., and Ostrand, T. (1989). Automatic generation of test scripts from formal test specifications. In Proceedings of the Third IEEE Symposium on Software Testing, Analysis, and Verification, pages 210-218, KeyWest, FL. ACM SIGSOFT 89.
27. Barr, E., Harman, M., McMinn, P., Shahbaz, M., and Yoo, S. (2015a). The oracle problem in software testing: A survey. IEEE Transactions on Software Engineering, 41(5):507-525.
28. Barr, E., Harman, M., McMinn, P., Shahbaz, M., and Yoo, S. (2015b). Repository of publications on the test oracle problem. Online. http://crestweb.cs.ucl.ac.uk/resources/oracle_repository, last access: February 2016.
29. Bauer, J. A. and Finger, A. B. (1979). Test plan generation using formal grammars. In Fourth International Conference on Software Engineering, pages 425-432, Munich, Germany.
30. Beck, K., Beedle, M., van Bennekum, A., Cockburn, A., Cunningham, W., Fowler, M., Grenning, J., Highsmith, J., Hunt, A., Jeffries, R., Kern, J., Marick, B., Martin, R. C., Mellor, S., Schwaber, K., Sutherland, J., and Thomas, D. (2001). The agile manifesto. Online Report. http://agilemanifesto.org, last access:July 2016.
31. Beizer, B. (1984). Software System Testing and Quality Assurance. Van Nostrand, New York, NY.
32. Beizer, B. (1990). Software Testing Techniques. Van Nostrand Reinhold, Inc, New York, NY, 2nd edition.
33. Beust, C. and Suleiman, H. (2008). Next Generation Java Testing: TestNG and Advanced Concepts. Addison-Wesley, Upper Saddle River, NJ.
34. Binder, R. V. (1994). Design for testability in object-oriented systems. Communications of the ACM, 37(9):87-101.
35. Binder, R. (2000). Testing Object-oriented Systems. Addison-Wesley Publishing Company Inc., New York, NY.
36. Bird, D. L. and Munoz, C. U. (1983). Automatic generation of random self-checking test cases. IBM Systems Journal, 22(3):229-345.
37. Black, P., Okun, V., and Yesha, Y. (2000). Mutation operators for specifications. In Fifteenth IEEE International Conference on Automated Software Engineering, pages 81-88.
38. Bloch, J. (2008). Effective Java: Second Edition. Addison-Wesley Publishing Company Inc, Boston, MA.
39. Blum, M. and Kannan, S. (1989). Designing programs that check their work. In Twenty-first ACM Symposium on the Theory of Computing, pages 86-97.
40. Blumenstyk, M. (2006). Web application development -Bridging the gap between QA and development. StickyMinds.com. www.stickyminds. com/s.asp?F=S3658_ART_2 last access: February 2016.
41. Borzovs, J., Kalninš, A., and Medvedis, I. (1991). Automatic construction of test sets: Practical approach. In Lecture Notes in Computer Science, Vol 502, pages 360-432. Springer-Verlag.
42. Bowser, J. H. (1988). Reference manual for Ada mutant operators. Technical report GIT-SERC-88/02, Georgia Institute of Technology.
43. Boyer, R. S., Elpas, B., and Levitt, K. N. (1975). Select-A formal system for testing and debugging programs by symbolic execution. In Proceedings of the International Conference on Reliable Software. SIGPLAN Notices, vol. 10, no. 6.
44. Bray, T., Paoli, J., and Sperberg-McQueen, C. M. (1998). Extensible markup language (XML) 1.0. W3C recommendation. www.w3.org/TR/REC-xml, last access: July 2016.
45. Briand, L. and Labiche, Y. (2001). A UML-based approach to system testing. In Proceedings of the Fourth International Conference on the Unified Modeling Language (UML '01), pages 194-208, Toronto, Canada.
46. Briand, L., Feng, J., and Labiche, Y. (2002). Using genetic algorithms and coupling measures to devise optimal integration test orders. In Proceedings of the 14th International Conference on Software Engineering and Knowledge Engineering, pages 43-50, Ischia, Italy.
47. Briand, L., Labiche, Y., and Wang, Y. (2003). An investigation of graph-based class integration test order strategies. IEEE Transactions on Software Engineering, 29(7):594-607.
48. Briand, L. C., Penta, M. D., and Labiche, Y. (2004). Assessing and improving state-based class testing: A series of experiments. IEEE Transaction on Software Engineering, 30(11):770-793.
49. Standard for Software Component Testing (BS 7925-2). British Standards Institute. www.ruleworks.co.uk/testguide/BS7925-2.htm, last access: February 2016.
50. British Computer Society, S. I. G. i. S. T. (2001). Standard for Software Component Testing, Working Draft 3.4. British Computer Society. www.testingstandards.co.uk/ComponentTesting.pdf, last access:July 2016.
51. Brownlie, R., Prowse, J., and Phadke, M. S. (1992). Robust testing of ATandT PMX/StarMAIL using OATS. ATandT Technical Journal, 71(3):41-47.
52. Brun, Y., Holmes, R., Ernst, M. D., and Notkin, D. (2011). Proactive detection of collaboration conflicts. In Proceedings of the 13th European Software Engineering Conference and the 19th ACM SIGSOFT Symposium on Foundations of Software Engineering, pages 168-178, Szeged, Hungary.
53. Budd, T. A. (1980). Mutation Analysis of Program Test Data. PhDthesis, Yale University, New Haven, CT.
54. Budd, T. A. and Angluin, D. (1982). Two notions of correctness and their relation to testing. Acta Informatica, 18(1):31-45.
55. Budd, T. A. and Lipton, R. J. (1978). Proving LISP programs using test data. In Digest for the Workshop on Software Testing and Test Documentation, pages 374-403, Ft. Lauderdale, FL.
56. Budd, T. and Sayward, F. (1977). Users guide to the Pilot mutation system. Technical report 114, Department of Computer Science, Yale University.
57. Budd, T. A., Lipton, R. J., DeMillo, R.A., and Sayward, F. G. (1979). Mutation analysis. Technical report GIT-ICS-79/08, School of Information and Computer Science, Georgia Institute of Technology, Atlanta, GA.
58. Burdy, L., Cheon, Y., Cok, D. R., Ernst, M. D., Kiniry, J. R., Leavens, G. T., Leino, K. R. M., and Poll, E. (2005). An overview of JML tools and applications. International Journal on Software Tools for Technology Transfer, 7:212-232.
59. Burr, K. and Young, W. (1998). Combinatorial test techniques:Table-based automation, test generation and code coverage. In Proceedings of the International Conference on Software Testing, Analysis, and Review (STAR’98), San Diego, CA.
60. Burroughs, K., Jain, A., and Erickson, R. L. (1994). Improved quality of protocol testing through techniques of experimental design. In Proceedings of the IEEE International Conference on Communications (Supercomm/ICC’94), pages 745-752, New Orleans, LA.
61. Buy, U., Orso, A., and Pezze, M. (2000). Automated testing of classes. In Proceedings of the 2000 International Symposium on Software Testing, and Analysis (ISSTA '00), pages 39-48, Portland, OR. IEEE Computer Society Press.
62. Cheatham, T. E., Holloway, G. H., and Townley, J. A. (1979). Symbolic evaluation and the analysis of programs. IEEE Transactions on Software Engineering, 5(4).
63. Chen, T. Y. and Lau, M. F. (2001). Test case selection strategies based on boolean specifications. Software Testing, Verification, and Reliability, 11(3):165-180, Wiley.
64. Chen, T. Y., Poon, P. L., Tang, S. F., and Tse, T. H. (2004). On the identification of categories and choices for specification-based test case generation. Information and Software Technology, 46(13):887-898.
65. Chen, T. Y., Tang, S. F., Poon, P. L., and Tse, T. H. (2005). Identification of categories and choices in activity diagrams. In Fifth International Conference on Quality Software (QSIC 2005), pages 55-63, Melbourne, Australia.
66. Chen, T. Y., Tse, T. H., and Zhou, Z. Q. (2001). Fault-based testing in the absence of an oracle. In Proceedings of the 25th Annual International Computer Software and Applications Conference (COMPSAC 2001), pages 172-178.
67. Chen, T. Y., Tse, T. H., and Zhou, Z. Q. (2011). Semi-proving:An integrated method for program proving, testing, and debugging. IEEE Transactions on Software Engineering, 37(1):109-125.
68. Cherniavsky, J. C. (1979). On finding test data sets for loop free programs. Information Processing Letters, 8(2):106-107.
69. Cherniavsky, J. C. and Smith, C. H. (1986). A theory of program testing with applications. Proceedings of the Workshop on Software Testing, pages 110-121.
70. Chilenski, J. and Richey, L. A. (1997). Definition for a masking form ofmodified condition decision coverage (MCDC). Technical report, Boeing, Seattle, WA.
71. Chilenski, J. J. (2003). Personal communication.
72. Chilenski, J. J. and Miller, S. P. (1994). Applicability of modified condition/decision coverage to software testing. Software Engineering Journal, 9(5):193-200.
73. Chow, T. (1978). Testing software designs modeled by finite-state machines. IEEE Transactions on Software Engineering, SE-4(3):178-187.
74. Clarke, L. A. (1976). A system to generate test data and symbolically execute programs. IEEE Transactions on Software Engineering, 2(3):215-222.
75. Clarke, L. A. and Richardson, D. J. (1985). Applications of symbolic evaluation. Journal of Systems and Software, 5(1):15-35.
76. Clarke, L. A., Podgurski, A., Richardson, D. J., and Zeil, S. J. (1985). A comparison of data flow path selection criteria. In Proceedings of the Eighth International Conference on Software Engineering, pages 244-251, London, UK. IEEE Computer Society Press.
77. Clarke, L. A., Podgurski, A., Richardson, D. J., and Zeil, S. J. (1989). Aformal evaluation of data flow path selection criteria. IEEE Transactions on Software Engineering, 15:1318-1332.
78. Cohen, D. M., Dalal, S. R., Fredman, M. L., and Patton, G. C. (1997). The AETG system: An approach to testing based on combinatorial design. IEEE Transactions on Software Engineering, 23(7):437-444.
79. Cohen, D. M., Dalal, S. R., Kajla, A., and Patton, G. C. (1994). The automatic efficient test generator (AETG) system. In Proceedings of Fifth International Symposium on Software Reliability Engineering (ISSRE’94), pages 303-309, Los Alamitos, CA.
80. Cohen, D. M., Dalal, S. R., Parelius, J., and Patton, G. C. (1996). The combinatorial design approach to automatic test generation. IEEE Software, pages 83-88.
81. Cohen, M. B., Gibbons, P. B., Mugridge, W. B., and Colburn, C. J. (2003). Constructing test cases for interaction testing. In Proceedings of the 25th International Conference on Software Engineering, (ICSE’03), pages 38-48. IEEE Computer Society Press.
82. Extensible markup language (XML) 1.0 (second edition)-W3C recommendation. www.w3.org/XML/#9802xml10, last access: July 2016.
83. Constantine, L. L. and Yourdon, E. (1979). Structured Design. Prentice-Hall, Englewood Cliffs, NJ.
84. Cooper, A. (1995). About Face: The Essentials of User Interface Design. Hungry Minds, New York, NY.
85. Copeland, L. (2003). A Practitioner’s Guide to Software Test Design. Artech House Publishers, Norwood, MA.
86. Dalal, S. R., Jain, A., Karunanithi, N., Leaton, J. M., and Lott, C. M. (1998). Model-based testing of a highly programmable system. In Proceedings of 9th International Symposium in Software Engineering (ISSRE’98), pages 174-178, Paderborn, Germany.
87. Dalal, S. R., Jain, A., Karunanithi, N., Leaton, J. M., Lott, C. M., Patton, G. C., and Horowitz, B. M. (1999). Model-based testing in practice. In Proceedings of 21st International Conference on Software Engineering (ICSE’99), pages 285-294, Los Angeles, CA. ACM Press.
88. Daran, M. and Thévenod-Fosse, P. (1996). Software error analysis: A real case study involving real faults and mutations. ACM SIGSOFT Software Engineering Notes, 21(3):158-177.
89. Darringer, J. A. and King, J. C. (1978). Applications of symbolic execution to program testing. IEEE Computer, 11(4).
90. Delamaro, M., Maldonado, J. C., and Mathur, A. P. (2001). Interface mutation: An approach for integration testing. IEEE Transactions on Software Engineering, 27(3):228-247.
91. Delamaro, M. E. and Maldonado, J. C. (1996). Proteum-A tool for the assessment of test adequacy for C programs. In Proceedings of the Conference on Performability in Computing Systems (PCS 96), pages 79-95, New Brunswick, NJ.
92. DeMillo, R. A. and Offutt, J. (1993). Experimental results from an automatic test case generator. ACM Transactions on Software Engineering Methodology, 2(2):109-127.
93. DeMillo, R. A. and Offutt, J. (1991). Constraint-based automatic test data generation. IEEE Transactions on Software Engineering, 17(9):900-910.
94. DeMillo, R. A., Guindi, D. S., King, K. N., McCracken, W. M., and Offutt, J. (1988). An extended overview of the Mothra software testing environment. In Proceedings of the IEEE Second Workshop on Software Testing, Verification, and Analysis, pages 142-151, Banff, Alberta.
95. DeMillo, R. A., Lipton, R. J., and Perlis, A. J. (1979). Social processes and proofs of theorems and programs. Communications of the ACM, 22(5).
96. DeMillo, R. A., Lipton, R. J., and Sayward, F. G. (1978). Hints on test data selection: Help for the practicing programmer. IEEE Computer, 11(4):34-41.
97. DeMillo, R.A., McCracken, W. M., Martin, R. J., and Passafiume, J. F. (1987). Software Testing and Evaluation. Benjamin/Cummings, Menlo Park, CA.
98. MIL-STD-498: Software Development and Documentation. Department of Defense.
99. DOD-STD-2167A: Defense System Software Development. Department of Defense.
100. Deutsch, M. S. (1982). Software Verification and Validation Realistic Project Approaches. Prentice-Hall, Englewood Cliffs, New Jersey, NJ.
101. Duncan, A. G. and Hutchison, J. S. (1981). Using attributed grammars to test designs and implementations. In Proceedings of the 5th International Conference on Software Engineering (ICSE 5), pages 170-177, San Diego, CA. IEEE Computer Society Press.
102. Dupuy, A. and Leveson, N. (2000). An empirical evaluation of the MC/DC coverage criterion on the HETE-2 satellite software. In Proceedings of the Digital Aviations Systems Conference (DASC).
103. Durelli, V. H., Offutt, J., Li, N., and Delamaro, M. (2016). What to expect of predicates: An empirical analysis of predicates in real world programs.
104. Dustin, E., Rashka, J., and Paul, J. (1999). Automated Software Testing: Introduction, Management, and Performance. Addison-Wesley Professional, New York, NY.
105. Eckhardt Jr., D. E. and Lee, L. D. (1988). Fundamental differences in the reliability of N-modular redundancy and N-version programming. The Journal of Systems and Software, 8(4):313-318.
106. Edelman, A. (1997). The mathematics of the Pentium division bug. SIAM Review, 39:54-67. www.siam.org/journals/sirev/39-1/29395.html, July 2016.
107. Fairley, R. E. (1975). An experimental program testing facility. IEEE Transactions on Software Engineering, SE-1:350-3571.
108. Feathers, M. (2004). Working Effectively with Legacy Code. Prentice-Hall, Upper Saddle River, NJ.
109. Ferguson, R. and Korel, B. (1996). The chaining approach for software test data generation. ACM Transactions on Software Engineering Methodology, 5(1):63-86.
110. Forman, I. R. (1984). An algebra for data flow anomaly detection. In Proceedings of the Seventh International Conference on Software Engineering, pages 278-286. IEEE Computer Society Press.
111. Fosdick, L. D. and Osterweil, L. J. (1976). Data flow analysis in software reliability. ACM Computing Surveys, 8(3):305-330.
112. Fowler, M. (2004). Is design dead? Online blog. http://martinfowler.com/articles/designDead.html, last access: February 2016.
113. Fowler, M. (2005). The new methodology. Online blog. www.martinfowler. com/articles/newMethodology.html, last access: February 2016.
114. Fowler, M. (2007). Mocks aren’t stubs. Online blog. www.martinfowler. com/articles/mocksArentStubs.html, last access: February 2016.
115. Fowler, M., Beck, K., Brant, J., Opdyke, W., and Roberts, D. (1999). Refactoring: Improving the Design of Existing Code. Addison-Wesley Longman, Westford, MA.
116. Frankl, P. G. and Deng, Y. (2000). Comparison of delivered reliability of branch, data flow and operational testing: A case study. In Proceedings of the 2000 International Symposium on Software Testing, and Analysis (ISSTA '00), pages 124-134, Portland, OR. IEEE Computer Society Press.
117. Frankl, P. G. and Weiss, S. N. (1993). An experimental comparison of the effectiveness of branch testing and data flow testing. IEEE Transactions on Software Engineering, 19(8):774-787.
118. Frankl, P. G. and Weyuker, E. J. (1986). Data flow testing in the presence of unexecutable paths. In Proceedings of the Workshop on Software Testing, pages 4-13, Banff, Alberta. IEEE Computer Society Press.
119. Frankl, P. G. and Weyuker, E. J. (1988). An applicable family of data flow testing criteria. IEEE Transactions on Software Engineering, 14(10):1483-1498.
120. Frankl, P. G., Weiss, S. N., and Hu, C. (1997). All-uses versus mutation testing: An experimental comparison of effectiveness. Journal of Systems and Software, 38(3):235-253.
121. Frankl, P. G., Weiss, S. N., and Weyuker, E. J. (1985). ASSET: A system to select and evaluate tests. In Proceedings of the Conference on Software Tools, New York, NY. IEEE Computer Society Press.
122. Freedman, R. S. (1991). Testability of software components. IEEE Transactions on Software Engineering, 17(6):553-564.
123. Fujiwara, S., Bochman, G., Khendek, F., Amalou, M., and Ghedasmi, A. (1991). Test selection based on finite state models. IEEE Transactions on Software Engineering, 17(6):591-603.
124. Gallagher, L., Offutt, J., and Cincotta, T. (2007). Integration testing of object-oriented components using finite state machines. Software Testing, Verification, and Reliability, Wiley, 17(1):215-266.
125. Gargantini, A. and Fraser, G. (2010). Generating minimal fault detecting test suites for boolean expressions. In AMOST 2010 -6th Workshop on Advances in Model Based Testing, pages 37-45, Paris, France.
126. Geist, R., Offutt, J., and Harris, F. (1992). Estimation and enhancement of real-time software reliability through mutation analysis. IEEE Transactions on Computers, 41(5):550-558. Special Issue on Fault-Tolerant Computing.
127. Girgis, M. R. and Woodward, M. R. (1985). An integrated system for program testing using weak mutation and data flow analysis. In Proceedings of the Eighth International Conference on Software Engineering, pages 313-319, London, UK. IEEE Computer Society Press.
128. Godefroid, P., Klarlund, N., and Sen, K. (2005). DART:Directed automated random testing. In 2005 ACM SIGPLAN conference on Programming Language Design and Implementation, pages 213-223, Chicago, IL.
129. Goldberg, A., Wang, T. C., and Zimmerman, D. (1994). Applications of feasible path analysis to program testing. In Proceedings of the 1994 IEEE International Symposium on Software Testing, and Analysis, pages 80-94, Seattle, WA. ACM Press.
130. Gonenc, G. (1970). A method for the design of fault-detection experiments. IEEE Transactions on Computers, C-19:155-558.
131. Goodenough, J. B. and Gerhart, S. L. (1975). Toward a theory of test data selection. IEEE Transactions on Software Engineering, 1(2).
132. Gourlay, J. S. (1983). A mathematical framework for the investigation of testing. IEEE Transactions on Software Engineering, 9(6):686-709.
133. Grindal, M. (2007). Evaluation of Combination Strategies for Practical Testing. PhD thesis, Skövde University / Linkoping University, Skövde, Sweden.
134. Grindal, M. and Offutt, J. (2007). Input parameter modeling for combination strategies. In IASTED International Conference on Software Engineering (SE 2007), Innsbruck, Austria. ACTA Press.
135. Grindal, M., Offutt, J., and Andler, S. F. (2005). Combination testing strategies: A survey. Software Testing, Verification, and Reliability, 15(2):97-133, Wiley.
136. Grindal, M., Lindström, B., Offutt, J., and Andler, S. F. (2006). An evaluation of combination testing strategies. Empirical Software Engineering, 11(4):583-611.
137. Grindal, M., Offutt, J., and Mellin, J. (2007). Conflict management when using combination strategies for software testing. In Australian Software Engineering Conference (ASWEC 2007), pages 255-264, Melbourne, Australia.
138. Grochtmann, M. and Grimm, K. (1993). Classification trees for partition testing. Software Testing, Verification, and Reliability, 3(2):63-82, Wiley.
139. Grochtmann, M., Grimm, K., and Wegener, J. (1993). Toolsupported test case design for black-box testing by means of the classification-tree editor. In Proceedings of the 1st European International Conference on Software Testing Analysis and Review (EuroSTAR 1993), pages 169-176, London, UK.
140. Guo, H.-F. and Qiu, Z. (2013). Automatic grammar-based test generation. In Testing Software and Systems, volume LNCS 8254, pages 17-32. Springer-Verlag.
141. Halbwachs, N. (1998). Synchronous programming of reactive systems -A tutorial and commented bibliography, LNCS 1427. In Tenth International Conference on Computer-Aided Verification, pages 1-16. Springer-Verlag.
142. Hamlet, R. (1981). Reliability theory of program testing. Acta Informatica, Springer-Verlag, pages 31-43.
143. Hamlet, R. G. (1977). Testing programs with the aid of a compiler. IEEE Transactions on Software Engineering, 3(4):279-290.
144. Hampton, M. and Petithomme, S. (2007). Leveraging a commercial mutation analysis tool for research. In Third IEEE Workshop on Mutation Analysis (Mutation 2007), pages 203-209, Windsor, UK.
145. Hanford, K. V. (1970). Automatic generation of test cases. IBM Systems Journal, 4:242-257.
146. Hanks, J. M. (1980). Testing COBOL programs by mutation: Volume I-introduction to the CMS.1 system, volume II -CMS.1 system documentation. Technical report GIT-ICS-80/04, Georgia Institute of Technology.
147. Harman, M., Jia, Y., and Langdon, W. B. (2010). How higher order mutation helps mutation testing (keynote). In 5th International Workshop on Mutation Analysis (Mutation 2010), Paris, France.
148. Harrold, M. J. and Rothermel, G. (1994). Performing data flow testing on classes. In Symposium on Foundations of Software Engineering, pages 154-163, New Orleans, LA. ACM SIGSOFT.
149. Harrold, M. J. and Rothermel, G. (1998). Empirical studies of a safe regression test selection technque. IEEE Transactions on Software Engineering, 24(6):401-419.
150. Harrold, M. J. and Soffa, M. L. (1991). Selecting and using data for integration testing. IEEE Software, 8(2):58-65.
151. Heller, E. (1995). Using design of experiment structures to generate software test cases. In Proceedings of the 12th International Conference on Testing Computer Software, pages 33-41, New York, NY. ACM.
152. Henninger, K. (1980). Specifiying software requirements for complex systems: New techniques and their applications. IEEE Transactions on Software Engineering, SE-6(1):2-12.
153. Herman, P. (1976). A data flow analysis approach to program testing. Australian Computer Journal, 8(3):92-96.
154. Hetzel, B. (1988). The Complete Guide to Software Testing. Wiley-QED, second edition.
155. Horgan, J. R. and London, S. (1991). Data flow coverage and the C languages. In Proceedings of the Fourth IEEE Symposium on Software Testing, Analysis, and Verification, pages 87-97, Victoria, British Columbia, Canada.
156. Horgan, J. R. and London, S. (1992). ATAC: A data flow coverage testing tool for C. In Proceedings of the Symposium of Quality Software Development Tools, pages 2-10, NewOrleans, LA. IEEE Computer Society Press.
157. Horgan, J. R. and Mathur, A. P. (1990). Weak mutation is probably strong mutation. Technical report SERC-TR-83-P, Software Engineering Research Center, Purdue University, West Lafayette, IN.
158. Howden, W. E. (1975). Methodology for the generation of program test data. IEEE Transactions on Software Engineering, SE-24.
159. Howden, W. E. (1976). Reliability of the path analysis testing strategy. IEEE Transactions on Software Engineering, 2(3):208-215.
160. Howden, W. E. (1977). Symbolic testing and the DISSECT symbolic evaluation system. IEEE Transactions on Software Engineering, 3(4).
161. Howden, W. E. (1978). Theoretical and empirical studies of program testing. IEEE Transactions on Software Engineering, 4(4):293-298.
162. Howden, W. E. (1982). Weak mutation testing and completeness of test sets. IEEE Transactions on Software Engineering, 8(4):371-379.
163. Howden, W. E. (1985). The theory and practice of function testing. IEEE Software, 2(5).
164. Howden, W. E. (1987). Functional Program Testing and Analysis. McGraw-Hill Book Company, New York, NY.
165. Huang, J. C. (1975). An approach to program testing. ACM Computing Surveys, 7(3):113-128.
166. Huller, J. (2000). Reducing time to market with combinatorial design method testing. In Proceedings of the 10th Annual International Council on Systems Engineering (INCOSE’00), Minneapolis, MN.
167. Hutchins, M., Foster, H., Goradia, T., and Ostrand, T. (1994). Experiments on the effectiveness of dataflow-and controlflow-based test adequacy criteria. In Proceedings of the Sixteenth International Conference on Software Engineering, pages 191-200, Sorrento, Italy. IEEE Computer Society Press.
168. Huth, M. and Ryan, M. D. (2000). Logic in Computer Science:Modelling and Reasoning About Systems. Cambridge University Press, Cambridge, UK.
169. IEEE Standard for Software and System Test Documentation. Institute of Electrical and Electronic Engineers, New York. IEEE Std 829-2008.
170. Ince, D. C. (1987). The automatic generation of test data. The Computer Journal, 30(1):63-69.
171. Jasper, R., Brennan, M., Williamson, K., Currier, B., and Zimmerman, D. (1994). Test data generation and feasible path analysis. In Proceedings of the 1994 IEEE International Symposium on Software Testing, and Analysis, pages 95-107, Seattle, WA. ACM Press.
172. Jazequel and Meyer, B. (1997). Design by contract: The lessons of Ariane. Computer, 30(1):129-130.
173. Jia, Y. and Harman, M. (2008). Constructing subtle faults using higher order mutation testing. In Eighth IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM 2008), pages 249-258, Beijing, China.
174. Jia, Y. and Harman, M. (2011). An analysis and survey of the development of mutation testing. IEEE Transactions of Software Engineering, 37(5):649-678.
175. Jin, Z. and Offutt, J. (1998). Coupling-based criteria for integration testing. Software Testing, Verification, and Reliability, 8(3):133-154, Wiley.
176. Jones, J. A. and Harrold, M. J. (2003). Test-suite reduction and prioritizaion for modified condition / decision coverage. IEEE Transactions on Software Engineering, 29(3):195-209.
177. Jones, B. F., Eyres, D. E., and Sthamer, H. H. (1998). A strategy for using genetic algorithms to automate branch and fault-based testing. The Computer Journal, 41(2):98-107.
178. Just, R., Jalali, D., Inozemtseva, L., Ernst, M. D., Holmes, R., and Fraser, G. (2014). Are mutants a valid substitute for real faults in software testing? In Proceedings of the Symposium on the Foundations of Software Engineering (FSE), pages 654-665, Hong Kong.
179. Kaminski, G. (2012). Applications of Logic Coverage Criteria and Logic Mutation to Software Testing. PhD thesis, George Mason University, Fairfax, VA.
180. Kaminski, G. and Ammann, P. (2009). Using logic criterion feasibility to reduce test set size while guaranteeing fault detection. In 2nd IEEE International Conference on Software Testing, Verification and Validation (ICST 2009), pages 356-365, Denver, CO.
181. Kaminski, G. and Ammann, P. (2010). Applications of optimization to logic testing. In CSTVA 2010 -2nd Workshop on Constraints in Software Testing, Verification and Analysis, pages 331-336, Paris, France.
182. Kaminski, G. and Ammann, P. (2011). Reducing logic test set size while preserving fault detection. Journal of Software Testing, Verification and Reliability, 21(3):155-193, Wiley. Special issue from the 2009 International Conference on Software Testing, Verification and Validation.
183. Kaminski, G., Ammann, P., and Offutt, J. (2013). Improving logic-based testing. Journal of Systems and Software, 86:2002-2012.
184. Kaner, C., Falk, J., and Nguyen, H. Q. (1999). Testing Computer Software. John Wiley and Sons, New York, NY, second edition.
185. Kim, S., Clark, J. A., and McDermid, J. A. (2000). Investigating the effectiveness of object-oriented strategies with the mutation method. In Proceedings of Mutation 2000: Mutation Testing in the Twentieth and the Twenty First Centuries, pages 4-100, San Jose, CA. Wiley’s Software Testing, Verification, and Reliability, December 2001.
186. Kim, Y. G., Hong, H. S., Cho, S. M., Bae, D. H., and Cha, S. D. (1999). Test cases generation from UML state diagrams. IEE Proceedings-Software, 146(4):187-192.
187. King, K. N. and Offutt, J. (1991). A Fortran language system for mutation-based software testing. Software-Practice and Experience, 21(7):685-718.
188. Knight, J. C. and Leveson, N. G. (1986). An experimental evaluation of the assumption of independence in multiversion programming. IEEE Transactions on Software Engineering, SE-12(1):86-109.
189. Knutson, C. and Carmichael, S. (2000). Safety first: Avoiding software mishaps. www.embedded.com/design/safety-andsecurity/4399493/Safety-First-Avoiding-Software-Mishaps, last access: February 2016.
190. Errors in education info system cause massive confusion. Online. www.koreatimes.co.kr/www/news/nation/2011/07/117_91459.html, last access: February 2016.
191. Korel, B. (1990a). Automated software test data generation. IEEE Transactions on Software Engineering, 16(8):870-879.
192. Korel, B. (1990b). A dynamic approach of test data generation. In Conference on Software Maintenance-1990, pages 311-317, San Diego, CA.
193. Korel, B. (1992). Dynamic method for software test data generation. Software Testing, Verification, and Reliability, 2(4):203-213, Wiley.
194. Koskela, L. (2008). Test Driven: Practical TDD and Acceptance TDD for Java Developers. Manning Publications Company, Greenwich, CT.
195. Krug, S. (2000). Don’t Make Me Think! A Common Sense Approach to Web Usability. New Riders Publishing, San Francisco, CA.
196. Kuhn, D. R. (1999). Fault classes and error detection capability of specification-based testing. ACM Transactions on Software Engineering Methodology, 8(4):411-424.
197. Kuhn, D. R. and Reilly, M. J. (2002). An investigation of the applicability of design of experiments to software testing. In Proceedings of the 27th NASA/IEE Software Engineering Workshop, NASA Goodard Space Flight Center, MD, USA. NASA/IEEE.
198. Kuhn, D. R., Wallace, D. R., and Jr., A.M. G. (2004). Software fault interactions and implications for software testing. IEEE Transactions on Software Engineering, 30(6):418-421.
199. Kung, D., Gao, J., Hsia, P., Toyoshima, Y., and Chen, C. (1995). A test strategy for object-oriented programs. In 19th Computer Software and Applications Conference (COMPSAC 95), pages 239-244, Dallas, TX. IEEE Computer Society Press.
200. Laski, J. (1990). Data flow testing in STAD. Journal of Systems and Software, 12:3-14.
201. Laski, J. and Korel, B. (1983). A data flow oriented program testing strategy. IEEE Transactions on Software Engineering, SE-9(3):347-354.
202. Lau, M. F. and Yu, Y. T. (2005). An extended fault class hierarchy for specification-based testing. ACM Transactions on Software Engineering Methodology, 14(3):247-276.
203. Lee, S. C. and Offutt, J. (2001). Generating test cases for XMLbasedWeb component interactions using mutation analysis. In Proceedings of the 12th IEEE International Symposium on Software Reliability Engineering, pages 200-209, Hong Kong, China.
204. Legard, H. and Marcotty, M. (1975). A genealogy of control structures. Communications of the ACM, 18:629-639.
205. Lei, Y. and Tai, K. C. (1998). In-parameter-order: A test generation strategy for pair-wise testing. In Proceedings of the Third IEEE High Assurance Systems Engineering Symposium, pages 254-261. IEEE.
206. Lei, Y. and Tai, K. C. (2001). A test generation strategy for pairwise testing. Technical Report TR-2001-03, Department of Computer Science, North Carolina State University, Raleigh.
207. Leveson, N. and Turner, C. S. (1993). An investigation of the Therac-25 accidents. IEEE Computer, 26(7):18-41.
208. Li, N. and Offutt, J. (2014). An empirical analysis of test oracle strategies for model-based testing. In 7th IEEE International Conference on Software Testing, Verification and Validation (ICST 2014), Cleveland, Ohio.
209. Li, N. and Offutt, J. (2016). Test oracle strategies for model-based testing. Under minor revision.
210. Li, N., Praphamontripong, U., and Offutt, J. (2009). An experimental comparison of four unit test criteria: Mutation, edge-pair, all-uses and prime path coverage. In Fifth IEEE Workshop on Mutation Analysis (Mutation 2009), Denver, CO.
211. Li, Z., Harman, M., and Hierons, R. M. (2007). Meta-heuristic search algorithms for regression test case prioritization. IEEE Transactions on Software Engineering, 33(4):225-237.
212. Lions, J. L. (1996). Ariane 5 flight 501 failure: Report by the inquiry board. http://sunnyday.mit.edu/accidents/Ariane5accidentreport.html, last access:February 2016.
213. Lipton, R. (1991). New directions in testing. In Distributed Computing and Cryptography, DIMACS Series in Discrete Mathematics and Theoretical Computer Science, volume 2, pages 191-202, Providence, RI.
214. Liskov, B. and Guttag, J. (2001). Program Development in Java: Abstraction, Specification, and Object-Oriented Design. Addison-Wesley Publishing Company Inc., New York, NY.
215. Littlewood, B. and Miller, D. R. (1989). Conceptual modeling of coincident failures in multiversion software. IEEE Transactions on Software Engineering, 15(12):1596-1614.
216. Ma, Y.-S., Kwon, Y.-R., and Offutt, J. (2002). Inter-class mutation operators for Java. In Proceedings of the 13th IEEE International Symposium on Software Reliability Engineering, pages 352-363, Annapolis, MD.
217. Ma, Y.-S., Offutt, J., and Kwon, Y.-R. (2005). MuJava: An automated class mutation system. Software Testing, Verification, and Reliability, 15(2):97-133, Wiley.
218. Malaiya, Y. K. (1995). Antirandom testing: Getting the most out of black-box testing. In International Symposium on Software Reliability Engineering (ISSRE’95), pages 86-95, Toulouse, France.
219. Malloy, B. A., Clarke, P. J., and Lloyd, E. L. (2003). A parameterized cost model to order classes for class-based testing of C++ applications. In Proceedings of the 14th IEEE International Symposium on Software Reliability Engineering, Denver, CO.
220. Mandl, R. (1985). Orthogonal latin squares: An application of experiment design to compiler testing. Communications of the ACM, 28(10):1054-1058.
221. Marick, B. (1991). The weak mutation hypothesis. In Proceedings of the Fourth IEEE Symposium on Software Testing, Analysis, and Verification, pages 190-199, Victoria, British Columbia, Canada.
222. Marick, B. (1995). The Craft of Software Testing: Subsystem Testing, Including Object-Based and Object-Oriented Testing. Prentice-Hall, Englewood Cliffs, New Jersey, NJ.
223. Mathur, A. P. (1991). On the relative strengths of data flow and mutation based test adequacy criteria. In Proceedings of the Sixth Annual Pacific Northwest Software Quality Conference, Portland, OR. Lawrence and Craig.
224. Mathur, A. P. (2014). Foundations of Software Testing. Addison-Wesley Professional, Indianapolis, IN, second edition.
225. Mathur, A. P. and Wong, W. E. (1994). An empirical comparison of data flow and mutation-based test adequacy criteria. Software Testing, Verification, and Reliability, 4(1):9-31, Wiley.
226. Maurer, P. M. (1990). Generating testing data with enhanced contextfree grammars. IEEE Software, 7(4):50-55.
227. McCabe, T. J. (1976). A complexity measure. IEEE Transactions on Software Engineering, SE-2(4):308-320.
228. Meyer, B. (1997). Object-Oriented Software Construction. Prentice Hall, Upper Saddle River, NJ, second edition.
229. Miller, E. F. and Melton, R. A. (1975). Automated generation of testcase datasets. In Proceedings of the International Conference on Reliable Software, pages 51-58.
230. Min-sang, K. and Sang-soo, K. (2011). Education info system miscalculated grades. Online. http://joongangdaily.joins.com/article/view. asp?aid=2939367, last access: February 2016.
231. Minkel, J. R. (2008). 2003 northeast blackout-five years later. Scientific American.
232. Mitzenmacher, M. and Upfal, E. (2005). Probability and Computing: Randomized Algorithms and Probabilistic Analysis. Cambridge University Press, Cambridge, UK.
233. Moler, C. (1995). A tale of two numbers. SIAM News, 28(1).
234. Morell, L. J. (1984). A Theory of Error-Based Testing. PhDthesis, University of Maryland, College Park, MD. Technical Report TR-1395.
235. Morell, L. J. (1990). A theory of fault-based testing. IEEE Transactions on Software Engineering, 16(8):844-857.
236. Myers, G. (1979). The Art of Software Testing. John Wiley and Sons, New York, NY.
237. Naito, S. and Tsunoyama, M. (1981). Fault detection for sequential machines by transition tours. In Proceedings Fault Tolerant Computing Systems, pages 238-243. IEEE Computer Society Press.
238. Namin, A. S. and Kakarla, S. (2011). The use of mutation in testing experiments and its sensitivity to external threats. In Proceedings of the 2011 International Symposium on Software Testing and Analysis, pages 342-352, New York, NY. ACM.
239. Naur, P. and Randell, B., editors (1968). Software Engineering:Report of a Conference Sponsored by the NATO Science Committee. Scientific Affairs Division, NATO.
240. Nuseibeh, B. (1997). Who dunnit? IEEE Software, 14:15-16.
241. Offutt, J. (1988). Automatic Test Data Generation. PhD thesis, Georgia Institute of Technology, Atlanta, GA. Technical report GIT-ICS 88/28.
242. Offutt, J. (1992). Investigations of the software testing coupling effect. ACM Transactions on Software Engineering Methodology, 1(1):3-18.
243. Offutt, J. and Abdurazik, A. (1999). Generating tests from UML specifications. In Proceedings of the Second IEEE International Conference on the Unified Modeling Language (UML99), pages 416-429, Fort Collins, CO. Springer-Verlag Lecture Notes in Computer Science Volume 1723.
244. Offutt, J. and Alluri, C. (2014). An industrial study of applying input space partitioning to test financial calculation engines. Empirical Software Engineering Journal, 19:558-581.
245. Offutt, J. and Lee, S. D. (1994). An empirical evaluation of weak mutation. IEEE Transactions on Software Engineering, 20(5):337-344.
246. Offutt, J. and Pan, J. (1997). Detecting equivalent mutants and the feasible path problem. Software Testing, Verification, and Reliability, 7(3):165-192, Wiley.
247. Offutt, J., Lee, A., Rothermel, G., Untch, R., and Zapf, C. (1996a). An experimental determination of sufficient mutation operators. ACM Transactions on Software Engineering Methodology, 5(2):99-118.
248. Offutt, J., Pan, J., Tewary, K., and Zhang, T. (1996b). An experimental evaluation of data flow and mutation testing. Software-Practice and Experience, 26(2):165-176.
249. Offutt, J., Payne, J., and Voas, J. M. (1996c). Mutation operators for Ada. Technical report ISSE-TR-96-09, Department of Information and Software Engineering, George Mason University, Fairfax, VA. http://cs.gmu.edu/-tradmin/, last access: July 2016.
250. Offutt, J., Jin, Z., and Pan, J. (1999). The dynamic domain reduction approach to test data generation. Software-Practice and Experience, 29(2):167-193.
251. Offutt, J., Abdurazik, A., and Alexander, R. T. (2000). An analysis tool for coupling-based integration testing. In The Sixth IEEE International Conference on Engineering of Complex Computer Systems (ICECCS '00), pages 172-178, Tokyo, Japan. IEEE Computer Society Press.
252. Offutt, J., Liu, S., Abdurazik, A., and Ammann, P. (2003). Generating test data from state-based specifications. Software Testing, Verification, and Reliability, 13(1):25-53, Wiley.
253. Offutt, J., Ma, Y.-S., and Kwon, Y.-R. (2005). muJava home page. Online. https://cs.gmu.edu/-offutt/mujava/, last access: February 2016.
254. Olender, K. M. and Osterweil, L. J. (1986). Specification and static evaluation of sequencing constraints in software. In Proceedings of the Workshop on Software Testing, pages 2-9, Banff, Alberta. IEEE Computer Society Press.
255. Olender, K. M. and Osterweil, L. J. (1989). Cesar: A static sequencing constraint analyzer. In Proceedings of the Third Workshop on Software Testing, Verification and Analysis, pages 66-74, Key West, FL. ACM SIGSOFT.
256. Orso, A. and Pezze, M. (1999). Integration testing of procedural object oriented programs with polymorphism. In Proceedings of the Sixteenth International Conference on Testing Computer Software, pages 103-114, Washington DC. ACM SIGSOFT.
257. Osterweil, L. J. and Fosdick, L. D. (1974). Data flow analysis as an aid in documentation, assertion generation, validation, and error detection. Technical report cu-cs-055-74, Department of Computer Science, University of Colorado, Boulder, CO.
258. Ostrand, T. J. and Balcer, M. J. (1988). The categorypartition method for specifying and generating functional tests. Communications of the ACM, 31(6):676-686.
259. Paulk, M. C., Weber, C. V., Curtis, B., and Chrissis, M. B. (1995). The Capability Maturity Model: Guidelines for Improving the Software Process. Addison-Wesley Longman Publishing Co., Inc., Boston, MA.
260. Payne, A. J. (1978). A formalised technique for expressing compiler exercisers. ACM SIGPLAN Notices, 13(1):59-69.
261. Peterson, I. (1997). Pentium bug revisited. http://mtarchive.blogspot.com/2016/08/pentium-bug-revisited.htm, last access: August 2016.
262. Pezze, M. and Young, M. (2008). Software Testing and Analysis: Process, Principles, and Techniques. Wiley, Hoboken, NJ.
263. Pimont, S. and Rault, J. C. (1976). A software reliability assessment based on a structural behavioral analysis of programs. In Proceedings of the Second International Conference on Software Engineering, pages 486-491, San Francisco, CA.
264. Information technology research: Investing in our future. Technical report, National Coordination Office Computing, Information, and Communications. www.nitrd.gov/pitac/report/, last access: July 2016.
265. Piwowarski, P., Ohba, M., and Caruso, J. (1993). Coverage measure experience during function test. In Proceedings of 14th International Conference on Software Engineering (ICSE’93), pages 287-301, Los Alamitos, CA. ACM.
266. Prather, R. E. (1983). Theory of program testing-an overview. The Bell System Technical Journal, 62(10).
267. Purdom, P. (1972). A sentence generator for testing parsers. BIT, 12:366-375.
268. Ramamoorthy, C. V., Ho, S. F., and Chen, W. T. (1976). On the automated generation of program test data. IEEE Transactions on Software Engineering, 2(4):293-300.
269. Rapps, S. and Weyuker, E. J. (1985). Selecting software test data using data flow information. IEEE Transactions on Software Engineering, 11(4):367-375.
270. Rayadurgam, S. and Heimdahl, M. P. E. (2001). Coverage based test-case generation using model checkers. In 8th IEEE International Conference and Workshop on the Engineering of Computer Based Systems, pages 83-91.
271. Rice, D. (2008). Geekonomics, The Real Cost of Insecure Software. Pearson Education, Upper Saddle River, NJ.
272. Roper, M. (1994). Software Testing. International Software Quality Assurance Series. McGraw-Hill, Hightstown, NJ.
273. Rothermel, G. and Harrold, M. J. (1996). Analyzing regression test selection techniques. IEEE Transactions on Software Engineering, 22(8):529-551.
274. RTCA-DO-178B (1992). Software considerations in airborne systems and equipment certification.
275. RTI (2002). The economic impacts of inadequate infrastructure for software testing. Technical report 7007.011, NIST. www.nist.gov/director/progofc/report02-3.pdf, last access: July 2016.
276. Sabnani, K. and Dahbura, A. (1988). A protocol testing procedure. Computer Networks and ISDN Systems, 14(4):285-297.
277. Schneider, F. B. (1999). Trust in Cyberspace. National Academy Press, Washington, DC.
278. Sen, K., Marinov, D., and Agha, G. (2005). Cute: A concolic unit testing engine for C. In ACM 10th European Software Engineering Conference, pages 263-272, Lisbon, Portugal.
279. Sherwood, G. (1994). Effective testing of factor combinations. In Proceedings of the Third International Conference on Software Testing, Analysis, and Review (STAR94), Washington DC. Software Quality Engineering.
280. Shiba, T., Tsuchiya, T., and Kikuno, T. (2004). Using artificial life techniques to generate test cases for combinatorial testing. In Proceedings of 28th Annual International Computer Software and Applications Conference (COMPSAC’04), pages 72-77, Hong Kong, China. IEEE Computer Society Press.
281. Shrestha, K. and Rutherford, M. (2011). An empirical evaluation of assertions as oracles. In Proceedings of the Fourth IEEE International Conference on Software Testing, Verification and Validation, pages 110-119, Berlin, Germany. IEEE Computer Society.
282. Sommerville, I. (1992). Software Engineering. Addison-Wesley Publishing Company Inc., 9th edition.
283. Sprenkle, S., Pollock, L., Esquivel, H., Hazelwood, B., and Ecott, S. (2007). Automated oracle comparators for testing web applications. In The 18th IEEE International Symposium on Software Reliability Engineering, pages 117-126, Trollhattan, Sweden.
284. Staats, M., Gay, G., and Heimdahl, M. P. E. (2012). Automated oracle creation support, or: How I learned to stop worrying about fault propagation and love mutation testing. In Proceedings of the International Conference on Software Engineering, ICSE, pages 870-880, Piscataway, NJ. IEEE Press.
285. Staats, M., Whalen, M. W., and Heimdahl, M. P. E. (2011). Better testing through oracle selection. In Proceedings of the 33rd International Conference on Software Engineering (NIER Track), ICSE 2011, pages 892-895, Waikiki, Honolulu, HI. ACM.
286. Stevens, W. P., Myers, G. J., and Constantine, L. L. (1974). Structured design. IBM Systems Journal, 13(2):115-139.
287. Stocks, P. and Carrington, D. (1993). Test Templates:A Specification-Based Testing Framework. In Proceedings of the Fifteenth International Conference on Software Engineering, pages 405-414, Baltimore, MD.
288. Stocks, P. and Carrington, D. (1996). A framework for specification-based testing. IEEE Transactions on Software Engineering, 22(11):777-793.
289. Symantec (2007). Symantec internet security threat report, volume XII. Online: http://eval.symantec.com/mktginfo/enterprise/white_papers/entwhitepaper_ internet_security_threat_report_xii_09_2007.en-us.pdf, last access:July 2016.
290. Tai, K.-C. and Daniels, F. J. (1997). Test order for inter-class integration testing of object-oriented software. In The Twenty-First Annual International Computer Software and Applications Conference (COMPSAC '97), pages 602-607, Santa Barbara, CA. IEEE Computer Society Press.
291. Tai, K. C. and Lei, Y. (2002). A test generation strategy for pairwise testing. IEEE Transactions on Software Engineering, 28(1):109-111.
292. Tillmann, N. and de Halleux, J. (2008). Pex-white box test generation for.NET. In LNCS 4966: Second International Conference on Tests and Proofs, pages 134-153, Prato, Italy.
293. Tillmann, N. and Schulte, W. (2005). Parameterized unit tests. In Proceedings of the 10th ACM European Software Engineering Conference held jointly with 13th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pages 253-262, Lisbon, Portugal.
294. Tip, F. (1994). A survey of program slicing techniques. Technical report CSR-9438, Computer Science/Department of Software Technology, Centrum voor Wiskunde en Informatica.
295. Traon, Y. L., Jéron, T., Jézéquel, J.-M., and Morel, P. (2000). Efficient object-oriented integration and regression testing. IEEE Transactions on Reliability, 49(1):12-25.
296. Utting, M. and Legeard, B. (2006). Practical Model-Based Testing: A Tools Approach. Morgan Kaufman, Burlington, MA.
297. Vilkomir, S. A. and Bowen, J. P. (2002). Reinforced condition/decision coverage (RC/DC): A new criterion for software testing. In Proceedings of ZB2002: 2nd International Conference of Z and B Users, pages 295-313, Grenoble, France. Springer-Verlag, LNCS 2272.
298. Voas, J. M. (1992). PIE: A dynamic failure-based technique. IEEE Transactions on Software Engineering, 18(8).
299. Voas, J. M. and Miller, K. W. (1995). Software testability: The new verification. IEEE Software, 12(3):553-563.
300. Wah, K. S. H. T. (1995). Fault coupling in finite bijective functions. Software Testing, Verification, and Reliability, 5(1):3-47, Wiley.
301. Wah, K. S. H. T. (2000). A theoretical study of fault coupling. Software Testing, Verification, and Reliability, 10(1):3-46, Wiley.
302. Weiser, M. (1984). Program slicing. IEEE Transactions on Software Engineering, SE-10(4):352-357.
303. Weiss, S. N. (1989). What to compare when comparing test data adequacy criteria. ACM SIGSOFT Notes, 14(6):42-49.
304. Weyuker, E. (1980). The oracle assumption of program testing. In Thirteenth International Conference on System Sciences, pages 44-49, Honolulu, HI.
305. Weyuker, E. J. and Ostrand, T. J. (1980). Theories of program testing and the application of revealing subdomains. IEEE Transactions on Software Engineering, 6(3):236-246.
306. Weyuker, E., Goradia, T., and Singh, A. (1994). Automatically generating test data from a boolean specification. IEEE Transactions on Software Engineering, 20(5):353-363.
307. Weyuker, E. J., Weiss, S. N., and Hamlet, R. G. (1991). Data flow-based adequacy analysis for languages with pointers. In Proceedings of the Fourth IEEE Symposium on Software Testing, Analysis, and Verification, pages 74-86, Victoria, British Columbia, Canada.
308. White, L. J. (1987). Software testing and verification. In Yovits, M. C., editor, Advances in Computers, volume 26, pages 335-390. Academic Press, Inc, Boston, MA.
309. White, L. and Wiszniewski, B. (1991). Path testing of computer programs with loops using a tool for simple loop patterns. Software-Practice and Experience, 21(10):1075-1102.
310. Wijesekera, D., Sun, L., Ammann, P., and Fraser, G. (2007). Relating counterexamples to test cases in CTL model checking specifications. In A-MOST '07: Third ACM Workshop on the Advances in Model-Based Testing, co-located with ISSTA 2007, London, UK.
311. Wikipedia (2009). Software test documentation. Online. http://en.wiki.org/wiki/Software_test_documentation, last access: February 2016.
312. Wikipedia (2015). Sieve of Eratosthenes. Online. http://en.wikipedia. org/wiki/Sieve_of_Eratosthenes, last access: February 2016.
313. Williams, A. W. (2000). Determination of test configurations for pairwise interaction coverage. In Proceedings of the 13th International Conference on the Testing of Communicating Systems (TestCom 2000), pages 59-74, Ottawa, Canada.
314. Williams, A. W. and Probert, R. L. (1996). A practical strategy for testing pair-wise coverage of network interfaces. In Proceedings of the 7th International Symposium on Software Reliability Engineering (ISSRE96), White Plains, NY.
315. Williams, A. W. and Probert, R. L. (2001). A measure for component interaction test coverage. In Proceedings of the ACSI/IEEE International Conference on Computer Systems and Applications (AICCSA 2001), pages 304-311, Beirut, Lebanon.
316. Wong, W. E. and Mathur, A. P. (1995). Fault detection effectiveness of mutation and data flow testing. Software Quality Journal, 4(1):69-83.
317. Woodward, M. R. and Halewood, K. (1988). From weak to strong, dead or alive? An analysis of some mutation testing issues. In Proceedings of the IEEE Second Workshop on Software Testing, Verification, and Analysis, pages 152-158, Banff, Alberta.
318. Xie, Q. and Memon, A. (2007). Designing and comparing automated test oracles for GUI-based software applications. ACM Transaction on Software Engineering and Methodology, 16(1).
319. Xie, T. and Notkin, D. (2005). Checking inside the black box:Regression testing by comparing value spectra. IEEE Transactions on Software Engineering, 31(10):869-883.
320. Yilmaz, C., Cohen, M. B., and Porter, A. (2004). Covering arrays for efficient fault characterization in complex configuration spaces. In Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2004), pages 45-54, Boston, MA. ACM Software Engineering Notes.
321. Yin, H., Lebne-Dengel, Z., and Malaiya, Y. K. (1997). Automatic test generation using checkpoint encoding and antirandom testing. Technical Report CS-97-116, Colorado State University.
322. Yu, T., Srisa-an, W., and Rothermel, G. (2013). An empirical comparison of the fault-detection capabilities of internal oracles. In The 24th IEEE International Symposium on Software Reliability Engineering, ISSRE '13, Pasadena, CA.
323. Zhou, Z. Q., Xiang, S., and Chen, T. Y. (2015). Metamorphic testing for software quality assessment: A study of search engines. IEEE Transactions on Software Engineering, published online, September 2015.
324. Zhu, H. (1996). Aformal analysis of the subsume relation between software test adequacy criteria. IEEE Transactions on Software Engineering, 22(4):248-255.
325. Zhu, H., Hall, P. A. V., and May, J. H. R. (1997). Software unit test coverage and adequacy. ACM Computing Surveys, 29(4):366-427.