Comparing the Effectiveness of Software Testing Strategies

IEEE Transactions on Software Engineering

Volumn SE-13, Issue 12, 1987, Pages 1278-1296

  Selby, Richard W ^b  

^a UNIVERSITY OF MARYLAND   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

SOFTWARE FUNCTIONAL TESTING; SOFTWARE STRUCTURAL TESTING; SOFTWARE TESTING STRATEGIES;

COMPUTER SOFTWARE;

EID: 0023670041     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.1987.232881     Document Type: Article

References (56)

1. V. R. Basili and B. T. Perricone, “Software errors and complexity: An empirical investigation,” Commun. ACM, vol. 27, no. 1, pp. 42–52, 52, Jan. 1984.
2. V. R. Basili and R. W. Selby, “Data collection and analysis in software research and management,” in Proc. Amer. Statist. Ass. and Biometric Soc. Joint Statistical Meetings, Philadelphia, PA, Aug. 13-16, 16, 1984.
3. ——, “Comparing the effectiveness of software testing strategies,” Dep. Comput. Sci., Univ. Maryland, College Park, Tech. Rep. TR-1501, 1501, May 1985.
4. V. R. Basili, R. W. Selby, and D. H. Hutchens, “Experimentation in software engineering,” IEEE Trans. Software Eng., vol. SE-12, no. 7, pp. 733–743, July 1986.
5. V. R. Basili and A. J. Turner, SIMPL-T: A Structured Programming Language. Paladin House, 1976.
6. V. R. Basili and D. M. Weiss, “A methodology for collecting valid software engineering data,” IEEE Trans. Software Eng., vol. SE-10, no. 6, pp. 728–738, Nov. 1984.
7. G. E. P. Box, W. G. Hunter, and J. S. Hunter, Statistics for Exper-imenters. New York: Wiley, 1978.
8. T. A. Budd, R. J. Lipton, F. G. Sayward, and R. DeMillo, “The design of a prototype mutation system for program testing,” Proc. AFIPS Conf., vol. 47, pp. 623–627, 1978.
9. R. Cailliau and F. Rubin, “ACM forum: On a controlled experiment in program testing,” Commun. ACM, vol. 22, pp. 687–688, Dec. 1979.
10. L. A. Clarke, Program Chair, Proc. Workshop Software Testing, Banff, Alta., Canada, July 15-17, 1986.
11. V. Church, “Benchmark statistics for the VAX 11/780 and the IBM 4341,” Computer Sciences Corporation, Silver Spring, MD, Internal Memo, 1984.
12. W. G. Cochran and G. M. Cox, Experimental Designs. New York: Wiley, 1950. Group
13. J. C. Deprie, “Report from the IFIP Working Group on Terminology,” in Proc. 15th Annu. Int. Symp. Fault Tolerant Computing, University of Michigan, Ann Arbor, MI, June 19-21, 1985.
14. M. E. Fagan, “Design and code inspections to reduce errors in program development,” IBM Syst. J., vol. 15, no. 3, pp. 182–211, 1976.
15. K. A. Foster, “Error sensitive test cases,” IEEE Trans. Software Eng., vol. SE-6, no. 3, pp. 258–264, 1980.
16. P. G. Frankl and E. J. Weyuker, “Data flow testing in the presence of unexecutable paths,” in Proc. Workshop Software Testing, Banff, Alta., Canada, July 15-17, 1986, pp. 4–13.
17. M. R. Girgis and M. R. Woodward, “An experimental comparison of the error exposing ability of program testing criteria,” in Proc. Workshop Software Testing, Banff, Alta., Canada, July 15-17, 1986, pp. 64–73.
18. S. A. Gloss-Soler, “The DACS glossary: A bibliography of software engineering terms, Data & Analysis Center for Software,” Griffiss Air Force Base, NY, Rep. GLOS-1, Oct. 1979.
19. J. B. Goodenough and S. L. Gerhart, “Toward a theory of test data selection,” IEEE Trans. Software Eng., vol. SE-1, pp. 156–173, June 1975.
20. J. D. Gould, “Some psychological evidence on how people debug computer programs,” Int. J. Man-Machine Studies, vol. 7, pp. 151–182, 182, 1975.
21. J. D. Gould and P. Drongowski, “An exploratory study of computer program debugging,” Human Factors, vol. 16, no. 3, pp. 258–277, 1974.
22. W. C. Hetzel, “An expermental analysis of program verification problem solving capabilities as they relate to programmer efficiency,” comput. Personnel, vol. 3, no. 3, pp. 10–15, 1972.
23. W. C. Hetzel, “An expermental analysis of program verification methods,” Ph.D. dissertation, Univ. North Carolina, Chapel Hill, 1976.
24. W. E. Howden, “Symbolic testing and the DISSECT symbolic evaluation system,” IEEE Trans. Software Eng., vol. SE-3, no. 4, pp. 266–278, 1977.
25. —, “Algebraic program testing,” Acta Inform., vol. 10, 1978.
26. —, “An evaluation of the effectiveness of symbolic testing,” Soft-ware Practice and Experience, vol. 8, pp. 381–397, 1978.
27. ——, “Functional program testing,” IEEE Trans. Software Eng., vol. SE-6, pp. 162–169, Mar. 1980.
28. ——, “Applicability of software validation techniques to scientific programs,” ACM Trans. Program. Lang. Syst., vol. 2, no. 3, pp. 307–320, July 1980.
29. ——, “A survey of dynamic analysis methods,” in Tutorial: Software Testing & Validation Techniques, 2nd ed., E. Miller and W. E. Howden, den, Eds. Washington, DC: IEEE Computer Society Press, 1981, pp. 209–231.
30. S-S. V. Hwang, “An empirical study in functional testing, structural testing, and code reading inspection*,” Dep. Comput. Sci., Univ. Maryland, College Park, Scholarly Paper 362, Dec. 1981.
31. IEEE Standard Glossary of Software Engineering Terminology, IEEE, New York, Rep. IEEE-STD-729-1983, 1983.
32. Z. Jelinski and P. B. Moranda, “Applications of a probability-based model to a code reading experiment,” in Proc. IEEE Symp. Computer Software Reliability, New York, 1973, pp. 78–81.
33. K. Jensen and N. Wirth, Pascal User Manual and Report, 2nd ed. New York: Springer-Verlag, 1974.
34. W. L. Johnson, S. Draper, and E. Soloway, “An effective bug classification scheme must take the programmer into account,” in Proc. Workshop High-Level Debugging, Palo Alto, CA, 1983.
35. R. C. Linger, H. D. Mills, and B. I. Witt, Structured Programming: Theory and Practice. Reading, MA: Addison-Wesley, 1979.
36. P. R. McMullin and J. D. Gannon, “Evaluating a data abstraction testing system based on formal specifications,” Dep. Comput. Sci., Univ. Maryland, College Park, Tech. Rep. TR-993, Dec. 1980.
37. B. Meyer, “On formalism in specifications,” IEEE Software, vol. 2, pp. 6–26, Jan. 1985.
38. E. Miller and W. E. Howden, Tutorial: Software Testing & Validation Techniques, 2nd ed., IEEE Catalog No. EHO 180-0. Washington, D.C: IEEE Computer Society Press, 1981.
39. H. D. Mills, “Mathematical foundations for structural programming,” IBM Rep. FSL 72-6021, 1972.
40. —, “How to write correct programs and know it,” in Proc. Int. Conf. Reliable Software, Los Angeles, CA, 1975, pp. 363–370.
41. G. J. Myers, “A controlled experiment in program testing and code walkthroughs inspections,” Commun. ACM, pp. 760–768, Sept. 1978.
42. —, The Art of Software Testing. New York: Wiley, 1979.
43. P. Naur, “Programming by action clusters,” BIT, vol. 9, no. 3, pp. 250–258, 1969.
44. T. J. Ostrand and E. J. Weyuker, “Collecting and categorizing software error data in an industrial environment*,” J. Syst. Software, vol. 4, pp. 289–300, 1984.
45. D. J. Panzl, “Experience with automatic program testing,” i n Proc. NBS Trends and Applications, Nat. Bureau Standards, Gaithersburg, MD, May 28, 1981, pp. 25–28.
46. H. Scheffe, The Analysis of Variance. New York: Wiley, 1959.
47. R. W. Selby, “An empirical study comparing software testing techniques,” in Proc. Sixth Minnowbrook Workshop Software Performance Evaluation, Blue Mountain Lake, NY, July 19-22, 1983.
48. —, “Evaluations of software technologies: Testing, CLEAN-ROOM, ROOM, and metrics,” Ph.D. dissertation, Dep. Comput. Sci., Univ. Maryland, College Park, Tech. Rep. TR-1500, 1985.
49. —— , “Combining software testing strategies: An empirical evaluation,” in Proc. Workshop Software Testing, Banff, Alta., Canada, July 15-17, 1986, pp. 82–91.
50. R. W. Selby, V. R. Basili, and F. T. Baker, “Cleanroom software development: An empirical evaluation,” IEEE Trans. Software Eng., vol. SE-13, pp. 1027–1037, Sept. 1987.
51. R. W. Selby, V. R. Basili, J. Page, and F. E. McGarry, “Evaluating software testing strategies,” in Proc. Ninth Annu. Software Eng. Workshop, NASA/GSFC, Greenbelt, MD, Nov. 1984.
52. L. G. Stucki, “New directions i n automated tools for improving software quality,” in Current Trends in Programming Methodology, R. T. Yeh, Ed. Englewood Cliffs, N J : Prentice Hall, 1977.
53. P. M. Valdes and A. L. Goel, “An error-specific approach to testing,” in Proc. 8th Annu. Software Eng. Workshop, NASA/GSFC, Greenbelt, MD, Nov. 1983.
54. D. M. Weiss and V. R. Basili, “Evaluating software development by analysis of changes: Some data from the software engineering laboratory,” IEEE Trans. Software Eng., vol. SE-11, no. 2, pp. 157–168, 168, Feb. 1985.
55. M. R. Woodward, D. Hedley, and M. A. Hennell, “Experience with path analysis and testing of programs,” IEEE Trans. Software Eng., vol. SE-6, no. 3, pp. 278–286, May 1980.
56. M. V. Zelkowitz, R. T. Yeh, R. G. Hamlet, J. D. Gannon, and V. R. Basili, “Software engineering practices in the US and Japan,” Computer, vol. 17, no. 6, pp. 57–66, June 1984.