Evaluating testing methods by delivered reliability

IEEE Transactions on Software Engineering

Volumn 24, Issue 8, 1998, Pages 586-601

  Frankl, Phyllis G ^a   Hamlet, Richard G ^a   Littlewood, Bev ^a  

^a POLYTECHNIC UNIVERSITY   (United States)

Author keywords

Debugging; Reliability; Software testing; Statistical testing theory

Indexed keywords

EID: 0001134764     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/32.707695     Document Type: Article

References (26)

1. V. Basili and S. Green, "Software Process Evolution at the SEL," IEEE Software, pp. 58-66,1994.
2. B. Beizer, "The Cleanroom Process Model: A Critical Examination," Proc. 13th Ann. Pacific Northwest Software Quality Conf., pp. 148-173, Portland, Ore., 1995.
3. R.W. Butler and G.B. Finelli, "The Infeasibility of Quantifying the Reliability of Life-Critical Real-Time Software," IEEE Trans. Software Eng., vol. 19, no. 1, pp. 3-12, Jan. 1993.
4. T.Y. Chen and Y.T. Yu, "On the Relationship between Partition and Random Testing," IEEE Trans. Software Eng., vol. 20, no. 12, pp. 977-980, Dec. 1994.
5. T.Y. Chen and Y.T. Yu, "On the Expected Number of Failures Detected by Subdomain Testing and Random Testing," IEEE Trans. Software Eng., vol. 22, no. 2, pp. 109-119, Feb. 1996.
6. R.H. Cobb and H.D. Mills, "Engineering Software Under Statistical Quality Control," IEEE Software, pp. 44-54, Nov. 1990.
7. S.R. Dalai, J.R. Morgan, and J.R. Kettenring, "Reliable Software and Communication: Software Quality, Reliability, and Safety," Proc. 15th ICSE, pp. 425-435, Baltimore, Md., 1993.
8. J. Duran and S. Ntafos, "An Evaluation of Random Testing," IEEE Trans. Software Eng., vol. 10, pp. 438-444,1984.
9. P.G. Frankl and S.N. Weiss, "An Experimental Comparison of the Effectiveness of Branch Testing and Data Flow Testing," IEEE Trans. Software Eng., vol. 19, no. 8, pp. 774-787, Aug. 1993.
10. P.G. Frankl and EJ. Weyuker, "Provable Improvements on Branch Testing," IEEE Trans. Software Eng., vol. 19, no. 10, pp. 962-975, Oct. 1993.
11. S. Gerhart, D. Craigen, and T. Ralston, "Observations on Industrial Practice Using Formal Methods," Proc. 15th Int'l Conf. Software Eng., pp. 24-33, Baltimore, Md., 1993.
12. D. Hamlet and R. Taylor, "Partition Testing Does Not Inspire Confidence," IEEE Trans. Software. Eng., vol. 16, pp. 1,402-1,411, Dec. 1990.
13. J. Morgan, S. London, and M. Lyu, "Achieving Software Quality with Testing Coverage," Computer, vol. 27, pp. 60-69,1994.
14. M. Hutchins, H. Foster, T. Goradia, and T. Ostrand, "Experiments on the Effectiveness of Dataflow-and Controlflow-Based Test Adequacy Criteria," Proc. 16th Int'l Conf. Software Eng., May 1994.
15. Z. Jelinski and P.B. Moranda, "Software Reliability Research," Statistical Computer Performance Evaluation, pp. 465-4184. New York: Academic Press, 1972.
16. B. Jeng and EJ. Weyuker, "Analyzing Partition Testing Strategies," IEEE Trans. Software Eng., vol. 17, pp. 703-711,1991.
17. L. Lauterbach and W. Randall, "Experimental Evaluation of Six Test Techniques.," COMPASS '89, pp. 36-41, Gaithersburg, Md., 1989.
18. N. Li and Y.K. Malaiya, "On Input Profile Selection for Software Testing," Proc. Fifth Int'l Symp. Software Reliability Eng., pp. 196205,1994.
19. B. Littlewood, "Stochastic Reliability Growth: A Model with Applications to Computer Software and Hardware Design," IEEE Trans. Reliability, vol. 30, no. 4, pp. 313-320, Oct. 1981.
20. B. Littlewood and L. Strigini, "Validation of Ultra-High Dependability for Software-Based Systems," Comm. ACM, vol. 36, no. 11, pp. 69-80,1993.
21. J.D. Musa, "Operational Profiles in Software-Reliability Engineering," IEEE Software, pp. 14-32,1993.
22. P. Piwowarski, M. Ohba, and J. Caruso, "Coverage Measurement Experience During Function Test," Proc. 15th Int'l Conf. Software Eng., pp. 287-301, Baltimore, Md., 1993.
23. M. Pizza and L. Strigini, "The Effect of Program Variability on Debugging Efficiency-Preliminary Results," technical report, Centre for Software Reliability, 1998.
24. R. Selby, V.R. Basili, and T. Baker, "Cleanroom Software Development: An Empirical Evaluation," IEEE Trans. Soft. Eng., vol. 13, no. 9, pp. 1,027-1,037, Sept. 1987.
25. M.Z. Tsoukalas, J.W. Duran, and S.C. Ntafos, "On Some Reliability Estimation Problems in Random and Partition Testing," IEEE Trans. Software Eng., vol. 19, pp. 687-697, July 1993.
26. W.E. Wong, J.R. Horgan, S. London, and A.P. Mathur, "Effect of Test Set Size and Block Coverage on The Fault Detection Effectiveness," Technical Report SERC-TR-153-P, SERC, 1994.