Estimation of software reliability by stratified sampling

ACM Transactions on Software Engineering and Methodology

Volumn 8, Issue 3, 1999, Pages 263-283

  Podgurski, Andy ^a   Masri, Wassim ^a   McCleese, Yolanda ^a   Wolff, Francis G ^a   Yang, Charles ^b  

^a Case Western Reserve University   (United States)

^b USA   (United States)

Author keywords

Beta testing; Cluster analysis; D.2.2 Software Engineering : Design Tools and Techniques; D.2.5 Software Engineering : Testing and Debugging; D.4.5 Operating Systems : Reliability; Operational testing; Software reliability; Software testing

Indexed keywords

EID: 0000150763     PISSN: 1049331X     EISSN: None     Source Type: Journal    
DOI: 10.1145/310663.310667     Document Type: Article

References (29)

1. ADRION, W. R., BRANSTAD, M. A., AND CHERNIAVSKY, J. C. 1982. Verification, validation, and testing of computer software. ACM Comput. Surv. 14, 2 (June), 159-192.
2. ANDERBERG, M. R. 1973. Cluster Analysis for Applications. Academic Press, Inc., New York, NY.
3. BROWN, J. R. AND LIPOW, M. 1975. Testing for software reliability. In Proceedings of the International Conference on Reliable Software (Los Angeles, CA, Apr.). 518-527.
4. COCHRAN, W. G. 1977. Sampling Techniques. John Wiley & Sons, Inc., New York, NY.
5. DEMILLO, R. A., LIPTON, R. J., AND SAYWARD, F. G. 1978. Hints on test data selection: Help for the practising programmer. Computer 11, 4, 34-41.
6. DURAN, J. W. AND WIORKOWSKI, J. J. 1980. Quantifying software validity by sampling. IEEE Trans. Reliab. R-29, 2 (June), 141-144.
7. DURAN, J. W. AND NTAFOS, S. C. 1984. An evaluation of random testing. IEEE Trans. Softw. Eng. SE-10, 4 (July), 438-444.
8. FRANKL, P. G. AND WEYUKER, E. J. 1993a. A formal analysis of the fault detecting ability of testing methods. IEEE Trans. Softw. Eng. 19, 3, 202-213.
9. FRANKL, P. G. AND WEYUKER, E. J. 1993b. Provable improvements on branch testing. IEEE Trans. Softw. Eng. 19, 10 (Oct.), 962-975.
10. HAMLET, D. AND TAYLOR, R. 1990. Partition testing does not inspire confidence. IEEE Trans. Softw. Eng. 16, 12 (Dec. 1990), 1402-1411.
11. HARROLD, M. J., MCGREGOR, J. D., AND FITZPATRICK, K. J. 1992. Incremental testing of object-oriented class structures. In Proceedings of the 14th International Conference on Software Engineering (ICSE '92, Melbourne, Australia, May 11-15), T. Montgomery, Ed. ACM Press, New York, NY, 68-80.
12. HOWDEN, W. E. 1980. Functional program testing. IEEE Trans. Softw. Eng. SE-6, 2 (Mar.), 162-169.
13. KAUFMAN, L. AND ROUSSEEUW, P. J. 1990. Finding Groups in Data. John Wiley & Sons, Inc., New York, NY.
14. KERNIGHAN, B. W. AND RITCHIE, D. M. 1988. The C Programming Language. Prentice Hall Press, Upper Saddle River, NJ.
15. MILLER, K. W., MORELL, L. J., NOONAN, R. E., PARK, S. K., NICOL, D. M., MURRILL, B. W., AND VOAS, J. M. 1992. Estimating the probability of failure when testing reveals no failures. IEEE Trans. Softw. Eng. 18, 1 (Jan. 1992), 33-43.
16. MUSA, J. D. 1993. Operational profiles in software-reliability engineering. IEEE Softw. (Mar.), 14-32.
17. NELSON, E. N. 1978. Estimating software reliability from test data. Mircroelectron. Reliab. 17, 67-74.
18. NEYMAN, J. 1934. On two different aspects of the representative method: The method of stratified sampling and the method of purposive selection. J. Royal Stat. Soc. B. 97, 558-606.
19. PODGURSKI, A. AND YANG, C. 1993. Partition testing, stratified sampling, and cluster analysis. SIGSOFT Softw. Eng. Notes 18, 5 (Dec. 1993), 169-181.
20. RAPPS, S. AND WEYUKER, E. J. 1985. Selecting software test data using data flow information. IEEE Trans. Softw. Eng. SE-11, 4 (Apr.), 367-375.
21. RICHARDSON, D. J. AND CLARKE, L. A. 1981. A partition analysis method to increase program reliability. In Proceedings of the 5th International Conference on Software Engineering (Los Alamitos, CA). 244-253.
22. SARNDAL, C.-E., SWENSSON, B., AND WRETMAN, J. 1992. Model Assisted Survey Sampling. Springer-Verlag, New York, NY.
23. SCHICK, G. J. AND WOLVERTON, R. W. 1978. An analysis of competing software reliability models. IEEE Trans. Softw. Eng. SE-4, 2 (Mar.), 104-120.
24. TAYLOR, R. N., LEVINE, D. L., AND KELLY, C. D. 1992. Structural testing of concurrent programs. IEEE Trans. Softw. Eng. 18, 3 (Mar. 1992), 206-215.
25. THAYER, T. A., LIPOW, M., AND NELSON, E. C. 1976. Software reliability. Tech. Rep. TRW-SS-76-03. TRW.
26. THÉVENOD-FOSSE, P. AND WAESELYNCK, H. 1993. STATEMATE applied to statistical software testing In Proceedings of the 1993 International Symposium on Software Testing and Analysis (ISSTA, Cambridge, MA, June 28-30, 1993), T. Ostrand and E. Weyuker, Eds. ACM Press, New York, NY, 99-109.
27. THOMPSON, S. K. 1992. Sampling. John Wiley & Sons, Inc., New York, NY.
28. TSOUKALAS, M. Z., DURAN, J. W., AND NTAFOS, S. 1993. On some reliability estimation problems in random and partition testing. IEEE Trans. Softw. Eng. 19, 7 (July), 687-697.
29. WEYUKER, E. J. AND JENG, B. 1991. Analyzing partition testing strategies. IEEE Trans. Softw. Eng. 17, 7 (July 1991), 703-711.