When to Stop Testing for Large Software Systems with Changing Code

IEEE Transactions on Software Engineering

Volumn 20, Issue 4, 1994, Pages 318-323

  Dalal, Siddhartha R ^a   McIntosh, Allen A ^a  

^a TELCORDIA TECHNOLOGIES   (United States)

Author keywords

graphical methods; Optimal stopping rule based on cost; software fault detection; software metrics; software reliability model; statistical inference

Indexed keywords

ECONOMICS; OPTIMIZATION; RELIABILITY; STATISTICAL METHODS; TESTING;

CHANGING CODES; LARGE SOFTWARE SYSTEMS; OPTIMAL STOPPING RULE; SOFTWARE FAULT DETECTION; SOFTWARE RELIABILITY MODELS;

COMPUTER SOFTWARE;

EID: 0028413121     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/32.277579     Document Type: Article

References (18)

1. W. S. Cleveland, “Robust locally weighted regression and smoothing scatterplots,” J. Am. Statist. Assoc., vol. 74, pp. 829–836, 1979.
2. S. R. Dalal and C. L. Mallows, “Buying with exacting confidence,” Annals of Applied Probability, vol. 2, pp. 752–765, Mar. 1992.
3. —, “Some graphical aids for deciding when to stop testing software,” IEEE J. Select. Areas Commun., Special Issue on Telecommunications Software Quality and Productivity, vol. 8, pp. 169–175, Feb. 1990.
4. —, “When should one stop testing software?” J. Am. Statist, Assoc., vol. 83, pp. 872–879, 1988.
5. —, “When to s top testing software? Some exact and asymptotic results,” in Bayesian Analysis in Statistics and Econometrics, P. Goel, Ed. New York: Springer-Verlag, 1990, pp. 267–276.
6. S. R. Dalal and A. A. McIntosh, “Reliability modeling and when to stop testing for large software systems in the presence of code churn,” Bellcore internal memorandum, 1992.
7. W. Ehrlich, B. Prasanna, J. Stampfel, and J. Wu, “Determining the cost of a stop-test decision,” IEEE Software, pp. 33–42, 1993.
8. D. M. Gay, “Algorithm 611: Subroutines for unconstrained minimization using a model/trust-region approach,” ACM Trans. Mathematical Software, vol. 9, no. 4, pp. 503–524, 1983.
9. A. L. Goel and K. Okumoto, “Time-dependent error-detection rate model for software reliability and other performance measures,” IEEE Trans. Reliability, vol. R-28, pp. 206–211, 1979.
10. Z. Jelinski and P. B. Moranda, “Software reliability research,” in Statistical Computer Performance Evaluation, W. Frieberger, Ed. New York: Academic, 1972, pp. 465–484.
11. W. Kremer, “Birth-death and bug counting,” IEEE Trans. Reliability, vol. R-32, no. 1, pp. 37–47, Jan. 1983.
12. N. Langberg and N. D. Singpurwalla, “A unification of some software reliability models,” SIAM J. Sci. Stat. Comput., vol. 36, no. 1, pp. 781–790, Jan. 1985.
13. B. Littlewood, “Stochastic reliability-growth: A model for fault-removal in computer programs and hardware design,” IEEE Trans. Reliability, vol. R-30, no. 4, pp. 313–320, Apr. 1981.
14. J. D. Musa, A. Iannino, and K. Okumoto, Software Reliability: Measurement, Prediction, Application. New York: McGraw-Hill, 1987.
15. J. D. Musa, “A theory of software reliability and its applications,” IEEE Trans. Software Eng., vol. SE-1, pp. 312–327, 1975.
16. M. C. J. van Pul, “Statistical Analysis of Software Reliability Models,” Chapter 7, Ph.D. dissertation, Univ. of Utrecht, 1993.
17. M. C. J. Van Put, “Statistical models in software reliability,” Proc. 23rd Symp. Interface, 1991, pp. 106–109.
18. N. Singpurwalla, “Determining an optimal time interval for testing and debugging software,” IEEE Trans. Software Eng., vol. 17, pp. 313–319, Apr. 1991.