The Measurement and Management of Software Reliability

Proceedings of the IEEE

Volumn 68, Issue 9, 1980, Pages 1131-1143

  Musa, John D ^a  

^a LUCENT TECHNOLOGIES   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER SOFTWARE;

EID: 0019057405     PISSN: 00189219     EISSN: 15582256     Source Type: Journal    
DOI: 10.1109/PROC.1980.11812     Document Type: Article

References (68)

1. L. Thomas, The Medusa and the Snail. New York: Viking Press, 1979, pp. 37-40.
2. B. W. Boehm, I. R. Brown, and M. lipow, “Quantitative evaluation of software quality,” in Proc. 2nd Int Conf. Software Engineering (San Francisco, CA), October 13-15, pp. 592-605, 1976.
3. C Nokes, “Availability and reliability of teleprocessing systems,” Comput. Commun., vol. 1 no. 1, pp. 33-41, Feb. 1978.
4. D. K. Lloyd and M. Lipow, Reliability: Management, Methods, and Mathematics, 2nd ed. Redondo Beach, CA, published by the authors, 1977, ch. 9.
5. M. L. Shooman, Probabilistic Reliability: An Engineering Approach. New York: McGraw-Hill, 1968, ch. 3.
6. S. A Gloss-Soler, Ed., The DACS Glossary-A Bibliography of Software Engineering Terms, Data and Analysis Center for Software Rep. GLOS-1, Rome Air Development Center, Rome, NY.
7. J R. E. Barlow and F. Broschan, Mathematical Theory of Reliability. New York: Wiley, 1965, p. 7.
8. B. Littlewood, “How to measure software reliability and how not to,” in Proc. 3rd Int. Conf. Software Engineering (Atlanta, GA), pp. 39-45, May 10-12 1978.
9. H. Hecht, “Measurement, estimation, and prediction of software reliability,” in Software Engineering Technology-Volume 2. Maidenhead, Berkshire, England: Infotech International, 1977, pp. 209-224. (Also in NASA Rep. CR145135, Jan. 1977.)
10. G. R. Hudson, Program errors as a birth and death process, System Development Corp. Rep. SP-3011, Dec. 4, 1967.
11. Z. Jelinski and P. B. Moranda, “Software reliability research,” in Statistical Computer Performance Evaluation, W. Freiberger, Ed. New York: Academic Press, 1972, pp. 465-484.
12. P. Moranda, “Predictions of software reliability during debugging,” in Proc. Ann. Reliability and Maintainability Symp. (Washington, DC), pp. 327-332, Jan. 1975.
13. M. Shooman, “Probabilistic models for software reliability prediction,” in Statistical Computer Performance Evaluation, W. Freidberger, Ed. New York: Academic Press, 1972, pp. 485-502.
14. M. L. Shooman and S. Natarajan, “Effect of manpower deployment and bug generation on software error models,” in Proc. Symp. Computer Software Engineering (New York), pp. 155-170, 1976.
15. G. J. Schick and R W. Wolverton, “Assessment of software reliability,” presented at 11th Annual Meeting of German Operations Research Society, Hamburg, Germany, Sept. 6-8, 1972; in Proc Operations Research, Physica-Verlag, Wurzburg-Wien, 1973, pp. 395-422.
16. “An analysis of competing software reliability models,” IEEE Trans. Software Eng., vol. SE-4, pp. 104-120, Mar. 1978.
17. N. F. Schneidewind, “An approach to software in reliability prediction and quality control,” in 1972 Fall Joint Comput. Conf., AFIPS Conf. Proc, vol 41. Montvale, NJ: AFIPS Press, pp. 837-847.
18. “A methodology for software reliability prediction and quality control,” NTIS Rep. AD 754377.
19. “Analysis of error processes in computer software,” in Proc. 1975 Int. Conf. Reliable Software (Los Angeles, CA), pp. 337-346, Apr. 21-23, 1975.
20. J. D. Musa, “A theory of software reliability and its application,” IEEE Trans. Software Eng., vol. SE-1, pp. 312-327, Sept. 1975.
21. H. Hecht, “Comparison of calendar-time-based and execution-time-based software reliability measurements,” to be published.
22. J. D. Musa, “Validity of the execution time theory of software reliability, IEEE Trans. Rel, vol R-28, pp. 181-191, Aug. 1979.
23. Software reliability data, report available from Data and Analysis Center for Software, Rome Air Development Center, Rome, NY.
24. L Miyamoto, “Software reliability in online real time environment,” in Proc. 1975 Int. Conf. Reliable Software (Los Angeles, CA), pp. 195-203, Apr. 21-23, 1975.
25. J. D. Musa, “Software reliability measurement,” in Software phenomenology: Working papers of the Software Life Cycle Management Workshop (Airlie, VA), pp. 427-451, Aug. 21-23, 1977; J. Syst. Software, to be published.
26. “The use of software reliability measures in project management,” in Proc. COMPSAC 78 (Chicago, IL), pp. 493-498, Nov. 13-16 1978.
27. P. Hamilton and J. D. Musa, “Measuring reliability of computation center software,” in Proc. 3rd Int. Conf. Software Engineering (Atlanta, GA), pp. 29-36, May 10-12 1978.
28. J. D. Musa, “Software reliability measures applied to system engineering,” in 1979 Nat. Computer Conf. Proc. (New York), pp. 941-946, June 4-7 1979.
29. B. Littlewood and J. L. Verrall, “A Bayesian reliability growth model for computer software,” J. Roy. Stat. Soc.-Series C, vol. 22 no. 3, pp. 332-346, 1973.
30. “A Bayesian reliability growth model for computer software,” in Proc. 1973 IEEE Symp. Computer Software Reliability (New York), pp. 70-77, Apr. 30-May 2 1973.
31. “A Bayesian reliability model with a stochastically monotone failure rate,” IEEE Trans. Rel, vol. R-23, pp. 108-114, June 1974.
32. B. Littlewood, “What makes a reliable program-Few bugs, or a small failure rate?,” submitted for Nat. Computer Conf. 1980.
33. A. L. Goel and K. Okumoto, Bayesian software prediction models, Rome Air Development Center, Rome, NY, Rep. RADC-TR-78-155 (5 volumes).
34. “Time-dependent error-detection rate model for software reliability and other performance measures,” IEEE Trans. Rel., vol. R-28, pp. 206-211, Aug. 1979.
35. W. H. MacWilliams, “Reliability of large, real time control software systems,” in Proc. 1973 IEEE Symp. Computer Software Reliability (New York), pp. 1-6, Apr. 30-May 2, 1973.
36. T. A. Thayer, Software reliability study, Rome Air Development Center, Rome, NY, Rep. RADC-TR-76-238.
37. J. D. Musa and A Iannino, “Estimation of software reliability of sequentially integrated systems,” to be published.
38. M. A. Herndon and N. T. Keenan, “Analysis of error remediation expenditures during validation,” in Proc. Int. Conf. Software Engineering (Atlanta, GA), pp. 202-206, May 10-12, 1978.
39. R. W. Motley and W. D. Brooks, Statistical Prediction of Programming errors, Rome Air Development Center, Rome, NY, Rep. RADC-TR-77-175.
40. K. Akiyama, “An example of software system debugging,” in Proc. IFIPS Congr. 1971. Amsterdam: The Netherlands, North-Holland, 1971, pp. 353-359.
41. A. R. Feuer and E. B. Fowlkes, “Some results from an empirical study of computer software,” in Proc. 4th. Int Conf. Software Engineering (Munich, Germany), pp. 351-355, Sept. 17-19, 1979.
42. A. Endres, “An analysis of errors and their causes in system programs,” in Proc. 1975 Int. Conf. Reliable Software (Los Angeles, CA), pp. 327-336, Apr. 21-23, 1975.
43. M. H. Halstead, Elements of Software Science. New York: Elsevier North Holland, 1977.
44. Y. Funami and M. H. Halstead, “A software physics analysis of Akiyama's debugging data,” in Proc. Symp. Computer Software Engineering (New York), pp. 133-138, 1976.
45. L. M Ottenstein, “Quantitative estimates of debugging requirements,” IEEE Trans. Software Eng., vol. SE-5, pp. 504-514, Sept. 1979.
46. N. F. Schneidewind, H. M. Hoffman, “An experiment in software error data collection and analysis,” IEEE Trans. Software Engineering, vol. SE-5, pp. 276-286, May 1979.
47. L. A. Belady, “On software complexity,” in Proc. Workshop Quantitative Software Models (Kiamesha Lake, NY), pp. 90-94, Oct. 9-11, 1979.
48. B. Curtis, “In search of software complexity,” in Proc. Workshop, Quantitative Software Models (Kiamesha Lake, NY), pp. 95-106, Oct. 9-11, 1979.
49. J. C Rault, “An approach towards reliable software,” in Proc. 4th. Int. Conf. Software Engineering (Munich, Germany), pp. 220-230, Sept. 17-19, 1979.
50. M Lipow and T. A. Thayer, “Prediction of software failures,” in Proc. Symp. Reliability Maintainability, 1977, pp. 489-494.
51. C. E. Martin, A model for estimating the number of residual errors in COBOL programs, Ph.D. dissertation, Auburn Univ., Auburn, AL, June 1977.
52. N. F. Schneidewind, “The use of simulation in the evaluation of software,” Comput., vol. 10, no. 4, pp. 47-53, Apr. 1977.
53. J. D. Musa, Program for software reliability and system test schedule estimation-user's guide, available from the author.
54. J. D. Musa and P. A Hamilton, Program for software reliability and system test schedule estimation-program documentation, available from the author.
55. B. Littlewood, “MTBF is meaningless in software reliability,” IEEE Trans. Rel. (corres.), vol. 24, p. 82, Apr. 1975.
56. J. D. Musa, private communication to B. Littlewood.
57. A. Iannino, private communication to J. D. Musa and B. Littlewood.
58. B. Littlewood, “Validation of a software reliability model,” in Proc. 2nd Software Life Cycle Management Workshop (Atlanta, GA), pp. 146-152, Aug. 21-22, 1978.
59. “A semi-Markov model for software reliability with failure costs,” in Proc. Symp. Computer Software Engineering (New York), pp. 281-300, 1976.
60. A. K. Trivedi and M. L. Shooman, Computer Software Reliability: Many-State Markov Modeling Techniques, Polytechnic Inst. New York, Rep. POLY-EE/EP-75-005/EER 116, Mar. 1975.
61. A. Costes, C. Landrault, and J. C LaPrie, “Reliability and availability models for maintained systems featuring hardware failures and design faults,” IEEE Trans. Comput., vol. C-27, pp. 548-560, June 1978.
62. J. D. Musa, “Software reliability modeling-Where are we and where should we be going?,” in Proc. 4th NASA Software Engineering Workshop (Greenbelt, MD), Nov. 19, 1979, to be published.
63. H. D. Mills, “On the statistical validation of computer programs,” IBM Federal Systems Div., Gaithersburg, MD, Rep. FSC-72-6015, 1972.
64. S. L. Basin, Estimation of Software Error Rates via Capture-Recapture Sampling: A Critical Review. Palo Alto, CA, Science Applications, Sept. 1973.
65. B. Rudner, “Seeding/tagging estimation of software errors: Models and estimates,” Rome Air Development Center, Rome, NY, Rep. RADC-TR-77-15.
66. T. Gilb, Software Metrics. Cambridge, MA: Winthrop, 1977, p. 28.
67. L. A. Belady and M. M. Lehman, “A model of large program development,” IBM Syst. J., vol. 15, no. 3, pp. 225-252, 1976.
68. A. N. Sukert, “Empirical validation of three software error prediction models,” IEEE Trans. Rel., vol. R-28, no. 3, pp. 199-205, Aug. 1979.