Emulation of software faults: A field data study and a practical approach

IEEE Transactions on Software Engineering

Volumn 32, Issue 11, 2006, Pages 849-867

  Durães, João A ^a   Madeira, Henrique S ^b  

^a POLYTECHNIC INSTITUTE OF COIMBRA   (Portugal)

^b UNIVERSITY OF COIMBRA   (Portugal)

Author keywords

Fault injection; Software faults; Software reliability

Indexed keywords

COMPUTER PROGRAMMING; COMPUTER SOFTWARE PORTABILITY; COMPUTER SYSTEMS; FAULT TOLERANCE;

FAULT INJECTION; SOFTWARE FAULT INJECTION TECHNIQUE (G-SWFIT); SOFTWARE FAULTS;

SOFTWARE RELIABILITY;

EID: 33947314499     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.2006.113     Document Type: Article

References (50)

1. M. Kalyanakrishnam, Z. Kalbarczyk, and R. Iyer, "Failure Data Analysis of a LAN of Windows NT Based Computers," Proc. Symp. Reliable Distributed Database Systems (SRDS-18), pp. 178-187, 1999.
2. I. Lee and R.K. Iyer, "Software Dependability in the Tandem GUARDIAN System," IEEE Trans. Software Eng., vol. 21, no. 5, pp. 455-467, May 1995.
3. M. Sullivan and R. Chillarege, "Comparison of Software Defects in Database Management Systems and Operating Systems," Proc. 22nd IEEE Fault Tolerant Computing Symp. (FTCS 22), pp. 475-484, July 1992.
4. J. Gray, "A Census of Tandem Systems Availability between 1985 and 1990," IEEE Trans. Reliability, vol. 39, no. 4, pp. 409-418, Oct. 1990.
5. J. Musa, Software Reliability Engineering. McGraw-Hill, 1996.
6. M.R. Lyu, Handbook of Software Reliability Engineering. IEEE Computer Society Press & McGraw-Hill, 1996.
7. J. Christmansson and P. Santhanam, "Error Injection Aimed at Fault Removal in Fault Tolerance Mechanisms-Criteria for Error Selection Using Field Data on Software Faults," Proc. Seventh IEEE Int'l Symp. Software Reliability Eng. (ISSRE '96), Oct./Nov. 1996.
8. J. Voas, F. Charron, G. McGraw, K. Miller, and M. Friedman, "Predicting How Badly 'Good' Software can Behave," IEEE Software, vol. 14, no. 4, 1997.
9. W.T. Ng and P.M. Chen, "Systematic Improvement of Fault Tolerance in the RIO File Cache," Proc. IEEE Fault Tolerant Computing Symp. (FTCS), June 1999.
10. A. Brown and D. Patterson, "Toward Availability Benchmark: A Case Study of Software RAID Systems," Proc. USENIX Ann. Technical Conf., pp 263-276, June 2000.
11. M. Vieira and H Madeira, "A Dependability Benchmark for OLTP Application Environments," Proc. 29th Int'l Conf. Very Large Databases (VLDB '03), Sept. 2003.
12. J. Zhu, J. Mauro, and I. Pramanick, "R3-A Framework for Availability Benchmarking," Proc. Int'l Conf. Dependable Systems and Networks (DSN '03), pp. B86-B87, 2003.
13. S. Lightstone, J. Hellerstein, W. Tetzlaff, P. Janson, E. Lassettre, C. Norton, B. Rajaraman, and L. Spainhower, "Toward Benchmarking Autonomic Computing Maturity," Proc. First IEEE Conf. Industrial Automatics (INDIN '03), Aug. 2003.
14. K. Kanoun, J. Arlat, D. Costa, M.D. Cin, P. Gil, J.C. Laprie, H. Madeira, and N. Suri, "DBench: Dependability Benchmarking," Proc. Supplement of the IEEE/IFIP Int'l Conf. Dependable Systems and Networks (DSN '01), 2001.
15. J. Christmansson and R. Chillarege, "Generation of an Error Set that Emulates Software Faults," Proc. 26th IEEE Fault Tolerant Computing Symp. (FTCS 26), pp. 304-313, June 1996.
16. J. Durães and H. Madeira, "Characterization of Operating Systems Behavior in the Presence of Faulty Device Drivers through Software Fault Emulation," Proc. Pacific Rim Int'l Symp. Dependable Computing (PRDC '02), pp. 201-209, 2002.
17. J. Durães, M. Vieira, and H. Madeira, "Web Server Dependability Benchmaking," Proc. Conf. Computer Safety (SAFECOMP), 2004.
18. H. Madeira, M. Vieira, and D. Costa, "On the Emulation of Software Faults by Software Fault Injection," Proc. Int'l Conf. Dependable Systems and Networks (DSN '00), pp. 417-426, 2000.
19. R. Chillarege, "Orthogonal Defect Classification," Handbook of Software Reliability Eng., chapter 9, IEEE CS Press, McGraw-Hill, 1995.
20. R. DeMillo, D. Guindi, W. McCracken, A. Offut, and K. King, "An Extended Overview of the Mothra Software Testing Environment," Proc. ACM SIGSOFT/IEEE Second Workshop Software Testing, Verification, and Analysis, pp. 142-151, July 1988.
21. S. Yamada, M. Ohba, and S. Osaki, "S-Shaped Software Reliability Growth Models and Their Application," IEEE Trans. Reliability, vol. 33, no. 4, pp. 289-292, Oct. 1984.
22. J.K. Chaar, M.J. Halliday, I.S. Bhandari, and R. Chillarege, "In-Process Evaluation for Software Inspection and Test," IEEE Trans. Software Eng., vol. 19, no. 11, pp. 1055-1070, Nov. 1993.
23. T. Khoshgoftaar et al. "Process Measures for Predicting Software Quality," Proc. High Assurance Systems Eng. Workshop (HASE '97), 1997.
24. J.P. Hudepohl et al., "EMERALD: A Case Study in Enhancing Software Reliability," Proc. IEEE Eighth Int'l Symp. Software Reliability Eng. (ISSRE '98), pp. 85-91, Nov. 1998.
25. M. Sullivan and R. Chillarege, "Software Defects and Their Impact on Systems Availability - A Study of Field Failures on Operating Systems," Proc. 21st IEEE Fault Tolerant Computing Symp. (FTCS 21), pp. 2-9, June 1991.
26. R.K. Iyer, "Experimental Evaluation," Proc. 25th IEEE Symp. Fault Tolerant Computing (FTCS 25), pp. 115-132, June 1995.
27. R. Chillarege, I.S. Bhandari, J.K. Chaar, M.J. Halliday, D. Moebus, B. Ray, and M. Wong, "Orthogonal Defect Classification - A Concept for In-Process Measurement," IEEE Trans. Software Eng., vol. 18, no. 11, pp. 943-956, Nov. 1992.
28. T. Budd, "Mutation Analysis: Ideas, Examples, Problems, and Prospects," Computer Program Testing, pp. 129-134, 1981.
29. M. Daran and P. Thévenod-Fosse, "Software Error Analysis: A Real Case Study Involving Real Faults and Mutations," Proc. Third Symp. Software Testing and Analysis (ISSTA '03), pp. 158-171, Jan. 1996.
30. J. Arlat et al., "Fault Injection and Dependability Evaluation of Fault Tolerant Systems," IEEE Trans. Computers, vol. 42, no. 8, pp. 919-923, Aug. 1993.
31. G. Kanawati, N. Kanawati, and J. Abraham, "FERRARI: A Tool for the Validation of System Dependability Properties," Proc. 22th IEEE Fault Tolerant Computing Symp. (FTCS 22), pp. 336-344, June 1992.
32. T. Tsai and R.K. Iyer, "An Approach to Benchmarking of Fault-Tolerant Commercial Systems," Proc. 26th IEEE Fault Tolerant Computing Symp. (FTCS 26), pp. 314-323, June 1996.
33. J. Carreira, H. Madeira, and J.G. Silva, "Xception: Software Fault Injection and Monitoring in Processor Functional Units," IEEE Trans. Software Eng., vol. 24, no. 2, Feb. 1998.
34. J. Aidemark, J. Vinter, P. Folkesson, and J. Karlsson, "Goofi: Generic Object-Oriented Fault Injection Tool," Proc. Int'l Conf. Dependable Systems and Networks (DSN '01), July 2001.
35. J.-C. Fabre, F. Salles, M. Moreno, and J. Arlat, "Assessment of COTS Microkernels by Fault Injection," Proc. Seventh IFIP Working Conf. Dependable Computing for Critical Applications (DCCA '99), pp. 25-44, 1999.
36. P. Koopman, J. Sung, C. Dingman, D. Siewiorek, and T. Marz, "Comparing Operating Systems Using Robustness Benchmarks," Proc. 16th Int'l Symp. Reliable Distributed Systems (SRDS '97), pp. 72-79, 1997.
37. J. Hudak, B. Suth, D. Siewiorek, and Z. Segall, "Evaluation and Comparison of Fault-Tolerant Software Techniques," IEEE Trans. Reliability, vol. 42, no. 2, pp. 190-204, June 1993.
38. W. Kao, "Experimental Study of Software Dependability," PhD thesis, Technical Report CRHC-94-16, Dept. of Computer Science, Univ. of Illinois at Urbana-Champaign, Illinois, 1994.
39. J. Bieman, D. Dreilinger, and L. Lin, "Using Fault Injection to Increase Test Coverage," Proc. Seventh IEEE Int'l Symp. Software Reliability Eng. (ISSRE '96), Oct./Nov. 1996.
40. D. Blough and T. Torii, "Fault-Injection-Based Testing of Fault-Tolerant Algorithms in Message-Passing Parallel Computers," Proc. 27th IEEE Fault Tolerant Computing Symp. (FTCS 27), pp. 258-267, June 1997.
41. W.T. Ng, C.M. Aycock, and P.M. Chen, "Comparing Disk and Memory's Resistance to Operating System Crashes," Proc. Seventh IEEE Int'l Symp. Software Reliability Eng. (ISSRE '96), 1996.
42. D. Costa, T. Rilho, and H. Madeira, "Joint Evaluation of Performance and Robustness of a COTS DBMS through Fault-Injection," Proc. Dependable Systems and Networks Conf. (DSN '00), June 2000.
43. D. Costa, T. Rilho, M. Vieira, and H. Madeira, "ESFFI-A Novel Technique for the Emulation of Software Faults in COTS Components," Proc. Eighth Ann. IEEE Int'l Conf. Eng. of Computer-Based Systems (ECBS '01), Apr. 2001.
44. J. Durães and H. Madeira, "Emulation of Software Faults by Selective Mutations at Machine-Code Level," Proc. 13th IEEE Int'l Symp. Software Reliability Eng. (ISSRE '02), Nov. 2002.
45. J. Durães and H. Madeira, "Definition of Software Fault Emulation Operators: A Field Data Study," Proc. IEEE/IFIP Int'l Conf. Dependable Systems and Networks, Dependable Computing, and Comm. (DSN-DCC '03), June 2003.
46. J. Durães and H. Madeira, "Generic Faultloads Based on Software Faults for Dependability Benchmarking," Proc. IEEE Int'l Symp. Dependable Systems and Networks (DSN '04), 2004.
47. Intel Architecture Software Developer's Manual, Vol. 2: Instruction Set Reference, Intel, 1997.
48. J. Voas and G. McGraw, Software Fault Injection: Inoculating Programs against Errors. John Wiley and Sons, 1997.
49. R.A. DeMillo, R.J. Lipton, and F.G. Sayward, "Hints on Test Data Selection: Help for the Practicing Programmer," Computer, vol. 11, no. 4, pp. 34-41, Apr. 1978.
50. R.E. Kirk, Statistics: An Introduction, fourth ed. Assessment Systems Corp., 1999.