Efficient testing of recovery code using fault injection

ACM Transactions on Computer Systems

Volumn 29, Issue 4, 2011, Pages

  Marinescu, Paul D ^a   Candea, George ^a  

^a EPFL   (Switzerland)

Author keywords

Automated testing; Fault injection

Indexed keywords

AUTOMATED TECHNIQUES; AUTOMATED TESTING; CRITICAL PARTS; FAULT INJECTION; FAULT INJECTION ENGINES; INJECTION TEST; MYSQL DATABASE; NAME SERVERS; PROGRAM BINARY; REPLICATION SYSTEMS; SIDE EFFECT; SOFTWARE SYSTEMS; SOURCE CODES; VERSION CONTROL SYSTEM;

CODES (SYMBOLS); SOFTWARE TESTING;

RECOVERY;

EID: 84857182468     PISSN: 07342071     EISSN: 15577333     Source Type: Journal    
DOI: 10.1145/2063509.2063511     Document Type: Article

References (63)

1. AHO, A. V., SETHI, R., AND ULLMAN, J. D. 1986. Compilers: Principles, Techniques, and Tools. Addison-Wesley Longman Publishing Co., Inc., Boston, MA.
2. APACHE. 2010. Apache Benchmark (AB). http://httpd.apache.org/docs/2.0/ programs/ab.html.
3. ARLAT, J., AGUERA, M., AMAT, L., CROUZET, Y., FABRE, J.-C., LAPRIE, J.-C., MARTINS, E., AND POWELL, D. 1990. Fault injection for dependability validation: A methodology and some applications. IEEE Trans. Softw. Engin. 16, 2.
4. BAIRAVASUNDARAM, L. N., RUNGTA, M., AGRAWAL, N., ARPACI-DUSSEAU, A. C., ARPACI-DUSSEAU, R. H., AND SWIFT, M. M. 2008. Analyzing the effects of disk-pointer corruption. In Proceedings of the International Conference on Dependable Systems and Networks.
5. BARBOSA, R., SILVA, N., DURAES, J., AND MADEIRA, H. 2007. Verification and validation of (real time) COTS products using fault injection techniques. In Proceedings of the International Conference on Commercial-off-the-Shelf-Based Software Systems.
6. BARTON, J., CZECK, E., SEGALL, Z., AND SIEWIOREK, D. 1990. Fault injection experiments using FIAT. IEEE Trans. Comput. 39, 4.
7. BIEMAN, J. M., DREILINGER, D., AND LIN, L. 1996. Using fault injection to increase software test coverage. In Proceedings of the International Symposium on Software Reliability Engineering.
8. BIND. 2010a. BIND aborts in dst api.c. https://lists.isc.org/pipermail/ bind-users/2010-January/078493.html.
9. BIND. 2010b. BIND crashes in statschannel.c. https://lists.isc.org/ pipermail/bind-users/2010-January/078428.html.
10. BISOLFATI, E., MARINESCU, P. D., AND CANDEA, G. 2010. Studying application-library interaction and behavior with LibTrac. In Proceedings of the International Conference on Dependable Systems and Networks.
11. BROADWELL, P. A., SASTRY, N., AND TRAUPMAN, J. 2002. FIG: A prototype tool for online verification of recovery mechanisms. In Proceedings of the Workshop on Self-Healing, Adaptive and Self-Managed Systems.
12. CADAR, C., DUNBAR, D., AND ENGLER, D. R. 2008. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In Proceedings of the Symposuim on Operating System Design and Implem.
13. CANDEA, G., DELGADO, M., CHEN, M., AND FOX, A. 2003. Automatic failure-path inference: A generic introspection technique for software systems. In Proceedings of the Workshop on Internet Applications.
14. CASTRO, M. AND LISKOV, B. 1999. Practical Byzantine fault tolerance. In Proceedings of the Symposium on Operating Systems Design and Implementation.
15. CHILLAREGE, R. AND BOWEN, N. S. 1989. Understanding large system failures - a fault injection experiment". In Intl. Symp. on Fault-Tolerant Computing.
16. CHIPOUNOV, V., KUZNETSOV, V., AND CANDEA, G. 2011. S2E: A platform for in-vivo multi-path analysis of software systems. In Proceedings of the International Conference on Architectural Support for Programming Languages and Operating Systems.
17. COOK, W. R. 2007. Applescript. In Proceedings of the Conference on History of Programming Languages.
18. CPYTHON. 2011. http://www.python.org/.
19. CURRY, T. W. 1994. Profiling and tracing dynamic library usage via interposition. In Proceedings of the USENIX Summer Technical Conference.
20. DAWSON, S., JAHANIAN, F., AND MITTON, T. 1997. Experiments on six commercial TCP implementations using a software fault injection tool. Softw. Pract. Exper. 27.
21. DOWSON, M. 1997. The Ariane 5 software failure. ACM SIGSOFT Softw. Engin. Notes 22, 2.
22. ELSA. 2009. http://www.eecs.berkeley.edu/~smcpeak/elkhound/sources/elsa/. (Accessed on 3/09).
23. FU, C., RYDER, B. G., MILANOVA, A., AND WONNACOTT, D. 2004. Testing of Java web services for robustness. In Proceedings of the International Symposium on Software Testing and Analysis.
24. GCOV 2010. GCC coverage testing tool. http://gcc.gnu.org/onlinedocs/gcc/ Gcov.html.
25. GIT. 2010a. Git crashes on make test. http://marc.info/?l=git&m= 125985479417107.
26. GIT. 2010b. Git fails when running commands in wrong environment. http://marc.info/?l=git&m=125986795807036.
27. GIT. 2010c. Git unchecked malloc's. http://marc.info/?l=git&m= 126298802319662.
28. GUNAWI, H. S., RUBIO-GONZÁLEZ, C., ARPACI-DUSSEAU, A. C., ARPACI-DUSSEAU, R. H., AND LIBLIT, B. 2008. EIO: Error handling is occasionally correct. In Proceedings of the USENIX Conference on File and Storage Technologies.
29. GUNAWI, H. S., DO, T., JOSHI, P., HELLERSTEIN, J. M., ARPACI-DUSSEAU, A. C., ARPACI-DUSSEAU, R. H., AND SEN, K. 2010. Towards automatically checking thousands of failures with micro-specifications. Tech. rep. UCB/EECS-2010-98, University of California.
30. GUO, Z., WANG, X., TANG, J., LIU, X., XU, Z., WU, M., KAASHOEK, M. F., AND ZHANG, Z. 2008. R2: An application-level kernel for record and replay. In Proceedings of the Symposium on Operating Systems Design and Implementation.
31. HADOOP FI. 2010. Hadoop Fault Injection framework. http://hadoop.apache. org/hdfs/docs/r0.21.0/faultinject-framework.html.
32. HOLODECK. 2010. Win32 fuzz testing and fault injection. http://www.securityinnovation.com/holodeck/.
33. HUNT, G. AND BRUBACHER, D. 1999. Detours: Binary Interception of Win32 Functions. In Proceedings of the USENIX Windows NT Symposium.
34. JOHANSSON, A., SURI, N., AND MURPHY, B. 2007. On the impact of injection triggers for OS robustness evaluation. In Proceedings of the International Symposium on Software Reliability Engineering.
35. KANAWATI, G. A., KANAWATI, N. A., AND ABRAHAM, J. A. 1995. FERRARI: A flexible software-based fault and error injection system. IEEE Trans. Comput. 44, 2.
36. KILLIAN, C., ANDERSON, J. W., JHALA, R., AND VAHDAT, A. 2007. Life, death, and the critical transition: Finding liveness bugs in systems code. In Proceedings of the Symposium on Networked Systems Design and Implementation.
37. KOOPMAN, P., SUNG, J., DINGMAN, C., SIEWIOREK, D., AND MARZ, T. 1997. Comparing operating systems using robustness benchmarks. In Proceedings of the International Symposium on Software Reliability Engineering.
38. LCOV. 2010. LTP gcov extension. http://ltp.sourceforge.net/coverage/lcov. php.
39. LETHBRIDGE, T. C., SINGER, J., AND FORWARD, A. 2003. How software engineers use documentation: The state of the practice. IEEE Softw. 20, 6.
40. LI, X., MARTIN, R., NAGARAJA, K., NGUYEN, T. D., AND ZHANG, B. 2002. Mendosus: A san-based faultinjection test-bed for the construction of highly available network services. In Proceedings of the Workshop on Novel Uses of System Area Networks.
41. LIBDWARF. 2010. Libdwarf. http://reality.sgiweb.org/davea/dwarf.html.
42. MARINESCU, P. D., BANABIC, R., AND CANDEA, G. 2010. An extensible technique for high-precision testing of recovery code. In Proceedings of the USENIX Annual Technical Conference.
43. MYSQL. 2009. MySQL crashes due to bus error during shutdown. http://bugs.mysql.com/bug.php?id=42109.
44. MYSQL. 2010a. MySQL crashes due to double unlock. http://bugs.mysql.com/ bug.php?id=53268.
45. MYSQL. 2010b. MySQL crashes due to error while reading errmsg.sys. http://bugs.mysql.com/bug.php?id=53393.
46. MYSQL. 2010c. MySQL InnoDB crashes during shutdown. http://bugs.mysql. com/bug.php?id=52546.
47. MYSQL. 2010d. http://www.mysql.com/.
48. NG, W. T. AND CHEN, P. M. 1999. The systematic improvement of fault tolerance in the Rio file cache. In Proceedings of the International Symposium on Fault-Tolerant Computing.
49. PIDGIN. 2009. Pidgin SIGABRTs on memory alloc. http://developer.pidgin. im/ticket/8672.
50. PIDGIN. 2010. Pidgin. http://www.pidgin.im.
51. PRABHAKARAN, V., BAIRAVASUNDARAM, L. N., AGRAWAL, N., GUNAWI, H. S., ARPACI-DUSSEAU, A. C., AND ARPACI-DUSSEAU, R. H. 2005. IRON file systems. In Proceedings of the Symposium on Operating Systems Principles.
52. PRASAD, M. AND CHIUEH, T. 2003. A binary rewriting defense against stack-based buffer overflow attacks. In Proceedings of the USENIX Annual Technical Conference.
53. RUBIO-GONZÁLEZ, C. AND LIBLIT, B. 2010. Expect the unexpected: Error code mismatches between documentation and the real world. In Proceedings of the Workshop on Program Analysis for Software Tools and Engineering.
54. SINGER, J. 1998. Practices of software maintenance. In Proceedings of the International Conference on Software Maintenance.
55. SLOWINSKA, A., STANCESCU, T., AND BOS, H. 2011. Howard: A dynamic excavator for reverse engineering data structures. In Proceedings of the Network and Distributed System Security Symposium.
56. STOTT, D. T., FLOERING, B., KALBARCZYK, Z., AND IYER, R. K. 2000. A framework for assessing dependability in distributed systems with lightweight fault injectors. In Proceedings of the International Computer Performance and Dependability Symposium.
57. SÜßKRAUT, M. AND FETZER, C. 2006. Automatically finding and patching bad error handling. In Proceedings of the European Dependable Computing Conference.
58. SYSBENCH. 2010. http://sysbench.sourceforge.net.
59. TESTAPI. 2010. Library of test and utility APIs. http://testapi.codeplex. com/.
60. TSAI, T. K. AND IYER, R. K. 1995. Measuring fault tolerance with the FTAPE fault injection tool. In Proceedings of the International Conference on Modelling Techniques and Tools for Computer Performance Evaluation.
61. V8 JAVASCRIPT ENGINE. 2011. http://code.google.com/p/v8/.
62. WEIMER, W. AND NECULA, G. C. 2008. Exceptional situations and program reliability. ACM Trans. Program. Lang. Syst. 30, 2.
63. ZHANG, J., ZHAO, R., AND PANG, J. 2007. Parameter and return-value analysis of binary executables. In Proceedings of the Annual Intrenational Computer Software and Applications Conference.