All file systems are not created equal: On the complexity of crafting crash-consistent applications

Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2014

Volumn , Issue , 2014, Pages 433-448

  Pillai, Thanumalayan Sankaranarayana ^a   Chidambaram, Vijay ^a   Alagappan, Ramnatthan ^a   Al Kiswany, Samer ^a   Arpaci Dusseau, Andrea C ^a   Arpaci Dusseau, Remzi H ^a  

^a UNIVERSITY OF WISCONSIN MADISON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER OPERATING SYSTEMS; FILE ORGANIZATION; SYSTEMS ANALYSIS;

APPLICATION LEVEL; CONSISTENCY PROTOCOL; DISTRIBUTED SYSTEMS; KEY-VALUE STORES; LINUX FILE SYSTEM; UPDATE PROTOCOL; VERSION CONTROL SYSTEM; VIRTUALIZATION SOFTWARE;

DISTRIBUTED DATABASE SYSTEMS;

EID: 84958019168     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (54)

1. Necessary step(s) to synchronize filename operations on disk. http://austingroupbugs.net/view.php?id=672.
2. Tool and Results: Application Crash Vulnerabilities. http://research.cs.wisc.edu/adsl/Software/alice/.
3. Apache. Apache Zookeeper. http://zookeeper.apache.org/.
4. Remzi H. Arpaci-Dusseau and Andrea C. Arpaci-Dusseau. Operating Systems: Three Easy Pieces. Arpaci-Dusseau Books, 0.8 edition, 2014.
5. Steve Best. JFS Overview. http://jfs.sourceforge.net/project/pub/jfs.pdf, 2000.
6. Remy Card, Theodore Ts'o, and Stephen Tweedie. Design and Implementation of the Second Extended Filesystem. In First Dutch International Symposium on Linux, Amsterdam, Netherlands, December 1994.
7. Donald D Chamberlin, Morton M Astrahan, Michael W Blasgen, James N Gray, W Frank King, Bruce G Lindsay, Raymond Lorie, James W Mehl, Thomas G Price, Franco Putzolu, et al. A history and evaluation of system r. Communications of the ACM, 24(10):632-646, 1981.
8. Donald D Chamberlin, Arthur M Gilbert, and Robert A Yost. A history of system r and sql/data system. In VLDB, pages 456-464, 1981.
9. Vijay Chidambaram, Thanumalayan Sankaranarayana Pillai, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. Optimistic Crash Consistency. In Proceedings of the 24th ACM Symposium on Operating Systems Principles (SOSP'13), Nemacolin Woodlands Resort, Farmington, Pennsylvania, October 2013.
10. Vijay Chidambaram, Tushar Sharma, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. Consistency Without Ordering. In Proceedings of the 10th USENIX Symposium on File and Storage Technologies (FAST'12), pages 101-116, San Jose, California, February 2012.
11. Chris Mason. Btrfs Mailing List. Re: Ordering of directory operations maintained across system crashes in Btrfs? http://www.spinics.net/lists/linux-btrfs/msg32215.html, 2014.
12. Jeremy Condit, Edmund B. Nightingale, Christopher Frost, Engin Ipek, Benjamin Lee, Doug Burger, and Derrick Coetzee. Better I/O Through Byte-addressable, Persistent Memory. In Proceedings of the 22nd ACM Symposium on Operating Systems Principles (SOSP'09), Big Sky, Montana, October 2009.
13. Jonathan Corbet. Solving the Ext3 Latency Problem. http://lwn.net/Articles/328363/, 2009.
14. Jonathan Corbet. That massive filesystem thread. http://lwn.net/Articles/326471/, March 2009.
15. Heming Cui, Gang Hu, Jingyue Wu, and Junfeng Yang. Verifying systems rules using rule-directed symbolic execution. In Proceedings of the eighteenth international conference on Architectural support for programming languages and operating systems, pages 329-342. ACM, 2013.
16. Christopher Frost, Mike Mammarella, Eddie Kohler, Andrew de los Reyes, Shant Hovsepian, Andrew Matsuoka, and Lei Zhang. Generalized File System Dependencies. In Proceedings of the 21st ACM Symposium on Operating Systems Principles (SOSP'07), pages 307-320, Stevenson, Washington, October 2007.
17. Bill Gallagher, Dean Jacobs, and Anno Langen. A High-performance, Transactional Filestore for Application Servers. In Proceedings of the 2005 ACM SIGMOD International Conference on Management of Data (SIGMOD'05), pages 868-872, Baltimore, Maryland, June 2005.
18. Gregory R. Ganger and Yale N. Patt. Metadata Update Performance in File Systems. In Proceedings of the 1st Symposium on Operating Systems Design and Implementation (OSDI'94), pages 49-60, Monterey, California, November 1994.
19. GNU. GNU Database Manager (GDBM). http://www.gnu.org.ua/software/gdbm/gdbm.html, 1979.
20. Google. LevelDB. https://code.google.com/p/leveldb/, 2011.
21. Robert Hagmann. Reimplementing the Cedar File System Using Logging and Group Commit. In Proceedings of the 11th ACM Symposium on Operating Systems Principles (SOSP'87), Austin, Texas, November 1987.
22. Dave Hitz, James Lau, and Michael Malcolm. File System Design for an NFS File Server Appliance. In Proceedings of the USENIX Winter Technical Conference (USENIX Winter'94), San Francisco, California, January 1994.
23. HyperSQL. HSQLDB. http://www.hsqldb.org/.
24. Dean Kuo. Model and verification of a data manager based on aries. ACM Trans. Database Syst., 21(4):427-479, December 1996.
25. Oren Laadan, Nicolas Viennot, Chia-Che Tsai, Chris Blinn, Junfeng Yang,, and Jason Nieh. Pervasive Detection of Process Races in Deployed Systems. In Proceedings of the 23rd ACM Symposium on Operating Systems Principles (SOSP'11), Cascais, Portugal, October 2011.
26. Linus Torvalds. Git. http://git-scm.com/, 2005.
27. Linus Torvalds. Git Mailing List. Re: what's the current wisdom on git over NFS/CIFS? http://marc.info/?l=git&m=124839484917965&w=2, 2009.
28. Lanyue Lu, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau, and Shan Lu. A Study of Linux File System Evolution. In Proceedings of the 11th USENIX Symposium on File and Storage Technologies (FAST'13), San Jose, California, February 2013.
29. Cris Pedregal Martin and Krithi Ramamritham. Toward formalizing recovery of (advanced) transactions. In Advanced Transaction Models and Architectures, pages 213-234. Springer, 1997.
30. Chris Mason. The Btrfs Filesystem. oss.oracle. com/projects/btrfs/dist/documentation/ btrfs-ukuug.pdf, September 2007.
31. Matt Mackall. Mercurial. http://mercurial.selenic.com/, 2005.
32. Marshall K. McKusick, William N. Joy, Sam J. Leffler, and Robert S. Fabry. A Fast File System for UNIX. ACM Transactions on Computer Systems, 2(3):181-197, August 1984.
33. C. Mohan, D. Haderle, B. Lindsay, H. Pirahesh, and P. Schwarz. ARIES: A Transaction Recovery Method Supporting Fine-Granularity Locking and Partial Rollbacks Using Write-Ahead Logging. ACM Transactions on Database Systems, 17(1):94-162, March 1992.
34. C Mohan, Bruce Lindsay, and Ron Obermarck. Transaction management in the r* distributed database management system. ACM Transactions on Database Systems (TODS), 11(4):378-396, 1986.
35. Vijayan Prabhakaran, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. Analysis and Evolution of Journaling File Systems. In Proceedings of the USENIX Annual Technical Conference (USENIX'05), pages 105-120, Anaheim, California, April 2005.
36. Abhishek Rajimwale, Vijay Chidambaram, Deepak Ramamurthi, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. Coerced Cache Eviction and Discreet-Mode Journaling: Dealing with Misbehaving Disks. In Proceedings of the International Conference on Dependable Systems and Networks (DSN'11), Hong Kong, China, June 2011.
37. Hans Reiser. ReiserFS. www.namesys.com, 2004.
38. Mendel Rosenblum and John Ousterhout. The Design and Implementation of a Log-Structured File System. ACM Transactions on Computer Systems, 10(1):26-52, February 1992.
39. Frank Schmuck and Jim Wylie. Experience with transactions in quicksilver. In ACM SIGOPS Operating Systems Review, volume 25, pages 239-253. ACM, 1991.
40. Konstantin Shvachko, Hairong Kuang, Sanjay Radia, and Robert Chansler. The Hadoop Distributed File System. In Proceedings of the 26th IEEE Symposium on Mass Storage Systems and Technologies (MSST'10), Incline Village, Nevada, May 2010.
41. Stewart Smith. Eat My Data: How everybody gets file I/O wrong. In OSCON, Portland, Oregon, July 2008.
42. R. P. Spillane, S. Gaikwad, E. Zadok, C. P. Wright, and M. Chinni. Enabling transactional file access via lightweight kernel extensions. In Proceedings of the Seventh USENIX Conference on File and Storage Technologies (FAST'09), pages 29-42, San Francisco, CA, February 2009. USENIX Association.
43. SQLite. SQLite transactional SQL database engine. http://www.sqlite.org/.
44. Sun Microsystems. ZFS: The last word in file systems. www.sun.com/2004-0914/feature/, 2006.
45. Adan Sweeney, Doug Doucette, Wei Hu, Curtis Anderson, Mike Nishimoto, and Geoff Peck. Scalability in the XFS File System. In Proceedings of the USENIX Annual Technical Conference (USENIX'96), San Diego, California, January 1996.
46. Symas. Lightning Memory-Mapped Database (LMDB). http://symas.com/mdb/, 2011.
47. Thanumalayan Sankaranarayana Pillai, Vijay Chidambaram, Jooyoung Hwang, Andrea C. Arpaci-Dusseau, Remzi H. Arpaci-Dusseau. Towards Efficient, Portable Application-Level Consistency. In Proceedings of the 9th Workshop on Hot Topics in Dependable Systems (HotDep'13), Farmington, PA, November 2013.
48. The Open Group. POSIX.1-2008 IEEE Std 1003.1. http://pubs.opengroup.org/onlinepubs/9699919799/, 2013.
49. The PostgreSQL Global Development Group. PostgreSQL. http://www.postgresql.org/.
50. Theodore Ts'o and Stephen Tweedie. Future Directions for the Ext2/3 Filesystem. In Proceedings of the USENIX Annual Technical Conference (FREENIX Track), Monterey, California, June 2002.
51. Stephen C. Tweedie. Journaling the Linux ext2fs File System. In The Fourth Annual Linux Expo, Durham, North Carolina, May 1998.
52. VMWare. VMWare Player. http://www.vmware.com/products/player.
53. Junfeng Yang, Can Sar, and Dawson Engler. EXPLODE: A Lightweight, General System for Finding Serious Storage System Errors. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI'06), Seattle, Washington, November 2006.
54. Mai Zheng, Joseph Tucek, Dachuan Huang, Feng Qin, Mark Lillibridge, Elizabeth S Yang, Bill W Zhao, and Shashank Singh. Torturing Databases for Fun and Profit. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI'14), Broomfield, CO, October 2014.