Survey on NoSQL for management of big data

Ruan Jian Xue Bao/Journal of Software

Volumn 24, Issue 8, 2013, Pages 1786-1803

  Shen, De Rong ^a   Yu, Ge ^a   Wang, Xi Te ^a   Nie, Tie Zheng ^a   Kou, Yue ^a  

^a NORTHEASTERN UNIVERSITY   (China)

Author keywords

Big data management; Key value storage; NoSQL

Indexed keywords

APPLICATION-ORIENTED; DATA CONSISTENCY; DATA MANAGEMENT SYSTEM; KEY-VALUE STORAGES; NOSQL; REPLICA STRATEGY; RESEARCH CHALLENGES; SYSTEM ARCHITECTURES;

DATABASE SYSTEMS; ELASTICITY; INFORMATION MANAGEMENT; MODELS; RESEARCH; SURVEYS;

DIGITAL STORAGE;

EID: 84883014425     PISSN: 10009825     EISSN: None     Source Type: Journal    
DOI: 10.3724/SP.J.1001.2013.04416     Document Type: Article

References (97)

1. Big data. 2011. http://en.wikipedia.org/wiki/Big_data
2. Zhou X F, Lu J H, Li C P, Du X Y. The challenges of big data from the perspective of data management. Communications of the China Computer Federation, 2012, 8(9): 16-21 (in Chinese).
3. Li G J. The scientific value of big data research. Communications of the China Computer Federation, 2012, 8(9): 8-15 (in Chinese).
4. The Internet Analysis Salon. Big data is coming. 2011 (in Chinese). http://www.techxue.com/portal.php?mod=view&aid=55
5. Informatica. Big data unleased. 2011. http://www.informatica.com/downloads/1601_big_data_wp.pdf
6. Ma S, Li J X, Hu C M. The challenge and thinking of big data science and engineering. Communications of the China Computer Federation, 2012, 8(9): 22-28 (in Chinese).
7. Rys M. scalable SQL. Communications of the ACM, 2011, 54(6): 48-53. [doi: 10.1145/1953122.1953141]
8. NoSQL. 2011. http://zh.wikipedia.org/wiki/NoSQL
9. NoSQL. 2011. http://nosql-databases.org/
10. Campbell D G, Kakivaya G, Ellis N. Extreme scale with full SQL language support in Microsoft SQL azure. In: Proc. of the SIGMOD. New York: ACM Press, 2010. 1021-1024. [doi: 10.1145/1807167.1807280]
11. Brantner M, Florescu D, Graf D, Kossmann D, Kraska T. Building a database on S3. In: Proc. of the SIGMOD. New York: ACM Press, 2008. 251-264. [doi: 10.1145/1376616.1376645]
12. Curino C, Jones E P C, Popa R A, Malviya N, Wu E, Madden S, Balakrishnan H, Zeldovich N. Relational cloud: A database as a service for the cloud. In: Proc. of the CIDR. 2011. 235-240.
13. NOSQL. 2009. http://nosql.eventbrite.com/
14. Nosqleast conference. 2010. https://nosqleast.com/2009/
15. Chang F, Dean J, Ghemawat S, Hsieh W C, Wallach D A, Burrows M, Chandra T, Fikes A, Grubwr R E. Bigtable: A distributed storage system for structured data. In: Proc. of the OSDI. New York: ACM Press, 2006.
16. Borthaku D. The hadoop distributed file system: Architecture and design. 2009. http://hadoop.apache.org/common/docs/r0.18.0/ hdfs_design.pdf
17. Hbase Development Team. Hbase: Bigtable-like structured storage for hadoop HDFS. 2009. http://wiki.apache.org/hadoop/Hbase
18. DeCandia G, Hastorun D, Jampani M, Kakulapati G, Lakshman A, Pilchin A, Sivasubramanian S, Vosshall P, Vogels W. Dynamo: Amazon's highly available key-value store. In: Proc. of the SOSP. New York: ACM Press, 2001. 205-220. [doi: 10.1145/1294261. 1294281]
19. Lakshman A, Malik P. Cassandra-A decentralized structured storage system. 2009. http://www.cs.cornell.edu/projects/ladis2009/ papers/lakshman-ladis2009.pdf
20. The Document Collection of Tokyocabinet/Tokyotyrant. 2010 (in Chinese). http://www.162cm.com/p/tokyotyrant.html#toc5
21. Apache CouchDB: The apache CouchDB project. 2010. http://couchdb.apache.org/
22. MongoDB. 2010. http://www.mongodb.org
23. Redis. 2010. http://redis.io/
24. Gilbert S, Lynch N. Brewer's conjecture and the feasibility of consistent, available, partition-tolerant Web services. ACM SIGACT News, 2002, 33(2): 51-59. [doi: 10.1145/564585.564601]
25. Pritchett D. BASE: An acid alternative. 2008. http://queue.acm.org/detail.cfm?id=1394128
26. Agrawal R, Ailamaki A, Bernstein P A, Brewer E A, Carey M J, Chaudhuri S, Doan A, Florescu D, Franklin M J, Garcia-Molina H, Gehrke J, Gruenwald L, Hass L M, Halevy A, Hellerstein J M, Ioannidis Y E, Korth H K, Kossmann D, Madden S, Magoulas R, Ooi B C, O'Reilly T, Ramakrishnan R, Sarawagi S, Stonebraker M, Szalay A S, Weikum G. The claremont report on database research. Communications of the ACM, 2009, 52(6): 56-65. [doi: 10.1145/1516046.1516062]
27. Dean J, Ghemawat S. MapReduce: Simplified data processing on large clusters. In: Proc. of the OSDI. New York: ACM Press, 2004. 1-13.
28. Stonebraker M, Cattell R. 10 rules for scalable performance in 'simple operation' datastores. Communications of the ACM, 2011, 54(6): 72-80. [doi: 10.1145/1953122.1953144]
29. Baker J, Bond C, Corbett J C, Furman J J, Khorlin A, Larson J, Léon J M, Li Y W, Lloyd A, Yushprakh V. Megastore: Providing scalable, highly available storage for interactive services. In: Proc. of the CIDR. 2011. 223-234.
30. Das S, Agrawal D, Abbadi A E. G-Store: A scalable data store for transactional multi-key access in the cloud. In: Proc. of the SoCC. New York: ACM Press, 2010. [doi: 10.1145/1807128.1807157]
31. Vo H T, Chen C, Ooi B C. Towards elastic transactional cloud storage with range query support. Proc. of the VLDB Endowment, 2010, 3(1-2): 506-514.
32. Wang S, Sa S X. Introduction to Database System. 4th ed., Beijing: Higher Education Press, 2006 (in Chinese).
33. Yang F, Shanmugasundaram J, Yerneni R. A scalable data platform for a large number of small applications. In: Proc. of the CIDR. 2009.
34. Petersen K, Spreitzer M J, Terry D B, Theimer M M, Demers A J. Flexible update propagation for weakly consistent replication. In Proc. of the SOSP. New York: ACM Press, 1997. 288-301. [doi: 10.1145/268998.266711]
35. Vogels W. Eventually consistent. Communications of the ACM, 2009, 52(1): 40-44. [doi: 10.1145/1435417.1435432]
36. Lamport L. The part-time parliament. ACM Trans. on Computer Systems, 1998, 16(2): 133-169. [doi: 10.1145/279227.279229]
37. Levandoski J J, Lomet D, Mokbel M F, Zhao K K. Deuteronomy: Transaction support for cloud data. In: Proc. of the CIDR. 2011. 123-133.
38. Curino C, Jones E, Zhang Y, Wu E, Madden S. Relational cloud: The case for a database service. Technical Report, MIT-CSAIL-TR-2010-014, Massachusetts Institute of Technology, 2010.
39. Das S, Agarwal D, Abbadi A E. ElasTraS: An elastic, scalable, and self managing transactional database for the cloud. Technical Report, 2010-4, Santa Barbara: University of California, 2010.
40. Das S, Agrawal D, Abbadi A E. ElasTraS: An elastic transactional data store in the cloud. In: Proc. of the USENIX HotCloud Workshop. 2009.
41. Wei Z, Pierre G, Chi C H. CloudTPS: Scalable transactions for Web applications in the cloud. Technical Report, IR-CS-053, Amsterdam: Vrije Universiteit, 2010.
42. Tatemura J, Po O, Hsiung W P, Hacigümüş H. Partiqle: An elastic SQL engine over key-value stores. In: Proc. of the SIGMOD. New York: ACM Press, 2012. 629-632.
43. Konstantinou I, Angelou E, Tsoumakos D. TIRAMOLA: Elastic NoSQL provisioning through a cloud management platform. In: Proc. of the SIGMOD. New York: ACM Press, 2012. 725-728. [doi: 10.1145/2213836.2213943]
44. Okcan A, Riedewald M. Processing Theta-joins using MapReduce. In: Proc. of the SIGMOD. New York: ACM Press, 2011. 949-960. [doi: 10.1145/1989323.1989423]
45. Nehme R, Bruno N. Automated partitioning design in parallel database systems. In: Proc. of the SIGMOD. New York: ACM Press, 2011. 1137-1148. [doi: 10.1145/1989323.1989444]
46. Das S, Nishimura S, Agrawal D, Abbadi A E. Live database migration for elasticity in a multitenant database for cloud platforms. Technical Report, 2010-09, Department of Computer Science, University of California at Santa Barbara, 2010.
47. Elmore A J, Das S, Agrawal D, Abbadi A E. Zephyr: Live migration in shared nothing databases for elastic cloud platforms. In: Proc. of the SIGMOD. New York: ACM Press, 2011. 301-312. [doi: 10.1145/1989323.1989356]
48. Kraska T, Hentschel M, Alonso G, Kossmann D. Consistency rationing in the cloud: Pay only when it matters. Proc. of the VLDB Endowment, 2009, 2(1): 253-264.
49. Gao L, Dahlin M, Nayate A, Zheng J D, Lyengar A. Application specific data replication for edge services. In: Proc. of the WWW. New York: ACM, 2003. 449-460. [doi: 10.1145/775152.775217]
50. Lu Y, Lu Y, Jiang H. Adaptive consistency guarantees for large-scale replicated services. In: Proc. of the NAS. IEEE, 2008. 89-96. [doi: 10.1109/NAS.2008.64]
51. Yu H, Vahdat A. Design and evaluation of a continuous consistency model for replicated services. In: Proc. of the OSDI. New York: ACM Press, 2000. 305-318.
52. Debnath B, Sengupta S, Li J. SkimpyStash: RAM space skimpy key-value store on flash-based storage. In: Proc. of the SIGMOD. New York: ACM, 2011. 25-36. [doi: 10.1145/1989323.1989327]
53. Anand A, Muthukrishnan C, Kappes S, Akella A, Nath S. Cheap and large CAMs for high performance data-intensive networked systems. In: Proc. of the NSDI. 2010. 29.
54. Andersenl D G, Franklin J, Kaminsky M, Phanishayee A, Tan L, Vasudevan V. FAWN: A fast array of wimpy nodes. In: Proc. of the SOSP. New York: ACM Press, 2009. [doi: 10.1145/1629575.1629577]
55. Debnath B, Sengupta S, Li J. ChunkStash: Speeding up inline storage deduplication using flash memory. In: Proc. of the USENIX. 2010.
56. Debnath B, Sengupta S, Li J. FlashStore: High throughput persistent key-value store. In: Proc. of the VLDB. Morgan Kaufmann, ACM, 2010. 1414-1425.
57. Nykiel T, Potamias M, Mishra C, Kollios G, Koudas N. MRShare: Sharing across multiple queries in MapReduce. In: Proc. of the VLDB. Morgan Kaufmann, ACM, 2010. 494-505.
58. Dittrich J, Quiané-Ruiz J A, Jindal A, Kargin Y, Setty V, Schad J. Hadoop++: Making a yellow elephant run like a cheetah (without it even noticing). In: Proc. of the VLDB. Morgan Kaufmann, ACM, 2010. 518-527.
59. Lin Y, Agrawal D, Chen C. Llama: Leveraging columnar storage for scalable join processing in the MapReduce framework. In: Proc. of the SIGMOD. New York: ACM Press, 2011. 961-972. [doi: 10.1145/1989323.1989424]
60. Borthakur D, Sarma J S, Gray J. Apache hadoop goes realtime at facebook. In: Proc. of the SIGMOD. New York: ACM Press, 2011. 1071-1080. [doi: 10.1145/1989323.1989438]
61. Silberstein A, Sears R, Zhou W C, Cooper B F. A batch of PNUTS: Experiences connecting cloud batch and serving systems. In: Proc. of the SIGMOD. New York: ACM Press, 2011. 1101-1112. [doi: 10.1145/1989323.1989441]
62. Kossmann D, Kraska T, Loesing S, Merkli S, Mittal R, Pfaffhauser F. Cloudy: A modular cloud storage system. In: Proc. of the VLDB. Morgan Kaufmann, ACM, 2010. 1533-1536.
63. O'reilly. Cassandra: The definitive Guide. 2010. http://ishare.iask.sina.com.cn/f/17348687
64. Wu S, Jiang D W, Ooi B C, Wu K L. Efficient B-tree based indexing for cloud data processing. In: Proc. of the VLDB. Morgan Kaufmann, ACM, 2010.
65. Wang J B, Wu S, Gao H, Li J Z, Ooi B C. Indexing multi-dimensional data in a cloud system. In: Proc. of the SIGMOD. New York: ACM Press, 2010. 591-602. [doi: 10.1145/1807167.1807232]
66. Tsatsanifos G, Sacharidis D, Sellis T. MIDAS: Multi-Attribute indexing for distributed architecture systems. In: Proc. of the SSTD. Heidelberg: Springer-Verlag, 2011. 168-185. [doi: 10.1007/978-3-642-22922-0_11]
67. Escriva R, Wong B, Gün Sirer E G. HyperDex: A distributed, searchable key-value store. In: Proc. of the SIGCOMM. 2012. 1-12. [doi: 10.1145/2377677.2377681]
68. Meng B P, Wang T J, Li H Y, Yang D Q. Regional bitmap index: A secondary index for data management in cloud computing environment. Chinese Journal of Computers, 2012, 35(11): 2306-2316 (in Chinese with English abstract).
69. Aguilera M K, Golab W, Shah M A. A practical scalable distributed B-tree. In: Proc. of the VLDB. Morgan Kaufmann, ACM, 2008.
70. Zhang X Y, Ai J, Wang Z Y, Lu J H, Meng X F. An efficient multi-dimensional index for cloud data management. In: Proc. of the CloudDB. New York: ACM Press, 2009. 17-24. [doi: 10.1145/1651263.1651267]
71. Cooper B F, Ramakrishnan R, Srivastava U, Silberstein A, Bohannon P, Jacobsen H A, Puz N, Weaver D, Yerneni R. PNUTS: Yahoo!'s hosted data serving platform. Proc. of the VLDB Endowment, 2008, 1(2): 1277-1288.
72. Amer-Yahia S, Halevy A, Alonso G, Kossmann D, Markl V, Doan A H, Weikum G. Databases and Web 2.0 panel at VLDB 2007. SIGMOD Record, 2008, 37(1): 49-52. [doi: 10.1145/1374780.1374794]
73. Bernstein P A, Cseri I, Dani N, Ellis N, Kalhan A, Kakivaya G, Lomet D B, Manne R, Novik L, Talius T. Adapting Microsoft SQL server for cloud computing. In: Proc. of the ICDE. IEEE, 2011. 1255-1263. [doi: 10.1109/ICDE.2011.5767935]
74. Gray J, Lamport L. Consensus on transaction commit. ACM Trans. on Database System, 2006, 31(1): 133-160. [doi: 10.1145/11328 63.1132867]
75. Aguilera M K, Merchant A, Shah M, Veitch A, Karamanolis C. Sinfonia: A new paradigm for building scalable distributed systems. In: Proc. of the SOSP. New York: ACM Press, 2007. [doi: 10.1145/1294261.1294278]
76. Thomson A, Diamond T, Weng S C. Calvin: Fast distributed transactions for partitioned database systems. In: Proc. of the SIGMOD. New York: ACM Press, 2012. [doi: 10.1145/2213836.2213838]
77. Das S, Nishimura S, Agrawal D, Abbadi A E. Albatross: Lightweight elasticity in shared storage databases for the cloud using live data migration. In: Proc. of the VLDB. Morgan Kaufmann, ACM, 2011. 494-505.
78. Clark C, Fraser K, Hand S, Hansen J G, Jul E, Limpach C, Pratt I, Warfield A. Live migration of virtual machines. In: Proc. of the NSDI. 2005. 273-286.
79. Liu H K, Jin H, Liao X F, Hu L T, Yu C. Live migration of virtual machine based on full system trace and replay. In: Proc. of the HPDC. 2009. 101-110. [doi: 10.1145/1551609.1551630]
80. De Witt D, Gray J. Parallel database systems: The future of high performance database systems. Communications of the ACM, 1992, 35(6): 85-98. [doi: 10.1145/129888.129894]
81. Ganguly S, Goel A, Silberschatz A. Efficient and accurate cost models for parallel query optimization. In: Proc. of the PODS. New York: ACM Press, 1996. 172-181. [doi: 10.1145/237661.237707]
82. Isard M, Budiu M, Yu Y. Dryad: Distributed data-parallel programs from sequential building blocks. In: Proc. of the EuroSys. New York: ACM Press, 2007. 59-72. [doi: 10.1145/1272996.1273005]
83. Yang H C, Dasdn A, Hsiao R L, Parker D S. Map-Reduce-Merge: Simplified relational data processing on large clusters. In: Proc. of the SIGMOD. New York: ACM Press, 2007. [doi: 10.1145/1247480.1247602]
84. Chaiken R, Jenkins B, Larson P Å, Ramsey B, Shakib D, Weaver S, Zhou J R. Scope: Easy and efficient parallel processing of massive data sets. In: Proc. of the VLDB. New York: ACM Press, 2008. 1265-1276.
85. Curino C, Jones E, Zhang Y, Madden S. Schism: A workload-driven approach to database replication and partitioning. Proc. of the VLDB Endowment, 2010, 3(1-2): 48-57.
86. Karypis G, Kumar V. A fast and high quality multilevel scheme for partitioning irregular graphs. SIAM Journal on Scientific Computing, 1998, 20(1): 359-392. [doi: 10.1137/S1064827595287997]
87. Ghandeharizadeh S, De Witt D J. Hybrid-Range partitioning strategy: A new declustering strategy for multiprocessor database machines. In: Proc. of the VLDB. 1990. 481-492.
88. Gufler B, Augsten N, Reiser A, Kemper A. Load balancing in MapReduce based on scalable cardinality estimates. In: Proc. of the ICDE. IEEE, 2012. [doi: 10.1109/ICDE.2012.58]
89. Kwon YC, Balazinska M, Howe B, Rolia J. SkewTune: Mitigating skew in MapReduce applications. In: Proc. of the SIGMOD. New York: ACM Press, 2012. 25-36. [doi: 10.1145/2213836.2213840]
90. Chen C, Chen G, Jiang D W, Ooi B C, Vo H T, Wu S, Xu Q Q. Providing scalable database services on the cloud. In: Proc. of the WISE. Heidelberg: Springer-Verlag, 2010. 1-19.
91. Vogels W. Eventually consistent. 2008. http://www.allthingsdistributed.com/ 2008/12/eventually_consistent.html
92. Rao J, Shekita E J, Tata S. Using Paxos to build a scalable, consistent, and highly available datastore. In: Proc. of the VLDB. 2011. 243-354.
93. Burrows M. The chubby lock service for loosely-coupled distributed system. In: Proc. of the OSDI. 2006. 335-350.
94. Reed B, Junqueira F P. A simple totally ordered broadcast protocol. In: Proc. of the Workshop on LADIS. 2008. 1-6. [doi: 10.1145/ 1529974.1529978]
95. Vigfusson Y, Chockler G. Clouds at the crossroads research perspectives. Spring, 2010, 16(3): 10-13. [doi: 10.1145/1734160.17341 65]
96. Lin Z Y, Lai Y X, Lin C, Xie Y, Zou Q. Research on Cloud Databases. Ruan Jian Xue Bao/Journal of Software, 2012, 23(5): 1148-1166 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/4195.htm [doi: 10.3724/SP.J.1001.2012.04195]
97. Wang Y J, Sun W D, Zhou S, Pei X Q, Li X Y. Key Technologies of distributed storage for cloud computing. Ruan Jian Xue Bao/Journal of Software, 2012, 23(4): 962-986 (in Chinese with English abstract). http://www.jos.org.cn/1000-9825/4175.htm [doi: 10. 3724/SP.J.1001.2012.04175]