High availability, elasticity, and strong consistency for massively parallel scans over relational data

VLDB Journal

Volumn 23, Issue 4, 2014, Pages 627-652

  Unterbrunner, Philipp ^a   Alonso, Gustavo ^b   Kossmann, Donald ^b  

^a SnowFlake Computing   (United States)

^b ETH ZURICH   (Switzerland)

Author keywords

Atomic multicast; Dynamic partial replication; High availability; Scan only query processing

Indexed keywords

HARDWARE; INFORMATION SYSTEMS;

HIGH AVAILABILITY; INDUSTRY USE-CASES; MANAGEMENT SYSTEMS; MASSIVELY PARALLELS; MULTICAST PROTOCOL; PARTIAL REPLICATION; ROBUST PERFORMANCE; STRONG CONSISTENCY;

MULTICASTING;

EID: 84904536046     PISSN: 10668888     EISSN: 0949877X     Source Type: Journal    
DOI: 10.1007/s00778-013-0343-9     Document Type: Article

References (79)

1. Aguilera, M. K., Chen, W., Toueg, S.: Heartbeat: a timeout-free failure detector for quiescent reliable communication. In: Proceedings of WDAG '97, pp. 126-140 (1997).
2. Aguilera, M. K., et al.: Reconfiguring replicated atomic storage: a tutorial. Bull. EATCS 102, 84-108 (2010).
3. Amir, Y., Danilov, C., Stanton, J.: A low latency, loss tolerant architecture and protocol for wide area group communication. In: Proceedings of ICDSN '00, pp. 327-336 (2000).
4. Amir, Y., et al.: The totem single-ring ordering and membership protocol. ACM Trans. Comput. Syst. 13, 311-342 (1995).
5. Armstrong, J.: Erlang. CACM 53, 68-75 (2010).
6. Arumugam, S., et al.: The datapath system: a data-centric analytic processing engine for large data warehouses. In: Proceedings of SIGMOD '10, pp. 519-530 (2010).
7. Ashmawy, K.: Partial live migration in scan-based database systems. Master's thesis, ETH Zurich (2011).
8. Baker, J., et al.: Megastore: providing scalable, highly available storage for interactive services. In: Proceedings of CIDR '11, pp. 223-234 (2011).
9. Bernstein, P. A., Hadzilacos, V., Goodman, N.: Concurrency Control and Recovery in Database Systems. Addison-Wesley, Reading, MA (1987).
10. Birman, K.: A history of the virtual synchrony replication model. In: Charron-Bost, B., Pedone, F., Schiper, A. (eds.) Replication: Theory and Practice, pp. 91-120. Springer, Berlin (2010).
11. Birman, K. P., et al.: Overcoming cap with consistent soft-state replication. IEEE Comput. Mag. 12, 50-58 (2012).
12. Birman, K. P., Renesse, R. V.: Reliable distributed computing with the ISIS toolkit. IEEE Computer Society Press, Silver Spring, MD (1994).
13. Brewer, E. A.: Towards robust distributed systems. In: Proceedings of PODC '00, p. 7 (2000).
14. Candea, G., Polyzotis, N., Vingralek, R.: A scalable, predictable join operator for highly concurrent data warehouses. PVLDB 2(1), 277-288 (2009).
15. Cattell, R.: Scalable sql and nosql data stores. SIGMOD Rec. 39(4), 12-27 (2010).
16. Chandra, T. D., Griesemer, R., Redstone, J.: Paxos made live: an engineering perspective. In: Proceedings of PODC '07, pp. 398-407 (2007).
17. Chandra, T. D., Toueg, S.: Unreliable failure detectors for reliable distributed systems. JACM 43(2), 225-267 (1996).
18. Chandrasekaran, S. et al.: Telegraphcq: an architectural status report. IEEE Data Eng. Bull. 26(1), 11-18 (2003).
19. Chandrasekaran, S., Franklin, M. J.: Streaming queries over streaming data. In Proceedings of VLDB '02, pp. 203-214 (2002).
20. Chang, F., et al.: Bigtable: a distributed storage system for structured data. ACM Trans. Comput. Syst. 26, 4: 1-4: 26 (2008).
21. Chockler, G. V., et al.: Reconfigurable distributed storage for dynamic networks. J. Parallel Distrib. Comput. 69(1), 100-116 (2009).
22. Chockler, G. V., Keidar, I., Vitenberg, R.: Group communication specifications: a comprehensive study. ACM Comput. Surv. 33(4), 1-43 (2001).
23. Cooper, B. F., et al.: Pnuts: Yahoo!'s hosted data serving platform. PVLDB 1(1), 1277-1288 (2008).
24. Corbett, J. C., et al.: Spanner: Google's globally-distributed database. In: Proceedings of OSDI '12, pp. 251-264 (2012).
25. Das, S., Agrawal, D., Abbadi, A. E.: Elastras: an elastic transactional data store in the cloud. In: Proceedings of HotCloud'09 (2009).
26. Das, S., Agrawal, D., Abbadi, A. E.: G-store: a scalable data store for transactional multi key access in the cloud. In: Proc. SoCC '10, pp. 163-174 (2010).
27. DeCandia, G., et al.: Dynamo: Amazon's highly available key-value store. In: Proc. SOSP '07, pp. 205-220 (2007).
28. Défago, X., Schiper, A., Urbán, P.: Total order broadcast and multicast algorithms: taxonomy and survey. ACM Comput. Surv. 36(4), 372-421 (2004).
29. Demers, A., et al.: Epidemic algorithms for replicated database maintenance. In: Proc. PODC '87, pp. 1-12 (1987).
30. Dept, B. B., Ban, B.: Adding group communication to Java in a non-intrusive way using the ensemble toolkit. Technical report, Cornell University (1997).
31. Dolev, D., Malki, D.: The transis approach to high availability cluster communication. CACM 39, 64-70 (1996).
32. Fabret, F., et al.: Filtering algorithms and implementation for very fast publish/subscribe systems. In: Proceedings of SIGMOD '01, pp. 115-126 (2001).
33. Fekete, A., Ramamritham, K.: Consistency models for replicated data. In: Charron-Bost, B., Pedone, F., Schiper, A. (eds.) Replication: Theory and Practice. Springer, Berlin, pp. 1-17 (2010).
34. Fernandez, P. M.: Red brick warehouse: a read-mostly rdbms for open smp platforms. In: Proceedings of SIGMOD '94, p. 492 (1994).
35. Fitzpatrick, B.: Distributed caching with memcached. Linux J. 124, 5 (2004).
36. Giannikis, G., Alonso, G., Kossmann, D.: Shareddb: killing one thousand queries with one stone. PVLDB 5(6), 526-537 (2012).
37. Giannikis, G., Makreshanski, D., Alonso, G., Kossmann, D.: Workload optimization using shareddb. In: Proceedings of SIGMOD '13, pp. 1045-1048 (2013).
38. Gilbert, S., Lynch, N.: Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services. SIGACT News 33(2), 51-59 (2002).
39. Glendenning, et al.: Scalable consistency in scatter. In: Proceedings of SOSP '11, pp. 15-28 (2011).
40. Graefe, G.: Volcano-an extensible and parallel query evaluation system. IEEE Trans. Knowl. Data Eng. 6(1), 120-135 (1994).
41. Gray J., et al.: The dangers of replication and a solution. In: Proceedings of SIGMOD '96, pp. 173-182 (1996).
42. Gray, J., Lamport, L.: Consensus on transaction commit. ACM Trans. Database Syst. 31(1), 133-160 (2006).
43. Guerraoui, R., Schiper, A.: The generic consensus service. IEEE Trans. Softw. Eng. 27(1), 29-41 (2001).
44. Habeeb, M.: A Developer's Guide to Amazon SimpleDB. Addison-Wesley Professional, Reading, MA (2010).
45. Hadzilacos, V., Toueg, S.: A modular approach to fault-tolerant broadcasts and related problems. Technical report, Cornell University (1994).
46. Harizopoulos, S., Shkapenyuk, V., Ailamaki, A.: Qpipe: a simultaneously pipelined relational query engine. In: Proceedings of SIGMOD '05, pp. 383-394 (2005).
47. Herlihy, M. P., Wing, J. M.: Linearizability: a correctness condition for concurrent objects. ACM Trans. Program. Lang. Syst. 12(3), 463-492 (1990).
48. Hildenbrand, S.: Scaling out column stores: data, queries, and transactions. Ph. D. thesis, Diss. 20314, ETH Zurich (2012).
49. Hunt, P., Konar, M., Junqueira, F. P., Reed, B.: Zookeeper: wait-free coordination for internet-scale systems. In: Proceedings of USENIXATC '10 (2010).
50. Johansson, E., et al.: The development of the hipe system: design and experience report. Int. J. Softw. Tools Tech. Transf. 4(4), 421-436 (2003).
51. Junqueira, F. P., Reed, B. C., Serafini, M.: Zab: high-performance broadcast for primary-backup systems. In: Proceedings of DISC '09, pp. 245-256 (2009).
52. Ktari, S., Zoubert, M., Hecker, A., Labiod, H.: Performance evaluation of replication strategies in dhts under churn. In: Proceedings of MUM '07, pp. 90-97 (2007).
53. Lakshman, A., Malik, P.: Cassandra: a decentralized structured storage system. SIGOPS Oper. Syst. Rev. 44, 35-40 (2010).
54. Lamport, L.: The part-time parliament. ACM Trans. Comput. Syst. 16(2), 133-169 (1998).
55. Lamport, L., Malkhi, D., Zhou, L.: Stoppable paxos. Technical report, Microsoft Research (2008).
56. Lamport, L., Malkhi, D., Zhou, L.: Reconfiguring a state machine. SIGACT News 41(1), 63-73 (2010).
57. Lang, C. A., et al.: Increasing buffer-locality for multiple relational table scans through grouping and throttling. In: Proceedings of ICDE '07, pp. 1136-1145 (2007).
58. Lynch, N., Shvartsman, A. A.: Rambo: a reconfigurable atomic memory service for dynamic networks. In: Proceedings of DISC '02, pp. 173-190 (2002).
59. Ostrowski, K., Birman, K., Dolev, D.: Quicksilver scalable multicast (qsm). In: Proceedings of NCA '08, pp. 9-18 (2008).
60. Peng, D., Dabek, F.: Large-scale incremental processing using distributed transactions and notifications. In: Proceedings of OSDI '10, pp. 1-15 (2010).
61. Plattner, H.: A common database approach for oltp and olap using an in-memory column database. In: Proceedings of SIGMOD '09, pp. 1-2 (2009).
62. Prisco, R. D., Lampson, B.: Revisiting the paxos algorithm. In: Proceedings of WDAG '97, pp. 111-125 (1997).
63. Qiao, L.: Main-memory scan sharing for multi-core cpus. PVLDB 1(1), 610-621 (2008).
64. Raman, V., et al.: Constant-time query processing. In: Proceedings of ICDE '08, pp. 60-69 (2008).
65. Rao, J., Shekita, E. J., Tata, S.: Using paxos to build a scalable, consistent, and highly available datastore. PVLDB 4, 243-254 (2011).
66. Schiper, N., Sutra, P., Pedone, F.: P-store: genuine partial replication in wide area networks. In: Proceedings of SRDS '10, pp. 214-224 (2010).
67. Schneider, F. B.: Implementing fault-tolerant services using the state machine approach: a tutorial. ACM Comput. Surv. 22(4), 299-319 (1990).
68. Schütt, T., Schintke, F., Reinefeld, A.: Scalaris: reliable transactional p2p key/value store. In: Proceedings of ERLANG '08, pp. 41-48 (2008).
69. Sellis, T. K.: Multiple-query optimization. ACM Trans. Database Syst. 13(1), 23-52 (1988).
70. Stoica, I., et al.: Chord: a scalable peer-to-peer lookup service for internet applications. In: Proceedings of SIGCOMM '01, pp. 149-160 (2001).
71. Thomson, A., Abadi, D. J.: The case for determinism in database systems. PVLDB 3(1-2), 70-80 (2010).
72. Thomson, A., et al.: Calvin: fast distributed transactions for partitioned database systems. In: Proceedings of SIGMOD '12, pp. 1-12 (2012).
73. Unterbrunner, P.: Elastic, reliable, and robust storage and query processing with Crescando/RB. Ph. D. thesis, Diss. 20272, ETH Zurich (2012).
74. Unterbrunner, P., Giannikis, G., Alonso, G., Fauser, D., Kossmann, D.: Predictable performance for unpredictable workloads. PVLDB 2(1), 706-717 (2009).
75. van Renesse, R., et al.: Horus: a flexible group communication system. CACM 39, 76-83 (1996).
76. van Renesse, R., Schneider, F. B.: Chain replication for supporting high throughput and availability. In: Proceedings of OSDI '04, pp. 91-104 (2004).
77. Vo, H. T., Chen, C., Ooi, B. C.: Towards elastic transactional cloud storage with range query support. PVLDB 3, 506-514 (2010).
78. Wei, Z., Pierre, G., Chi, C.-H.: CloudTPS: scalable transactions for web applications in the cloud. IEEE Trans. Serv. Comput. 5(4), 525-539 (2012).
79. Zukowski, M., et al.: Cooperative scans: dynamic bandwidth sharing in a dbms. In: Proceedings of VLDB '07, pp. 723-734 (2007).