Efficient bulk insertion into a distributed ordered table

Proceedings of the ACM SIGMOD International Conference on Management of Data

Volumn , Issue , 2008, Pages 765-778

  Silberstein, Adam ^a   Cooper, Brian F ^a   Srivastava, Utkarsh ^a   Vee, Erik ^a   Yemeni, Ramana ^a   Ramakrishnan, Raghu ^a  

^a YAHOO RESEARCH   (United States)

Author keywords

Bulk loading; Distributed and parallel databases; Ordered tables

Indexed keywords

INTERNET;

BULK LOADING; DISTRIBUTED AND PARALLEL DATABASES; LARGE CLUSTERS; NP-HARD; NUMBER OF DATUMS; ORDERED TABLES; PROTOTYPE SYSTEMS; THROUGHPUT GAINS; VE TECHNIQUES;

PARTITIONS (BUILDING);

EID: 57149122176     PISSN: 07308078     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1376616.1376693     Document Type: Conference Paper

References (27)

1. GridFTP. http://www.globus.org/grid-software/- data/gridftp.php ftp://ftp.php.
2. Hacmp for system p servers, http://www-O3.ibm.com/systems/p / software/hacmp/index.html.
3. Oracle real application clusters 11g. http: //www.oracle.com/technology/ products/database/clustering/index.html.
4. Scalability and performance with oracle 11g database. http://www.oracle,com/technology/deploy/-performance/pdf/db-perfscale-11gr1-twp. pdf, 2007.
5. M. K. Aguilera, A. Merchant, M. Shah, A. Veitch, and C. Karamanolis. Sinfonia: A new paradigm for building scalable distributed systems. In Proc. SOSP, October 2007.
6. P. Bernstein, N. Dani, B. Khessib, R, Manne, and D. Shutt. Data management issues in supporting large-scale web services. IEEE Data Engineering Bulletin, December 2006.
7. L. Breslau, P. Cao, L. Fan, G. Phillips, and S. Shenker. Web caching and zipf-like distributions: Evidence and implications. In Proc. INFOCOM, 1999.
8. F. Chang et al. Bigtable: A distributed storage system for structured data, In OSDL 2006.
9. C. Cliekuri and S. Khanna. A polynomial time approximation scheme for the multiple knapsack problem. SIAM Journal on Computing, 35(3):713-728, 2003.
10. B. F. Cooper, R. Ramakrishnan, U. Srivastava, A. Silberstein, P. Bohannon, H. Jacobsen, N. Puz, D. Weaver, and R. Yemeni. PNUTS: Yahoo!'s hosted data serving platform. Technical report, Yahoo! Research, 2008.
11. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. In Proc. OSDI December 2004.
12. G. DeCandia, D. Hastomn, M. Jampani, G. Kakulapati, A. Lakshman, A. Pilchin, S. Sivasubramanian, P. Vosshall, and W. Vogels. Dynamo: Amazon's highly available key-value store. In Proc. SOSP, October 2007.
13. D. J. DeWitt, J. F. Naughton, and D. A. Schneider. Parallel sorting on a shared-nothing architecture using probabilistic splitting. In Proc. Conference on Parallel and Distributed Information Systems, pages 280-291, 1991.
14. G. Graefe. B-tree indexes for high update rates. SIGMOD Record, 35(1):39-44, March 2006.
15. C. S. Group. Community systems research at yahoo! SIGMOD Record, 36(3):47-54, September 2007.
16. S. Hung and J. Fisk. An algorithm for 0-1 multiple knapsack problems. In Naval Res. Logist. Quart., pages 571-579, 1978. [17] H. Jagadish, P. Narayan, S. Seshadri, S. Sudarsfian, and R. Kanneganti. Incremental organization for data recording and warehousing. In Proc. Very Large Databases, August 1997.
17. R. Loulou and E. Michaelides. New greedy-like heuristics for the multidimensional 0-1 knapsack problem. Operations Research, 27:1101-1114, 1979.
18. S. Martello and P. Toth. Knapsack Problems: Algorithms and Computer Implementations. John Wiley and Sons, 1990.
19. S. Mishra. Loading bulk data into a partitioned table. http: / /www.microsoft.com/tecfmet/prodtecfmol/sql/-bestpractice/- loading-bulk-data- partitioned-table.mspx, September 2006.
20. C. Mohan and I. Narang. Algorithms for creating indexes for very large tables without quiescing updates. In Proc. SIGMOD, June 1992.
21. P. O'Neil, E. Cheng, D. Gawlick, and E. O'Neil. The log-sructured merge-tree. In Acta Informatica, volume 33, pages 351-385, 1996.
22. S. Senju and Y. Toyoda. An approach to linear programming with 0-1 variables. Management Science, 15(4):B196-B207, 1968.
23. S. Seshadri and J. Naughton. Sampling issues in parallel database systems. In Proc. Extending Database Technology, March 1992.
24. J. van den Bercken, B. Seeger, and P. Widmayer. A generic approach to bulk loading multidimensional index structures. In Proc. Very Large Databases, August 1997.
25. M. Vasquez and J. Hao. A hybrid approach for the 01 multidimensional knapsack problem. In IJCAI, pages 328-333, 2001.
26. S. A. Weil, S. A. Brandt, E. L. Miller, D. D. E. Long, and C. Maltzahn. Ceph: A scalable, high-performance distributed file system. In Proc. OSDI, 2006.
27. J. Wiener and J. Naughton. Oodb bulk loading revisited: The partitioned-list approach. In Proc. Very Large Databases, September 1995.