Online maintenance of very large random samples on flash storage

VLDB Journal

Volumn 19, Issue 1, 2010, Pages 67-90

  Nath, Suman ^a   Gibbons, Phillip B ^b  

^a MICROSOFT RESEARCH   (United States)

^b INTEL CORPORATION   (United States)

Author keywords

Flash storage; Random sample; Semi random writes; Sensor networks

Indexed keywords

COMPUTING DEVICES; DATA STORAGE; DATA STREAM; DATA UPDATE; EMBEDDED SENSORS; ENERGY CONSUMPTION; ENERGY EFFICIENT; FLASH CARD; FLASH STORAGE; FUNDAMENTAL PROBLEM; MAGNETIC DISK; MEMORY FOOTPRINT; ONLINE MAINTENANCE; ORDERS OF MAGNITUDE; RANDOM SAMPLE; SEMI-RANDOM; SEMI-RANDOM WRITES; SENSOR READINGS; WIDE SPECTRUM;

ELECTRIC LOAD FORECASTING; LAPTOP COMPUTERS; MAGNETIC STORAGE; SENSOR NETWORKS; TELECOMMUNICATION EQUIPMENT; WIRELESS NETWORKS;

FLASH MEMORY;

EID: 75949104652     PISSN: 10668888     EISSN: 0949877X     Source Type: Journal    
DOI: 10.1007/s00778-009-0164-z     Document Type: Article

References (34)

1. Agrawal, N., Prabhakaran, V., Wobber, T., Davis, J.D., Manasse, M., Panigrahy, R.: Design tradeoffs for SSD performance. In: USENIX Annual Technical Conference (2008)
2. Babcock, B., Chaudhuri, S., Das, G.: Dynamic sample selection for approximate query processing. In: ACM SIGMOD International Conference on Management of Data (2003)
3. Birrell, A., Isard, M., Thacker, C., Wobber, T.: A design for high- performance flash disks. SIGOPS Oper. Syst. Rev. 41(2), 88-93 (2007)
4. Bobineau, C., Bouganim, L., Pucheral, P., Valduriez, P.: PicoDBMS: Scaling down database techniques for the smartcard. In: International Conference on Very Large Data Bases (VLDB) (2000)
5. Bouganim, L., Jónsson, B., Bonnet, P.: uFLIP: Understanding flash IO patterns. In: Fourth Biennial Conference on Innovative Data Systems Research (CIDR) (2009)
6. Chaudhuri, S., Das, G., Datar, M., Motwani, R., Narasayya, V.R.: Overcoming limitations of sampling for aggregation queries. In: IEEE International Conference on Data Engineering (ICDE) (2001)
7. Diao, Y., Ganesan, D., Mathur, G., Shenoy, P.: Rethinking data management for storage-centric sensor networks. In: Third Biennial Conference on Innovative Data Systems Research (CIDR) (2007)
8. Douglis, F., Cáceres, R., Kaashoek, F., Li, K., Marsh, B., Tauber, J.A.: Storage alternatives for mobile computers. In: USENIX Conference on Operating Systems Design and Implementation (OSDI) (1994)
9. Fan, C.T., Muller, M.E., Rezucha, I.: Development of sampling plans by using sequential (item by item) selection techniques and digital computers. J. Am. Stat. Assoc. 57(298), 387-402 (1962)
10. Gibbons, P.B., Tirthapura, S.: Estimating simple functions on the union of data streams. In: ACM Symposium on Parallel Algorithms and Architectures (SPAA) (2001)
11. Hachman, M.: New Samsung notebook replaces hard drive with flash. http://www.extremetech.com/article2/0,1558,1966644,00.asp, May 2006
12. Intel-Corporation. Understanding the Flash Translation Layer (FTL) specification. http://www.embeddedfreebsd.org/Documents/Intel-FTL.pdf (1998)
13. Janukowicz, J., Reinsel, D.: SSDs: The other primary storage alternative. IDC White Paper (2008)
14. Jermaine, C., Datta, A., Omiecinski, E.: A novel index supporting high volume data warehouse insertion. In: International Conference on Very Large Data Bases (VLDB) (1999)
15. Jermaine, C., Pol, A., Arumugam, S.: Online maintenance of very large random samples. In:ACM SIGMOD International Conference on Management of Data (2004)
16. Kim, H., Ahn, S.: BPLRU: a buffer management scheme for improving random writes in flash storage. In: USENIX Conference on File and Storage Technologies (FAST) (2008)
17. Kim, J., Kim, J.M., Noh, S.H., Min, S.L., Cho, Y.: A space-efficient flash translation layer for compact flash systems. IEEE Trans. Consumer Electron. 48(2), 366-375 (2002)
18. Lee, J., Kim, S., Kwon, H., Hyun, C., Ahn, S., Choi, J., Lee, D., Noh, S.H.: Block recycling schemes and their cost-based optimization in NAND flash memory based storage system. In: ACM/IEEE Inter- national Conference on Embedded Software (EMSOFT) (2007)
19. Lee, S.-W., Moon, B.: Design of flash-based DBMS: an in-page logging approach. In: ACM SIGMOD International Conference on Management of Data (2007)
20. Mathur, G., Desnoyers, P., Ganesan, D., Shenoy, P.: Capsule: an energy-optimized object storage system for memory-constrained sensor devices. In: ACM International Conference on Embedded Networked Sensor Systems (SenSys) (2006)
21. Miller, P.: Simple Techannounces 512 GB and 256 GB 3.5-inch SSD drives. http://www.engadget.com/2007/04/18/ (2007)
22. Moteiv Corporation. Tmote sky platform. http://www.moteiv.com/community/Tmote_Sky_Downloads (2007)
23. Nath, S., Kansal, A.: FlashDB: dynamic self-tuning database for NAND flash. In: ACM/IEEE International Conference on Information Processing in Sensor Networks (IPSN) (2007)
24. Olken, F., Rotem, D., Xu, P.: Random sampling from hash files. SIGMOD Rec., 19(2), 375-386 (1990)
25. O'Neil, P., Cheng, E., Gawlick, D., O'Neil, E.: The log-structured merge-tree (LSM-tree). Acta Inf. 33(4), 351-385 (1996)
26. Pugh, W.: Skip lists: a probabilistic alternative to balanced trees. Commun. ACM 33(6), 668-676 (1990)
27. Reinsel, D., Janukowicz, J.: Datacenter SSDs: Solid footing for growth. Samsung white paper. http://www.samsung.com/global/ business/semiconductor/products/flash/ssd/pdf/datacenter_ssds.pdf (2008)
28. SyCard. CF extend 180 CompactFlash Flexible Extender Card. http://www.sycard.com/cfext180.html (2008)
29. Vitter, J.S.: Random sampling with a reservoir. ACM Trans. Math. Softw. 11(1), 37-57 (1985)
30. Vitter, J.S.: An efficient algorithm for sequential random sampling. ACM Trans. Math. Softw. 13(1), 58-67 (1987)
31. Vitter, J.S.: External memory algorithms and data structures. ACM Comput. Surv. 33(2), 209-271 (2001)
32. Wu, C.-H., Chang, L.-P., Kuo, T.-W.: An efficient R-tree implementation over flash-memory storage systems. In: ACM International Symposium on Advances in Geographic Information Systems (GIS) (2003)
33. Yahoo!-Finance. Zeus-IOPS solid state drives surge to 512GB. http://biz.yahoo.com/pz/070418/117663.html (2007)
34. Zeinalipour-Yazti, D., Lin, S., Kalogeraki, V., Gunopulos, D., Najjar, W.A.: Microhash: an efficient index structure for flash- based sensor devices. In: USENIX Conference on File and Storage Technologies (FAST) (2005)