Metadata traces and workload models for evaluating big storage systems

Proceedings - 2012 IEEE/ACM 5th International Conference on Utility and Cloud Computing, UCC 2012

Volumn , Issue , 2012, Pages 125-132

  Abad, Cristina L ^a,b,c   Luu, Huong ^a   Roberts, Nathan ^b   Lee, Kihwal ^b   Lu, Yi ^a   Campbell, Roy H ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b YAHOO INC   (United States)

^c ESCUELA SUPERIOR POLITÉCNICA DEL LITORAL   (Ecuador)

Author keywords

Big data; HDFS; MDS; Metadata; Storage

Indexed keywords

BIG DATUM; EVALUATION APPROACH; FILE SYSTEMS; HADOOP DISTRIBUTED FILE SYSTEMS; HDFS; LEAST RECENTLY USED; MDS; METADATA MANAGEMENT; METADATA WORKLOADS; NAMESPACES; REAL TRACE; STORAGE SYSTEMS; SYNTHETIC WORKLOADS; WORK-LOAD MODELS;

CLOUD COMPUTING; DIGITAL STORAGE; ENERGY STORAGE; FILE ORGANIZATION;

METADATA;

EID: 84874278017     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/UCC.2012.27     Document Type: Conference Paper

References (22)

1. ABAD, C., LU, Y., AND CAMPBELL, R. DARE: Adaptive data replication for efficient cluster scheduling. In IEEE CLUSTER (2011).
2. ABAD, C., LUU, H., LU, Y., AND CAMPBELL, R. Metadata workloads for testing Big storage systems. Tech. rep., UIUC, 2012. http://hdl.handle.net/2142/ 30013.
3. ABAD, C., ROBERTS, N., LU, Y., AND CAMPBELL, R. A storage-centric analysis of MapReduce workloads: File popularity, temporal locality and arrival patterns. In Proc. IEEE IISWC (2012).
4. AGRAWAL, ARPACI-DUSSEAU, AND ARPACI-DUSSEAU. Generating realistic Impressions for file-system benchmarking. TOS 5 (2009).
5. AGRAWAL, N., BOLOSKY, W., DOUCEUR, J., AND LORCH, J. A five-year study of file-system metadata. TOS 3 (2007).
6. ALAM, S., EL-HARAKE, H., HOWARD, K., STRINGFELLOW, N., AND VERZELLONI, F. Parallel I/O and the metadata wall. In PDSW (2011).
7. ANANTHARANAYANAN, G., GHODSI, A., WANG, A., BORTHAKUR, D., KANDULA, S., SHENKER, S., AND STOICA, I. PACMan: Coordinated memory caching for parallel jobs. In Usenix NSDI (2012).
8. CHEN, Y., SRINIVASAN, K., GOODSON, G., AND KATZ, R. Design implications for enterprise storage systems via multi-dimensional trace analysis. In SOSP (2011).
9. DAYAL, S. Characterizing HEC storage systems at rest. Tech. Rep. CMU-PDL-08-109, CMU, 2008.
10. DEVULAPALLI, A., AND OHIO, P. File creation strategies in a distributed metadata file system. In IPDPS (2007).
11. GEIST, A. Paving the road to exascale. SciDAC Review, 16 (2010).
12. LEUNG, SHAO, BISSON, PASUPATHY, AND MILLER. Spyglass: Fast, scalable metadata search for large-scale storage systems. In FAST'09.
13. MCKUSICK, K., AND QUINLAN, S. GFS: Evolution on fast-forward. Commun. ACM 53 (2010).
14. NICOLAE, B., ANTONIU, G., AND BOUGÉ, L. Enabling high data throughput in desktop grids through decentralized data and metadata management: The blobseer approach. In EuroPar09 (2009).
15. PATIL, S., AND GIBSON, G. Scale and concurrency of GIGA+: File system directories with millions of files. In Usenix FAST (2011).
16. SHVACHKO. HDFS scalability: Limits to growth. Usenix; login (2010).
17. TANG, FU, CHERKASOVA, AND VAHDAT. MediSyn: A synthetic streaming media service workload generator. In NOSSDAV (2003).
18. TARASOV, V., BHANAGE, S., ZADOK, E., AND SELTZER, M. Benchmarking file system benchmarking. In HotOS (2011).
19. TARASOV, V., KUMAR, K., MA, J., HILDEBRAND, D., POVZNER, A., KUENNING, G., AND ZADOK, E. Extracting flexible, replayable models from large block traces. In Usenix FAST (2012).
20. VIJAYAKUMAR, K., MUELLER, F., MA, X., AND ROTH, P. Scalable I/O tracing and analysis. In PDSW (2009).
21. WELCH, B., UNANGST, M., ABBASI, Z., GIBSON, G., MUELLER, B., SMALL, J., ZELENKA, J., AND ZHOU, B. Scalable performance of the Panasas parallel file system. In Usenix FAST (2008).
22. ZAHARIA, BORTHAKUR, SENSARMA, ELMELEEGY, SHENKER, AND STOICA. Delay scheduling: A simple technique for achieving locality and fairness in cluster scheduling. In EuroSys (2010).