Compression-aware I/O performance analysis for big data clustering

Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

Volumn , Issue , 2012, Pages 45-52

  Xue, Zhenghua ^a   Li, Jianhui ^a   Zhang, Yang ^a   Shen, Geng ^a   Xu, Qian ^b   Shao, Jing ^a  

^a INSTITUTE OF GEOLOGY AND GEOPHYSICS   (China)

^b BAIDU INC   (China)

Author keywords

Big data clustering; Compression contribution model; I O bottleneck

Indexed keywords

BIG DATUM; CLUSTER COMPUTER; COMPUTATIONAL PROCESS; CONTRIBUTION MODEL; DATA VOLUME; I/O BOTTLENECK; I/O PERFORMANCE; K-MEANS ALGORITHM; PERFORMANCE IMPROVEMENTS;

CLUSTER ANALYSIS; CLUSTER COMPUTING; CLUSTERING ALGORITHMS; COMPUTER SYSTEMS PROGRAMMING;

DATA COMPRESSION;

EID: 84866606377     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2351316.2351323     Document Type: Conference Paper

References (24)

1. D. Abadi, S. Madden, and M. Ferreira. Integrating compression and execution in column-oriented database systems. In Proceedings of the 2006 ACM SIGMOD international conference on Management of data, SIGMOD '06, pages 671-682, New York, NY, USA, 2006. ACM.
2. P. K. Agarwal and N. H. Mustafa. k-means projective clustering. In Proceedings of the twenty-third ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, PODS '04, pages 155-165, New York, NY, USA, 2004. ACM.
3. C. C. Aggarwal, J. L. Wolf, P. S. Yu, C. Procopiuc, and J. S. Park. Fast algorithms for projected clustering. In Proceedings of the 1999 ACM SIGMOD international conference on Management of data, SIGMOD '99, pages 61-72, New York, NY, USA, 1999. ACM.
4. C. C. Aggarwal and P. S. Yu. Redefining clustering for high-dimensional applications. IEEE Trans. on Knowl. and Data Eng., 14(2):210-225, Mar. 2002.
5. D. C. Anderson, J. S. Chase, S. Gadde, A. J. Gallatin, K. G. Yocum, and M. J. Feeley. Cheating the i/o bottleneck: network storage with trapeze/myrinet. In Proceedings of the annual conference on USENIX Annual Technical Conference, ATEC '98, pages 12-12, Berkeley, CA, USA, 1998. USENIX Association.
6. Y. Chen, A. Ganapathi, and R. H. Katz. To compress or not to compress - compute vs. io tradeoffs for mapreduce energy efficiency. In Proceedings of the first ACM SIGCOMM workshop on Green networking, Green Networking '10, pages 23-28, New York, NY, USA, 2010. ACM.
7. Y.-J. Chiang and C. T. Silva. I/o optimal isosurface extraction (extended abstract). In Proceedings of the 8th conference on Visualization '97, VIS '97, pages 293-ff., Los Alamitos, CA, USA, 1997. IEEE Computer Society Press.
8. J. M. del Rosario, R. Bordawekar, and A. Choudhary. Improved parallel i/o via a two-phase run-time access strategy. SIGARCH Comput. Archit. News, 21(5):31-38, Dec. 1993.
9. Dhillon. Method and system for clustering data in parallel in a distributed-memory multiprocessor system.
10. I. S. Dhillon and D. S. Modha. A data-clustering algorithm on distributed memory multiprocessors. In Revised Papers from Large-Scale Parallel Data Mining, Workshop on Large-Scale Parallel KDD Systems, SIGKDD, pages 245-260, London, UK, UK, 2000. Springer-Verlag.
11. R. L. Ferreira Cordeiro, C. Traina, Junior, A. J. Machado Traina, J. López, U. Kang, and C. Faloutsos. Clustering very large multi-dimensional datasets with mapreduce. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, KDD '11, pages 690-698, New York, NY, USA, 2011. ACM.
12. G. A. Gibson. Redundant disk arrays: Reliable, parallel secondary storage. Technical report, Berkeley, CA, USA, 1999.
13. Z. Huang. The apache software foundation. http://mahout.apache.org/.
14. A. K. Jain, M. N. Murty, and P. J. Flynn. Data clustering: a review. ACM Comput. Surv., 31(3):264-323, Sept. 1999.
15. J. Lee, M. Winslett, X. Ma, and S. Yu. Enhancing data migration performance via parallel data compression. In Proceedings of the 16th International Parallel and Distributed Processing Symposium, IPDPS '02, pages 142-, Washington, DC, USA, 2002. IEEE Computer Society.
16. K.-H. Lee, Y.-J. Lee, H. Choi, Y. D. Chung, and B. Moon. Parallel data processing with mapreduce: a survey. SIGMOD Rec., 40(4):11-20, Jan. 2012.
17. Y. Li and S. M. Chung. Parallel bisecting k-means with prediction clustering algorithm. J. Supercomput., 39(1):19-37, Jan. 2007.
18. J. Ousterhout, P. Agrawal, D. Erickson, C. Kozyrakis, J. Leverich, D. Mazìeres, S. Mitra, A. Narayanan, D. Ongaro, G. Parulkar, M. Rosenblum, S. M. Rumble, E. Stratmann, and R. Stutsman. The case for ramcloud. Commun. ACM, 54(7):121-130, July 2011.
19. J. Ousterhout and F. Douglis. Beating the i/o bottleneck: a case for log-structured file systems. SIGOPS Oper. Syst. Rev., 23(1):11-28, Jan. 1989.
20. V. S. Pai, P. Druschel, and W. Zwaenepoel. Io-lite: a unified i/o buffering and caching system. ACM Trans. Comput. Syst., 18(1):37-66, Feb. 2000.
21. A. K. H. Tung, X. Xu, and B. C. Ooi. Curler: finding and visualizing nonlinear correlation clusters. In Proceedings of the 2005 ACM SIGMOD international conference on Management of data, SIGMOD '05, pages 467-478, New York, NY, USA, 2005. ACM.
22. B. Welton, D. Kimpe, J. Cope, C. M. Patrick, K. Iskra, and R. Ross. Improving i/o forwarding throughput with data compression. In Proceedings of the 2011 IEEE International Conference on Cluster Computing, CLUSTER '11, pages 438-445, Washington, DC, USA, 2011. IEEE Computer Society.
23. W. Zhao, H. Ma, and Q. He. Parallel k-means clustering based on mapreduce. In Proceedings of the 1st International Conference on Cloud Computing, CloudCom '09, pages 674-679, Berlin, Heidelberg, 2009. Springer-Verlag.
24. M. Zukowski, S. Heman, N. Nes, and P. Boncz. Super-scalar ram-cpu cache compression. In Proceedings of the 22nd International Conference on Data Engineering, ICDE '06, pages 59-, Washington, DC, USA, 2006. IEEE Computer Society.