DisCo: Distributed Co-clustering with Map-Reduce: A case study towards petabyte-scale end-to-end mining

Proceedings - IEEE International Conference on Data Mining, ICDM

Volumn , Issue , 2008, Pages 512-521

  Papadimitriou, Spiros ^a  

^a IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CO-CLUSTERING; COLLABORATIVE FILTERING; COMMODITY HARDWARE; DATA SETS; DATA STORAGE; DISTRIBUTED DATA; DISTRIBUTED PROCESSING; EXECUTION ENGINE; GRAPH MINING; LARGE DATASETS; MANAGEMENT COMPONENTS; MINING PROCESS; MINING TASKS; OPEN SOURCES; PRE-PROCESSING; REAL-WORLD APPLICATION; TEXT MINING;

COBALT COMPOUNDS; DATA MINING; DATA PROCESSING; INFORMATION MANAGEMENT; MINING;

CLUSTER ANALYSIS;

EID: 67149126890     PISSN: 15504786     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICDM.2008.142     Document Type: Conference Paper

References (38)

1. Hadoop. http://hadoop.apache.org/core/.
2. HBase. http://hadoop.apache.org/hbase/.
3. PIG. http://incubator.apache.org/pig/.
4. A. Banerjee, I. Dhillon, J. Ghosh, S. Nerugu, and D. S. Modha. A generalized maximum entropy approach to Bregman co-clustering and matrix approximation. In KDD, 2004.
5. CFS. Lustre file system. http://www.lustre.org/.
6. D. Chakrabarti, S. Papadimitriou, D. Modha, and C. Faloutsos. Fully automatic cross-associations. In KDD, 2004.
7. F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes, and R. E. Gruber. Bigtable: A distributed storage system for structured data. In OSDI, 2006.
8. Y. Cheng and G.M. Church. Biclustering of expression data. In ISMB, 2000.
9. H. Cho, I. Dhillon, Y. Guan, and S. Sra. Minimum sumsquared residue co-clustering of gene expression data. In SDM, 2004.
10. C.-T. Chu, S. K. Kim, Y.-A. Lin, Y. Yu, G. R. Bradski, A. Y. Ng, and K. Olukotun. Map-Reduce for machine learning on multicore. In NIPS, 2006.
11. A. Das, M. Datar, A. Garg, and S. Rajaram. Google news personalization: Scalable online collaborative filtering. In WWW, 2007.
12. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. In OSDI, 2004.
13. J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. CACM, 51(1), 2008.
14. D. J. DeWitt and M. Stonebraker. MapReduce: A major step backwards. http://www.databasecolumn.com/2008/01/ mapreduce-a-major-step-back.html.
15. I. S. Dhillon, S. Mallela, and D. S. Modha. Informationtheoretic co-clustering. In KDD, 2003.
16. U. Fayyad. From mining the web to inventing the new sciences underlying the internet. http://www.sigkdd.org/kdd2007/program.html#invited2, 2008.
17. B. Gedik, H. Andrade, K.-L. Wu, P. S. Yu, and M. Doo. SPADE: The System S declarative stream processing engine. In SIGMOD, 2008.
18. T. George and S. Merugu. A scalable collaborative filtering framework based on co-clustering. In ICDM, 2005.
19. S. Ghemawat, H. Gobioff, , and S.-T. Leung. The Google file system. In SOSP, 2003.
20. G. A. Gibson, D. F. Nagle, K. Amiri, J. Butler, F.W. Chang, H. Gobioff, C. Hardin, E. Riedel, D. Rochberg, and J. Zelenka. A cost-effective, high-bandwidth storage architecture.In ASPLOS, 1998.
21. L. Glimcher and G. Agrawal. FREERIDE-G: Enabling distributed processing of large datasets. In HPDC, 2008.
22. R. Grossman. Data mining FAQ. http://www.rgrossman.com/dm.htm.
23. R. L. Grossman and Y. Gu. Data mining using high performance clouds: Experimental studies using sector and sphere. In KDD, 2008.
24. J. A. Hartigan. Direct clustering of a data matrix. J. Am. Stat. Assoc., 67(337), 1972.
25. M. Isard, M. Budiu, Y. Yu, A. Birrell, and D. Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In EuroSys, 2007.
26. Kosmix. Kosmos Distributed File System (KFS). http://kosmosfs. sourceforge.net/.
27. J. Leskovec and E. Horvitz. Planetary-scale views on an instant-messaging network. In WWW, 2008.
28. H. Li and N. Abe. Word clustering and disambiguation based on co-occurence data. In COLING-ACL, 1998.
29. S. C.Madeira and A. L. Oliveira. Biclustering algorithms for biological data analysis: A survey. IEEE/ACM TCBB, 1, 2004.
30. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig Latin: A not-so-foreign language for data processing. In SIGMOD, 2008.
31. R. Pike, S. Dorward, R. Griesemer, and S. Quinlan. Interpreting the data: Parallel analysis with sawzall. Sci. Prog. J., 13(4), 2005.
32. E. Riedel, C. Faloutsos, G. A. Gibson, and D. Nagle. Active disks for large-scale data processing. IEEE Computer, 2001.
33. F. Schmuck and R. Haskin. GPFS: A shared-disk file system for large computing clusters. In FAST, 2002.
34. M. Seltzer. Beyond relational databases. CACM, 51(7), 2008.
35. M. Stonebraker and U. Cetintemel. One size fits all: An idea whose time has come and gone. In ICDE, 2005.
36. S. A. Weil, S. A. Brandt, E. L. Miller, D. D. Long, and C. Maltzahn. Ceph: A scalable, high-performance distributed file system. In OSDI, 2006.
37. C. K. Wensel. Cascading. http://www.cascading.org/.
38. Zvents. Hypertable. http://hypertable.org/.