Design of a next generation sampling service for large scale data analysis applications

Proceedings of the International Conference on Supercomputing

Volumn , Issue , 2005, Pages 91-100

  Wang, H ^a   Parthasarathy, S ^a   Ghoting, A ^a   Tatikonda, S ^a   Buehrer, G ^a   Kurc, T ^a   Saltz, J ^a  

^a Ohio State University   (United States)

Author keywords

Data centers; Data mining; Parallel I O; Sampling

Indexed keywords

DATA REDUCTION; DATA STORAGE EQUIPMENT; DATA TRANSFER; DATABASE SYSTEMS; SAMPLING;

BOOTSTRAPPING-BASED TECHNIQUES; DATA CENTERS; DATA SETS; PARALLEL I/O;

DATA MINING;

EID: 32844469575     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1088149.1088162     Document Type: Conference Paper

References (34)

1. B. Babcocket al. Models and issues in data stream systems. In ACM Symposium on Principles of Database Systems, 2002.
2. M. Cannataro and D. Talia. Knowledge grid an architecture for distributed knowledge discovery. In CACM, Vol. 46, No. 1, pp. 89-93, 2003.
3. P. Cams, W. Ligon, R. Ross, and R. Thakur, Pvfs: A parallel file system for linux clusters. In Proceedings of the Annual Linux Showcase and Conference, 2000.
4. Ling Tony Chen and Doron Rotem. Declustering objects for visualization. In Proceedings of the 19th International Conference on Very Large Data Bases, pages 85-96, Dublin, Ireland, August 1993.
5. A. Chervenak, I. Foster, C. Kesselman, C. Salisbury, and S. Tuecke. The Data Grid: Towards an Architecture For the Distributed Management and Analysis of Large Scientific Datasets, 2001.
6. Peter F. Corbett and Dror G. Feitelson. The Vesta parallel file system. ACM Transactions on Computer Systems, 14(3):225-264, August 1996.
7. Thomas H. Cormen, Charles E. Leiserson, and Ronald L. Rivest. Introduction to Algorithms. McGraw Hill, 1990.
8. David DeWitt and Jim Gray. Parallel database systems: the future of high performance database systems. Communications of the ACM, 35(6):85-98, June 1992.
9. Thomas G. Dietterich. An experimental comparison of three methods for constructing ensembles of decision trees: Bagging, boosting, and randomization. Machine Learning, 40(2):139-157, 2000.
10. P. Domingos and G. Hulten. A general method for scaling up machine learning algorithms and its applications to clustering. In Proceedings of the International Conference on Machine Learning, 2001.
11. H. C. Du and J. S. Sobolewski. Disk allocation for Cartesian product files on multiple-disk systems. ACM Transactions onDatabase Systems, 7(1):82-101, March 1982.
12. Christos Faloutsos and Pravin Bhagwat. Declustering using fractals. In the 2nd International Conference on Parallel and Distributed Information Systems, pages 18-25, San Diego, CA, January 1993.
13. M. T. Fang, R. C. T. Lee, and C. C. Chang. The idea of de-clustering and its applications. In Proceedings of the 12th VLDB Conference, pages 181-188, 1986.
14. J. Han and M. Kamber. Data Mining: Concepts and Techniques. Morgan Kauffman, 2001.
15. C. Jermaine, A. Pol, and S. Arumugam. Online maintenance of very large random samples. In Proceedings of the International Conference on Management of Data, 2004.
16. G. John and P. Langley. Static versus dynamic sampling for data mining. In Proceedings of the International Conference on Knowledge Discovery and Data Mining, 1996.
17. David Kotz. Disk-directed I/O for MIMD multiprocessors. In Proceedings of the 1994 Symposium on Operating Systems Design and Implementation, pages 61-74. ACM Press, November 1994.
18. Duen-Ren Liu and Shashi Shekhar. A similarity graph-based approach to declustering problems and its applications towards parallelizing grid files. In Proceedings of the International Conference on Data Engineering, pages 373-381, Taipei, Taiwan, March 1995. IEEE Computer Society Press.
19. M. Mahoney and P. Chan. Learning rules for anomaly detection of hostile network traffic. In Proceedings of the International Conference on Data Mining, 2003.
20. John M. May. Parallel I/O for High Performance Computing. Morgan Kaufmann Publishers, 2000.
21. Bongki Moon, Anurag Acharya, and Joel Saltz. Study of scalable declustering algorithms for parallel grid files. In Proceedings of the Tenth International Parallel Processing Symposium. IEEE Computer Society Press, April 1996.
22. Nils Nieuwejaar and David Kotz. The Galley parallel file system. In Proceedings of the 1996 International Conference on Supercomputing, pages 374-381. ACM Press, May 1996.
23. F. Olken and D. Rotem. Random sampling from database files: A survey. In Proceedings of the International Conference on Scientific and Statistical Database Management, 1990.
24. J. Pan, C. Faloutsos, and S. Seshan. Fastcars: Fast, correlation-aware sampling for network data mining. In Proceeding of the IEEE GlobeCom Global Internet Symposium, 2002.
25. S. Parthasarathy. Efficient progressive sampling for association rules. In Proceedings of the International Conference on Data Mining, 2002.
26. Jean-Pierre Prost, Richard Treumann, Richard Hedges, Bin Jia, and Alice Koniges. MPI-IO/GPFS, an optimized implementation of MPI-IO on top of GPFS. In Proceedings of the 3001 ACM/IEEE SC01 Conference. ACM Press, November 2001.
27. F. Provost, D. Jensen, and T. Oates. Efficient progressive sampling. In Proceedings of the International Conference on Knowledge Discovery and Data Mining, 1999.
28. F. Provost and V. Kolluri. A survey of methods for scaling up inductive algorithms. Data Mining and Knowledge Discovery, 1999.
29. K. E. Seamons, Y. Chen, P. Jones, J. Jozwiak, and M. Winslett. Server-directed collective I/O in Panda. In Proceedings of Supercomputing '95, San Diego, CA, December 1995. IEEE Computer Society Press.
30. X. Shen and A. Choudhary. A distributed multi-storage i/o system for high performance data intensive computing. In International Symposium on Cluster Computing and the Grid (CCGrid 2002), May 2002.
31. Rajeev Thakur, Alok Choudhary, Rajesh Bordawekar, Sachin More, and Sivaramakrishna Kuditipudi. Passion: Optimized I/O for parallel applications. IEEE Computer, 29(6):70-78, June 1996.
32. H. Toivonen. Sampling large databases for associations. In Proceedings of the International Conference on Very Large Databases, 1996.
33. J. Vitter. Random sampling with a reservoir. ACM Transactions on Mathematical Software, 1985.
34. M. Zaki, S. Parthasarathy, M. Ogihara, and W. Li. New algorithms for fast discovery of association rules. In Proceedings of the International Conference on Knowledge Discovery and Data Mining, 1997.