Exploiting idle cycles to execute data mining applications on clusters of PCs

Journal of Systems and Software

Volumn 80, Issue 5, 2007, Pages 778-790

  Senger, Hermes ^a   Hruschka, Eduardo R ^a   Silva, Fabrício A B ^a   Sato, Liria M ^b   Bianchini, Calebe P ^b   Jerosch, Bruno F ^a  

^a CATHOLIC UNIVERSITY OF SANTOS   (Brazil)

^b UNIVERSITY OF SÃO PAULO   (Brazil)

Author keywords

Commodity clusters; Load sharing; Parallel and distributed data mining

Indexed keywords

DISTRIBUTED COMPUTER SYSTEMS; JAVA PROGRAMMING LANGUAGE; PARALLEL PROCESSING SYSTEMS; PERSONAL COMPUTERS; RESOURCE ALLOCATION; USER INTERFACES;

COMMODITY CLUSTERS; LOAD SHARING; PARALLEL AND DISTRIBUTED DATA MINING;

DATA MINING;

EID: 33847734336     PISSN: 01641212     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jss.2006.05.035     Document Type: Article

References (39)

1. Baraglia, R., Laforenza, D., Orlando, S., Palmerini, P., Perego, R., 2000. Implementation issues in the design of I/O intensive data mining applications on clusters of workstations. In: Proceedings of 3rd Workshop on High Perf. Data Mining, in conjunction with IPDPS'2000, Lecture Notes in Computer Science (LNCS), vol. 1800.
2. Bouchenak S., Hagimont D., Krakowiak S., De Palma N., and Boyer F. Experiences implementing efficient Java thread serialization, mobility and persistence. Software - Practice & Experience 34 4 (2004) 355-393
3. Brunner R.K., and Kalé L.V. Adapting to load on workstation clusters. Proceedings of the Seventh Symposium on the Frontiers of Massively Parallel Computation (1999), IEEE Computer Society Press 106-112
4. Canataro M., and Talia D. The knowledge grid. Commun. ACM 46 1 (2003)
5. Casavant T.L., and Kuhl J.G. A taxonomy of scheduling in general-purpose distributed computing systems. IEEE Trans. Software Eng. 14 2 (1988)
6. Cassandras C. Discrete Event Systems: Modeling and Performance Analysis (1993), Aksen Associates and IRWIN
7. Celis, S., Musicant, D.R., 2002. Weka-parallel: machine learning in parallel. Technical Report, Carleton College Computer Science. Available from: .
8. Czajkowski, K., Foster, I., Karonis, N., Kesselman, C., Martin, S., Smith, W., Tuecke, S., 1998. A resource management architecture for metacomputing systems. In: Proceedings of IPPS/SPDP9́8 Workshop on Job Scheduling Strategies for Parallel Processing, pp. 62-82.
9. Czajkowski, K., Foster, I., Kesselman, C., Sander, V., Tuecke, S., 2002. SNAP: a protocol for negotiating service level agreements and coordinating resource management in distributed systems. In: 8th Workshop on Job Scheduling Strategies for Parallel Processing, Edinburgh, Scotland.
10. Devarakonda M.V., and Iyer R.K. Predictability of process resource usage: a measurement-based study on Unix. IEEE Trans. Software Eng. 15 12 (1989) 1579-1586
11. Dinda, P., O'Halloran, D., 1998. The statistical properties of host load. In: Proceedings of 4th Workshop on Languages, Compilers, and Run-time Systems for Scalable Computers.
12. Duda R.O., Hart P.E., and Stork D.G. Pattern Classification. second ed. (2001), John Wiley & Sons
13. Elnozahy, E.N., Johnson, D.B., Zwaenpoel, W., 1992. The performance of consistent checkpointing. In: Proceedings of the 11th IEEE Symposium on Reliable Distributed Systems, Houston, TX.
14. Fayyad U.M., Shapiro G.P., and Smyth P. From data mining to knowledge discovery: an overview. In: Fayyad U.M., Shapiro G.P., Smyth P., and Uthurusamy R. (Eds). Advances in Knowledge Discovery and Data Mining (1996), MIT Press 1-37
15. Ferrari D., and Zhou S. An empirical investigation of load indices for load balancing applications. Proceedings of International Symposium on Computer Performance Modeling, Measurement and Evaluation (1987), North-Holland, Amsterdam 515-528
16. Frank, E., Witten, I.H., 1998. Generating accurate rule sets without global optimization. In: Proceedings of 15th International Conference on Machine Learning.
17. Freitas A.A., and Lavington S.H. Mining Very Large Databases with Parallel Processing (1998), Kluwer Academic Publishers
18. Garbacki, P., Biskupski, B., Bal, H., 2005. Transparent fault tolerance for grid applications. In: Proceedings of the European Grid Conference (EGC2005), Amsterdam, The Netherlands.
19. Giersch A., Robert Y., and Vivien F. Scheduling tasks sharing files on heterogeneous master-slave platforms. J. Syst. Architect. 52 (2006) 88-104
20. Giersch, A., Robert, Y., Vivien, F., 2004. Scheduling tasks sharing files from distributed repositories. Technical Report No. 5214, INRIA, France.
21. Giersh A., Robert Y., and Vivien F. Scheduling tasks sharing files on heterogeneous master-slave platforms. Proceedings of the 12th Euromicro Workshop in Parallel, Distributed and Network-based Processing (2004), IEEE CS Press 364-371
22. Hand D., Mannila H., and Smyth P. Principles of Data Mining (2001), MIT Press, Cambridge, MA
23. Jain R. The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling (1991), Wiley-Interscience, New York
24. Lawall J.L., and Muller G. Efficient incremental checkpointing of Java programs. Proceedings of the 2000 International Conference on Dependable Systems and Networks (DSN 2000) (2000), IEEE Computer Society Press
25. Liu, K., Kargupta, H. Distributed data mining bibliography website. Available from: .
26. Lumb I., and Smith C. Scheduling attributes and platform LSF. In: Nabrzyski J., Scopf J.M., and Weglarz J. (Eds). Grid Resource Management: State of the Art and Future Directions (2003), Kluwer Academic Publishers
27. Maheswaram, M., Ali, S., Siegel, H.J., Hengsen, D., Freund, R., 1999. Dynamic matching and scheduling of a class of independent tasks onto heterogeneous computing systems. In: 8th Heterogeneous Computing Workshop (HCW'99).
28. Merz, C.J., Murphy, P.M. UCI repository of machine learning databases. Available from: , University of California, Irvine, CA.
29. Napper J., Alvisi L., and Vin H. A fault-tolerant Java virtual machine. Proceedings of the International Conference on Dependable Systems and Networks (DSN 2003) (2003), IEEE Computer Society Press
30. Rosenwald A., et al. The use of molecular profiling to predict survival after chemotherapy for diffuse large-B-cell lymphoma. New Engl. J. Med. 346 25 (2002) 1937-1947
31. In: Schnekenburger T., and Stellner G. (Eds). Load Distribution for Parallel Applications (1997), Teubner, Germany
32. Senger H., and Sato L.M. Load distribution for heterogeneous and non-dedicated clusters based on dynamic monitoring and differentiated services. Proceedings of IEEE International Conference on Cluster Computing (Cluster'03), Hong Kong (2003), IEEE CS Press, Los Alamitos 199-206
33. Senger H., Hruschka E.R., Silva F.A.B., Sato L.M., Bianchini C.P., and Esperidião M.D. Inhambu: data mining using idle cycles in clusters of PCs. IFIP International Conference on Network and Parallel Computing 2004, Wuhan, China, October 2004. Lecture Notes in Computer Science (LNCS) vol. 3222 (2004), Springer-Verlag, Berlin 213-220
34. Senger H., Silva F.A.B., and Nascimento W. Hierarchical scheduling of independent tasks with shared files. International Workshop on Grid Testbeds. Proceedings of the IEEE International Symposium on Cluster Computing and the Grid Workshops (CCGRIDW 2006) (2006), IEEE CS Press, Los Alamitos
35. Silva F.A.B., Carvalho S., Senger H., Hruschka E.R., and Farias C.R.G. Running data mining applications on the grid: a bag-of-tasks approach. Proceedings of International Conference on Computer Science and its Applications (ICCSA), Assisi, Italy. LNCS vol. 3044 (2004), Springer-Verlag, Berlin, Heidelberg, New York 168-177
36. Silva F.A.B., Carvalho S., and Hruschka E.R. A scheduling algorithm for running bag-of-tasks data mining application on the grid. Proceedings of the Euro-Par 2004, Pisa, Italy, 2004. LNCS vol. 3419 (2004), Springer-Verlag, Berlin, Heidelberg, New York 254-262
37. Stone, N., Simmel, D., Kielmann, T., GWD-I, T., 2004. An architecture for grid checkpoint recovery services and a GridCPR API. Grid Checkpoint Recovery Working Group. Global Grid Forum. Available from: .
38. The Condor Project Website. Available from: .
39. Witten I.H., and Frank E. Data mining: practical machine learning tools with Java implementations (2000), Morgan Kaufmann, San Francisco