A survey of methods for distributed machine learning

Progress in Artificial Intelligence

Volumn 2, Issue 1, 2013, Pages 1-11

  Peteiro Barral, Diego ^a   Guijarro Berdiñas, Bertha ^a  

^a UNIVERSITY OF A CORUÑA   (Spain)

Author keywords

Data fragmentation; Distributed learning; Large scale learning; Machine learning; Scalability

Indexed keywords

INTELLIGENT SYSTEMS; LEARNING SYSTEMS; MACHINE LEARNING; SCALABILITY;

AMOUNT OF INFORMATION; DATA FRAGMENTATION; DISTRIBUTED ENVIRONMENTS; DISTRIBUTED LEARNING; DISTRIBUTED MACHINE LEARNING; LARGE-SCALE LEARNING; LEARNING PROCESS; REAL APPLICATIONS;

LEARNING ALGORITHMS;

EID: 85049799378     PISSN: 21926352     EISSN: 21926360     Source Type: Journal    
DOI: 10.1007/s13748-012-0035-5     Document Type: Review

References (42)

1. School of Information and Management and Systems. How much information? http://www2. sims. berkeley. edu/research/projects/how-much-info/internet. html (2000). Accessed 27 Sept 2010.
2. D-Lib Magazine. A research library based on the historical collections of the Internet Archive. http://www. dlib. org/dlib/february06/arms/02arms. html (2006). Accessed 27 Oct 2010.
3. Catlett, J.: Megainduction: machine learning on very large databases. PhD thesis, School of Computer Science, University of Technology, Sydney, Australia (1991).
4. Bottou, L., Bousquet, O.: The tradeoffs of large scale learning. Adv. Neural Inf. Process. Syst. 20, 161-168 (2008).
5. Sonnenburg, S., Ratsch, G., Rieck, K.: Large scale learning with string kernels. In: Bottou, L., Chapelle, O., DeCoste, D., Weston, J. (eds.) Large Scale Kernel Machines, pp. 73-104. MIT Press, Cambridge (2007).
6. Moretti, C., Steinhaeuser, K., Thain, D., Chawla, N. V.: Scaling up classifiers to cloud computers. In: Proceedings of the 8th IEEE International Conference on Data Mining (ICDM), pp. 472-481 (2008).
7. Krishnan, S., Bhattacharyya, C., Hariharan, R.: A randomized algorithm for large scale support vector learning. In: Proceedings of Advances in Neural Information Processing Systems (NIPS), pp. 793-800 (2008).
8. Raina, R., Madhavan, A., Ng., A. Y.: Large-scale deep unsupervised learning using graphics processors. In: Proceedings of the 26th Annual International Conference on Machine Learning (ICML), pp. 873-880 (2009).
9. Provost, F., Kolluri, V.: A survey of methods for scaling up inductive algorithms. Data Min. Knowl. Discov. 3(2), 131-169 (1999).
10. Giordana, A., Neri, F.: Search-intensive concept induction. Evol. Comput. 3(4), 375-416 (1995).
11. Anglano, C., Botta, M.: Now g-net: learning classification programs on networks of workstations. IEEE Trans. Evol. Comput. 6(5), 463-480 (2002).
12. Rodríguez, M., Escalante, D. M., Peregrín, A.: Efficient distributed genetic algorithm for rule extraction. Appl. Soft Comput. 11(1), 733-743 (2011).
13. Lopez, L. I., Bardallo, J. M., De Vega, M. A., Peregrin, A.: Regal-tc: a distributed genetic algorithm for concept learning based on regal and the treatment of counterexamples. Soft Comput. 15(7), 1389-1403 (2011).
14. Trelles, O., Prins, P., Snir, M., Jansen, R. C.: Big data, but are we ready? Nat. Rev. Genetics 12(3), 224-224 (2011).
15. Stoica, I.: A berkeley view of big data: algorithms, machines and people. In: UC Berkeley EECS Annual Research, Symposium (2011).
16. LaValle, S., Lesser, E., Shockley, R., Hopkins, M. S., Kruschwitz, N.: Big data, analytics and the path from insights to value. MIT Sloan Manag. Rev. 52(2), 21-32 (2011).
17. Borthakur, D.: The hadoop distributed file system: architecture and design. Hadoop Project Website 11, 21 (2007).
18. Caragea, D., Silvescu, A., Honavar, V.: Analysis and synthesis of agents that learn from distributed dynamic data sources. In: Wermter, S., Austin, J., Willshaw D. J. (eds.) Emergent Neural Computational Architectures Based on Neuroscience, pp. 547-559. Springer-Verlag, Berlin (2001).
19. Tsoumakas, G., Vlahavas, I.: Distributed data mining. In: Erickson J. (ed.) Database Technologies: Concepts, Methodologies, Tools, and Applications, pp. 157-171. IGI Global, Hershey (2009).
20. Kargupta, H., Byung-Hoon, D. H., Johnson, E.: Collective Data Mining: A New Perspective Toward Distributed Data Analysis. In: Kargupta, H., Chan, P. (eds.) Advances in Distributed and Parallel Knowledge Discovery, AAAI Press/The MIT Press, Menlo Park (1999).
21. Dietterich, T.: Ensemble methods in machine learning. In: Gayar, N. E., Kittler, J., Roli, F. (eds.) Multiple classifier systems, pp. 1-15. Springer, New York (2000).
22. Guo, Y., Sutiwaraphun, J.: Probing knowledge in distributed data mining. In: Zhong, N., Zhou, L. (eds.) Methodologies for Knowledge Discovery and Data Mining, pp. 443-452. Springer, Berlin (1999).
23. Hansen, L. K., Salamon, P.: Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 12(10), 993-1001 (1990).
24. Chan, P. K., Stolfo, S. J.: Toward parallel and distributed learning by meta-learning. In: AAAI Workshop in Knowledge Discovery in Databases, pp. 227-240 (1993).
25. Kittler, J.: Combining classifiers: a theoretical framework. Pattern Anal. Appl. 1(1), 18-27 (1998).
26. Ho, T. K., Hull, J. J., Srihari, S. N.: Decision combination in multiple classifier systems. IEEE Trans. Pattern Anal. Mach. Intell. 16(1), 66-75 (1994).
27. Kittler, J., Hatef, M., Duin, R. P. W., Matas, J.: On combining classifiers. IEEE Trans. Pattern Anal. Mach. Intell. 20(3), 226-239 (1998).
28. Wolpert, D. H.: Stacked generalization. Neural Netw. 5(2), 241-259 (1992).
29. Breiman, L.: Pasting small votes for classification in large databases and on-line. Mach. Learn. 36(1), 85-103 (1999).
30. Breiman. L.: Out-of-bag estimation. Technical report. Available at ftp://ftp. stat. berkeley. edu/pub/users/breiman/OOBestimation. ps (1996).
31. Chawla, N., Hall, L., Bowyer, K., Moore, T., Kegelmeyer, W.: Distributed pasting of small votes. In: Gayar, N. E., Kittler, J., Roli, F. (eds.) Multiple Classifier Systems, pp. 52-61. Springer, New York (2002).
32. Tsoumakas G., Vlahavas, I.: Effective stacking of distributed classifiers. In: ECAI, pp. 340-344 (2002).
33. Lazarevic, A., Obradovic, Z.: Boosting algorithms for parallel and distributed learning. Distrib. Parallel Databases 11(2), 203-229 (2002).
34. Freund, Y., Schapire, R. E.: Experiments with a new boosting algorithm. In: International Conference on Machine Learning, pp. 148-156. Morgan Kaufmann Publishers, Inc., San Francisco (1996).
35. Hand, D. J., Mannila, H., Smyth, P.: Principles of data mining. The MIT press, Cambridge (2001).
36. Tsoumakas, G., Katakis, I., Vlahavas, I.: Effective voting of heterogeneous classifiers. In: Machine Learning: ECML, pp. 465-476 (2004).
37. Woods, K., Kegelmeyer, W. P. Jr., Bowyer, K.: Combination of multiple classifiers using local accuracy estimates. IEEE Trans. Pattern Anal. Mach. Intell. 19(4), 405-410 (1997).
38. Gama, J., Rodrigues, P. P., Sebastião, R.: Evaluating algorithms that learn from data streams. In: Proceedings of the 2009 ACM symposium on Applied Computing (ACM), pp. 1496-1500 (2009).
39. Urban, P., Défago, X., Schiper, A.: Neko: a single environment to simulate and prototype distributed algorithms. In: 15th International Conference on Information Networking, pp. 503-511. IEEE (2001).
40. Tsoumakas, G., Angelis, L., Vlahavas, I.: Clustering classifiers for knowledge discovery from physically distributed databases. Data Knowl. Eng. 49(3), 223-242 (2004).
41. Sonnenburg, S., Franc, V., Yom-Tov, E., Sebag, M.: PASCAL large scale Learning challenge. In: 25th International Conference on Machine Learning (ICML2008) Workshop. http://largescale. first. fraunhofer. de. J. Mach. Learn. Res. 10, 1937-1953 (2008).
42. Peteiro-Barral, D., Bolon-Canedo, V., Alonso-Betanzos, A., Guijarro-Berdinas, B., Sanchez-Marono, N.: Scalability analysis of filter-based methods for feature selection. In: Howlett R. (ed.) Advances in Smart Systems Research, vol. 2, no. 1, pp. 21-26. Future Technology Publications, Shoreham-by-sea, UK (2012).