Parallel approaches to machine learning - A comprehensive survey

Journal of Parallel and Distributed Computing

Volumn 73, Issue 3, 2013, Pages 284-292

  Upadhyaya, Sujatha R ^a  

^a INFOSYS TECHNOLOGIES LTD   (India)

Author keywords

Distributed and parallel machine learning; GPU; Map reduce

Indexed keywords

ARTIFICIAL INTELLIGENCE; GRAPHICS PROCESSING UNIT; IMAGE PROCESSING; LEARNING SYSTEMS; PARALLEL ARCHITECTURES; PROGRAM PROCESSORS;

COMPLEX PROCESSES; GRAPH TRAVERSALS; IMPROVE PERFORMANCE; MAP-REDUCE; PARALLEL APPLICATION; PARALLEL MACHINE; PERFORMANCE ENHANCEMENTS; TIME-PERIODS;

LEARNING ALGORITHMS;

EID: 84880780351     PISSN: 07437315     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jpdc.2012.11.001     Document Type: Article

References (85)

1. R. Agrawal, et al., Fast discovery of association rules, in: Advances in Knowledge Discovery and Data Mining, AAAI Press, 1996.
2. R. Agrawal, J. Shafer, Parallel mining of association rules, in: IEEE TKDE, 1996.
3. Y. Ben-Haim, E. Yom-Tov, A streaming parallel decision tree algorithm, Journal of Machine Learning (2010).
4. J. Bolz, I. Farmer, E. Grinspun, P. Schröder, Sparse matrix solvers on the GPU: conjugate gradients and multigrid, in: ACM SIGGRAPH, 2003.
5. L. Bottou, O. Chapelle, D. DeCoste, J. Weston (Eds.), Large Scale Kernel Machines, The MIT Press, 2007.
6. K.W. Bowyer, L. Hall, T. Moore, N. Chawla, A parallel decision tree builder for mining very large visualization datasets, in: IEEE Conference on Systems, Man and Cybernetics, 2000.
7. L. Boxer, R. Miller, A. Rau-Chaplin, Scalable parallel algorithms for geometric pattern recognition, Journal of Parallel and Distributed Computing (1999).
8. G. Buehrer, S. Parthasarathy, S. Tatikonda, Toward terabyte pattern mining - an architecture conscious solution, in: 12th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming, 2007.
9. F. Cao, A.K.H. Tung, A. Zhou, Scalable clustering using graphics processors, in: WAIM 2006, 2006.
10. A. Carpenter, CUSVM: a CUDA implementation of support vector classification and regression, 2009. http://patternsonascreen.net/cuSVMDesc.pdf.
11. B. Catanzaro, N. Sundaram, K. Keutzer, Fast support vector machine training and classification on graphics processors, in: ICML 2008, 2008.
12. B. Catanzaro, N. Sundaram, K. Keutzer, A map reduce framework for programming graphics processors, in: PACT 2008, 2008.
13. J.M. Cavanagh, T.E. Potok, X. Cui, Parallel latent semantic analysis using graphics processor unit, in: 11th Annual Conference Companion on Genetic and GECCO'09, Evolutionary Computation Conference: Late Breaking Papers, 2009.
14. S. Celis, D.R. Musicant, Weka parallel: machine learning in parallel, Technical Report, Carlton College, CS TR 2002, 2002.
15. J.ST. Charles, R.M. Patio, T.E. Potok, X. Cui, Flocking-based document clustering on the graphics processing unit, US department of energy, Journal of Undergraduate Research (2006).
16. S. Che, M. Boyer, J. Meng, D. Tarjan, J.W. Sheaffer, K. Skadron, A performance study of general-purpose applications on graphics processors using CUDA, Journal of Parallel and Distributed Computing (2008) 350-371.
17. D. Cheung, et al., A fast distributed algorithm for mining association rules, in: International Conference Parallel and Distributed Information Systems, 1996.
18. D. Cheung, K. Hu, S. Xia, Asynchronous parallel algorithm for mining association rules on shared-memory multi-processors, in: ACM Symposium on Parallel algorithms and Architectures, 1998.
19. D. Cheung, Y. Xiao, Effect of data skewness in parallel mining of association rules, in: Pacific-Asia Conference Knowledge Discovery and Data Mining, 1998.
20. C. Chu, S.K. Kim, Y. Lin, Y. Yu, G. Bradski, A.Y. Ng, K. Olukotun, Map-reduce for machine learning on multicore, in: NIPS, 2006.
21. M. Craus, A. Archip, A generalized parallel algorithm for frequent itemset mining, in: 12th WSEAS International Conference on Computers, 2008.
22. X. Cui, J. Beaver, J.S. Charles, T. Potok, The GPU enhanced parallel computing for large scale data clustering, in: Book Chapter of GPU Computing Gems 2010, Springer, 2010.
23. G. Dahl, A. McAvinney, T. Newhall, Parallelizing neural networks training for cluster systems', in: PDCN, 2008.
24. A. DI Blas, T. Kaldeywey,Why graphics processors will transform database processing? Data monster, 2009. http://spectrum.ieee.org/computing/software/ data-monster/0.
25. M. El-Hajj, O. Zaiane, Parallel leap: large-scale maximal, pattern mining in a distributed environment, in: ICPADS, 2006.
26. R. Fang, B. He, M. Lu, K. Yang, N.K. Govindaraju, Q. Luo, P.V. Sander, Gpuqp: query co-processing using graphics processors, in: ACM SIGMOD 2007.
27. W. Fang, K.K. Lau, M. Lu, X. Xiao, C.K. Lam, P.Y. Yang, B. He, Q. Luo, P.V. Sander, K. Yang, GPUMiner: parallel data mining on graphics processors, Technical Report HKUST-CS08-07, Oct 2008.
28. W. Fang, M. Lu, X. Xiao, B. He, Q. Luo, Frequent itemset mining on graphics processors, in: 4th International Workshop on Data Management On New Hardware, 2009.
29. P. Farber, K. Asanovic, Parallel neural network training on multi-spert, in: IEEE 3rd International Conference on Algorithms and Architectures for Parallel Processing, 1997.
30. M. Geary, H. Lee, D. Signorelli, J. Vaughan, Implementing extremely randomized trees in CUDA, Technical Report, Stanford University, 2009.
31. A. Ghoting, G. Buehrer, S. Parthasarathy, D. Kim, A. Nguyen, Y. Chen, P. Dubey, Cache-conscious frequent pattern mining on a modern processor, in: VLDB, 2005.
32. N. Govindaraju, J. Gray, R. Kumar, D. Manocha, GPUTeraSort: high performance graphics co-processor sorting for large database management, in: ACM SIGMOD, 2006.
33. N.K. Govindaraju, B. Lloyd, W. Wang, M. Lin, D. Manocha, Fast computation of database operations using graphics processors, in: ACM SIGMOD, 2004.
34. N.K. Govindaraju, N. Raghuvanshi, M. Henson, D. Tuft, D. Manocha, A cache-efficient sorting algorithm for database and data mining computations using graphics processors, Technical Report, University of North Carolina, 2005.
35. N.K. Govindaraju, N. Raghuvanshi, D. Manocha, Fast and approximate stream mining of quantiles and frequencies using graphics processors, in: ACM SIGMOD, 2005.
36. H.P. Graf, E. Cosatto, L. Bottou, I. Durdanovic, Vladimir Vapnik, Parallel support vector machine: the cascade SVM, in: NIPS, 2005.
37. S. Guha, S. Krishnan, S. Venkatasubramanian, Tutorial.: data visualization and mining using the GPU, KDD, 2005.
38. E.H. Han, G. Karypis, V. Kumar, Scalable parallel data mining for association rules, in: Proceeding of the ACM Conference Management of Data, 1997.
39. B. He, W. Fang, Q. Luo, N.K. Govindaraju, T. Wang, Mars: a map reduce framework on graphics processors, in: PACT 2008, 2008.
40. B. He, N.K. Govindaraju, Q. Luo, B. Smith, Efficient gather and scatter operations on graphics processors, in: SC 2007, 2007.
41. R. Jin, G. Agarwal, Communication and memory efficient parallel decision tree construction, in: Third SIAM Conference on Data Mining, 2003.
42. M.V. Joshi, G. Garypis, V. Kumar, ScalParC: a new scalable and efficient parallel classification algorithm for mining large datasets, in: International Parallel Processing Symposium, 1998.
43. E.S. Larsen, D. Mcallister, Fast matrix multiplies using graphics hardware, in: SC 2001, 2001.
44. J. Lin, C. Dyer, Data-Intensive Text Processing with Map Reduce, Morgan Kaufmann Book, 2010.
45. L. Liu, E. Li, Y. Zhang, Z. Tang, Optimization of frequent itemset mining on multiple-core processor, in: VLDB 2007, 2007.
46. H. Li, Y. Wang, D. Zhang, M. Zhang, E. Chang, PFP: parallel FP-growth for query recommendation, in: ACM Recommender Systems 2008.
47. L.N. Long, A. Gupta, Scalable massively parallel artificial neural networks, in: AIAA 2005, 2005.
48. B. Lu, M. Ito, Task decomposition and module combination based on class relations: a modular neural network for pattern classification, IEEE Transactions on Neural Networks 10 (5) (1999).
49. D.L. Ly, V. Paprotski, D. Yen, Neural networks on GPUS: restricted Boltzmann machines, Technical Report, University of Toronto, 2008.
50. A. Mooley, K. Murthy, H. Singh, DisMaRC: a distributed map reduce framework on CUDA, Technical Report, University of Texas, 2009.
51. A. Mueller, Fast sequential and parallel algorithms for association rule mining, a comparison, Technical Report, 1995.
52. G.J. Narlikar, A parallel, multi-threaded decision tree builder, Technical Report, CMU, CMU-CS-98-184, 1998.
53. M. Oldroyd, Optimization of Massively Parallel Neural Networks, Fultus Publishing, ISBN: 1596820101, 2004.
54. J.D. Owens, D. Luebke, N. Govindaraju, M. Harris, J. Kruger, A.E. Lefohn, T.J. Purcell, A survey of general-purpose computation on graphics hardware, in: EUROGRAPHICS 05, 2005.
55. J.D. Owens, D. Luebke, N. Govindaraju, M. Harris, J. Kruger, A.E. Lefohn, T.J. Purcell, A survey of general-purpose computation on graphics hardware, Computer Graphics Forum (2007).
56. B. Panda, J.S. Herbach, S. Basu, R.J. Bayardo, Massively parallel learning of tree ensembles with map reduce, in: VLDB, 2009.
57. J.S. Park, M. Chen, P.S. Yu, Efficient parallel data mining for association rules, in: ACM International Conference Information and Knowledge Management, 1995.
58. J.S. Park, M. Chen, P.S. Yu, An effective hash-based algorithm for mining association rules, in: ACM SIGMOD1995, 1995.
59. S. Parthasarathy, M.J. Zaki, M. Ogihara, W. Li, Parallel data mining for association rules on shared memory systems, Journal of Knowledge and Information Systems (2001).
60. R.U. Pedersen, M. Schoeberl, Short paper: object oriented machine learning with a multicore real-time java processor, in: JTRES 10, 2010.
61. N. Pilkington, H. Zen, An implementation of decision tree-based context clustering on graphics processing units, in: Interspeech 2010, 2010.
62. A.H. P. Nguyen, T.B. Ho, A distributed algorithm for mining association rules, in: The Third International Conference on Parallel and Distributed Computing, Applications and Technologies, 2002.
63. R.D. Prabhu, Gneuron: parallel neural networks with GPU, in: HiPC 2007, 2007.
64. M.C. Schats, C. Trapnell, Fast exact string matching on the GPU, Technical Report, Stanford, 2009.
65. U. Seiffert, Artificial neural networks on massively parallel computer hardware, in: ESANN'02, 2002.
66. T. Sharp, Implementing decision trees and forests on a GPU, in: ECCV 2008, 2008.
67. T. Shintani, M. Kitsuregawa, Hash-based parallel algorithms for mining association rules, in: International Conference on Parallel and Distributed Information Systems, 1996.
68. B.V. Srinivasan, Q. Hu, R. Duraiswami, GPUML: graphical processors for speeding kernel machines, in: Workshop on High Performance Analytics, Algorithms, Implementations and Applications, Seoul Conference on Data Mining, 2010.
69. K.K. Thearling, Massively parallel architectures and algorithms for time series analysis, in: L. Nadel, D. Stien (Eds.), Lectures in Complex Systems, Addison-Wesley, 1995.
70. A. Veloso, M. Erick Otey, S. Parthasarathy, W. Meira Jr., Parallel and distributed frequent itemset mining on dynamic datasets, in: HiPC, 2003.
71. Y. Wen, B. Lu, in: J. Wang, X. Liao, Z. Yi (Eds.), Hierarchical and Parallel Method for Training Support Vector Machines, in: ISNN LNCS, vol. 3496, 2005, pp. 881-886.
72. I. Wesley-Smith, A parallel artificial neural network implementation, in: NCUR, 2006.
73. R. Wu, B. Zhang, M. Hsu, GPU accelerated large scale analytics, Technical Report, 2009-38, 2009.
74. Y. Xia, X. Feng, V.K. Prasanna, Parallel evidence propagation on multicore processors, in: PACT 2009, 2009.
75. L. Yang, Pruning and visualizing generalized association rules in parallel coordinates, in: IEEE TKDE, 2005.
76. Y. Ye, C.-C. Chiang, A parallel apriori algorithm for frequent itemsets mining, in: SERA, 2006.
77. M.J. Zaki, Data mining parallel and distributed association mining: a survey, in: IEEE Concurrency, 1999.
78. M.J. Zaki, et al., Parallel data mining for association rules on shared-memory multi-processors, in: Supercomputing'96, 1996.
79. M.J. Zaki, et al., Parallel algorithms for fast discovery of association rules, in: Data Mining and Knowledge Discovery, 1997.
80. M.J. Zaki, et al., Parallel algorithms for discovery of association rules, in: KDD, 1997.
81. M.J. Zaki, C. Ho, R. Agrawal, Parallel classification for data mining on shared-memory multiprocessors, in: ICDE, 1999.
82. M. Zechner, M. Granitzer, Accelerating K means on the graphics processor via CUDA, in: First International Conference on Intensive Applications and Services, INTENSIVE'09, 2009.
83. Y. Zhang, F. Mueller, X. Cui, T.E. Potok, Data-intensive document clustering on GPU clusters, Journal of Parallel and Distributed Computing (2010).
84. Y. Zhang, F. Mueller, X. Cui, T. Potok, Large-scale multi-dimensional document clustering on GPU clusters, in: IEEE International Parallel & Distributed Processing Symposium IPDPS 2010, 2010.
85. Y. Zhang, F. Mueller, X.T. Potok, GPU accelerated text mining, in: Workshop on Exploiting Parallelism using GPUs and Other Hardware-Assisted Methods, March 2009.