GPU-accelerated parallel hierarchical extreme learning machine on flink for big data

IEEE Transactions on Systems, Man, and Cybernetics: Systems

Volumn 47, Issue 10, 2017, Pages 2740-2753

  Chen, Cen ^a,b   Li, Kenli ^a,b   Ouyang, Aijia ^a,b,c   Tang, Zhuo ^a,b   Li, Keqin ^a,b,d  

^a HUNAN UNIVERSITY   (China)

^b National Supercomputing Center   (China)

^c ZUNYI NORMAL COLLEGE   (China)

^d STATE UNIVERSITY OF NEW YORK   (United States)

Author keywords

Big data; deep learning (DL); flink; GPGPU; hierarchical extreme learning machine (H ELM); parallel

Indexed keywords

APPROXIMATION ALGORITHMS; ARTIFICIAL INTELLIGENCE; CACHE MEMORY; CLUSTER COMPUTING; CLUSTERING ALGORITHMS; COMPUTER ARCHITECTURE; COMPUTER GRAPHICS; DATA HANDLING; GRAPHICS PROCESSING UNIT; KNOWLEDGE ACQUISITION; LEARNING ALGORITHMS; LEARNING SYSTEMS; MAPPING; NETWORK LAYERS; OPTIMIZATION; PROGRAM PROCESSORS;

COMPUTING PLATFORM; COMPUTING POWER; EXTREME LEARNING MACHINE; FEED-FORWARD NETWORK; JAVA VIRTUAL MACHINES; LARGE-SCALE DATASETS; PARTITIONING STRATEGIES; UNIVERSAL APPROXIMATION;

BIG DATA;

EID: 85018957128     PISSN: 21682216     EISSN: 21682232     Source Type: Journal    
DOI: 10.1109/TSMC.2017.2690673     Document Type: Article

References (37)

1. J. Kim and W. Lee, "Stochastic decision making for adaptive crowdsourcing in medical big-data platforms," IEEE Trans. Syst., Man, Cybern., Syst., vol. 45, no. 11, pp. 1471-1476, Nov. 2015
2. L. Liu and H. Jia, "Trust evaluation via large-scale complex serviceoriented online social networks," IEEE Trans. Syst., Man, Cybern., Syst., vol. 45, no. 11, pp. 1402-1412, Nov. 2015
3. G.-B. Huang, H. Zhou, X. Ding, and R. Zhang, "Extreme learning machine for regression and multiclass classification," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 42, no. 2, pp. 513-529, Apr. 2012
4. G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, "Extreme learning machine: A new learning scheme of feedforward neural networks," in Proc. IEEE Int. Joint Conf. Neural Netw., vol. 2. Budapest, Hungary, 2004, pp. 985-990
5. G.-B. Huang, Q.-Y. Zhu, and C.-K. Siew, "Extreme learning machine: Theory and applications," Neurocomputing, vol. 70, nos. 1-3, pp. 489-501, 2006
6. G.-B. Huang, L. Chen, and C.-K. Siew, "Universal approximation using incremental constructive feedforward networks with random hidden nodes," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 879-892, Jul. 2006
7. G.-B. Huang, M.-B. Li, L. Chen, and C.-K. Siew, "Incremental extreme learning machine with fully complex hidden nodes," Neurocomputing, vol. 71, nos. 4-6, pp. 576-583, 2008
8. J. Tang, C. Deng, and G.-B. Huang, "Extreme learning machine for multilayer perceptron," IEEE Trans. Neural Netw. Learn. Syst., vol. 27, no. 4, pp. 809-821, Apr. 2016
9. J. Dean and S. Ghemawat, "MapReduce: Simplified data processing on large clusters," Commun. ACM, vol. 51, no. 1, pp. 107-113, 2008
10. Y. Xun, J. Zhang, and X. Qin, "FiDoop: Parallel mining of frequent itemsets using mapreduce," IEEE Trans. Syst., Man, Cybern., Syst., vol. 46, no. 3, pp. 313-325, Mar. 2016
11. W.-P. Ding, C.-T. Lin, M. Prasad, S.-B. Chen, and Z.-J. Guan, "Attribute equilibrium dominance reduction accelerator (DCCAEDR) based on distributed coevolutionary cloud and its application in medical records," IEEE Trans. Syst., Man, Cybern., Syst., vol. 46, no. 3, pp. 384-400, Mar. 2016
12. Flink Programming Guide. Accessed on Jul. 1, 2016. [Online]. Available: http://flink.apache.org/ (2016)
13. M. Zaharia et al., "Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing," in Proc. 9th USENIX Conf. Netw. Syst. Design Implement., San Jose, CA, USA, 2012, p. 2
14. Cudnn. Accessed on Jul. 1, 2016. [Online]. Available: https://developer.nvidia.com/cudnn (2016)
15. A. Coates et al., "Deep learning with COTS HPC systems," in Proc. Int. Conf. Mach. Learn., Atlanta, GA, USA, 2013, pp. 1337-1345
16. C. Chen, K. Li, A. Ouyang, Z. Tang, and K. Li, "Gflink: An in-memory computing architecture on heterogeneous CPU-GPU clusters for big data," in Proc. Int. Conf. Parallel Process., Philadelphia, PA, USA, 2016, pp. 542-551
17. N.-Y. Liang, G.-B. Huang, P. Saratchandran, and N. Sundararajan, "A fast and accurate online sequential learning algorithm for feedforward networks," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1411-1423, Nov. 2006
18. H.-J. Rong, G.-B. Huang, N. Sundararajan, and P. Saratchandran, "Online sequential fuzzy extreme learning machine for function approximation and classification problems," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 39, no. 4, pp. 1067-1072, Aug. 2009
19. G. Huang, S. Song, J. N. D. Gupta, and C. Wu, "Semi-supervised and unsupervised extreme learning machines," IEEE Trans. Cybern., vol. 44, no. 12, pp. 2405-2417, Dec. 2014
20. Q. He, T. Shang, F. Zhuang, and Z. Shi, "Parallel extreme learning machine for regression based on MapReduce," Neurocomputing, vol. 102, pp. 52-58, Feb. 2013
21. B.Wang, S. Huang, J. Qiu, Y. Liu, and G. Wang, "Parallel online sequential extreme learning machine based on MapReduce," Neurocomputing, vol. 149, pp. 224-232, Feb. 2015
22. J. Xin et al., "Elm: Distributed extreme learning machine with MapReduce," World Wide Web, vol. 17, no. 5, pp. 1189-1204, 2014
23. R. A. V. D. Geijn and J. Watts, "SUMMA: Scalable universal matrix multiplication algorithm," Concurrency Comput. Pract. Exp., vol. 9, no. 4, p. 255-274, 1997
24. D. Schmidt, G. Ostrouchov, W.-C. Chen, and P. Patel, "Tight coupling of R and distributed linear algebra for high-level programming with big data," in Proc. SC Companion High Perform. Comput. Netw. Stor. Anal. (SCC), Salt Lake City, UT, USA, 2012, pp. 811-815
25. J. Choi, J. J. Dongarra, R. Pozo, and D. W. Walker, "ScaLAPACK: A scalable linear algebra library for distributed memory concurrent computers," in Proc. 4th Symp. Front. Massively Parallel Comput., McLean, VA, USA, 1992, pp. 120-127
26. S. Seo et al., "HAMA: An efficient matrix computation with the mapreduce framework," in Proc. 2nd Int. Conf. Cloud Comput. CloudCom, Indianapolis, IN, USA, Nov./Dec. 2010, pp. 721-726
27. Cublas Programming Guide. Accessed on Nov. 1, 2016. [Online]. Available: http://docs.nvidia.com/cuda/cublas/index.html (2016)
28. Cusparse Programming Guide. Accessed on Nov. 1, 2016. [Online]. Available: http://docs.nvidia.com/cuda/cusparse/index.html (2016)
29. K. Li, W. Yang, and K. Li, "Performance analysis and optimization for SpMV on GPU using probabilistic modeling," IEEE Trans. Parallel Distrib. Syst., vol. 26, no. 1, pp. 196-205, Jan. 2015
30. W. Yang, K. Li, Z. Mo, and K. Li, "Performance optimization using partitioned SpMV on GPUs and multicore CPUs," IEEE Trans. Comput., vol. 64, no. 9, pp. 2623-2636, Sep. 2015
31. T. Poggio and F. Girosi, "Networks for approximation and learning," Proc. IEEE, vol. 78, no. 9, pp. 1481-1497, Sep. 1990
32. H. White, Artificial Neural Networks: Approximation and Learning Theory. Cambridge, MA, USA: Blackwell, 1992
33. G.-B. Huang and L. Chen, "Convex incremental extreme learning machine," Neurocomputing, vol. 70, nos. 16-18, pp. 3056-3062, 2007
34. G. Huang, G.-B. Huang, S. Song, and K. You, "Trends in extreme learning machines: A review," Neural Netw., vol. 61, pp. 32-48, Jan. 2015
35. Y. Bengio, "Learning deep architectures for AI," Found. Trends Mach. Learn., vol. 2, no. 1, pp. 1-127, 2009
36. A. Beck and M. Teboulle, "A fast iterative shrinkage-thresholding algorithm for linear inverse problems," SIAM J. Imag. Sci., vol. 2, no. 1, pp. 183-202, 2009
37. Y. Bengio, A. Courville, and P. Vincent, "Representation learning: A review and new perspectives," IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 8, pp. 1798-1828, Aug. 2013