Scaling Support Vector Machines on modern HPC platforms

Journal of Parallel and Distributed Computing

Volumn 76, Issue , 2015, Pages 16-31

  You, Yang ^a   Fu, Haohuan ^a   Song, Shuaiwen Leon ^b   Randles, Amanda ^c   Kerbyson, Darren ^b   Marquez, Andres ^b   Yang, Guangwen ^a   Hoisie, Adolfy ^b  

^a TSINGHUA UNIVERSITY   (China)

^b PACIFIC NORTHWEST NATIONAL LABORATORY   (United States)

^c LAWRENCE LIVERMORE NATIONAL LABORATORY   (United States)

Author keywords

Dynamic modeling; Machine learning models; Multi many core architectures; Optimization techniques; Performance analysis; Support Vector Machine

Indexed keywords

ARTIFICIAL INTELLIGENCE; BIG DATA; COMPUTER ARCHITECTURE; DATA MINING; DESIGN; DYNAMIC MODELS; LEARNING SYSTEMS; MICROPROCESSOR CHIPS; PROGRAM PROCESSORS; SCHEDULING;

CROSS PLATFORM PERFORMANCE; HIGH PERFORMANCE COMPUTING; MACHINE LEARNING MODELS; MANY-CORE ARCHITECTURE; MULTI CORE AND MANY CORES; MULTI-LEVEL PARALLELISM; OPTIMIZATION TECHNIQUES; PERFORMANCE ANALYSIS;

SUPPORT VECTOR MACHINES;

EID: 85027956569     PISSN: 07437315     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jpdc.2014.09.005     Document Type: Article

References (47)

1. C. Benthin, I. Wald, S. Woop, M. Ernst, and W. Mark Combining single and packet-ray tracing for arbitrary ray distributions on the intel mic architecture IEEE Trans. Vis. Comput. Graphics 18 9 2012 1438 1448
2. R.D. Blumofe, C.F. Joerg, B.C. Kuszmaul, C.E. Leiserson, K.H. Randall, and Y. Zhou Cilk: An Efficient Multithreaded Runtime System, Vol. 30 1995 ACM
3. L.J. Cao, S. Keerthi, C.-J. Ong, J. Zhang, U. Periyathamby, X.J. Fu, and H. Lee Parallel sequential minimal optimization for the training of support vector machines Neural Networks, IEEE Transactions on 17 4 2006 1039 1049
4. B. Catanzaro, N. Sundaram, and K. Keutzer Fast support vector machine training and classification on graphics processors Proceedings of the 25th International Conference on Machine Learning 2008 ACM 104 111
5. E.Y. Chang Psvm: parallelizing support vector machines on distributed computers Foundations of Large-Scale Multimedia Information Management and Retrieval 2011 Springer 213 230
6. C.-C. Chang, and C.-J. Lin Libsvm: a library for support vector machines ACM Trans. Intel. Syst. Technol. 2 3 2011 27
7. G. Chrysos, Knights corner, intels first many integrated core (mic) architecture product, in: Hot Chips, vol. 24, 2012.
8. G. Chrysos, Intel xeon phi coprocessor - the architecture (2014). URL http://software.intel.com/en-us/articles/intel-xeon-phi-coprocessor-codename-knights-corner.
9. R. Collobert, S. Bengio, and Y. Bengio A parallel mixture of svms for very large scale problems Neural Comput. 14 5 2002 1105 1114
10. C. Cortes, and V. Vapnik Support-vector networks Mach. Learn. 20 3 1995 273 297
11. L. Dagum, and R. Menon Openmp: an industry standard api for shared-memory programming Comput. Sci. Eng., IEEE 5 1 1998 46 55
12. J. Dongarra, Top500 june 2013 list. (2012). http://www.top500.org/lists/2013/06/.
13. ®many integrated core architecture High Performance Computing and Simulation (HPCS), 2012 International Conference on 2012 IEEE 365 366
14. R.-E. Fan, P.-H. Chen, and C.-J. Lin Working set selection using second order information for training support vector machines J. Mach. Learn. Res. 6 2005 1889 1918
15. H.P. Graf, E. Cosatto, L. Bottou, I. Dourdanovic, and V. Vapnik Parallel support vector machines: The cascade svm Advances in neural information processing systems 17 2004 521 528
16. I. Guyon, S. Gunn, A. Ben-Hur, and G. Dror Result analysis of the nips 2003 feature selection challenge Adv. Neural Inf. Process. Syst. 2004 545 552
17. ®many integrated core architecture Euro-Par 2011: Parallel Processing Workshops 2012 Springer 375 384
18. S. Herrero-Lopez, J.R. Williams, and A. Sanchez Parallel multiclass classification using svms on gpus Proceedings of the 3rd Workshop on General-Purpose Computation on Graphics Processing Units 2010 ACM 2 11
19. M. Hu, and W. Hao A parallel approach for svm with multi-core cpu Computer Application and System Modeling (ICCASM), 2010 International Conference on, vol. 15 2010 10.1109/ICCASM.2010.5622595 pp. V15-373-V15-377
20. J.J. Hull A database for handwritten text recognition research IEEE Trans. Pattern Anal. Mach. Intell. 16 5 1994 550 554
21. M. Intel, Intel math kernel library, 2007.
22. T. Joachims Text Categorization with Support Vector Machines: Learning with Many Relevant Features 1998 Springer
23. T. Joachims, Making large scale svm learning practical.
24. S.S. Keerthi, S.K. Shevade, C. Bhattacharyya, and K.R.K. Murthy Improvements to platt's smo algorithm for svm classifier design Neural Comput. 13 3 2001 637 649
25. C. Leslie, E. Eskin, W.S. Noble, The spectrum kernel: A string kernel for svm protein classification, in: Proceedings of the Pacific Symposium on Biocomputing, vol. 7, Hawaii, USA., 2002, pp. 566-575.
26. T.-K. Lin, and S.-Y. Chien Support vector machines on gpu with sparse matrix format Machine Learning and Applications (ICMLA), 2010 Ninth International Conference on 2010 IEEE 313 318
27. X. Liu, M. Smelyanskiy, E. Chow, and P. Dubey Efficient sparse matrix-vector multiplication on x86-based many-core processors Proceedings of the 27th International ACM Conference on International Conference on Supercomputing 2013 ACM 273 282
28. G.S. Mann, and A. McCallum Simple, robust, scalable semi-supervised learning via expectation regularization Proceedings of the 24th International Conference on Machine Learning 2007 ACM 593 600
29. M.-S. group, Mic-svm v0.1 (2013). URL http://github.com/You1991/MICSVM.
30. B.L. Milenova, J.S. Yarmus, M.M. Campos, Svm in oracle database 10g: removing the barriers to widespread adoption of support vector machines, in: Proceedings of the 31st International Conference on Very Large Data Bases, VLDB Endowment, 2005, pp. 1152-1163.
31. T. Mudge Power: A first-class architectural design constraint Computer 34 4 2001 52 58
32. NVIDIA Nvidia fermi compute architecture whitepaper (2009). URL http://www.nvidia.com/content/PDF.
33. Nvidia Nvidia kepler gk110 architecture whitepaper (2012). URL http://www.nvidia.com/content/PDF.
34. E. Osuna, R. Freund, and F. Girosi An improved training algorithm for support vector machines Neural Networks for Signal Processing [1997] VII. Proceedings of the 1997 IEEE Workshop 1997 IEEE 276 285
35. J.C. Platt, 12 fast training of support vector machines using sequential minimal optimization.
36. R. Rahman, Intel xeon phi coprocessor vector microarchitecture (2014). URL http://software.intel.com/en-us/articles/intel-xeon-phi-coprocessor-vector-microarchitecture.
37. N. Satish, C. Kim, J. Chhugani, A. Nguyen, V. Lee, D. Kim, and P. Dubey Fast sort on cpus, gpus and intel mic architectures, Tech. Rep., Technical Report, Intel 2010
38. N. Satish, C. Kim, J. Chhugani, H. Saito, R. Krishnaiyer, M. Smelyanskiy, M. Girkar, and P. Dubey Can traditional programming bridge the ninja performance gap for parallel computing applications? Proceedings of the 39th International Symposium on Computer Architecture 2012 IEEE Press 440 451
39. B. Schölkopf, and A.J. Smola Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond 2002 MIT press
40. S. Sonnenburg, V. Franc, E. Yom-Tov, M. Sebag, Pascal large scale learning challenge, in: 25th International Conference on Machine Learning (ICML2008) Workshop. http://www.largescale. first. fraunhofer. de. J. Mach. Learn. Res, vol. 10, 2008, pp. 1937-1953.
41. C. Su, D. Li, D.S. Nikolopoulos, K.W. Cameron, B.R.d. Supinski, and E.A. Leon Model-based, memory-centric performance and power optimization on numa multiprocessors Proceedings of the 2012 IEEE International Symposium on Workload Characterization (IISWC) IISWC '12 2012 IEEE Computer Society Washington, DC, USA 164 173 10.1109/IISWC.2012.6402921
42. F.E. Tay, and L. Cao Application of support vector machines in financial time series forecasting Omega 29 4 2001 309 317
43. S. Tyree, J.R. Gardner, K.Q. Weinberger, K. Agrawal, J. Tran, Parallel support vector machines in practice, arXiv preprint arXiv:1404.1066.
44. V.N. Vapnik, and S. Kotz Estimation of Dependences Based on Empirical data, Vol. 41 1982 Springer-Verlag New York
45. Z. Wang, and M.F. O'Boyle Mapping parallelism to multi-cores: a machine learning based approach Proceedings of the 14th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming PPoPP '09 2009 ACM New York, NY, USA 75 84 10.1145/1504176.1504189 URL http://doi.acm.org/10.1145/1504176.1504189
46. S. Williams, L. Oliker, R. Vuduc, J. Shalf, K. Yelick, and J. Demmel Optimization of sparse matrix-vector multiplication on emerging multicore platforms Parallel Comput. 35 3 2009 178 194
47. Y. You, S. Song, H. Fu, A. Marquez, M. Dehnavi, K. Barker, K.W. Cameron, A. Randles, and G. Yang Mic-svm: Designing a highly efficient support vector machine for advanced modern multi-core and many-core architectures Parallel & Distributed Processing (IPDPS), 2014 IEEE 28th International Symposium on 2014 IEEE