Hybrid MPI/OpenMP parallel linear support vector machine training

Journal of Machine Learning Research

Volumn 10, Issue , 2009, Pages 1937-1953

  Woodsend, Kristian ^a   Gondzio, Jacek ^a  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

Author keywords

Hybrid parallelism; Interior point method; Largescale learning; Linear SVM training

Indexed keywords

AUGMENTED SYSTEM MATRIX; HESSIAN MATRICES; HYBRID PARALLELISM; INTERIOR POINT METHOD; INTERIOR POINT METHODS; LARGESCALE LEARNING; LINEAR SUPPORT VECTOR MACHINES; LINEAR SVM TRAINING; MACHINE LEARNING TECHNOLOGY; PARALLEL IMPLEMENTATIONS; PARALLEL PROCESSOR; QUADRATIC OPTIMIZATION PROBLEMS; TRAINING PROCESS;

EDUCATION; GEARS; IMAGE RETRIEVAL; LARGE SCALE SYSTEMS; MULTILAYER NEURAL NETWORKS; PROBLEM SOLVING;

SUPPORT VECTOR MACHINES;

EID: 70349417901     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (30)

1. Antoine Bordes, Léon Bottou, Patrick Gallinari, and Jason Weston. Solving multiclass support vector machines with LaRank. In Zoubin Ghahramani, editor, Proceedings of the 24th International Machine Learning Conference, pages 89-96, Corvallis, Oregon, 2007. OmniPress. URL http://leon.bottou.org/papers/ bordes-2007.
2. Alfredo Buttari, Julien Langou, Jakub Kurzak, and Jack Dongarra. A class of parallel tiled linear algebra algorithms for multicore architectures. Parallel Computing, 35(1):38-53, January 2009.
3. Edward Chang, Kaihua Zhu, Hao Wang, Hongjie Bai, Jian Li, Zhihuan Qiu, and Hang Cui. Parallelizing support vector machines on distributed computers. In J.C. Piatt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems 20, pages 257-264, Cambridge, MA, 2008. MIT Press.
4. Ronan Collobert, Samy Bengio, and Yoshua Bengio. A parallel mixture of SVMs for very large scale problems. Neural Computation, 14(5): 1105-1114, 2002.
5. Jian-xiong Dong, Adam Krzyzak, and Ching Suen. A fast parallel optimization for training support vector machine. In P. Perner and A. Rosenfeld, editors, Proceedings of 3rd International Conference on Machine Learning and Data Mining, pages 96-105, Leipzig, Germany, 2003. Springer Lecture Notes in Artificial Intelligence.
6. Igor Durdanovic, Eric Cosatto, and Hans-Peter Graf. Large scale parallel SVM implementation. In Léon Bottou, Olivier Chapelle, Dennis DeCoste, and Jason Weston, editors, Large Scale Kernel Machines, chapter 5, pages 105-138 MIT Press, 2007.
7. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research, 9:1871-1874, 2008.
8. Michael Ferris and Todd Munson. Interior point methods for massive support vector machines. SIAM Journal on Optimization, 13(3):783-804, 2003. (Pubitemid 36971112)
9. Shai Fine and Katya Scheinberg. Efficient SVM training using low-rank kernel representations. Journal of Machine Learning Research, 2:243-264, 2002.
10. Glenn Fung and Olvi L. Mangasarian. Proximal support vector machine classifiers. In F. Provost and R. Srikant, editors, Proceedings KDD-2001: Knowledge Discovery and Data Mining, August 26-29, 2001, San Francisco, CA, pages 77-86, New York, 2001. Asscociation for Computing Machinery.
11. Jacek Gondzio and Andreas Grothey. Parallel interior point solver for structured quadratic programs: Application to financial planning problems. Annals of Operations Research, 152(1): 319-339,2007.
12. Jacek Gondzio and Robert Sarkissian. Parallel interior point solver for structured linear programs. Mathematical Programming, 96(3):561-584, 2003.
13. Kazushige Goto and Robert A. van de Geijn. Anatomy of high-performance matrix multiplication. ACM Trans. Math. Softw., 34(3): 1-25, 2008.
14. Hans Peter Graf, Eric Cosatto, Léon Bottou, Igor Dourdanovic, and Vladimir Vapnik. Parallel support vector machines: the Cascade SVM. In Lawrence Saul, Yair Weiss, and Léon Bottou, editors, Advances in Neural Information Processing Systems. MIT Press, 2005. volume 17.
15. José R. Herrero. A Framework for Efficient Execution of Matrix Computations. PhD thesis, Universität Politécnica de Catalunya (UPC), July 2006.
16. Bo Kagström, Per Ling, and Charles van Loan. GEMM-based level 3 BLAS: high-performance model implementations and performance evaluation benchmark. ACM Trans. Math. Softw., 24 (3):268-302, 1998.
17. Glenn R. Luecke, Jae H. Yun, and Philip W. Smith. Performance of parallel cholesky factorization algorithms using bias. The Journal of Supercomputing, 6(3):315-329, December 1992.
18. MPI-Forum. MPI: A Message-Passing Interface Standard. University of Tennessee, Knoxville, Tennessee, 1.1 edition, June 1995. URL http://www.mpi-forum.org.
19. OpenMP Architecture Review Board. OpenMP Application Program Interface, 3.0 edition, May 2008. URL http://www.openmp.org.
20. Edgar Osuna, Robert Freund, and Federico Girosi. An improved training algorithm for support vector machines. In J. Principe, L. GiIe, N. Morgan, and E. Wilson, editors, Neural Networks for Signal Processing VII -Proceedings of the 1997 IEEE Workshop, pages 276-285. IEEE, 1997.
21. John Piatt. Fast training of support vector machines using sequential minimal optimization. In Bernhard Schölkopf, Christopher. J. C. Burges, and Alexander. J. Smola, editors, Advances in Kernel Methods: Support Vector Learning, pages 185-208. MIT Press, 1999.
22. Rolf Rabenseifner and Gerhard Wellein. Comparison of parallel programming models on clusters of SMP nodes. In Hans Georg Bock, Ekaterina Kostina, Hoang Xuan Phu, and Rolf Rannacher, editors, Modelling, Simulation and Optimization of Complex Processes: Proceedings of the International Conference on High Performance Scientific Computing, March 10-14, 2003, Hanoi, Vietnam, pages 409-426. Springer, 2003.
23. Lorna Smith and Mark Bull. Development of mixed mode MPI / OpenMP applications. Scientific Programming, 2001.
24. Soeren Sonnenburg, Vojtech Franc, Elad Yom-Tov, and Michèle Sebag. PASCAL large scale learning challenge, 2008. URL http ://largescale.first. fraunhof er.de/.
25. Amund Tveit and Havard Engum. Parallelization of the incremental proximal support vector machine classifier using a heap-based tree topology. Technical report, IDI, NTNU, Trondheim, 2003.
26. Vladimir Vapnik. Statistical Learning Theory. Wiley, 1998.
27. Kristian Woodsend and Jacek Gondzio. Exploiting separability in large-scale support vector machine training. Technical Report MS-07-1002, School of Mathematics, University of Edinburgh, August 2007. Submitted for publication. Available at http://www.maths.ed.ac.uk/-gondz10/reports/wgSVM.html.
28. Stephen J. Wright. Primal-Dual Interior-Point Methods. S.I.A.M., 1997.
29. Gaetano Zanghirati and Luca Zanni. A parallel solver for large quadratic programs in training support vector machines. Parallel Computing, 29(4):535-551, 2003.
30. Luca Zanni, Thomas Serafini, and Gaetano Zanghirati. Parallel software for training large scale support vector machines on multiprocessor systems. Journal of Machine Learning Research, 7: 1467-1492, 2006.