Parallel software for training large scale support vector machines on multiprocessor systems

Journal of Machine Learning Research

Volumn 7, Issue , 2006, Pages 1467-1492

  Zanni, Luca ^a   Serafini, Thomas ^a   Zanghirati, Gaetano ^b  

^a UNIVERSITY OF MODENA AND REGGIO EMILIA   (Italy)

^b UNIVERSITY OF FERRARA   (Italy)

Author keywords

Decomposition techniques; Gradient projection methods; Large scale quadratic programs; Parallel computation; Support vector machines

Indexed keywords

BENCHMARKING; CODES (SYMBOLS); COMPUTATION THEORY; COMPUTER SOFTWARE; ITERATIVE METHODS; MULTIPROCESSING SYSTEMS; PROBLEM SOLVING; QUADRATIC PROGRAMMING;

DECOMPOSITION TECHNIQUES; GRADIENT PROJECTION METHODS; LARGE SCALE QUADRATIC PROGRAMS; MULTIPROCESSOR SYSTEMS; PARALLEL COMPUTATION; SUPPORT VECTOR MACHINES;

LEARNING SYSTEMS;

EID: 33745784205     PISSN: 15337928     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (37)

1. Jonathan Barzilai and Jonathan M. Borwein. Two-point step size gradient methods. IMA Journal of Numerical Analysis, 8(1):141-148, 1988.
2. Dimitri P. Bertsekas. Nonlinear Programming. Athena Scientific, Belmont, MA, second edition, 1999.
3. Ernesto G. Birgin, José Mario Martinez, and Marcos Raydan. Nonmonotone spectral projected gradient methods on convex sets. SIAM Journal on Optimization, 10(4):1196-1211, 2000.
4. Bernhard E. Boser, Isabelle Guyon, and Vladimir N. Vapnik. A training algorithm for optimal margin classifiers. In David Haussler, editor, Proceedings of the 5th Annual ACM Workshop on Computational Learning Theory, pages 144-152. ACM Press, Pittsburgh, PA, 1992.
5. Chih-Chung Chang and Chih-Jen Lin. LIBSVM: a library for support vector machines, 2001. URL http://www.csie.ntu.edu.tw/~cjlin/libsvm.
6. Pai-Hsuen Chen, Rong-En Fan, and Chih-Jen Lin. A study on SMO-type decomposition methods for support vector machines. Technical report, Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan, 2005.
7. To appear on IEEE Transaction on Neural Networks, 2006.
8. Ronan Collobert and Samy Bengio. SVMTorch: Support vector machines for large-scale regression problems. Journal of Machine Learning Research, 1:143-160,2001.
9. Ronan Collobert, Samy Bengio, and Yoshua Bengio. A parallel mixture of SVMs for very large scale problems. Neural Computation, 14(5):1105-1114, 2002.
10. Nello Cristianini and John Shawe-Taylor. An Introduction to Support Vector Machines and other Kernel-Based Learning Methods. Cambridge University Press, 2000.
11. Yu-Hong Dai and Roger Fletcher. New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds. Mathematical Programming, 106(3):403-421, 2006.
12. Yu-Hong Dai and Roger Fletcher. Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming. Numerische Mathematik, 100(1):21-47, 2005.
13. Jian-Xiong Dong, Adam Krzyzak, and Ching Y. Suen. A fast parallel optimization for training support vector machine. In P. Perner and A. Rosenfeld, editors, Proceedings of 3rd International Conference on Machine Learning and Data Mining, volume 17, pages 96-105. Springer Lecture Notes in Artificial Intelligence, Leipzig, Germany, 2003.
14. Rong-En Fan, Pai-Hsuen Chen, and Chih-Jen Lin. Working set selection using second order information for training Support Vector Machines. Journal of Machine Learning Research, 6:1889-1918, 2005.
15. Hans Peter Graf, Eric Cosatto, Léon Bottou, Igor Dourdanovic, and Vladimir N. Vapnik. Parallel support vector machines: the Cascade SVM. In Lawrence Saul, Yair Weiss, and Léon Bottou, editors, Advances in Neural Information Processing Systems, volume 17. MIT Press, 2005.
16. Chih-Wei Hsu and Chih-Jen Lin. A simple decomposition method for support vector machines. Machine Learning, 46:291-314, 2002.
17. Don Hush and Clint Scovel. Polynomial-time decomposition algorithms for support vector machines. Machine Learning, 51:51-71, 2003.
18. Throstem Joachims. Making large-scale SVM learning practical. In Bernard Schölkopf, C.J.C. Burges, and Alex Smola, editors, Advances in Kernel Methods - Support Vector Learning. MIT Press, Cambridge, MA, 1998.
19. S. Sathiya Keerthi and Elmer G. Gilbert. Convergence of a generalized SMO algorithm for SVM classifier design. Machine Learning, 46:351-360, 2002.
20. Chih-Jen Lin. On the convergence of the decomposition method for support vector machines. IEEE Transactions on Neural Networks, 12:1288-1298, 2001a.
21. Chih-Jen Lin. Linear convergence of a decomposition method for support vector machines. Technical report, Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan, 2001b.
22. Chih-Jen Lin. Asymptotic convergence of an SMO algorithm without any assumptions. IEEE Transactions on Neural Networks, 13:248-250, 2002.
23. Message Passing Interface Forum. MPI: A message-passing interface standard (version 1.2). International Journal of Super computing Applications, 8(3/4), 1995. URL http://www.mpi-forum.org. Also available as Technical Report CS-94-230, Computer Science Dept., University of Tennesse, Knoxville, TN.
24. Bruce A. Murtagh and Michael A. Saunders. MINOS 5.5 user's guide. Technical report, Department of Operation Research, Stanford University, Stanford CA, 1998.
25. Edgar Osuna, Robert Freund, and Girosi Federico. Training support vector machines: an application to face detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR97), pages 130-136. IEEE Computer Society, New York, 1997.
26. light algorithm. Optimization Methods and Software, 20:317-334, 2005.
27. Panos M. Pardalos and Naina Kovoor. An algorithm for a singly constrained class of quadratic programs subject to upper and lower bounds. Mathematical Programming, 46:321-328, 1990.
28. John C. Platt. Fast training of support vector machines using sequential minimal optimization. In Bernard Schölkopf, C.J.C. Burges, and Alex Smola, editors, Advances in Kernel Methods -Support Vector Learning. MIT Press, Cambridge, MA, 1998.
29. Valeria Ruggiero and Luca Zanni. A modified projection algorithm for large strictly convex quadratic programs. Journal of Optimization Theory and Applications, 104(2):281-299, 2000a.
30. Valeria Ruggiero and Luca Zanni. Variable projection methods for large convex quadratic programs. In Donato Trigiante, editor, Recent Trends in Numerical Analysis, volume 3 of Advances in the Theory of Computational Mathematics, pages 299-313. Nova Science Publisher, 2000b.
31. Thomas Serafini and Luca Zanni. On the working set selection in gradient projection-based decomposition techniques for support vector machines. Optimization Methods and Software, 20:583-596, 2005.
32. Thomas Serafini, Gaetano Zanghirati, and Luca Zanni. Gradient projection methods for quadratic programs and applications in training support vector machines. Optimization Methods and Software, 20:353-378, 2005.
33. Alex J. Smola. pr_LOQO optimizer, 1997. URL http://www.kernel-machines.org/code/prloqo.tar.gz.
34. Ivor W. Tsang, James T. Kwok, and Pak-Ming Cheung. Core vector machines: fast SVM training on very large data sets. Journal of Machine Learning Research, 6(4):363-392, 2005.
35. Vladimir N. Vapnik. Statistical Learning Theory. John Wiley and Sons, New York, 1998.
36. Gaetano Zanghirati and Luca Zanni. A parallel solver for large quadratic programs in training support vector machines. Parallel Computing, 29:535-551, 2003.
37. Luca Zanni. An improved gradient projection-based decomposition technique for support vector machines. Computational Management Science, 3(2):131-145, 2006.