Random projections for linear support vector machines

ACM Transactions on Knowledge Discovery from Data

Volumn 8, Issue 4, 2014, Pages

  Paul, Saurabh ^a   Boutsidis, Christos ^b   Magdon Ismail, Malik ^a   Drineas, Petros ^a  

^a RENSSELAER POLYTECHNIC INSTITUTE   (United States)

^b YAHOO RESEARCH   (United States)

Author keywords

Classification; dimensionality reduction; support vector machines

Indexed keywords

CLASSIFICATION (OF INFORMATION); DIMENSIONALITY REDUCTION; MATRIX ALGEBRA;

D-DIMENSIONAL SPACES; DIMENSION REDUCTION TECHNIQUES; HIGH PROBABILITY; LINEAR SUPPORT VECTOR MACHINES; MINIMUM ENCLOSING BALL; RANDOM PROJECTIONS; RELATIVE ERRORS; SYNTHETIC DATA;

SUPPORT VECTOR MACHINES;

EID: 84908246259     PISSN: 15564681     EISSN: 1556472X     Source Type: Journal    
DOI: 10.1145/2641760     Document Type: Conference Paper

References (32)

1. D. Achlioptas. 2003. Database-friendly random projections: Johnson-Lindenstrauss with binary coins. Journal of Computer and System Sciences 66, 4 (2003), 671-687.
2. N. Ailon and B. Chazelle. 2006. Approximate nearest neighbors and the fast Johnson-Lindenstrauss transform. In Proceedings of the 38th Annual ACM Symposium on Theory of Computing. 557-563.
3. N. Ailon and E. Liberty. 2008. Fast dime nsion reduction using Rademacher series on dual BCH codes. In Proceedings of the 19th Annual ACM-SIAM Symposium on Discrete Algorithms. 1-9.
4. J. L. Balcazar, Y. Dai, and O. Watanabe. 2001. A random sampling technique for training support vector machines. In Proceedings of the 12th International Conference on Algorithmic Learning Theory. 119-134.
5. J. L. Balczar, Y. Dai, and O.Watanabe. 2002. Provably fast support vec tor regression using random sampling. In Proceedings of SIAM Workshop in Discrete Mathematics and Data Mining.
6. A. Blum. 2006. Random projection, margins, kernels, and feature-selection. In Proceedings of the International Conference on Subspace, Latent Structure and Feature Selection. 52-68.
7. D. Cai, X. He, J. Han, and H.-J. Zhang. 2006. Orthogonal laplacianfaces for face recognition. IEEE Transactions on Image Processing 15, 11 (2006), 3608-3614.
8. C.-C. Chang and C.-J. Lin. 2011. LIBSVM: A library for support vector machines. ACMTransactions on Intelligent Systems and Technology 2, 3 (2011), 27:1-27:27. Software available at http://www.csie.ntu.edu.tw/?cjlin/libsvm.
9. K. L. Clarkson and D. W. Woodruff. 2013. Low rank approximation and regression in input sparsity time. In Proceedings of the 45th ACM Symposium on the Theory of Computing.
10. K. Crammer and Y. Singer. 2000. On the learnability and design of output codes for multi-class problems. In Computational Learning Theory. 35-46.
11. N. Cristianini and J. Shawe-Taylor. 2000. Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press.
12. S. Dasgupta,A. Gupta 2003. An elementary proof of a theorem of Johnson and Lindenstrauss Random Structures and Algorithms 22, 1, 2003, 60-65.
13. D. Davidov, E. Gabrilovich, and S. Markovitch. 2004. Parameterized generation of labeled datasets for text categorization based on a hiera rchical directory. In Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. 250-257. .
14. P. Drineas, M. W. Mahoney, S. Muthukrishnan, and T. Sarlos. 2011. Faster least squares approximation. Numerische Mathematik 117, 2 (2011), 219-249.
15. R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. 2008. LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research (2008), 1871-1874.
16. P. Indyk and R. Motwani. 1998. Approximate nearest neighbors: Towards remo ving the curse of dimensionality. In Proceedings of the 30th Annual ACM Symposium on Theory of Computing. 604-613.
17. V. Jethava, K. Suresh, C. Bhattacharyya, and R. Hariharan. 2009. Randomized algorithms for large scale SVMs. CoRR abs/0909.3609 (2009). http://arxiv.org/abs/0909.3609.
18. S. Krishnan, C. Bhattacharyya, and R. Hariharan. 2008. A randomized algorithm for large scale support vector learning. In Advances in 20th Neural Information Processing Systems. 793-800.
19. D. D. Lewis, Y. Yang, T. G. Rose, and F. Li. 2004. RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research (2004), 361-397.
20. J. Z. Li, D. M. Absher, H. Tang, A. M. Southwick, A. M. Casto, S. Ramachandran, H. M. Cann, G. S. Barsh, M. Feldman, L. L. Cavalli-Sforza, and R. M. Myers. 2008. Worldwide human relationships inferred from genome-wide patterns of variation. Science 319, 5866 (2008), 1100-1104.
21. P. Li, T. J. Hastie, and W. K. Church. 2006. Very sparse random projections. In Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 287-296.
22. A. Magen and A. Zouzias. 2011. Low rank matrix-valued Chernoff bounds and approximate matrix multiplication. In Proceedings of the 22nd Annual ACM-SIAM Symposium on Discrete Algorithms. 1422-1436.
23. X. Meng and M. W. Mahoney. 2013. Low-distortion subspace embeddings in input-sparsity time and applications to robust linear regression. In Proceedings of the 45th ACM Symposium on the Theory of Computing.
24. J. Nelson and H. L. Nguyen. 2013. OSNAP: Faster numerical linear algebra algorithms via sparser subspace embeddings. In Proceedings of the 54th Annual IEEE Symposium on Foundations of Computer Science (FOCS'13).
25. P. Paschou, J. Lewis, A. Javed, and P. Drineas. 2010. Ancestry informative markers for fine-scale individual assignment to worldwide populations. Journal of Medical Genetics 47, 12 (2010), 835-47.
26. S. Paul, C. Boutsidis, M. Magdon-Ismail, and P. Drineas. 2013. Random projections for support vector machines. In Proceedings of the 16th International Conference on Artificial Intelligence & Statistics, JMLR W&CP. 498-506.
27. D. T. Ross, U. Scherf, M. B. Eisen, C. M. Perou, P. Spellman, V. Iyer, S. S. Jeffrey,M. Van de Rijn, M.Waltham, A. Pergamenschikov, J. C. F Lee, D. Lashkari, D. Shalon, T. G. Myers, J. N. Weinstein, D. Botstein, and P. O. Brown. 2000. Systematic variation in gene expression patterns in human cancer cell lines. Nature Genetics 24, 3 (2000), 227-234.
28. Q. Shi, J. Petterson, G. Dror, J. Langford, A. Smola, and S. V. N. Vishwanathan. 2009. Hash kernels for structured data. Journal of Machine Learning Research 10 (2009), 2615-2637.
29. Q. Shi, C. Shen, R. Hill, and A. V. D. Hengel. 2012. Is margin preserved after random projection?. In Proceedings of 29th International Conference on Machine Learning. 591-598.
30. V. N. Vapnik and A. Chervonenkis. 1971. On the uniform convergence of relative frequencies of events to their probabilities. Theory of Probability and its Applications 16 (1971), 264-280.
31. V. N. Vapnik. 1998. Statistical learning theory. Theory of Probability and its Applications 16 (1998), 264-280.
32. L. Zhang, M. Mahdavi, R. Jin, and T. Yang. 2013. Recovering optimal solution by dual random projection. In Proceedings of the Conference on Learning Theory (COLT'13) JMLR W&CP, Vol. 30. 135-157.