Recovering the optimal solution by dual random projection

Journal of Machine Learning Research

Volumn 30, Issue , 2013, Pages 135-157

  Zhang, Lijun ^a   Mahdavi, Mehrdad ^a   Jin, Rong ^a   Yang, Tianbao ^b   Zhu, Shenghuo ^c  

^a Michigan State University   (United States)

^b GE GLOBAL RESEARCH   (United States)

^c NEC LABORATORIES AMERICA   (United States)

Author keywords

Dual solution; Low rank; Primal solution; Random projection

Indexed keywords

ALGORITHMS; MATRIX ALGEBRA; OPTIMAL SYSTEMS; OPTIMIZATION;

CLASSIFICATION PERFORMANCE; DUAL SOLUTIONS; HIGH DIMENSIONAL DATA; HIGH DIMENSIONAL SPACES; LOW RANK; LOW-DIMENSIONAL SUBSPACE; OPTIMIZATION PROBLEMS; RANDOM PROJECTIONS;

RECOVERY;

EID: 84898043777     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Conference Paper

References (38)

1. Dimitris Achlioptas. Database-friendly random projections: Johnson-lindenstrauss with binary coins. Journal of Computer and System Sciences, 66(4): 671-687, 2003.
2. Rosa I. Arriaga and Santosh Vempala. An algorithmic theory of learning: robust concepts and random projection. In Proceedings of the 40th Annual Symposium on Foundations of Computer Science, pages 616-623, 1999.
3. Maria-Florina Balcan, Avrim Blum, and Santosh Vempala. Kernels as features: On kernels, margins, and low-dimensional mappings. Machine Learning, 65(1): 79-94, 2006.
4. Ella Bingham and Heikki Mannila. Random projection in dimensionality reduction: applications to image and text data. In Proceedings of the 7th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 245-250, 2001.
5. Avrim Blum. Random projection, margins, kernels, and feature-selection. In Proceedings of the 2005 international conference on Subspace, Latent Structure and Feature Selection, pages 52-68, 2006.
6. J.M. Borwein, A.S. Lewis, J. Borwein, and AS Lewis. Convex analysis and nonlinear optimization: theory and examples. Springer New York, 2006.
7. Christos Boutsidis, Anastasios Zouzias, and Petros Drineas. Random projections for k-means clustering. In Advances in Neural Information Processing Systems 23, pages 298-306, 2010.
8. Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
9. Vladimir Braverman, Rafail Ostrovsky, and Yuval Rabani. Rademacher chaos, random eulerian graphs and the sparse johnson-lindenstrauss transform. ArXiv e-prints, arXiv: 1011.2590, 2010.
10. Emmanuel J. Candès and Michael B. Wakin. An introduction to compressive sampling. IEEE Signal Processing Magazine, 25(2): 21-30, 2008.
11. Nicolo Cesa-Bianchi and Gabor Lugosi. Prediction, Learning, and Games. Cambridge University Press, 2006.
12. Sanjoy Dasgupta and Yoav Freund. Random projection trees and low dimensional manifolds. In Proceedings of the 40th annual ACM symposium on Theory of computing, pages 537-546, 2008.
13. David L. Donoho. Compressed sensing. IEEE Transaction on Information Theory, 52: 1289-1306, 2006.
14. Xiaoli Zhang Fern and Carla E. Brodley. Random projection for high dimensional data clustering: a cluster ensemble approach. In Proceedings of the 20th International Conference on Machine Learning, pages 186-193, 2003.
15. Dmitriy Fradkin and David Madigan. Experiments with random projections for machine learning. In Proceedings of the 9th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 517-522, 2003.
16. Yoav Freund, Sanjoy Dasgupta, Mayank Kabra, and Nakul Verma. Learning the structure of manifolds using random projections. In Advances in Neural Information Processing Systems 20, pages 473-480, 2008.
17. Alex Gittens and Joel A. Tropp. Tail bounds for all eigenvalues of a sum of random matrices. ArXiv e-prints, arXiv: 1104.4513, 2011.
18. Navin Goel, George Bebis, and Ara Nefian. Face recognition experiments with random projection. In Proceedings of SPIE, pages 426-437, 2005.
19. Gene H. Golub and Charles F. Van Loan. Matrix computations, 3rd Edition. Johns Hopkins University Press, 1996.
20. Xin Guo and Ding-Xuan Zhou. An empirical feature-based learning algorithm producing sparse approximations. Applied and Computational Harmonic Analysis, 32(3): 389-400, 2012.
21. Isabelle Guyon and André Elisseeff. An introduction to variable and feature selection. Journal of Machine Learning Research, 3: 1157-1182, 2003.
22. N. Halko, P. G. Martinsson, and J. A. Tropp. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM Review, 53(2): 217-288, 2011.
23. Per Christian Hansen. Rank-deficient and discrete ill-posed problems: numerical aspects of linear inversion. Society for Industrial and Applied Mathematics, Philadelphia, PA, USA, 1998.
24. Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning. Springer Series in Statistics. Springer New York, 2009.
25. Elad Hazan and Satyen Kale. Beyond the regret minimization barrier: an optimal algorithm for stochastic strongly-convex optimization. In Proceedings of the 24th Annual Conference on Learning Theory (COLT), pages 421-436, 2011.
26. Elad Hazan, Tomer Koren, and Nati Srebro. Beating sgd: Learning svms in sublinear time. In Advances in Neural Information Processing Systems 24, pages 1233-1241, 2011.
27. Ming Ji, Tianbao Yang, Binbin Lin, Rong Jin, and Jiawei Han. A simple algorithm for semi-supervised learning with improved generalization error bound. In Proceedings of the 29th International Conference on Machine Learning (ICML-12), pages 1223-1230, 2012.
28. Samuel Kaski. Dimensionality reduction by random mapping: fast similarity computation for clustering. In Proceedings of the 1998 IEEE International Joint Conference on Neural Networks, Volume 1, pages 413-418, 1998.
29. Edo Liberty, Nir Ailon, and Amit Singer. Dense fast random projections and lean walsh transforms. In Proceedings of the 12th International Workshop on Randomization and Computation (RANDOM), pages 512-522, 2008.
30. Oldalric-Ambrym Maillard and Remi Munos. Linear regression with random projections. Journal of Machine Learning Research, 13: 2735-2772, 2012.
31. Rajeev Mowani and Prabhakar Raghavan. Randomized Algorithms. Cambridge University Press, 1995.
32. Yu. Nesterov. Smooth minimization of non-smooth functions. Mathematical Programming, 103(1): 127-152, 2005.
33. Saurabh Paul, Christos Boutsidis, Malik Magdon-Ismail, and Petros Drineas. Random projections for support vector machines. ArXiv e-prints, arXiv: 1211.6085, 2012.
34. Ali Rahimi and Benjamin Recht. Random features for large-scale kernel machines. In Advances in Neural Information Processing Systems 20, pages 1177-1184, 2008.
35. Shai Shalev-Shwartz and Yoram Singer. Online learning meets optimization in the dual. In Proceedings of 19th Annual Conference on Learning Theory (COLT), pages 423-437, 2006.
36. Qinfeng Shi, Chunhua Shen, Rhys Hill, and Anton van den Hengel. Is margin preserved after random projection? In Proceedings of the 29th International Conference on Machine Learning, 2012.
37. Santosh S. Vempala. The Random Projection Method. American Mathematical Society, 2004.
38. Shenghuo Zhu. A short note on the tail bound of wishart distribution. ArXiv e-prints, arXiv: 1212.5860, 2012.