Leveraged volume sampling for linear regression

Advances in Neural Information Processing Systems

Volumn 2018-December, Issue , 2018, Pages 2505-2514

  Derezinski, Michał ^a   Warmuth, Manfred K ^a   Hsu, Daniel ^b  

^a University of California Santa Cruz   (United States)

^b Columbia University ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

HIGH PROBABILITY; I.I.D. SAMPLINGS; LINEAR REGRESSION PROBLEMS; RANDOM MATRICES; REJECTION SAMPLINGS; SUM OF SQUARES; UNBIASED ESTIMATES; UNBIASED ESTIMATOR;

MATRIX ALGEBRA;

EID: 85064832658     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (33)

1. Nir Ailon and Bernard Chazelle. The fast Johnson-Lindenstrauss transform and approximate nearest neighbors. SIAM Journal on computing, 39(1):302-322, 2009.
2. Zeyuan Allen-Zhu, Yuanzhi Li, Aarti Singh, and Yining Wang. Near-optimal design of experiments via regret minimization. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 126-135, International Convention Centre, Sydney, Australia, 2017.
3. T Ando, Roger A. Horn, and Charles R. Johnson. The singular values of a Hadamard product: A basic inequality. Journal of Linear and Multilinear Algebra, 21(4):345-365, 1987.
4. Haim Avron and Christos Boutsidis. Faster subset selection for matrices and applications. SIAM Journal on Matrix Analysis and Applications, 34(4):1464-1499, 2013.
5. Joshua Batson, Daniel A Spielman, and Nikhil Srivastava. Twice-Ramanujan sparsifiers. SIAM Journal on Computing, 41(6):1704-1721, 2012.
6. L Elisa Celis, Amit Deshpande, Tarun Kathuria, and Nisheeth K Vishnoi. How to be fair and diverse? arXiv:1610.07183, October 2016.
7. L Elisa Celis, Vijay Keswani, Damian Straszak, Amit Deshpande, Tarun Kathuria, and Nisheeth K Vishnoi. Fair and diverse dpp-based data summarization. arXiv:1802.04023, February 2018.
8. N. Cesa-Bianchi, P. M. Long, and M. K. Warmuth. Worst-case quadratic loss bounds for on-line prediction of linear functions by gradient descent. IEEE Transactions on Neural Networks, 7(3):604-619, 1996. Earlier version in 6th COLT, 1993.
9. Chih-Chung Chang and Chih-Jen Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2:27:1-27:27, 2011. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.
10. Xue Chen and Eric Price. Condition number-free query and active learning of linear families. CoRR, abs/1711.10051, 2017.
11. Michał Derezinski ´ and Manfred K Warmuth. Unbiased estimates for linear regression via volume sampling. In Advances in Neural Information Processing Systems 30, pages 3087-3096, Long Beach, CA, USA, December 2017.
12. Michał Derezinski ´ and Manfred K. Warmuth. Reverse iterative volume sampling for linear regression. Journal of Machine Learning Research, 19(23):1-39, 2018.
13. Michał Derezinski ´ and Manfred K. Warmuth. Subsampling for ridge regression via regularized volume sampling. In Proceedings of the 21st International Conference on Artificial Intelligence and Statistics, 2018.
14. Amit Deshpande and Luis Rademacher. Efficient volume sampling for row/column subset selection. In Proceedings of the 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, FOCS'10, pages 329-338, Washington, DC, USA, 2010.
15. Amit Deshpande, Luis Rademacher, Santosh Vempala, and Grant Wang. Matrix approximation and projective clustering via volume sampling. In Proceedings of the Seventeenth Annual ACM-SIAM Symposium on Discrete Algorithm, SODA'06, pages 1117-1126, Philadelphia, PA, USA, 2006.
16. Petros Drineas, Malik Magdon-Ismail, Michael W. Mahoney, and David P. Woodruff. Fast approximation of matrix coherence and statistical leverage. J. Mach. Learn. Res., 13(1):3475-3506, December 2012.
17. 2 regression and applications. In Proceedings of the seventeenth annual ACM-SIAM symposium on Discrete algorithm, pages 1127-1136, 2006.
18. Valerii V. Fedorov, William J. Studden, and E. M. Klimko, editors. Theory of optimal experiments. Probability and mathematical statistics. Academic Press, New York, 1972.
19. Mike Gartrell, Ulrich Paquet, and Noam Koenigstein. Bayesian low-rank determinantal point processes. In Proceedings of the 10th ACM Conference on Recommender Systems, RecSys'16, pages 349-356, New York, NY, USA, 2016.
20. David Gross and Vincent Nesme. Note on sampling without replacing from a finite collection of matrices. arXiv:1001.2738, January 2010.
21. Nicholas JA Harvey and Neil Olver. Pipage rounding, pessimistic estimators and matrix concentration. In Proceedings of the twenty-fifth annual ACM-SIAM symposium on Discrete algorithms, pages 926-945. SIAM, 2014.
22. Wassily Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American statistical association, 58(301):13-30, 1963.
23. Alex Kulesza and Ben Taskar. k-DPPs: Fixed-Size Determinantal Point Processes. In Proceedings of the 28th International Conference on Machine Learning, pages 1193-1200. Omnipress, 2011.
24. Alex Kulesza and Ben Taskar. Determinantal Point Processes for Machine Learning. Now Publishers Inc., Hanover, MA, USA, 2012.
25. Yin Tat Lee and He Sun. Constructing linear-sized spectral sparsification in almost-linear time. In Foundations of Computer Science (FOCS), 2015 IEEE 56th Annual Symposium on, pages 250-269. IEEE, 2015.
26. Chengtao Li, Stefanie Jegelka, and Suvrit Sra. Polynomial time algorithms for dual volume sampling. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 5045-5054. 2017.
27. Michael W. Mahoney. Randomized algorithms for matrices and data. Found. Trends Mach. Learn., 3(2):123-224, February 2011.
28. Zelda E. Mariet and Suvrit Sra. Elementary symmetric polynomials for optimal experimental design. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 2136-2145. 2017.
29. Aleksandar Nikolov, Mohit Singh, and Uthaipon Tao Tantipongpipat. Proportional volume sampling and approximation algorithms for A-optimal design. arXiv:1802.08318, July 2018.
30. Robin Pemantle and Yuval Peres. Concentration of Lipschitz functionals of determinantal and other strong rayleigh measures. Combinatorics, Probability and Computing, 23(1):140-160, 2014.
31. Tamas Sarlos. Improved approximation algorithms for large matrices via random projections. In Proceedings of the 47th Annual IEEE Symposium on Foundations of Computer Science, FOCS'06, pages 143-152, Washington, DC, USA, 2006.
32. Joel A. Tropp. User-friendly tail bounds for sums of random matrices. Foundations of Computational Mathematics, 12(4):389-434, August 2012.
33. David P Woodruff. Sketching as a tool for numerical linear algebra. Foundations and Trends® in Theoretical Computer Science, 10(1-2):1-157, 2014.