Spectral sparsification and regret minimization beyond matrix multiplicative updates

Proceedings of the Annual ACM Symposium on Theory of Computing

Volumn 14-17-June-2015, Issue , 2015, Pages 237-245

  Allen Zhu, Zeyuan ^a   Liao, Zhenyu ^b   Orecchia, Lorenzo ^b  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b BOSTON UNIVERSITY   (United States)

Author keywords

Multiplicative weight updates; Regret minimization; Regularization; Spectral sparsification

Indexed keywords

COMPUTATION THEORY;

DENSITY MATRIX; FOLLOW THE REGULARIZED LEADERS; MULTIPLICATIVE UPDATES; NOVEL CONSTRUCTION; REGRET MINIMIZATION; REGULARIZATION; SPARSIFICATION; WEIGHT UPDATE;

MATRIX ALGEBRA;

EID: 84958765899     PISSN: 07378017     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2746539.2746610     Document Type: Conference Paper

References (45)

1. J. Abernethy and A. Rakhlin. Beating the adaptive bandit with high probability. In Proceedings of the 22nd Conference on Learning Theory - COLT' 09, pages 280-289, 2009.
2. J. D. Abernethy, E. Hazan, and A. Rakhlin. Interior-Point Methods for Full-Information and Bandit Online Learning. IEEE Transactions on Information Theory, 58(7):4164-4175, July 2012. An earlier version of this paper has appeared in COLT'08.
3. Z. Allen-Zhu and L. Orecchia. Using optimization to break the epsilon barrier: A faster and simpler width-independent algorithm for solving positive linear programs in parallel. In Proceedings of the 26th ACM-SIAM Symposium on Discrete Algorithms, SODA '15, 2015.
4. D. G. Anderson, M. Gu, and C. Melgaard. An Efficient Algorithm for Unweighted Spectral Graph Sparsification. ArXiv e-prints, abs/1410.4273, Oct. 2014.
5. S. Arora, E. Hazan, and S. Kale. Fast Algorithms for Approximate Semidefinite Programming using the Multiplicative Weights Update Method. In 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05), pages 339-348. IEEE, 2005.
6. S. Arora, E. Hazan, and S. Kale. The Multiplicative Weights Update Method: a Meta-Algorithm and Applications. Theory of Computing, 8:121-164, 2012.
7. S. Arora and S. Kale. A combinatorial, primal-dual approach to semidefinite programs. In Proceedings of the thirty-ninth annual ACM symposium on Theory of computing - STOC '07, page 227, New York, New York, USA, 2007. ACM Press.
8. A. Asadpour, M. X. Goemans, A. MAEdry, S. O. Gharan, and A. Saberi. An O(logn/loglogn)-approximation Algorithm for the Asymmetric Traveling Salesman Problem. In Proceedings of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms - SODA '10, pages 379-389, 2010.
9. J.-Y. Audibert, S. Bubeck, and G. Lugosi. Minimax policies for combinatorial prediction games. Proceedings of COLT 2011, 2011.
10. N. Bansal, U. Feige, R. Krauthgamer, K. Makarychev, V. Nagarajan, J. S. Naor, and R. Schwartz. Min-max Graph Partitioning and Small Set Expansion. In 2011 IEEE 52nd Annual Symposium on Foundations of Computer Science, pages 17-26. IEEE, Oct. 2011.
11. J. Batson, D. A. Spielman, and N. Srivastava. Twice-Ramanujan Sparsifiers. SIAM Review, 56(2):315-334, May 2014.
12. A. Ben-Tal and A. Nemirovski. Lectures on Modern Convex Optimization. Society for Industrial and Applied Mathematics, Jan. 2013.
13. A. A. Benczúr and D. R. Karger. Approximating s-t minimum cuts inO(n ) time. In Proceedings of the twenty-eighth annual ACM symposium on Theory of computing - STOC '96, pages 47-55, New York, New York, USA, 1996. ACM Press.
14. A. A. Benczúr and D. R. Karger. Randomized Approximation Schemes for Cuts and Flows in Capacitated Graphs. Technical report, July 2002.
15. R. Bhatia. Matrix Analysis, volume 169 of Graduate Texts in Mathematics. Springer New York, New York, NY, 1997.
16. S. Bubeck and N. Cesa-Bianchi. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and trends in machine learning, 5(1):1-122, 2012.
17. M. K. de Carli Silva, N. J. A. Harvey, and C. M. Sato. Sparse Sums of Positive Semidefinite Matrices. Technical report, July 2011.
18. N. Cesa-Bianchi and G. Lugosi. Prediction, Learning, and Games. Cambridge University Press, Cambridge, 2006.
19. M. Charikar, T. Leighton, S. Li, and A. Moitra. Vertex sparsifiers and abstract rounding algorithms. In 51th Annual IEEE Symposium on Foundations of Computer Science, FOCS 2010, October 23-26, 2010, Las Vegas, Nevada, USA, pages 265-274, 2010.
20. P. Christiano, J. A. Kelner, A. Madry, D. A. Spielman, and S.-H. Teng. Electrical flows, laplacian systems, and faster approximation of maximum flow in undirected graphs. In Proceedings of the 43rd annual ACM symposium on Theory of computing -STOC '11, page 273, New York, New York, USA, Oct. 2011. ACM Press.
21. C. Daskalakis, A. Deckelbaum, and A. Kim. Near-optimal no-regret algorithms for zero-sum games. In Proceedings of the Twenty-Second Annual ACM-SIAM Symposium on Discrete Algorithms -SODA '11, pages 235-254, 2011.
22. Y. Freund and R. E. Schapire. A desicion-theoretic generalization of on-line learning and an application to boosting. In Computational learning theory, pages 23-37. Springer, 1995.
23. N. Garg and J. Könemann. Faster and Simpler Algorithms for Multicommodity Flow and Other Fractional Packing Problems. SIAM Journal on Computing, 37(2):630-652, Jan. 2007.
24. E. Hazan. The convex optimization approach to regret minimization. In Optimization for machine learning, chapter 10, pages 287-304. MIT press, 2012.
25. E. Hazan, S. Kale, and S. Shalev-Shwartz. Near-optimal algorithms for online matrix prediction. In Proceedings of the 25th Annual Conference on Learning Theory - COLT '12, pages 38.1-38.13, 2012.
26. R. Jain, Z. Ji, S. Upadhyay, and J. Watrous. QIP = PSPACE. Journal of the ACM (JACM), 58(6):30, 2011.
27. W. B. Johnson and J. Lindenstrauss. Extensions of Lipschitz mappings into a Hilbert space. Contemporary Mathematics, 26:189-206, 1984.
28. J. A. Kelner, Y. T. Lee, L. Orecchia, and A. Sidford. An Almost-Linear-Time Algorithm for Approximate Max Flow in Undirected Graphs, and its Multicommodity Generalizations. In Proceedings of the 25th Annual ACM-SIAM Symposium on Discrete Algorithms - SODA '14, number 1 in STOC '14, Apr. 2014.
29. E. H. Lieb. Convex trace functions and the wigner-yanase-dyson conjecture. Advances in Mathematics, 11(3):267-288, 1973.
30. A. Naor. Sparse quadratic forms and their geometric applications [after Batson, Spielman and Srivastava]. Astérisque, 2012.
31. L. Orecchia. Fast Approximation Algorithms for Graph Partitioning using Spectral and Semidefinite-Programming Techniques. PhD thesis, EECS Department, University of California, Berkeley, May 2011.
32. L. Orecchia, S. Sachdeva, and N. K. Vishnoi. Approximating the exponential, the lanczos method and an O(m)-time spectral algorithm for balanced separator. In STOC '12. ACM Press, Nov. 2012.
33. R. Peng and D. A. Spielman. An efficient parallel solver for SDD linear systems. In Symposium on Theory of Computing, STOC 2014, New York, NY, USA, May 31 - June 03, 2014, pages 333-342, 2014.
34. S. A. Plotkin, D. B. Shmoys, and E. Tardos. Fast Approximation Algorithms for Fractional Packing and Covering Problems. Mathematics of Operations Research, 20(2):257-301, May 1995.
35. A. Rakhlin. Lecture notes on online learning. Draft, 2009. Available at http://www-st at.whart on.upenn.edu/~rakhlin/courses/stat991/papers/lecture-notes.pdf.
36. S. Shalev-Shwartz. Online learning: Theory, algorithms, and applications. PhD thesis, Hebrew University, 2007.
37. S. Shalev-Shwartz. Online Learning and Online Convex Optimization. Foundations and Trends in Machine Learning, 4(2):107-194, 2011.
38. J. Sherman. Breaking the multicommodity flow barrier for o(vlog n)-approximations to sparsest cut. In Proceedings of the 50th Annual IEEE Symposium on Foundations of Computer Science, FOCS '09, pages 363-372, 2009.
39. D. A. Spielman and N. Srivastava. Graph Sparsification by Effective Resistances. SIAM Journal on Computing, 40(6):1913-1926, Jan. 2011.
40. D. A. Spielman and S.-H. Teng. Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems. In Proceedings of the thirty-sixth annual ACM symposium on Theory of computing - STOC '04, page 81, New York, New York, USA, 2004. ACM Press.
41. D. A. Spielman and S.-H. Teng. Spectral Sparsification of Graphs. SIAM Journal on Computing, 40(4):981-1025, Jan. 2011.
42. D. Steurer. Fast SDP algorithms for constraint satisfaction problems. In Proceedings of the Twenty-First Annual ACM-SIAM Symposium on Discrete Algorithms - SODA '10, pages 684-697, 2010.
43. N. K. Vishnoi. personal communication, 2014.
44. M. Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. In Proceedings of the 20th International Conference on Machine Learning (ICML-03), pages 928-936, 2003.
45. A. Zouzias. A matrix hyperbolic cosine algorithm and applications. In Proceedings of the 39th International Colloquium Conference on Automata, Languages, and Programming - Volume Part I, ICALP'12, pages 846-858, Berlin, Heidelberg, 2012. Springer-Verlag.