Fundamental limits of online and distributed algorithms for statistical learning and estimation

Advances in Neural Information Processing Systems

Volumn 1, Issue January, 2014, Pages 163-171

  Shamir, Ohad ^a  

^a WEIZMANN INSTITUTE OF SCIENCE   (Israel)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; INFORMATION SCIENCE; LEARNING SYSTEMS; OPTIMIZATION; PROBLEM SOLVING; SEMANTICS;

COMMUNICATION CONSTRAINTS; DISTRIBUTED LEARNING; FIRST ORDER METHOD; MACHINE LEARNING APPROACHES; MULTI ARMED BANDIT; SMALL MEMORY FOOTPRINT; STATISTICAL LEARNING; STOCHASTIC OPTIMIZATION PROBLEMS;

LEARNING ALGORITHMS;

EID: 84937874425     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (31)

1. A. Agarwal, P. Bartlett, P. Ravikumar, and M. Wainwright. Information-theoretic lower bounds on the oracle complexity of stochastic convex optimization. Information Theory, IEEE Transactions on, 58(5):3235-3249, 2012.
2. A. Agarwal, O. Dekel, and L. Xiao. Optimal algorithms for online convex optimization with multi-point bandit feedback. In COLT, 2010.
3. Noga Alon, Yossi Matias, and Mario Szegedy. The space complexity of approximating the frequency moments. In STOC, 1996.
4. J.-Y. Audibert, S. Bubeck, and G. Lugosi. Minimax policies for combinatorial prediction games. In COLT, 2011.
5. P. Auer, N. Cesa-Bianchi, Y. Freund, and R. Schapire. The nonstochastic multiarmed bandit problem. SIAM Journal on Computing, 32(1):48-77, 2002.
6. M. Balcan, A. Blum, S. Fine, and Y. Mansour. Distributed learning, communication complexity and privacy. In COLT, 2012.
7. A. Balsubramani, S. Dasgupta, and Y. Freund. The fast convergence of incremental pca. In NIPS, 2013.
8. A. Berthet and P. Rigollet. Complexity theoretic lower bounds for sparse principal component detection. In COLT, 2013.
9. J. Bien and R. Tibshirani. Sparse estimation of a covariance matrix. Biometrika, 98(4):807-820, 2011.
10. S. Bubeck and N. Cesa-Bianchi. Regret analysis of stochastic and nonstochastic multi-armed bandit problems. Foundations and Trends in Machine Learning, 5(1):1-122, 2012.
11. N. Cesa-Bianchi and L. Gabor. Prediction, learning, and games. Cambridge University Press, 2006.
12. N. Cesa-Bianchi, S. Shalev-Shwartz, and O. Shamir. Efficient learning with partially observed attributes. The Journal of Machine Learning Research, 12:2857-2878, 2011.
13. S. Chien, K. Ligett, and A. McGregor. Space-efficient estimation of robust statistics and distribution testing. In ICS, 2010.
14. T. Cover and J. Thomas. Elements of information theory. John Wiley & Sons, 2006.
15. M. Crouch, A. McGregor, and D. Woodruff. Stochastic streams: Sample complexity vs. space complexity. In MASSIVE, 2013.
16. S. S. Dragomir. Upper and lower bounds for Csiszár's f-divergence in terms of the Kullback-Leibler distance and applications. In Inequalities for Csiszár f-Divergence in Information Theory. RGMIA Monographs, 2000.
17. S. Guha and A. McGregor. Space-efficient sampling. In AISTATS, 2007.
18. L. Kontorovich. Statistical estimation with bounded memory. Statistics and Computing, 22(5):1155-1164, 2012.
19. S. Mannor and O. Shamir. From bandits to experts: On the value of side-observations. In NIPS, 2011.
20. I. Mitliagkas, C. Caramanis, and P. Jain. Memory limited, streaming pca. In NIPS, 2013.
21. S. Muthukrishnan. Data streams: Algorithms and applications. Now Publishers Inc, 2005.
22. A. Nemirovsky and D. Yudin. Problem Complexity and Method Efficiency in Optimization. Wiley-Interscience, 1983.
23. M. Raginsky and A. Rakhlin. Information-based complexity, feedback and dynamics in convex programming. Information Theory, IEEE Transactions on, 57(10):7036-7056, 2011.
24. A. Rahimi and B. Recht. Random features for large-scale kernel machines. In NIPS, 2007.
25. Y. Seldin, P. Bartlett, K. Crammer, and Y. Abbasi-Yadkori. Prediction with limited advice and multiarmed bandits with paid observations. In ICML, 2014.
26. I. Taneja and P. Kumar. Relative information of types, Csiszár's f-divergence, and information inequalities. Inf. Sci., 166(1-4):105-125, 2004.
27. C. Williams and M. Seeger. Using the nyström method to speed up kernel machines. In NIPS, 2001.
28. D. Woodruff. The average-case complexity of counting distinct elements. In ICDT, 2009.
29. Y. Zhang, J. Duchi, M. Jordan, and M. Wainwright. Information-theoretic lower bounds for distributed statistical estimation with communication constraints. In NIPS, 2013.
30. Y. Zhang, J. Duchi, and M. Wainwright. Communication-efficient algorithms for statistical optimization. In NIPS, 2012.
31. H. Zou, T. Hastie, and R. Tibshirani. Sparse principal component analysis. Journal of computational and graphical statistics, 15(2):265-286, 2006.