Information-theoretic lower bounds for distributed statistical estimation with communication constraints

Advances in Neural Information Processing Systems

Volumn , Issue , 2013, Pages

  Zhang, Yuchen ^a   Duchi, John C ^a   Jordan, Michael I ^a,b   Wainwright, Martin J ^a,b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ESTIMATION; REGRESSION ANALYSIS; RISK PERCEPTION;

COMMUNICATION CONSTRAINTS; INFORMATION-THEORETIC LOWER BOUNDS; INTERACTIVE COMMUNICATIONS; LOCATION MODELS; LOWER BOUNDS; MINIMAX RISK; REGRESSION MODEL; STATISTICAL ESTIMATION;

COMMUNICATION;

EID: 84899033780     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. H. Abelson. Lower bounds on information transfer in distributed computations. Journal of the ACM, 27(2):384-392, 1980.
2. M.-F. Balcan, A. Blum, S. Fine, and Y. Mansour. Distributed learning, communication complexity and privacy. In Proceedings of the Twenty Fifth Annual Conference on Computational Learning Theory, 2012. URL http://arxiv.org/abs/ 1204.3514.
3. K. Ball. An elementary introduction to modern convex geometry. In S. Levy, editor, Flavors of Geometry, pages 1-58. MSRI Publications, 1997.
4. S. Boyd and L. Vandenberghe. Convex Optimization. CambridgeUniversity Press, 2004.
5. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1), 2011.
6. T. M. Cover and J. A. Thomas. Elements of Information Theory, Second Edition. Wiley, 2006.
7. O. Dekel, R. Gilad-Bachrach, O. Shamir, and L. Xiao. Optimal distributed online prediction using mini-batches. Journal of Machine Learning Research, 13:165-202, 2012.
8. J. C. Duchi and M. J.Wainwright. Distance-based and continuum fano inequalities with applications to statistical estimation. arXiv [cs.IT], to appear, 2013.
9. J. C. Duchi, A. Agarwal, and M. J. Wainwright. Dual averaging for distributed optimization: convergence analysis and network scaling. IEEE Transactions on Automatic Control, 57(3): 592-606, 2012.
10. J. C. Duchi, M. I. Jordan, and M. J. Wainwright. Local privacy and statistical minimax rates. arXiv:1302.3203 [math.ST], 2013. URL http://arXiv.org/abs/1302.3203.
11. S. Han and S. Amari. Statistical inference under multiterminal data compression. IEEE Transactions on Information Theory, 44(6):2300-2324, 1998.
12. I. A. Ibragimov and R. Z. Has'minskii. Statistical Estimation: Asymptotic Theory. Springer- Verlag, 1981.
13. E. Kushilevitz and N. Nisan. Communication Complexity. CambridgeUniversity Press, 1997.
14. E. L. Lehmann and G. Casella. Theory of Point Estimation, Second Edition. Springer, 1998.
15. Z.-Q. Luo. Universal decentralized estimation in a bandwidth constrained sensor network. IEEE Transactions on Information Theory, 51(6):2210-2219, 2005.
16. Z.-Q. Luo and J. N. Tsitsiklis. Data fusion with minimal communication. IEEE Transactions on Information Theory, 40(5):1551-1563, 1994.
17. R. McDonald, K. Hall, and G. Mann. Distributed training strategies for the structured perceptron. In North American Chapter of the Association for Computational Linguistics (NAACL), 2010.
18. J. N. Tsitsiklis. Decentralized detection. In Advances in Signal Processing, Vol. 2, pages 297-344. JAI Press, 1993.
19. A. B. Tsybakov. Introduction to Nonparametric Estimation. Springer, 2009.
20. Y. Yang and A. Barron. Information-theoretic determination of minimax rates of convergence. Annals of Statistics, 27(5):1564-1599, 1999.
21. A. C.-C. Yao. Some complexity questions related to distributive computing (preliminary report). In Proceedings of the Eleventh Annual ACM Symposium on the Theory of Computing, pages 209-213. ACM, 1979.
22. B. Yu. Assouad, Fano, and Le Cam. In Festschrift for Lucien Le Cam, pages 423-435. Springer-Verlag, 1997.
23. Y. Zhang, J. C. Duchi, and M. J. Wainwright. Communication-efficient algorithms for statistical optimization. In Advances in Neural Information Processing Systems 26, 2012.