A split-and-conquer approach for analysis of extraordinarily large data

Statistica Sinica

Volumn 24, Issue 4, 2014, Pages 1655-1684

  Chen, Xueying ^a   Xie, Min Ge ^a  

^a RUTGERS UNIVERSITY   (United States)

Author keywords

Big data; Combining results from independent analyses; Distributed computing; Generalized linear models; Large sample theory; Penalized regression

Indexed keywords

EID: 84946566592     PISSN: 10170405     EISSN: None     Source Type: Journal    
DOI: 10.5705/ss.2013.088     Document Type: Article

References (31)

1. Agarwal, A. and Duchi, J. C. (2012). Distributed delayed stochastic optimization. Decision and Control (CDC), 2012 IEEE 51st Annual Conference, 5451-5452.
2. Ahmed, A., Aly, M., Das, A., Smola, A. J. and Anastasakos, T. (2012). Web-scale multitask feature selection for behavioral targeting. Proceedings of the 21st ACM International Conference on Information and Knowledge Management, 1737-1741.
3. Andrews, G. R. (2000). Foundations of Multithreaded, Parallel, and Distributed Programming. Addison-Wesley.
4. Breheny, P. and Huang, J. (2011). Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection. Ann. Appl. Statist. 5, 232-253.
5. Chen, S. S., Donoho, D. L. and Saunders, M. A. (2001). Atomic decomposition by basis pursuit. SIAM Rev. 129-159.
6. Duchi, J. C., Agarwal, A. andWainwright, M. J. (2012). Dual averaging for distributed optimization: convergence analysis and network scaling, Automatic Control, IEEE Transactions, 57, 592-606.
7. Efron, B., Hastie, T., Johnstone, I. and Tibshirani, R. (2004). Least angle regression. Ann. Statist. 32, 407-451.
8. Fan, J., Guo, S. and Hao, N. (2010). Variance estimation using refitted cross-validation in ultrahigh dimensional regression. Arxiv preprint arXiv:1004.5178.
9. Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc. 96, 1348-1360.
10. Fan, J. and Lv, J. (2011). Non-concave penalized likelihood with NP-dimensionality. IEEE transaction on Information Theory 57, 5467-5484.
11. Fan, J., Samworth, R. and Wu, Y. (2009). Ultrahigh dimensional feature selection: beyond the linear model. J. Mach. Learn. Res. 10, 2013-2038.
12. Golub, G. H. and Van Loan, C. F. (1983). Matrix Computations. Johns Hopkins University, Press, Baltimore.
13. Liu, D. (2012). Combination of confidence distributions and an efficient approach for metaanalysis of heterogeneous studies. Ph.D thesis, Department of Statistics and Biostatistics, Rutgers University.
14. Mackey, L., Talwalkar, A. and Jordan, M. I. (2011). Divide-and-conquer matrix factorization. arXiv preprint arXiv:1107.0789.
15. Marcenko, V. A. and Pastur, L. A. (1967). Distribution of eigenvalues for some sets of random matrices. Sbornik: Mathematics 1, 457-483.
16. Meinshausen, N. and Buhlmann, P. (2010). Stability selection. J. Roy. Statist. Soc. Ser. B 72, 417-473.
17. Shah, R. and Samworth, R. J. (2013). Variable selection with error control: Another look at stability selection. J. Roy. Statist. Soc. Ser. B 75, 55-80.
18. Singh, K. and Xie, M. and Strawderman, W. E. (2005). Combining information from independent sources through confidence distributions. Ann. Statist. 33, 159-183.
19. Takemura, A. and Sheena, Y. (2005). Distribution of eigenvalues and eigenvectors of Wishart matrix when the population eigenvalues are infinitely dispersed and its application to minimax estimation of covariance matrix. J. Multivariate Anal. 94, 271-299.
20. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. J. Roy. Statist. Soc. Ser. B 58, 267-288.
21. Trefethen, L. N. and Bau III, D. (1997). Numerical Linear Algebra. SIAM.
22. Vrahatis, M. N. (1989). A short proof and a generalization of Miranda's existence theorem. Proc. Amer. Math. Soc, 107, 701-703.
23. Wainwright, M. J. (2009). Sharp thresholds for noisy and high-dimensional recovery of sparsity using 1-constrained quadratic programming (lasso). IEEE Transactions on Information Theory, 55, 2183-2202.
24. Xie, M. and Singh, K. and Strawderman, W. E. (2011). Confidence distributions and a unifying framework for meta-analysis. J. Amer. Statist. Assoc. 106, 320-333.
25. Yuan, M. and Lin, Y. (2006). Model selection and estimation in regression with grouped variables. J. Roy. Statist. Soc. Ser. B 68, 49-67.
26. Zhang, C. H. (2010). Nearly unbiased variable selection under minimax concave penalty. Ann. Statist. 38, 894-942.
27. Zhang, Y., Duchi, J. C. and Wainwright, M. J. (2012). Communication-efficient algorithms for statistical optimization. Decision and Control (CDC), 2012 IEEE 51st Annual Conference, 6792-6792.
28. Zhang, Y., Duchi, J. C. and Wainwright, M. J. (2013). Divide and conquer kernel ridge regression: A distributed algorithm with minimax optimal rates. arXiv preprint arXiv:1305.5029.
29. Zhang, C.-H. and Huang, J. (2008). The sparsity and bias of the Lasso selection in highdimensional linear regression. Ann. Statist. 36, 1567-1594.
30. Zhang, C.-H. and Zhang, S. S. (2014). Confidence intervals for low-dimensional parameters in high-dimensional linear models. J. Roy. Statist. Soc. Ser. B 76, 217-242.
31. Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. J. Roy. Statist. Soc. Ser. B 67, 301-320.