Distributed sparse linear regression

IEEE Transactions on Signal Processing

Volumn 58, Issue 10, 2010, Pages 5262-5276

  Mateos, Gonzalo ^a   Bazerque, Juan Andrés ^a   Giannakis, Georgios B ^a  

^a University of Minnesota   (United States)

Author keywords

Distributed linear regression; Lasso; parallel optimization; sparse estimation

Indexed keywords

CENTRAL PROCESSING UNITS; COGNITIVE RADIO; COMMUNICATION COST; CONTINUOUS VARIABLES; CONVERGENCE SPEED; DATA SETS; DISTRIBUTED LINEAR REGRESSION; DISTRIBUTED SCHEMES; ELASTIC NET; GLOBAL OPTIMALITY; INTER-AGENT COMMUNICATIONS; JOINT ESTIMATION; LASSO; LINEAR REGRESSION PROBLEMS; METHOD OF MULTIPLIERS; NOVEL ALGORITHM; NUMERICAL EXPERIMENTS; PARALLEL OPTIMIZATION; PARALLELIZATIONS; RADIO-FREQUENCY POWER; REGRESSION COEFFICIENT; SPARSE ESTIMATION; THRESHOLDING; TRAINING DATA; TRAINING SETS; UNDER-DETERMINED; WIRELESS COMMUNICATIONS;

ALGORITHMS; COMMUNICATION; ESTIMATION; LINEAR REGRESSION; OPTIMIZATION; PIPELINE PROCESSING SYSTEMS; PROGRAM PROCESSORS; WIRELESS TELECOMMUNICATION SYSTEMS;

CONVERGENCE OF NUMERICAL METHODS;

EID: 77956773514     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2010.2055862     Document Type: Article

References (40)

1. I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci, "Wireless sensor networks: a survey," Comput. Netw., vol.38, pp. 393-422, Mar. 2002.
2. S. Barbarossa, G. Scutari, and T. Battisti, "Cooperative sensing for cognitive radio using decentralized projection algorithms," in Proc. Workshop Signal Processing Advances in Wireless Communications, Perugia, Italy, Jun. 2009, pp. 116-120.
3. J. A. Bazerque and G. B. Giannakis, "Distributed spectrum sensing for cognitive radio networks by exploiting sparsity," IEEE Trans. Signal Process., vol.58, no.3, pp. 1847-1862, Mar. 2010.
4. D. P. Bertsekas, Nonlinear Programming, 2nd ed. Belmont, MA: Athena Scientific, 1999.
5. D. P. Bertsekas and J. N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods, 2nd ed. Belmont, MA: Athena Scientific, 1999.
6. L. Breiman, "Better subset regression using the nonnegative garrote," Technometrics, vol.37, pp. 373-384, Nov. 1995.
7. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol.20, pp. 33-61, 1998.
8. C. Clifton, M. Kantarcioglu, J. Vaidya, X. Lin, and M. Y. Zhu, "Tools for privacy preserving distributed data mining," ACM SIGKDD Explorations, vol.4, pp. 28-34, 2002.
9. A. G. Dimakis, S. Kar, J.M. F.Moura, M. G. Rabbat, and A. Scaglione, "Gossip algorithms for distributed signal processing," Mar. 2010 [Online]. Available: http://arxiv.org/abs/1003.5309
10. B. Efron, T. Hastie, I. M. Johnstone, and R. Tibshirani, "Least angle regression," Ann. Stat., vol.32, pp. 407-499, 2004.
11. J. Fan and R. Li, "Variable selection via nonconcave penalized likelihood and its oracle properties," J. Amer. Stat. Assoc., vol.96, pp. 1348-1360, 2001.
12. P. A. Forero, A. Cano, and G. B. Giannakis, "Consensus-based distributed support vector machines," J. Mach. Learn. Res., vol.11, pp. 1663-1707, May 2010.
13. J. Friedman, T. Hastie, H. Hofling, and R. Tibshirani, "Pathwise coordinate optimization," Ann. Appl. Stat., vol.1, pp. 302-332, 2007.
14. J. Friedman, T. Hastie, and R. Tibshirani, "Regularized paths for generalized linear models via coordinate descent," J. Stat. Softw., vol.33, 2010.
15. D. Gabay and B. Mercier, "A dual algorithm for the solution of nonlinear variational problems via finite-element approximations," Comput. Math. Appl., vol.2, pp. 17-40, 1976.
16. R. Glowinski and A. Marrocco, "Sur l'approximation, paréléments finis d'orde un, et la résolution par pénalisation-dualité d'une classe de problèmes de Dirichlet non-linéaires," (in French) Rev. Francaise d'Aut. Inf. Rech. Oper., vol.2, pp. 41-76, 1975.
17. T. Goldstein and S. Osher, "The split bregman method for L1 regularized problems," SIAM J. Imag. Sci., vol.2, pp. 323-343, 2009.
18. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, 2nd ed. New York: Springer, 2009.
19. X. Hong, C. Wang, H. Chen, and Y. Zhang, "Secondary spectrum access networks," IEEE Veh. Technol. Mag., vol.4, no.2, pp. 36-43, 2009.
20. A. Jadbabaie and S. Morse, "Coordination of groups of mobile autonomous agents using nearest neighbor rules," IEEE Trans. Autom, Control, vol.48, no.6, pp. 988-1001, 2003.
21. A. Nedic and D. P. Bertsekas, "Incremental subgradient methods for nondifferentiable optimization," SIAM J. Optim., vol.12, pp. 109-138, Jan. 2001.
22. A. Nedic and A. Ozdaglar, "Distributed subgradient methods for multiagent optimization," IEEE Trans. Autom. Control, vol.54, no.1, pp. 48-61, Jan. 2009.
23. R. Olfati-Saber, J. A. Fax, and R. M. Murray, "Consensus and cooperation in networked multi-agent systems," Proc. IEEE, vol.95, pp. 215-233, Jan. 2007.
24. J. B. Predd, S. R. Kulkarni, and H. V. Poor, "A collaborative training algorithm for distributed learning," IEEE Trans. Inf. Theory, vol.55, no.4, pp. 1856-1871, Apr. 2009.
25. M. G. Rabbat and R. D. Nowak, "Quantized incremental algorithms for distributed optimization," IEEE J. Sel. Areas Commun., vol.23, no.4, pp. 798-808, 2005.
26. M. G. Rabbat, R. D. Nowak, and J. A. Bucklew, "Generalized consensus computation in networked systems with erasure links," in Proc. Workshop Signal Processing Advances in Wireless Communications, New York, Jun. 2005, pp. 1088-1092.
27. S. S. Ram, A. Nedic, and V. V. Veeravalli, "Stochastic incremental gradient descent for estimation in sensor networks," in Proc. 41st Asilomar Conf. Signals, Systems, Computers, Pacific Grove, CA, 2007, pp. 582-586.
28. S. S. Ram, A. Nedic, and V. V. Veeravalli, "Distributed stochastic subgradient projection algorithms for convex optimization," Nov. 2008 [Online]. Available: http://arxiv.org/abs/0811.2595
29. I. D. Schizas, A. Ribeiro, and G. B. Giannakis, "Consensus in ad hoc WSNs with noisy links-Part I: Distributed estimation of deterministic signals," IEEE Trans. Signal Process., vol.56, no.1, pp. 350-364, Jan. 2008.
30. M. Schmidt, G. Fung, and R. Rosales, "Fast optimization methods for L1 regularization: A comparative study and two new approaches," in Proc. Euro. Conf. Machine Learning, Warsaw, Poland, 2007, pp. 286-297.
31. T. Stamey, J. Kabalin, I. Johnstone, F. Freiha, E. Redwine, and N.Yang, "Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of prostate ii radical prostatectomy treated patients," J. Urology, vol.16, pp. 1076-1083, 1989.
32. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Roy. Stat. Soc. B, vol.58, pp. 267-288, 1996.
33. R. Tibshirani, M. Saunders, R. Rosset, J. Zhu, and K. Knight, "Sparsity and smoothness via the fused lasso," J. Roy. Stat. Soc. B, vol.67, pp. 91-108, 2005.
34. P. Tseng, "Convergence of block coordinate descent method for nondifferentiable maximization," J. Optim. Theory Appl., vol.109, pp. 473-492, 2001.
35. S. J. Wright, R. D. Nowak, and M. A. T. Figueiredo, "Sparse reconstruction by separable approximation," IEEE Trans. Signal Process., vol.57, no.7, pp. 2479-2493, Jul. 2009.
36. T. Wu and K. Lange, "Coordinate descent algorithms for lasso penalized regression," Ann. Appl. Stat., vol.2, pp. 224-244, 2008.
37. L. Xiao and S. Boyd, "Fast linear iterations for distributed averaging," Syst. Control Lett., vol.53, pp. 65-78, Sep. 2004.
38. H. Zhu, G. B. Giannakis, and A. Cano, "Distributed in-network channel decoding," IEEE Trans. Signal Process., vol.57, no.10, pp. 3970-3983, Oct. 2009.
39. H. Zou, "The adaptive lasso and its oracle properties," J. Amer. Stat. Assoc., vol.101, pp. 1418-1429, 2006.
40. H. Zou and T. Hastie, "Regularization and variable selection via the elastic net," J. Roy. Stat. Soc. B, vol.67, pp. 301-320, 2005.