Supervised feature selection in graphs with path coding penalties and network flows

Journal of Machine Learning Research

Volumn 14, Issue , 2013, Pages 2449-2485

  Mairal, Julien ^a,b   Yu, Bin ^a,c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b INRIA RHÔNE ALPES   (France)

^c Department of Electrical Engineering and Computer Science ^*  (France)

Author keywords

Convex and non convex optimization; Graph sparsity; Network flow optimization

Indexed keywords

DIRECTED ACYCLIC GRAPH (DAG); GRAPH SPARSITY; LONG RANGE INTERACTIONS; NETWORK FLOW OPTIMIZATIONS; NONCONVEX OPTIMIZATION; PATH SELECTION PROBLEM; REGULARIZATION FUNCTION; SUPERVISED LEARNING PROBLEMS;

CONVEX OPTIMIZATION; FLOW GRAPHS; GENE EXPRESSION; GRAPH THEORY; OPTIMIZATION;

NETWORK CODING;

EID: 84885632749     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (62)

1. N. Ahmed, T. Natarajan, and K.R. Rao. Discrete cosine transform. IEEE Transactions on Computers, C-23(1):90-93, 1974.
2. R.K. Ahuja, T.L. Magnanti, and J.B. Orlin. Network Flows. Prentice Hall, 1993.
3. H. Akaike. Information theory and an extension of the maximum likelihood principle. In Second International Symposium on Information Theory, volume 1, pages 267-281, 1973.
4. F. Bach. Exploring large feature spaces with hierarchical multiple kernel learning. In Advances in Neural Information Processing Systems (NIPS), 2008.
5. F. Bach, R. Jenatton, J. Mairal, and G. Obozinski. Optimization with sparsity-inducing penalties. Foundation and Trends in Machine Learning, 4:1-106, 2012.
6. A. Barron, J. Rissanen, and B. Yu. The minimum description length principle in coding and modeling. IEEE Transactions on Information Theory, 44(6):2743-2760, 1998. (Pubitemid 128741906)
7. A. Beck and M. Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2(1):183-202, 2009.
8. E. Bernard, L. Jacob, J. Mairal, and J.-P. Vert. Efficient RNA isoform identification and quantification from RNA-Seq data with network flows. Technical Report hal-00803134, 2013.
9. D.P. Bertsekas. Network Optimization: Continuous and Discrete Models. Athena Scientific, 1998.
10. T. Blumensath and M.E. Davies. Iterative hard thresholding for compressed sensing. Applied and Computational Harmonic Analysis, 27(3):265-274, 2009.
11. M. Boguñá, R. Pastor-Satorras, A. D́iaz-Guilera, and A. Arenas. Models of social networks based on social distance attachment. Physical Review E, 70(5):056122, 2004.
12. S. P. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
13. Y. Boykov, O. Veksler, and R. Zabih. Fast approximate energy minimization via graph cuts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(11):1222-1239, 2001. (Pubitemid 33137959)
14. A. Buades, B. Coll, and J.M. Morel. A review of image denoising algorithms, with a new one. SIAM Multiscale Modelling and Simulation, 4(2):490, 2005.
15. V. Cehver, M. Duarte, C. Hedge, and R. G. Baraniuk. Sparse signal recovery using Markov random fields. In Advances in Neural Information Processing Systems (NIPS), 2008.
16. A. Chambolle and J. Darbon. On total variation minimization and surface evolution using parametric maximal flows. International Journal of Computer Vision, 84(3):288-307, 2009.
17. S.S. Chen, D.L. Donoho, and M.A. Saunders. Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing, 20(1):33-61, 1999. (Pubitemid 128689501)
18. X. Chen, Q. Lin, S. Kim, J. Pena, J.G. Carbonell, and E.P. Xing. Smoothing proximal gradient method for general structured sparse learning. In Proceedings of the Twenty-Seven Conference on Uncertainty in Artificial Intelligence (UAI), 2011.
19. H.Y. Chuang, E. Lee, Y.T. Liu, D. Lee, and T. Ideker. Network-based classification of breast cancer metastasis. Molecular Systems Biology, 3(140), 2007.
20. T.H. Cormen, C.E. Leiserson, R.L. Rivest, and C. Stein. Introduction to Algorithms. MIT Press, 2001.
21. C. Couprie, L. Grady, H. Talbot, and L. Najman. Combinatorial continuous maximum flow. SIAM Journal on Imaging Sciences, 4:905-930, 2011.
22. T.M. Cover and J.A Thomas. Elements of Information Theory. Wiley, 2006. 2nd edition.
23. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian. Image denoising by sparse 3D transformdomain collaborative filtering. IEEE Transactions on Image Processing, 16(8):2080-2095, 2007. (Pubitemid 47225460)
24. I. Daubechies, M. Defrise, and C. De Mol. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Communications on Pure and Applied Mathematics, 57(11): 1413-1457, 2004. (Pubitemid 39427442)
25. M. Elad and M. Aharon. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing, 54(12):3736-3745, 2006. (Pubitemid 44811686)
26. J. Fan and R. Li. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456):1348-1360, 2001. (Pubitemid 33695585)
27. L.R. Ford and D.R. Fulkerson. Maximal flow through a network. Canadian Journal ofMathematics, 8(3):399-404, 1956.
28. P. Gleiser and L. Danon. Community structure in jazz. Advances in Complex Systems, 6(4):565-573, 2003.
29. A.V. Goldberg. An Efficient Implementation of a Scaling Minimum-Cost Flow Algorithm. Journal of Algorithms, 22(1):1-29, 1997. (Pubitemid 127451786)
30. A.V. Goldberg and R.E. Tarjan. A new approach to the maximum flow problem. In Proceedings of the ACM Symposium on Theory of Computing, 1986.
31. R. Guimerà, L. Danon, A. D́iaz Guilera, F. Giralt, and A. Arenas. Self-similar community structure in a network of human interactions. Physical Review E, 68(6):065103, 2003.
32. D.S. Hochbaum. Complexity and algorithms for nonlinear optimization problems. Annals of Operations Research, 153(1):257-296, 2007.
33. H. Hoefling. A path algorithm for the fused lasso signal approximator. Journal of Computational and Graphical Statistics, 19(4):984-1006, 2010.
34. J. Huang, T. Zhang, and D. Metaxas. Learning with structured sparsity. Journal of Machine Learning Research, 12:3371-3412, 2011.
35. D.R. Hunter and K. Lange. A tutorial on MM algorithms. The American Statistician, 58(1):30-37, 2004. (Pubitemid 38260327)
36. L. Jacob, G. Obozinski, and J.-P. Vert. Group Lasso with overlap and graph Lasso. In Proceedings of the International Conference on Machine Learning (ICML), 2009.
37. R. Jenatton, J-Y. Audibert, and F. Bach. Structured variable selection with sparsity-inducing norms. Journal of Machine Learning Research, 12:2777-2824, 2011.
38. D.J.C. MacKay. Information Theory, Inference, and Learning Algorithms. Cambridge University Press, 2003.
39. J. Mairal. Sparse Coding for Machine Learning, Image Processing and Computer Vision. PhD thesis, Ecole Normale Supérieure de Cachan, 2010. http://tel.archives-ouvertes.fr/ tel-00595312.
40. J. Mairal. Optimization with first-order surrogate functions. In Proceedings of the International Conference on Machine Learning (ICML), 2013.
41. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman. Non-local sparse models for image restoration. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2009.
42. J. Mairal, F. Bach, J. Ponce, and G. Sapiro. Online learning for matrix factorization and sparse coding. Journal of Machine Learning Research, 11:19-60, 2010.
43. J. Mairal, R. Jenatton, G. Obozinski, and F. Bach. Convex and network flow optimization for structured sparsity. Journal of Machine Learning Research, 12:2649-2689, 2011.
44. S. Mallat and Z. Zhang. Matching pursuit in a time-frequency dictionary. IEEE Transactions on Signal Processing, 41(12):3397-3415, 1993.
45. N. Meinshausen and P. Bühlmann. Stability selection. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(4):417-473, 2010.
46. A. Nemirovsky and D. Yudin. Problem Complexity and Method Efficiency in Optimization. Wiley, 1983.
47. Y. Nesterov. Gradient methods for minimizing composite objective function. Technical report, CORE Discussion paper, 2007.
48. J. Nocedal and S.J. Wright. Numerical Optimization. Springer Verlag, 2006. 2nd edition.
49. G. Obozinski and F. Bach. Convex relaxations for combinatorial penalties. Technical Report arXiv:1205.1240v1, 2012.
50. M. R. Osborne, B. Presnell, and B. A. Turlach. On the Lasso and its dual. Journal of Computational and Graphical Statistics, 9(2):319-37, 2000.
51. J. Portilla, V. Strela, M. J. Wainwright, and E. P. Simoncelli. Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Transactions on Image Processing, 12(11):1338-1351, 2003.
52. F. Rapaport, A. Zinovyev, M. Dutreix, E. Barillot, and J.-P. Vert. Classification of microarray data using gene networks. BMC Bioinformatics, 8(1):35, 2007.
53. J. Rissanen. Modeling by shortest data description. Automatica, 14(5):465-471, 1978.
54. M. Schmidt, N. Le Roux, and F. Bach. Convergence rates of inexact proximal-gradient methods for convex optimization. In Advances in Neural Information Processing Systems (NIPS), 2011.
55. G. Schwarz. Estimating the dimension of a model. Annals of Statistics, 6(2):461-464, 1978.
56. R. Tibshirani. Regression shrinkage and selection via the Lasso. Journal of the Royal Statistical Society: Series B, 58(1):267-288, 1996.
57. B.A. Turlach, W.N. Venables, and S.J. Wright. Simultaneous variable selection. Technometrics, 47 (3):349-363, 2005. (Pubitemid 41164070)
58. M.H. Van De Vijver et al. A gene-expression signature as a predictor of survival in breast cancer. The New England Journal of Medicine, 347(25):1999-2009, 2002.
59. S. Wright, R. Nowak, and M. Figueiredo. Sparse reconstruction by separable approximation. IEEE Transactions on Signal Processing, 57(7):2479-2493, 2009.
60. M. Yuan and Y. Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B, 68:49-67, 2006. (Pubitemid 43415335)
61. P. Zhao, G. Rocha, and B. Yu. The composite absolute penalties family for grouped and hierarchical variable selection. Annals of Statistics, 37(6A):3468-3497, 2009.
62. H. Zou and T. Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B, 67(2):301-320, 2005. (Pubitemid 40465877)