Sensitivity computation of the ℓ1 minimization problem and its application to dictionary design of ill-posed problems

Inverse Problems

Volumn 25, Issue 9, 2009, Pages

  Horesh, L ^a   Haber, E ^b  

^a IBM T J WATSON RESEARCH CENTER   (United States)

^b EMORY UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMIC FRAMEWORK; BAYESIAN; DESIGN PROBLEMS; DICTIONARY CONSTRUCTIONS; ILL POSED PROBLEM; INVERSE SOLUTION; MATHEMATICAL FORMULATION; MINIMIZATION PROBLEMS; NON-TRIVIAL; OBSERVATION OPERATOR; OPTIMALITY CRITERIA; OPTIMIZATION STRATEGY; SALIENT FEATURES; SENSITIVITY COMPUTATION; SPARSE CODING; SPARSE REPRESENTATION; TRAINING MODEL;

DESIGN; DIFFERENTIAL EQUATIONS; MATHEMATICAL OPERATORS; MEDICAL IMAGING; MEDICAL PROBLEMS; OPTIMIZATION; SENSITIVITY ANALYSIS; SET THEORY;

INVERSE PROBLEMS;

EID: 70350630587     PISSN: 02665611     EISSN: 13616420     Source Type: Journal    
DOI: 10.1088/0266-5611/25/9/095009     Document Type: Article

References (76)

1. Aharon M, Elad M and Bruckstein A 2006 K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation IEEE Trans. Signal Process. 54 4311-22
2. Akcakaya M and Tarokh V 2007 Performance study of various sparse representation methods using redundant frames 41st Ann. Conf. Inf. Sci. Syst., 14-16 Mar 2007 (CISS '07) pp 726-9 (Pubitemid 351710509)
3. Alexandrov N and Dennis J E 1994 Algorithms for bilevel optimization AIAA/USAF/NASA/ISSMO 5th Symp. on Multidisciplinary Analysis and Optimization (Panama City Beach, FL) pp 810-16
4. Bard J F 2006 Practical Bilevel Optimization: Algorithms and Applications (Nonconvex Optimization and its Applications) (New York: Springer)
5. Bard J F and Moore J T 1990 A branch and bound algorithm for the bilevel programming problem SIAM J. Sci. Stat. Comput. 11 281-92
6. Bierman R and Singh R 2007 Influence of dictionary size on the lossless compression of microarray images 20th IEEE Int. Symp. on Computer-Based Medical Systems, 2007. CBMS '07 (20-22 June) pp 237-42
7. Bonnans J F and Shapiro A 2000 Perturbation Analysis of Optimization Problems (New York: Springer)
8. Bruckstein A M, Donoho D L and Elad M 2008 From sparse solutions of systems of equations to sparse modeling of signals and images SIAM Rev. 51 34-81
9. Burke J V, Lewis A S and Overton M L 2005 A robust gradient sampling algorithm for nonsmooth, nonconvex optimization SIAM J. Optim. 15 751-79
10. Candes E, Braun N and Wakin M 2007 Sparse signal and image recovery from compressive samples 4th IEEE Int. Symp. on Biomedical Imaging: From Nano to Macro, 12-15 April 2007 (ISBI 2007) pp 976-9
11. Chen J and Huo X 2005 Sparse representations for multiple measurement vectors (mmv) in an over-complete dictionary IEEE Int. Conf. Acoustics, Speech and Signal Processing, Proc. (ICASSP '05) (18-23 Mar 2005) vol 4 pp iv/257-iv/260
12. Chen S S, Donoho D L and Saunders M A 1998 Atomic decomposition by basis pursuit SIAM J. Sci. Comput. 20 33-61
13. Claerbout J and Muir F 1973 Robust modeling with erratic data Geophysics 38 826-44
14. Colson B, Marcotte P and Savard G 2007 An overview of bilevel optimization Ann. Oper. Res. 153 235-56
15. Dempe S and Gadhi N 2007 Necessary optimality conditions for bilevel set optimization problems J. Global Optim. 39 529-42
16. 1-norm solution is also the sparsest solution Commun. Pure Appl. Math. 59 797-829
17. Donoho D L and Elad M 2003 Optimally sparse representation in general (nonorthogonal) dictionaries via ell 1 minimization PNAS 100 2197-202
18. Duarte-Carvajalino J M and Sapiro G 2009 Learning to sense sparse signals: simultaneous sensing matrix and sparsifying dictionary optimization IEEE Trans. Image Process. 18 1395-408
19. Efron B, Hastie T, Johnstone I and Tibshirani R 2004 Least angle regression Ann. Stat. 32 407-99
20. Engan K, Husoy J H and Aase S O 2001 Frame based representation and compression of still images Int. Conf. Image Process., Oct 2001 vol 2 pp 427-30
21. Engan K and Skretting K 2002 A novel image denoising technique using overlapping frames Vis. Imaging Image Process. 364 265-70
22. Engan K, Skretting K and Hkon Husoy J 2007 Family of iterative ls-based dictionary learning algorithms, ils-dla, for sparse signal representation Digit. Signal Process. 17 32-49
23. Figueiredo M A T, Nowak R D and Wright S J 2007 Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems IEEE J. Sel. Top. Signal Process. 1 586-97
24. Fischer S, Cristöbal G and Redondo R 2006 Sparse overcomplete Gabor wavelet representation based on local competitions IEEE Trans. Image Process. 15 265-72
25. Fuchs J J and Guillemot C 2007 Fast implementation of a penalized sparse representations algorithm: applications in image denoising and coding. Conf. Record of the 41st Asilomar Conf. on Signals, Systems and Computers (ACSSC 2007) (4-7 Nov 2007) pp 508-512
26. Girolami M 2001 A variational method for learning sparse and overcomplete representations Neural Comput. 13 2517-32
27. Guanquan Z and Wensheng Z 2000 Parallel implementation of 2D prestack depth migration 4th Int. Conf./Exhibition on High Performance Computing in the Asia-Pacific Region (14-17 May 2000) vol 2 pp 970-5
28. Haber E, Ascher U M and Oldenburg D 2000 On optimization techniques for solving nonlinear inverse problems Inverse Problems 16 1263-80
29. Haber E and Tenorio L 2003 Learning regularization functionals: and a supervised training approach Inverse Problems 19 611-26
30. Hansen P C 1998 Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion (Philadelphia SIAM)
31. Johansen T A 1997 On Tikhonov regularization, bias and variance in nonlinear system identification Automatica 33 441-6
32. Jost P, Lesage S, Vandergheynst P and Gribonval R 2006 MOTIF: an efficient algorithm for learning translation invariant dictionaries
33. Kervrann C and Boulanger J 2006 Optimal spatial adaptation for patch-based image denoising IEEE Trans. Image Process. 15 2866-78
34. Koh K, Kim S J and Boyd S 2007 An interior-point method for large-scale l1-regularized logistic regression J. Mach. Learn. Res. 8 1519-55 (Pubitemid 47210273)
35. Kreutz-Delgado K, Murray J F, Rao B D, Engan K, Lee T W and Sejnowski T J 2003 Dictionary learning algorithms for sparse representation Neural Comput. 15 349-96
36. Lewicki M S and Olshausen B A 1998 Inferring sparse, overcomplete image codes using an efficient coding framework Proc. 1997 Conf. on Advances in neural information processing systems 10 (Denver, Colorado) (Cambridge, MA: MIT Press) pp 815-21
37. Lewicki M S and Sejnowski T J 2000 Learning overcomplete representations Neural Comput. 12 337-65
38. Liao H Y and Sapiro G 2008 Sparse representations for limited data tomography 5th IEEE Int. Symp. on Biomedical Imaging: From Nano to Macro (14-17 May, Paris, France)
39. Mailh B, Lesage S, Gribonval R and Bimbot F 2008 Shift-invariant dictionary learning for sparse representations: extending k-svd Proc. 16th European Signal Processing Conf., EUSIPCO 2008 (Aug 2008, Lausanne, Switzerland)
40. Mairal J, Bach F, Ponce J, Sapiro G and Zisserman A 2008 Discriminative learned dictionaries for local image analysis IEEE Conf. on Computer Vision and Pattern Recognition Anchorage (Alaska, USA)
41. Mairal J, Leordeanu M, Bach F, Hebert M and Ponce J 2008 Discriminative sparse image models for class-specific edge detection and image interpretation Eur. Conf. on Computer Vision (Marseille, France)
42. Malioutov D M, Cetin M and Willsky A S 2004 Optimal sparse representations in general overcomplete bases IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, ICASSP '04 (17-21 May, 2004) vol 2 pp ii-793-ii-796
43. Migdalas A, Pardalos P M and Värbrand P 1998 Multilevel optimization: Algorithms and applications Nonconvex Optimization and Its Applications vol 20 (Berlin: Springer)
44. Mishali M and Eldar Y C 2007 The continuous joint sparsity prior for sparse representations: Theory and applications 2nd IEEE Int. Workshop on Computational Advances in Multi-Sensor Adaptive Processing, CAMPSAP 2007 (12-14 Dec, 2007) pp 125-8
45. Monaci G and Vandergheynst P 2004 Learning structured dictionaries for image representation Int. Conf. on Image Processing (ICIP '04) vol 4 pp 2351-4
46. Murray J F and Kreutz-Delgado K 2006 Learning sparse overcomplete codes for images J. VLSI Signal Process. 45 97-110
47. Nagy J G and O'Leary D P 2003 Image deblurring: I can see clearly now Comput. Sci. Eng. 5 82-4
48. Nocedal J and Wright S 1999 Numerical Optimization (New York: Springer)
49. Olshausen B A and Field D J 1996 Natural image statistics and efficient coding Network 7 333-9
50. Olshausen B A and Field D J 1996 Emergence of simple-cell receptive field properties by learning a sparse code for natural images Nature 381 607-9
51. Olshausen B A and Field D J 1997 Sparse coding with an overcomplete basis set: a strategy employed by v1? Vis. Res. 37 3311-25
52. Overton M 2003 A robust matlab code for nonsmooth, nonconvex optimization http://www.cs.nyu.edu/overton/papers/gradsamp/alg/
53. O'Leary D P, Hansen P C and Nagy J G 2006 Deblurring Images: Matrices, Spectra and Filtering vol 75 (Philadelphia: SIAM)
54. Poggio T and Girosi F 1998 A sparse representation for function approximation Neural Comput. 10 1445-54
55. Qiangsheng L, Qiao W and Lenan W 2004 Size of the dictionary in matching pursuit algorithm IEEE Trans. Signal Process. 52 3403-8
56. Rakhlin A 2006 Applications of empirical processes in learning theory: algorithmic stability and generalization bounds PhD Thesis Dept. of Brain and Cognitive Sciences, Massachusetts Institute of Technology
57. Robinson S M 1974 Perturbed Kuhn-Tucker points and rates of convergence for a class of nonlinear-programming algorithms Math. Program 7 1-16
58. Rudin L, Osher S and Fatemi E 1992 Nonlinear total variation based noise removal algorithms Physica D 60 259-68
59. Ryen T, Aase S O and Husoy J H 2001 Finding sparse representation of quantized frame coefficients using 0-1 integer programming Proc. 2nd Int. Symp. on Image and Signal Processing and Analysis, ISPA 2001 (19-21 June 2001) pp 541-4
60. Saab R, Chartrand R and Yilmaz O 2008 Stable sparse approximations via nonconvex optimization IEEE Int. Conf. Acoustics, Speech and Signal Processing, ICASSP 2008 (31 Mar-4 Apr 2008) pp 3885-88
61. Shawe-Taylor J and Cristianini N 2000 Support Vector Machines and Other Kernel-based Learning Methods (Cambridge: Cambridge University Press)
62. Skretting K, Engan K, Husy J H and Aas S O 2001 Sparse representation of images using overlapping frames SCIA Bergen, Norway
63. Sukjoon P, Changsoo S, Dong-Joo M and Taeyoung H 2005 Refraction traveltime tomography using damped monochromatic wavefield Geophysics 70 U1-U7
64. Tikhonov A N 1963 Solution of incorrectly formulated problems and the regularization method Dokl. Akad. Nauk. SSSR 4 1035-8
65. Tikhonov A N 1963 Solution of incorrectly formulated problems and the regularization method Sov. Math-Dokl 151 501-4 (Engl. Transl.)
66. Tikhonov A N, Leonov A S and Yagola A G 1998 Nonlinear ill-posed problems (London: Chapman and Hall) (in English)
67. Tipping M E 2001 Sparse Bayesian learning and the relevance vector machine J. Mach. Learn. Res. 1 211-44
68. Tsaig Y and Donoho D L 2006 Breakdown of equivalence between the minimal l1-norm solution and the sparsest solution Signal Process. 86 533-48
69. van den Berg E and Friedlander M P 2007 In pursuit of a root Technical Report Department of Computer Science, University of British Columbia
70. Vapnik V 1995 The Nature of Statistical Learning Theory (Berlin: Springer)
71. Vicente L N and Calamai P H 1994 Bilevel and multilevel programming: a bibliography review J. Global Optim. 5 291-306
72. Vila-Forcn J E, Voloshynovskiy O, and Koval S and Pun T 2006 Facial image compression based on structured codebooks in overcomplete domain EURASIP J. Appl. Signal Process. 11
73. Vogel C 2001 Computational Methods for Inverse Problem (Philadelphia: SIAM)
74. Wang J and Sacchi M D 2007 High-resolution wave-equation amplitude-variation-with-ray-parameter (avp) imaging with sparseness constraints Geophysics 72 S11-S18
75. Whittall K P and Oldenburg D W 1992 Inversion of Magnetotelluric Data for a One-Dimensional Conductivity (Geophysical Monograph Series vol 5) (Tulsa, OK: Society of Exploration Geophysicists)
76. Yardibi T, Li J, Stoica P and Cattafesta L N 2008 Sparsity constrained deconvolution approaches for acoustic source mapping J. Acoust. Soc. Am. 123 2631-42