Family of iterative LS-based dictionary learning algorithms, ILS-DLA, for sparse signal representation

Digital Signal Processing: A Review Journal

Volumn 17, Issue 1, 2007, Pages 32-49

  Engan, Kjersti ^a   Skretting, Karl ^a   Husøy, John Håkon ^a  

^a UNIVERSITY OF STAVANGER   (Norway)

Author keywords

Compression; Dictionary design; Dictionary learning; Frame design; Least squares; Matching pursuit; Overcomplete dictionary; Signal representation; Sparse approximation

Indexed keywords

DATA STRUCTURES; DIGITAL FILTERS; ITERATIVE METHODS; LEARNING SYSTEMS; MATHEMATICAL TRANSFORMATIONS; SIGNAL SYSTEMS;

DICTIONARY DESIGN; DICTIONARY LEARNING; FILTER BANKS; SPARSE APPROXIMATION; SPARSE SIGNAL REPRESENTATION; VECTOR SELECTION ALGORITHMS;

LEARNING ALGORITHMS;

EID: 37849188832     PISSN: 10512004     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.dsp.2006.02.002     Document Type: Article

References (52)

1. G. Davis, Adaptive nonlinear approximations, Ph.D. thesis, New York University, September 1994
2. S.S. Chen, Basis pursuit, Ph.D. thesis, Stanford University, November 1995
3. Mallat S.G., and Zhang Z. Matching pursuits with time-frequency dictionaries. IEEE Trans. Signal Process. 41 (1993) 3397-3415
4. Goyal V.K., Vetterli M., and Thao N.T. Quantized overcomplete expansions in RN: Analysis, synthesis, and algorithms. IEEE Trans. Inform. Theory 44 (1998) 16-31
5. Cotter S., Rao B.D., Engan K., and Kreutz-Delgado K. Sparse solutions to linear inverse problems with multiple measurement vectors. IEEE Trans. Signal Process. 53 (2005) 2477-2488
6. Rao B.D., and Kreutz-Delgado K. An affine scaling methodology for best basis selection. IEEE Trans. Signal Process. 47 (1999) 187-200
7. B.D. Rao, K. Engan, K. Kreutz-Delgado, Diversity measure minimization based basis selection methods from noisy observations, IEEE Trans. Signal Process. (2000), submitted for publication
8. K. Skretting, J.H. Husøy, Partial search vector selection for sparse signal representation, in: NORSIG-03, Bergen, Norway, October 2003, available at: http://www.ux.his.no/~karlsk/
9. Olshausen B.A., and Field D.J. Sparse coding with an overcomplete basis set: A strategy employed in V1. Vision Res. 37 (1997) 3311-3325
10. Lewicki M.S., and Olshausen B.A. Probabilistic framework for the adaption and comparison of image codes. J. Opt. Soc. Amer. A Opt. Image Sci. Vision 16 7 (1999) 1587-1601
11. Lewicki M.S., and Sejnowski T.J. Learning overcomplete representations. Neural Comput. 12 (2000) 337-365
12. Lee T.W., Lewicki M.S., Girolami M., and Sejnowski T.J. Blind source separation of more sources than mixtures using overcomplete representations. IEEE Signal Process. Lett. 6 (1999) 87-90
13. Kreutz-Delgado K., Rao B., Engan K., Lee T.-W., and Sejnowski T. Learning overcomplete dictionaries and sparse representations. Neural Comput. (2000)
14. Chou Y.-T., Hwang W.-L., and Huang C.-L. Gain-shape optimized dictionary for matching pursuit video coding. Signal Process. 83 9 (2003) 1937-1943
15. Schmid-Saugeon P., and Zakhor A. Dictionary design for matching pursuit and application to motion-compensated video coding. IEEE Trans. Circuits Syst. Video Tech. 14 (2004) 880-886
16. M. Aharon, M. Elad, A. Bruckstein, The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation, IEEE Trans. Signal Process. (2005), in press
17. M. Aharon, M. Elad, A. Bruckstein, K-SVD: Design of dictionaries for sparse representations, in: Proceedings of SPARS '05, Rennes, France, November 2005
18. S. Lesage, R. Gribonval, F. Bimbot, L. Benaroya, Learning unions of orthonormal bases with thresholded singular value decomposition, in: International Conference on Acoustics Speech and Signal Processing, Philadelphia, USA, March 2005
19. P. Jost, P. Vandergheynst, S. Lesage, R. Gribonval, Learning redundant dictionaries with translation invariance property: The motif algorithm, in: Proceedings of SPARS '05, Rennes, France, November 2005
20. K. Engan, S.O. Aase, J.H. Husøy, Method of optimal directions for frame design, in: Proceedings of ICASSP '99, Phoenix, USA, March 1999, pp. 2443-2446
21. Engan K., Aase S.O., and Husøy J.H. Multi-frame compression: Theory and design. Signal Process. 80 (2000) 2121-2140
22. Aase S.O., Husøy J.H., Skretting K., and Engan K. Optimized signal expansions for sparse representation. IEEE Trans. Signal Process. 49 (2001) 1087-1096
23. K. Skretting, K. Engan, J.H. Husøy, S.O. Aase, Sparse representation of images using overlapping frames, in: Proceedings of SCIA '01, Bergen, Norway, June 2001
24. K. Skretting, J.H. Husøy, S.O. Aase, A simple design of sparse signal representations using overlapping frames, in: Proceedings of ISPA '01, 2001, pp. 424-428
25. Kreyszig E. Advanced Engineering Mathematics. sixth ed. (1988), Wiley, New York
26. Vetterli M., and Kovačević J. Wavelets and Subband Coding (1995), Prentice Hall, Englewood Cliffs, NJ
27. Malvar H.S., and Staelin D.H. The LOT: Transform coding without blocking effects. IEEE Trans. Acoust. Speech Signal Process. 37 (1989) 553-559
28. Natarajan B.K. Sparse approximate solutions to linear systems. SIAM J. Comput. 24 (1995) 227-234
29. Chng E.S., Chen S., and Mulgrew B. Efficient computational schemes for the orthogonal least squares algorithm. IEEE Trans. Signal Process. 43 (1995) 373-376
30. M. Gharavi-Alkhansari, T.S. Huang, A fast orthogonal matching pursuit algorithm, in: International Conference on Acoustics Speech and Signal Processing, Seattle, USA, May 1998, pp. 1389-1392
31. Cotter S.F., Adler J., Rao B.D., and Kreutz-Delgado K. Forward sequential algorithms for best basis selection. IEE Proc. Vision Image Signal Process. 146 (1999) 235-244
32. Chen S.S., Donoho D.L., and Saunders M.A. Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20 (1998) 33-61
33. Gorodnitsky I.F., and Rao B.D. Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm. IEEE Trans. Signal Process. 45 (1997) 600-616
34. Rao B.D., Engan K., Cotter S.F., Palmer J., and Kreutz-Delgado K. Subset selection in noise based on diversity measure minimization. IEEE Trans. Signal Process. 51 3 (2003) 760-770
35. D.L. Donoho, M. Elad, Maximal sparsity representation via l1 minimization, in: Proceedings of National Academy of Science, vol. 100, March 2003, pp. 2197-2202
36. Gribonval R., and Nielsen M. Sparse representations in unions of bases. IEEE Trans. Inform. Theory 49 (2003) 3320-3325
37. Tropp J.A. Greed is good: Algorithmic results for sparse approximation. IEEE Trans. Inform. Theory 50 (2004) 2231-2242
38. Fuchs J.J. Recovery of exact sparse representations in the presence of bounded noise. IEEE Trans. Inform. Theory 51 (2005) 3601-3608
39. K. Engan, Frame based signal representation and compression, Ph.D. thesis, University of Stavanger/NTNU, Stavanger, Norway, September 2000, available at: http://www.ux.uis.no/~kjersti/
40. K. Engan, J.R. Akselsen, Comparison of frame based coding (FBC) with multi stage vector quantization (MSVQ), in: Proceedings NORSIG '01, Trondheim, Norway, October 2001, pp. 123-128
41. Gersho A., and Gray R.M. Vector Quantization and Signal Compression (1992), Kluwer, Boston
42. M. Aharon, M. Elad, A. Bruckstein, On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them, J. Linear Algebra Appl. (2005), in press
43. M. Aharon, M. Elad, A. Bruckstein, K-SVD and its non-negative variant for dictionary design, in: Proceedings of the SPIE Conference Wavelets, vol. 5914, July 2005
44. K. Skretting, Sparse signal representation using overlapping frames, Ph.D. thesis, Stavanger University College/NTNU, Stavanger, Norway, 2002, available at: http://www.ux.his.no/~karlsk/
45. Skretting K., Husøy J.H., and Aase S.O. General design algorithm for sparse frame expansions. Elsevier Signal Process. 86 (2006) 117-126
46. K. Skretting, J.H. Husøy, Texture classification using sparse frame based representation, J. Appl. Signal Process., in press
47. J. Herredsvela, K. Engan, T.O. Gulsrud, K. Skretting, Texture classification using sparse representations by learned compound dictionaries, in: Proceedings of SPARS '05, Rennes, France, November 2005
48. Jalaleddine S.M.S., Hutchens C.G., Strattan R.D., and Coberly W.A. ECG data compression techniques-A unified approach. IEEE Trans. Biomed. Eng. 37 (1990) 329-343
49. Lu Z., Kim D.Y., and Pearlman W.A. Wavelet compression of ECG signals by the set partitioning in hierarchical trees algorithm. IEEE Trans. Biomed. Eng. 47 (2000) 849-856
50. Zigel Y., Cohen A., and Katz A. ECG signal compression using analysis by synthesis coding. IEEE Trans. Biomed. Eng. 47 (2000) 1308-1316
51. Nygaard R., Melnikov G., and Katsaggelos A.K. A rate distortion optimal ECG coding algorithm. IEEE Trans. Biomed. Eng. 48 (2001) 28-40
52. The MIT-BIH Arrhythmia Database CD-ROM, second ed., MIT Press, Cambridge, 1992