Learning sparse representation using iterative subspace identification

IEEE Transactions on Signal Processing

Volumn 58, Issue 6, 2010, Pages 3055-3065

  Gowreesunker, B Vikrham ^a   Tewfik, Ahmed H ^b  

^a TEXAS INSTRUMENTS   (United States)

^b University of Minnesota   (United States)

Author keywords

Block sparseness; Dictionary training; Iterative subspace identification; Overcomplete dictionary design; Shift tolerance; Sparse representation

Indexed keywords

COMPLEMENTARY OPTIMIZATION METHOD; DATA DRIVEN; DICTIONARY TRAINING; LEARNING SUBSPACES; OVERCOMPLETE DICTIONARIES; PRIOR KNOWLEDGE; SPARSE REPRESENTATION; SUBSPACE IDENTIFICATION; SUBSPACE IDENTIFICATION METHODS; TRAINING PROCEDURES; TRAINING TIME;

BLIND SOURCE SEPARATION; INDEPENDENT COMPONENT ANALYSIS; INTERSYMBOL INTERFERENCE;

LEARNING ALGORITHMS;

EID: 77952641783     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2010.2044251     Document Type: Article

References (52)

1. S. Mallat, A Wavelet Tour of Signal Processing, 2nd ed. New York: Academic, 1999.
2. D. L. Donoho and M. R. Duncan, "Digital curvelet transform: Strategy, implementation and experiments, " in Proc. Aerosense 2000, Wavelet Applications VII, 2000, pp. 12-29, SPIE.
3. D. L. Donoho, X. Huo, I. Jermyn, P. Jones, G. Lerman, O. Levi, and F. Natterer, "Beamlets and multiscale image analysis, " in Multiscale and Multiresolution Methods. New York: Springer, 2001, pp. 149-196.
4. E. Candes, Ridgelets: Theory and Applications. Stanford, CA: Department of Statistics, Stanford Univ., 1998.
5. S. Mallat and Z. Zhang, "Matching pursuit with time-frequency dictionaries, " IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397-3415, Dec. 1993.
6. T. Strohmer, "A unified approach to numerical algorithms for discrete Gabor expansions, " in Proc. SampTA-Sampling Theory and Applications, Aveiro/Portugal, 1997, pp. 297-302.
7. M. Aharon, M. Elad, and A. Bruckstein, "The k-svd: An algorithm for designing of overcomplete dictionaries for sparse representation, " IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, Nov. 2006.
8. B. Olshausen and D. Field, "Natural image statistics and efficient coding, " in Proc. Network: Comput. Neural Syst., 1996, vol. 7, pp. 333-339.
9. B. A. Olshausen and D. J. Field, "Sparse coding with an overcom-plete basis set: A strategy employed by v1?, " Vision Res., vol. 37, pp. 3311-3325, 1998.
10. M. S. Lewicki and B. A. Olshausen, "A probabilistic framework for the adaptation and comparison of image codes, " J. Opt. Soc. Amer. A, vol. 16, pp. 1587-1601, 1999.
11. B. V. Gowreesunker and A. H. Tewfik, "A novel subspace clustering method for dictionary design, " inProc. ICA, 2009, vol. 5441, pp. 34-41.
12. B. V. Gowreesunker and A. H. Tewfik, "A shift tolerant dictionary training method, " presented at the Signal Processing With Adaptive Sparse Structured Representations (SPARS), Saint Malo, France, 2009, INRIA Rennes-Bretagne Atlantique.
13. M. Zibulevsky and B. A. Pearlmutter, "Blind source separation by sparse decomposition in a signal dictionary, " Neural Comput., vol. 13, pp. 863-882, 2000.
14. P. Bofill and M. Zibulevsky, "Underdetermined blind source separation using sparse representation, " Signal Process., vol. 81, no. 11, pp. 2353-2362.
15. C. Fevotte and S. Godsill, "A Bayesian approach for blind separation of sparse sources, " IEEE Trans. Speech Audio Process., vol. 14, no. 6, pp. 2174-2188, Nov. 2006.
16. R. Gribonval, "Sparse decomposition of stereo signals with matching pursuit and application to blind separation of more than two sources from a stereo mixture, " in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing (ICASSP), May 2002, vol. 3, pp. 3057-3060.
17. S. Lesage, S. Krstulovic, and R. Gribonval, "Under-determined source separation: Comparison of two approaches based on sparse decomposition, " in Proc. Independent Component Analysis Signal Separation (ICA), Mar. 2006, pp. 633-640.
18. O. Yilmaz and S. Rickard, "Blind separation of speech mixtures via time-frequency masking, " IEEE Trans. Signal Process., vol. 52, no. 7, pp. 1830-1847, Jul. 2004.
19. B. V. Gowreesunker and A. H. Tewfik, "Two improved sparse decomposition methods for blind source separation, " in Proc. ICA, 2007, vol. 4666, Lecture Notes in Computer Science, pp. 365-372.
20. D. Wang and A. Tewfik, "Sparse representation of deformable 3D organs, " in Proc. ISBI, 2009, to be published.
21. B. Gowreesunker, A. Tewfik, V. A. Tadipatri, N. F. Ince, J. Ashe, and G. Pellizzer, "Overcoming measurement time variability in brain machine interfaces, " in Proc. 31st Annu. Int. Conf. IEEE Engineering Medicine Biology Soc. (EMBS), Sep. 2009, pp. 3134-3137.
22. W. Dai and O. Milenkovic, "Subspace pursuit for compressive sensing: Closing the gap between performance and complexity, " CoRR, vol. abs/0803. 0811, 2008 [Online]. Available: http://arxiv.org/abs/0803.0811
23. A. Ganesh, Z. Zhou, and Y. Ma, "Separation of subspace-sparse signal: Algorithm and conditions, " in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing (ICASSP), Apr. 2009.
24. Y. C. Eldar and H. Bolcskei, "Block-sparsity: Coherence and efficient recovery, " in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing (ICASSP), Apr. 2009.
25. S. Chen and D. Donoho, "Atomic decomposition by basis pursuit, " SIAM J. Scientif. Comput., vol. 20, no. 1, pp. 33-61, 1998.
26. Y. C. Pati, R. Rezaiifar, Y. C. P. R. Rezaiifar, and P. S. Krishnaprasad, "Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition, " in Proc. 27th Annu. Asilomar Conf. Signals, Systems, Computers, 1993, pp. 40-44.
27. M. Alghoniemy and A. H. Tewfik, "A sparse solution to the bounded subset selection problem: A network flow model approach, " in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing (ICASSP), May 2004, vol. 5, pp. 89-92.
28. S. Chen, S. A. Billings, and W. Luo, "Orthogonal least squares methods and their application to non-linear system identication, " Int. J. Control, vol. 50, no. 5, pp. 1873-1896, 1989.
29. S. Chen, C. Cowan, and P. Grant, "Orthogonal least squares learning algorithm for radial basis function networks, " IEEE Trans. Neural Netw., vol. 2, no. 2, pp. 302-309, 1991.
30. J. A. Tropp, A. C. Gilbert, and M. J. Strauss, "Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit, " Signal Process., vol. 86, no. 3, pp. 572-588, Mar. 2006.
31. J. A. Tropp, "Algorithms for simultaneous sparse approximation. Part II: Convex relaxation, " Signal Process., vol. 86, no. 3, pp. 589-602, Mar. 2006.
32. A. Gersho and R. M. Gray, Vector Quantization and Signal Compression. Norwell, MA: Kluwer Academic, 1991.
33. M. S. Lewicki, T. J. Sejnowski, and H. Hughes, "Learning overcom-plete representations, " Neural Comput., vol. 12, pp. 337-365, 1998.
34. Kreutz-Delgado, K. Murray, J. F. Rao, B. D. Engan, T.-W. K. Lee, and T. J. Sejnowski, "Dictionary learning algorithms for sparse representation, " Neural Comput., vol. 15, pp. 349-396, 2003.
35. K. Engan, S. O. Aase, and J. H. Husy, "Multi-frame compression: Theory and design, " EURASIP Signal Process., vol. 80, pp. 2121-2140, 2000.
36. K. Engan, B. D. Rao, and K. Kreutz-Delgado, "Frame design using FO-CUSS with method of optimal directions (MOD), " in Proc. NORSIG, Tronheim, Norway, Jun. 1999, pp. 65-69.
37. J. F. Murray and K. Kreutz-Delgado, "An improved FOCUSS-based learning algorithm for solving sparse linear inverse problems, " in Proc. Conf. Rec. 35rd Asilomar Conf. Signals, Systems, Computers, Nov. 2001.
38. K. Kreutz-Delgado and B. D. Rao, "FOCUSS-based dictionary learning algorithms, " in Proc. SPIE Wavelet Applications in Signal and Image Processing VIII, Bellingham, WA, Jul.-Aug. 2000, vol. 4119-53.
39. M. Yaghoobi, T. Blumensath, and M. E. Davies, "Dictionary learning for sparse approximations with the majorization method, " IEEE Trans. Signal Process., vol. 57, no. 6, pp. 2178-2191, Jun. 2009.
40. S. Lesage, R. Gribonval, F. Bimbot, and L. Benaroya, "Learning unions of orthonormal bases with thresholded singular value decomposition, " in Proc. Int. Conf. Acoustics, Speech, Signal Processing (ICASSP), Mar. 2005, vol. 5, pp. v/293-v/296.
41. M. G. Jafari and M. D. Plumbley, "An adaptive orthogonal sparsifying transform for speech signals, " in Proc. 3rd Int. Symp. Communications, Control, Signal Processing (ISCCSP), Malta, Mar. 12-14, 2008, pp. 786-790.
42. T. Blumensath and M. Davies, "Sparse and shift-invariant representations of music, " IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 1, pp. 50-57, Jan. 2006.
43. P. S. Bradley and O. L. Mangasarian, "k-plane clustering, " J. Global Optimiz., vol. 16, pp. 23-32, 2000.
44. P. Tseng, "Nearest q-flat to m points, " J. Optimiz. Theory Appl., vol. 105, pp. 249-252, 2000.
45. T. Blumensath and M. Davies, "On the difference between orthogonal matching pursuit and orthogonal least squares, " 2007 [Online]. Available: http://www.see.ed.ac.uk/~tblumens/papers/BDOMPvsOLS07.pdf
46. B. Mailhe, S. Lesage, R. Gribonval, and F. Bimbot, "Shift-invariant dictionary learning for sparse representations: Extending k-SVD, " presented at the 16th Eur. Signal Processing Conf. (EUSIPCO), Lausanne, Switzerland, Aug. 25-29, 2008.
47. P. Jost, P. Vandergheynst, S. Lesage, and R. Gribonval, "Motif: An efficient algorithm for learning translation invariant dictionaries, " in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing (ICASSP), May 2006, vol. 5, pp. V-V.
48. D. Wang and A. Tewfik, "In vivo tracking of 3d organs using spherical harmonics and subspace clustering, " in Proc. Int. Conf. Image Processing (ICIP), 2009.
49. Signal Separation Evaluation Campaign. [Online]. Available: http://sisec.wiki.irisa.fr
50. K-svd Matlab Toolbox. [Online]. Available: http://www.cs.tech-nion. ac.il/elad/software
51. B. Gowreesunker and A. Tewfik, "Blind source separation using monochannel overcomplete dictionaries, " in Proc. IEEE Int. Conf. Acoustics, Speech, Signal Processing (ICASSP), Apr. 4-31, 2008, pp. 33-36.
52. E. Vincent, C. Fevotte, and R. Gribonval, "Performance measurement in blind audio source separation, " IEEE Trans. Audio, Speech, Lang. Process., vol. 14, no. 4, pp. 1462-1469, 2006. (Pubitemid 46547636)