Recursive least squares dictionary learning algorithm

IEEE Transactions on Signal Processing

Volumn 58, Issue 4, 2010, Pages 2121-2130

  Skretting, Karl ^a   Engan, Kjersti ^a  

^a UNIVERSITY OF STAVANGER   (Norway)

Author keywords

Dictionary design; Dictionary learning; Sparse approximation

Indexed keywords

BASIC PROPERTIES; CONVERGENCE PROPERTIES; DICTIONARY LEARNING; EXCELLENT PERFORMANCE; FORGETTING FACTORS; OVERCOMPLETE DICTIONARIES; RECURSIVE LEAST SQUARES; RECURSIVE LEAST SQUARES ALGORITHMS; SPARSE APPROXIMATION; SPARSE APPROXIMATIONS; SPARSE SIGNAL REPRESENTATION; TRAINING SETS;

ADAPTIVE ALGORITHMS; ADAPTIVE FILTERING; ADAPTIVE FILTERS; CONVERGENCE OF NUMERICAL METHODS; VECTOR QUANTIZATION;

LEARNING ALGORITHMS;

EID: 77949403269     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2010.2040671     Document Type: Article

References (34)

1. K. Engan, B. Rao, and K. Kreutz-Delgado, "Frame design using FOCUSS with method of optimized directions (MOD)," in Proc. NORSIG'99, Oslo, Norway, Sep. 1999, pp. 65-69.
2. S. F. Cotter and B. D. Rao, "Application of total least squares (TLS) to the design of sparse signal representation dictionaries," in Proc. Asilomar Conf. on Signals, Syst. Comput., Monterey, CA, Nov. 2002, vol.1, pp. 963-966.
3. K. Kreutz-Delgado, J. F. Murray, B. D. Rao, K. Engan, T.-W. Lee, and T. J. Sejnowski, "Dictionary learning algorithms for sparse representation," Neural Comput., vol.15, no.2, pp. 349-396, Feb. 2003.
4. M. Aharon, M. Elad, and A. Bruckstein, "K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation," IEEE Trans. Signal Process. vol.54, no.11, pp. 4311-4322, 2006 [Online]. Available: http://dx.doi.org/10.1109/TSP.2006.881199
5. K. Engan, K. Skretting, and J. H. Husøy, "A family of iterative LS-based dictionary learning algorithms, ILS-DLA, for sparse signal representation," Digital Signal Process., vol.17, pp. 32-49, Jan. 2007.
6. B. Mailhé, S. Lesage, R. Gribonval, and F. Bimbot, "Shift-invariant dictionary learning for sparse representations: Extending {K-SVD}," in Proc. 16th Eur. Signal Process. Conf. (EUSIPCO-2008), Lausanne, Switzerland, Aug. 2008.
7. M. Yaghoobi, T. Blumensath, and M. E. Davies, "Dictionary learning for sparse approximations with the majorization method," IEEE Trans. Signal Process., vol.109, no.6, pp. 1-14, Jun. 2009.
8. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, "Online dictionary learning for sparse coding," in Proc. 26th Ann. Int. Conf. Machine Learning ICML'09, New York, Jun. 2009, pp. 689-696, ACM.
9. K. Engan, S. O. Aase, and J. H. Husøy, "Designing frames for matching pursuit algorithms," in Proc. ICASSP'98, Seattle, WA, May 1998, pp. 1817-1820.
10. K. Engan, S. O. Aase, and J. H. Husøy, "Method of optimal directions for frame design," in Proc. ICASSP'99, Phoenix, AZ, Mar. 1999, pp. 2443-2446.
11. K. Skretting, "Sparse signal representation using overlapping frames" Ph.D. dissertation,NTNU Trondheim and Høgskolen i Stavanger, , Oct. 2002 [Online]. Available: http://www.ux.uis.no/karlsk/, available at
12. K. Skretting, J. H. Husøy, and S. O. Aase, "General design algorithm for sparse frame expansions," Elsevier Signal Process., vol.86, pp. 117-126, Jan. 2006.
13. K. Skretting, J. H. Husøy, and S. O. Aase, "A simple design of sparse signal representations using overlapping frames," in Proc. 2nd Int. Symp. Image and Signal Process. Anal. ISPA01, Pula, Croatia, Jun. 2001, pp. 424-428 [Online]. Available: http://www.ux.uis.no/karlsk/
14. K. Skretting, K. Engan, J. H. Husøy, and S. O. Aase, "Sparse representation of images using overlapping frames," in Proc. 12th Scandinavian Conf. Image Anal. SCIA 2001, Bergen, Norway, Jun. 2001, pp. 613-620 [Online]. Available: http://www.ux.uis.no/karlsk/
15. D. R. Cox, "Note on grouping," J. Amer. Statist. Assoc., vol.52, pp. 543-547, 1957.
16. S. P. Lloyd, "Least squares quantization in PCM," IEEE Trans. Inf. Theory, pp. 129-137, Mar. 1982.
17. E. Forgy, "Cluster analysis of multivariate data: Efficiency vs. interpretability of classification," Biometrics, vol.21, p. 768, 1965.
18. Y. Linde, A. Buzo, and R. M. Gray, "An algorithm for vector quantizer design," IEEE Trans. Commun., vol.28, pp. 84-95, Jan. 1980.
19. J. MacQueen, "Some methods for classification and analysis of multivariate observations," in Proc. 5th Berkeley Symp. Math. Statist. Probabil., 1967, vol.I, pp. 281-296.
20. G. Davis, "Adaptive nonlinear approximations," Ph.D. dissertation, New York Univ., New York, Sep. 1994.
21. S. F. Cotter, J. Adler, B. D. Rao, and K. Kreutz-Delgado, "Forward sequential algorithms for best basis selection," Inst. Elect. Eng. Proc. Vis. Image Signal Process., vol.146, no.5, pp. 235-244, Oct. 1999.
22. K. Skretting and J. H. Husøy, "Partial search vector selection for sparse signal representation," in Proc. NORSIG-03, Bergen, Norway, Oct. 2003 [Online]. Available: http://www.ux.uis.no/karlsk/
23. S. G. Mallat and Z. Zhang, "Matching pursuit with time-frequency dictionaries," IEEE Trans. Signal Process., vol.41, no.12, pp. 3397-3415, Dec. 1993.
24. Y. C. Pati, R. Rezaiifar, and P. S. Krishnaprasad, "Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition," in Proc. Asilomar Conf. Signals Syst. Comput., Nov. 1993.
25. M. Gharavi-Alkhansari and T. S. Huang, "A fast orthogonal matching pursuit algorithm," in Proc. ICASSP'98, Seattle, WA, May 1998, pp. 1389-1392.
26. I. F. Gorodnitsky and B. D. Rao, "Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm," IEEE Trans. Signal Process., vol.45, no.3, pp. 600-616, Mar. 1997.
27. K. Engan, B. Rao, and K. Kreutz-Delgado, "Regularized FOCUSS for subset selection in noise," in Proc. NORSIG 2000, Sweden, Jun. 2000, pp. 247-250.
28. I. Daubechies, "Ten lectures on wavelets," in Soc. Indust. Appl. Math., Philadelphia, PA, 1992, notes from the 1990 CBMS-NSF Conference on Wavelets and Applications at Lowell, MA.
29. Z. Xie and J. Feng, "KFCE: A dictionary generation algorithm for sparse representation," Elsevier Signal Process., vol.89, pp. 2072-2079, Apr. 2009.
30. C. Darken and J. E. Moody, "Note on learning rate schedules for stochastic optimization," NIPS 1990, pp. 832-838.
31. M. Y. Mashor, "Improving the performance of k-means clustering algorithm to position the centres of RBF network," Int. J. Comput., Internet and Manage., pp. 1-15, Aug. 1998.
32. L. Bottou and Y. Bengio, "Convergence properties of the means algorithms," in Advances in Neural Information Processing Systems, G. Tesauro, D. Touretzky, and T. Leen, Eds. Cambridge: The MIT Press, 1995, vol.7, pp. 585-592 [Online]. Available: citeseer.nj.nec. com/bottou95convergence.html
33. J. Munkres, "Algorithms for the assignment and transportation problems," J. Soc. Indust. Appl. Math., vol.5, no.1, pp. 32-38, Mar. 1957.
34. M. Aharon, M. Elad, and A. M. Bruckstein, "On the uniqueness of overcomplete dictionaries, and a practicalway to retrieve them," Linear Algebra Appl., vol.416, pp. 58-67, 2006.