Sparse representation for computer vision and pattern recognition

Proceedings of the IEEE

Volumn 98, Issue 6, 2010, Pages 1031-1044

  Wright, John ^a,b   Ma, Yi ^a,b   Mairal, Julien ^c   Sapiro, Guillermo ^d   Huang, Thomas S ^a   Yan, Shuicheng ^e  

^a University of Illinois at Urbana Champaign   (United States)

^b MICROSOFT RESEARCH ASIA   (China)

^c ECOLE NORMALE SUPÉRIEURE   (France)

^d University of Minnesota   (United States)

^e NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Compressed sensing; Computer vision; Pattern recognition; Signal representations

Indexed keywords

COMPRESSED SENSING; PATTERN RECOGNITION; SEMANTICS;

HIGH-FIDELITY; REVIEW PAPERS; SEMANTIC INFORMATION; SIGNAL REPRESENTATIONS; SPARSE REPRESENTATION; SPARSE SIGNAL REPRESENTATION; STATE OF THE ART; TRAINING SAMPLE;

COMPUTER VISION;

EID: 77952717202     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2010.2044470     Document Type: Article

References (98)

1. J. Blackard, D. Dean, and C. Anderson, "Forest coverType dataset," 1998. [Online]. Available: http://kdd.ics.uci.edu/databases/ covertype/covertype.data.html
2. B. Liebe and B. Schiele, "Analyzing appearance and contour based methods for object categorization," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2003, pp. 409-415. [Online]. Available: http://tahiti.mis. informatik.tu-darmstadt.de/oldmis/Research/Projects/categorization/eth80-db. html.
3. Compressive Sensing Resources, Rice University, Accessed 2010. [Online]. Available: http://www.dsp.ece.rice.edu/cs/
4. M. Aharon, M. Elad, and A. M. Bruckstein, "The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representations," IEEE Trans. Signal Process., vol.54, no.11, pp. 4311-4322, Nov. 2006. (Pubitemid 44637761)
5. S. P. Awate and R. T. Whitaker, "Unsupervised, information- theoretic, adaptive image filtering for image restoration," IEEE Trans. Pattern Anal. Mach. Intell., vol.28, no.3, pp. 364-376, Mar. 2006. (Pubitemid 43159633)
6. P. Baheti and M. Neifeld, "Feature-specific structured imaging," Appl. Opt., vol.45, no.28, pp. 7382-7391, 2006. (Pubitemid 44719857)
7. P. Baheti and M. Neifeld, "Adaptive feature specific imaging: A face recognition example," Appl. Opt., vol.47, no.10, pp. 821-831, 2008.
8. P. Baheti and M. Neifeld, "Random projections based feature-specific structured imaging," Opt. Exp., vol.16, no.3, pp. 1764-1776, 2009. (Pubitemid 351240018)
9. R. Basri and D. Jacobs, "Lambertian reflection and linear subspaces," IEEE Trans. Pattern Anal. Mach. Intell., vol.25, no.3, pp. 218-233, Mar. 2003.
10. P. Belhumeur, J. Hespanda, and D. Kriegman, "Eigenfaces vs. fisherfaces: Recognition using class specific linear projection," IEEE Trans. Pattern Anal. Mach. Intell., vol.19, no.7, pp. 711-720, Jul. 1997. (Pubitemid 127762263)
11. M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation," Neural Comput., vol.15, pp. 1373-1396, 2003. (Pubitemid 37049796)
12. A. Bruckstein, D. Donoho, and M. Elad, "From sparse solutions of systems of equations to sparse modeling of signals and images," SIAM Rev., vol.51, no.1, pp. 34-81, 2009.
13. O. Bryt and M. Elad, "Compression of facial images using the K-SVD algorithm," J. Vis. Commun. Image Represent., vol.19, pp. 270-283, 2008.
14. A. Buades, B. Coll, and J. Morel, "A review of image denoising algorithms, with a new one," SIAM J. Multiscale Model. Simul., vol. 4, no. 2, pp. 490-530, 2005. (Pubitemid 43890582)
15. J. Cai, H. Ji, X. Liu, and Z. Shen, "Blind motion deblurring from a single image using sparse approximation," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2009, pp. 104-111.
16. 1-magic: Recovery of sparse signals via convex programming. [Online]. Available: http://www.acm.caltech. edu/l1magic/
17. E. Candès and T. Tao, "Decoding by linear programming," IEEE Trans. Inf. Theory, vol.51, no.12, pp. 4203-4215, Dec. 2005. (Pubitemid 41800353)
18. E. J. Candès, "Compressive sampling," in Proc. Int. Congr. Mathematicians, Madrid, Spain, 2006, vol.3, pp. 1433-1452.
19. 1 minimization," J. Fourier Anal. Appl., vol.14, no.5-6, pp. 877-905, 2008.
20. V. Cevher, A. C. Sankaranarayanan, M. F. Duarte, D. Reddy, R. G. Baraniuk, and R. Chellappa, "Compressive sensing for background subtraction," presented at the Eur. Conf. Comput. Vis., Marseille, France, Oct. 12-18, 2008.
21. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol.20, no.1, pp. 33-61, 1998.
22. 1-graph for image analysis," IEEE Trans. Image Process., 2010.
23. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, "Color image denoising by sparse 3D collaborative filtering with grouping constraint in luminance-chrominance space," in Proc. IEEE Int. Conf. Image Process., San Antonio, TX, Sep. 2007, pp. 313-316.
24. I. Daubechies, M. Defrise, and C. D. Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint," Commun. Pure Appl. Math., vol.57, pp. 1413-1457, 2004. (Pubitemid 39427442)
25. M. Dikmen and T. Huang, "Robust estimation of foreground in surveillance video by sparse error estimation," in Proc. 19th Int. Conf. Pattern Recognit., 2008. DOI: 10.1109/ICPR.2008.4761910.
26. Y. Dong, D. Liu, D. Dunson, and L. Carin, "Bayesian multi-task compressive sensing with Dirichlet process priors," IEEE Trans. Pattern Anal. Mach. Intell., 2009.
27. D. Donoho, "Neighborly polytopes and sparse solution of underdetermined linear equations," 2005. preprint.
28. 1-norm solution is also the sparsest solution," Commun. Pure Appl. Math., vol.59, no.6, pp. 797-829, 2006. (Pubitemid 43667226)
29. 1 minimization," Proc. Nat. Acad. Sci., pp. 2197-2202, Mar. 2003. (Pubitemid 36297476)
30. 1-norm minimization problems when the solution may be sparse, 2006, preprint. [Online]. Available: http://www.stanford. edu/tsaig/research.html
31. D. L. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol.52, no.4, pp. 1289-1306, Apr. 2006.
32. D. L. Donoho and M. Elad, "Optimal sparse representation in general (nonorthogoinal) dictionaries via l1 minimization," in Proc. Nat. Acad. Sci., Mar. 2003, vol.100, pp. 2197-2202. (Pubitemid 36297476)
33. J. M. Duarte-Carvajalino and G. Sapiro, "Learning to sense sparse signals: Simultaneous sensing matrix and sparsifying dictionary optimization," IEEE Trans. Image Process., vol.18, no.7, pp. 1395-1408, Jul. 2009.
34. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, "Least angle regression," Ann. Stat., vol.32, no.2, pp. 407-499, 2004. (Pubitemid 41250302)
35. M. Elad, "Optimized projections for compressed-sensing," IEEE Trans. Signal Process., vol.55, no.12, pp. 5695-5702, Dec. 2007.
36. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries," IEEE Trans. Image Process., vol.54, no.12, pp. 3736-3745, Dec. 2006. (Pubitemid 44811686)
37. E. Elhamifar and R. Vidal, "Sparse subspace clustering," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2009, pp. 2790-2797.
38. K. Engan, S. O. Aase, and J. H. Husoy, "Frame based signal compression using method of optimal directions (MOD)," in Proc. IEEE Int. Symp. Circuits Syst., Jul. 1999, vol.4, DOI: 10.1109/ISCAS.1999.779928.
39. M. A. T. Figueiredo, "Adaptive sparseness using Jeffreys prior," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2001, pp. 697-704.
40. J. Gemmeke and B. Cranen, "Noise robust digit recognition using sparse representations," in Int. Speech Commun. Assoc. Tutorial Res. Workshop, 2008.
41. J. Gemmeke and B. Cranen, "Using sparse representations for missing data imputation in noise robust speech recognition," in Proc. EUSIPCO, 2008. [Online]. Available: http://www.eurasip.org/Proceedings/Eusipco/ Eusipco2008/papers/1569102857.pdf.
42. A. Georghiades, P. Belhumeur, and D. Kriegman, "From few to many: Illumination cone models for face recognition under variable lighting and pose," IEEE Trans. Pattern Anal. Mach. Intell., vol.23, no.6, pp. 643-660, Jun. 2001. (Pubitemid 32585276)
43. R. Giryes, Y. C. Eldar, and M. Elad, "Automatic parameter setting for iterative shrinkage methods," in Proc. IEEE 25th Conv. Electron. Electr. Eng. Israel, Dec. 2008, pp. 820-824.
44. R. Gribonval and M. Nielsen, "Sparse representations in unions of bases," IEEE Trans. Inf. Theory, vol.49, no.12, pp. 3320-3325, Dec. 2003.
45. R. Grosse, R. Raina, H. Kwong, and A. Ng, "Shift-invariant sparse coding for audio classification," in Proc. Conf. Uncertainty Artif. Intell., 2007.
46. 1-minimizatoin: Methodology and convergence," SIAM J. Optim., vol.19, no.3, pp. 1107-1130, 2008.
47. X. Hang and F. Wu, "Sparse representation for classification of tumors using gene expression data," J. Biomed. Biotechnol., 2009, Article ID 403689, 6 p.
48. J. Hull, "A database for handwritten text recognition research," IEEE Trans. Pattern Anal. Mach. Intell., vol.16, no.5, pp. 550-554, May 1994.
49. I. Joliffe, Principal Component Analysis. New York: Springer-Verlag, 1986.
50. J. Kim, J. Choi, J. Yi, and M. Turk, "Effective representation using ICA for face recognition robust to local distortion and partial occlusion," IEEE Trans. Pattern Anal. Mach. Intell., vol.27, no.12, pp. 1977-1981, Dec. 2005. (Pubitemid 43051393)
51. B. Krishnapuram, L. Carin, M. A. T. Figueiredo, and A. J. Hartemink, "Sparse multinomial logistic regression: Fast algorithms and generalization bounds," IEEE Trans. Pattern Anal. Mach. Intell., vol.27, no.6, pp. 957-968, Jun. 2005. (Pubitemid 40889845)
52. K. Lee, J. Ho, and D. Kriegman, "Acquiring linear subspaces for face recognition under variable lighting," IEEE Trans. Pattern Anal. Mach. Intell., vol.27, no.5, pp. 684-698, May 2005. (Pubitemid 40608402)
53. S. Li, X. Hou, H. Zhang, and Q. Cheng, "Learning spatially localized, parts-based representation," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2001, vol.1, pp. 207-212. (Pubitemid 34063776)
54. X. Li, T. Jia, and R. Zhang, "Expression-insensitive 3D face recognition using sparse representation," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2009, pp. 2575-2582.
55. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Learning discriminative dictionaries for local image analysis," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2008, DOI: 10.1109/CVPR.2008. 4587652.
56. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Non-local sparse models for image restoration," in Proc. IEEE Int. Conf. Comput. Vis., 2009.
57. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Supervised dictionary learning," in Advances in Neural Information Processing Systems, vol.21, D. Koller, D. Schuurmans, Y. Bengio, and L. Bottou, Eds. Cambridge, MA: MIT Press, 2009.
58. J. Mairal, M. Elad, and G. Sapiro, "Sparse representation for color image restoration," IEEE Trans. Image Process., vol.17, no.1, pp. 53-69, Jan. 2008.
59. J. Mairal, M. Leordeanu, F. Bach, M. Hebert, and J. Ponce, "Discriminative sparse image models for class-specific edge detection and image interpretation," presented at the Eur. Conf. Comput. Vis., Marseille, France, Oct.12-18, 2008.
60. J. Mairal, G. Sapiro, and M. Elad, "Learning multiscale sparse representations for image and video restoration," SIAM Multiscale Modeling and Simulation, vol.7, no.1, pp. 214-241, Apr. 2008.
61. X. Mei, H. Ling, and D. Jacobs, "Sparse representation of cast shadows via l1-regularized least squares," in Proc. IEEE Int. Conf. Comput. Vis., 2009.
62. P. Nagesh and B. Li, "A compressive sensing approach for expression-invariant face recognition," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2009, pp. 1518-1525.
63. B. A. Olshausen and D. J. Field, "Sparse coding with an overcomplete basis set: A strategy employed by v1?" Vis. Res., vol.37, pp. 3311-3325, 1997.
64. G. Peyre, "Sparse modeling of textures," 2007, preprint ceremade 2007-2015
65. A. Quattoni, M. Collins, and T. Darrell, "Transfer learning for image classification with sparse prototype representation," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2008. DOI: 10.1109/CVPR.2008.4587637.
66. R. Raina, A. Battle, H. Lee, B. Packer, and A. Y. Ng, "Self-taught learning: Transfer learning from unlabeled data," in Proc. Int. Conf. Mach. Learn., 2007, pp. 759-766. (Pubitemid 47275135)
67. I. Ramirez, F. Lecumberry, and G. Sapiro, Sparse modeling with universal priors and learned incoherent dictionaries. [Online]. Available: http://www.ima.umn.edu/preprints/sep2009/2279.pdf
68. S. Rao, R. Tron, R. Vidal, and Y. Ma, "Motion segmentation via robust subspace separation in the presence of outlying, incomplete, and corrupted trajectories," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2008, DOI: 10.1109/CVPR.2008.4587437.
69. 1-minimization," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2009, pp. 2350-2357.
70. F. Rodriguez and G. Sapiro, "Sparse representations for image classification: Learning discriminative and reconstructive non-parametric dictionaries," Univ. Minnesota, Minneapolis, MN, Tech. Rep./IMA Preprint, Dec. 2007.
71. S. Roweis and L. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol.290, no.22, pp. 2323-2326, 2000. (Pubitemid 32041578)
72. K. Schnass and P. Vandergheynst, "Dictionary preconditioning for greedy algorithms," IEEE Trans. Signal Process., vol.56, no.5, pp. 1994-2002, May 2008.
73. M. Seeger, "Bayesian inference and optimal design in the sparse linear model," J. Mach. Learn. Res., vol.9, pp. 759-813, 2008.
74. J. Shi and J. Malik, "Normalized cuts and image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol.22, no.8, pp. 888-905, Aug. 2000.
75. A. Szlam and G. Sapiro, "Discriminative k-metrics," 2009, preprint.
76. D. Takhar, J. Laska, M. Wakin, M. Duarte, D. Baron, S. Sarvotham, K. Kelly, and R. Baraniuk, "A new compressive imaging camera architecture using optical domain compression," in Proc. Comput. Imaging IV SPIE Electron. Imaging, 2006, pp. 43-52.
77. J. Tenenbaum, V. de Silva, and J. Langford, "A global geometric framework for dimensionality reduction," Science, vol.290, no.5500, pp. 2319-2323, 2000. (Pubitemid 32041577)
78. R. Tibshirani, "Regression shrinkage and selection via the LASSO," J. R. Stat. Soc., vol.58, no.1, pp. 267-288, 1996.
79. M. E. Tipping, "Sparse bayesian learning and the relevance vector machine," J. Mach. Learn. Res., vol.1, pp. 211-244, 2001. (Pubitemid 33687203)
80. J. Tropp and S. Wright, "Computational methods for sparse solution of linear inverse problems," California Inst. Technol., Pasadena, CA, ACM Tech. Rep. 2009-1, 2009.
81. J. A. Tropp, "Greed is good: Algorithmic results for sparse approximation," IEEE Trans. Inf. Theory, vol.50, no.10, pp. 2231-2242, Oct. 2004.
82. M. Turk and A. Pentland, "Face recognition using Eigenfaces," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 1991, pp. 586-591.
83. A. Wagner, J. Wright, A. Ganesh, Z. Zhou, and Y. Ma, "Towards a practical face recognition system: Robust registration and illumination by sparse representation," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2009, pp. 597-604.
84. C. Wang, S. Yan, L. Zhang, and H. Zhang, "Multi-label sparse coding for automatic image annotation," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2009, pp. 1643-1650.
85. Y. Weiss, H. Chang, and W. Freeman, "Learning compressed sensing," in Proc. Allerton Conf. Commun. Control Comput., 2007. [Online]. Available: http://www.cs.huji.ac.il/~yweiss/allerton-final.pdf.
86. 1-minimization," IEEE Trans. Inf. Theory, 2009.
87. J. Wright, A. Yang, A. Ganesh, S. Sastry, and Y. Ma, "Robust face recognition via sparse representation," IEEE Trans. Pattern Anal. Mach. Intell., vol.30, no.2, pp. 210-227, Feb. 2009.
88. S. Yan and H. Wang, "Semi-supervised learning by sparse representation," in SIAM Int. Conf. Data Mining, pp. 792-801.
89. S. Yan, D. Xu, B. Zhang, Q. Yang, H. Zhang, and S. Lin, "Graph embedding and extensions: A general framework for dimensionality reduction," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 1, pp. 40-51, Jan. 2007. (Pubitemid 46415944)
90. A. Yang, R. Jafari, S. Sastry, and R. Bajcsy, "Distributed recognition of human actions using wearable motion sensor networks," J. Ambient Intell. Sensor Environ., vol.1, no.2, pp. 103-115, 2009.
91. A. Yang, S. Maji, K. Hong, P. Yan, and S. Sastry, "Distributed compression and fusion of nonnegative sparse signals for multiple-view object recognition," in Proc. Int. Conf. Inf. Fusion, 2009, pp. 1867-1874.
92. J. Yang, J. Wright, T. Huang, and Y. Ma, "Image superresolution as sparse representation of raw patches," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2008. DOI: 10.1109/CVPR.2008.4587647.
93. J. Yang, K. Yu, Y. Gong, and T. Huang, "Linear spatial pyramid matching using sparse coding for image classification," in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit., 2009, pp. 1794-1801.
94. L. Yang, Distance metric learning: A comprehensive survey. [Online]. Available: http://www.cse.msu.edu/~yangliu1/frame-survey-v2.pdf
95. W. Zhao, R. Chellappa, J. Phillips, and A. Rosenfeld, "Face recognition: A literature survey," ACM Comput. Surv., pp. 399-458, 2003.
96. Z. Zhou, A. Wagner, H. Mobahi, J. Wright, and Y. Ma, "Face recognition with contiguous occlusion using markov random fields," in Proc. IEEE Int. Conf. Comput. Vis., 2009.
97. X. Zhu, "Semi-supervised learning literature survey," Dept. Comput. Sci., Univ. Wisconsin, Madison, WI, Tech. Rep. 1530, 2005.
98. H. Zou, "The adaptive LASSO and its oracle properties," J. Amer. Stat. Assoc., vol.101, pp. 1418-1429, 2006.