Beyond sparsity: The role of L1-optimizer in pattern classification

Pattern Recognition

Volumn 45, Issue 3, 2012, Pages 1104-1118

  Yang, Jian ^a,b   Zhang, Lei ^c   Xu, Yong ^d   Yang, Jing Yu ^a  

^a NANJING UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

^b CALIFORNIA INSTITUTE OF TECHNOLOGY   (United States)

^c HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

^d HARBIN INSTITUTE OF TECHNOLOGY   (China)

Author keywords

Classifier; Feature extraction; Pattern classification; Sparse representation

Indexed keywords

CLASS LABELS; CLASSIFICATION PERFORMANCE; OPTIMIZERS; RULE BASED; SAMPLE SIZE PROBLEMS; SPARSE REPRESENTATION; TEST SAMPLES; TRAINING SAMPLE;

FEATURE EXTRACTION;

COMPUTATIONAL EFFICIENCY;

EID: 80055016825     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2011.08.022     Document Type: Article

References (53)

1. W.E. Vinje, and J.L. Gallant Sparse coding and decorrelation in primary visual cortex during natural vision Science 287 5456 2000 1273 1276 (Pubitemid 30112150)
2. B.A. Olshausen, and D.J. Field Sparse coding of sensory inputs Current Opinion in Neurobiology 14 4 2004 481 487 (Pubitemid 39078497)
3. T. Serre, Learning a Dictionary of Shape-Components in Visual Cortex: Comparison with Neurons, Humans and Machines, Ph.D. Dissertation, MIT, 2006.
4. E. Candès, J. Romberg, and T. Tao Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information IEEE Transactions on Information Theory 52 2 2006 489 509 (February) (Pubitemid 43174093)
5. E. Candès, and T. Tao Near optimal signal recovery from random projections: universal encoding strategies? IEEE Transactions on Information Theory 52 12 2006 5406 5425 (December) (Pubitemid 46445381)
6. D. Donoho Compressed sensing IEEE Transactions on Information Theory 52 4 2006 1289 1306 (April)
7. E.J. Candes, and M.B. Wakin An introduction to compressive sampling IEEE Signal Processing Magazine 25 2 2008 21 30 (21 March)
8. S. Mallat, and Z. Zhang Matching pursuits with time-frequency dictionaries IEEE Transactions on Signal Processing 41 12 1993 3397 3415
9. M. Aharon, M. Elad, and A. Bruckstein The K-SVD: an algorithm for designing of overcomplete dictionaries for sparse representation, IEEE Transactions on Signal Processing 54 11 2006 4311 4322 (Pubitemid 44637761)
10. J. Bobin, J.-L. Starck, J. Fadili, Y. Moudden, and D.L. Donoho Morphological component analysis: an adaptative thresholding strategy IEEE Transactions on Image Processing 16 11 2007 2675 2681 (Pubitemid 350147649)
11. K. Labusch, E. Barth, and T. Martinetz Simple method for high-performance digit recognition based on sparse coding IEEE Transactions on Neural Networks 19 11 2008
12. H. Zhou, T. Hastie, R. Tibshirani, Sparse Principle Component Analysis, Technical Report, Statistics Department, Stanford University, 2004.
13. A. dAspremont, L. El Ghaoui, M.I. Jordan, and G.R.G. Lanckriet A direct formulation for sparse PCA using semidefinite programming Advances in Neural Information Processing Systems (NIPS) 2004 (December)
14. B. Moghaddam, Y. Weiss, and S. Avidan Spectral bounds for sparse PCA: exact and greedy algorithms Advances in Neural Information Processing Systems 18 2005
15. B. Moghaddam, Y. Weiss, S., Avidan, Generalized spectral bounds for sparse LDA, in: ICML 06: Proceedings of the 23rd International Conference on Machine Learning, 2006, pp. 641648.
16. D. Cai, X. He, J. Han, Sparse projections over graph, in: Proceedings of AAAI Conference on Artificial Intelligence (AAAI-08), Chicago, Illinois, July 2008.
17. L.S. Qiao, S.C. Chen, and X.Y. Tan Sparsity preserving projections with applications to face recognition Pattern Recognition 43 1 2010 331 341
18. S. Yan, H. Wang, Semi-supervised learning by sparse representation, in: Proceedings of SIAM International Conference on Data Mining (SDM 2009) Sparks, Nevada, USA, 2009.
19. S. Yan, D. Xu, B. Zhang, H. Zhang, Q. Yang, and S. Lin Graph embedding and extension: a general framework for dimensionality reduction IEEE Transactions on Pattern Analysis and Machine Intelligence 29 1 2007 40 51 (Pubitemid 46415944)
20. J. Wright, Y. Ma, and J. Mairal Sparse representation for computer vision and pattern recognition Proceedings of IEEE 2009 (March)
21. J. Wright, A. Yang, A. Ganesh, S. Sastry, and Y. Ma Robust face recognition via sparse representation IEEE Transactions on Pattern Analysis and Machine Intelligence 31 2 2009 210 227
22. J. Yang, J. Wright, T. Huang, and Y. Ma Image superresolution as sparse representation of raw patches IEEE Conference on Computer Vision and Pattern Recognition 2008
23. J. Mairal, G. Sapiro, and M. Elad. Learning multiscale sparse representations for image and video restoration SIAM MMS 7 1 2008 214 241 (April)
24. K. Huang, and S. Aviyente Sparse representation for signal classification Neural Information Processing Systems 2006
25. J. Mairal, F. Bach, J. Ponce, G. Sapiro, A. Zisserman, Learning discriminative dictionaries for local image analysis, in: Proceedings of IEEE CVPR, 2008.
26. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman Supervised dictionary learning D. Koller, D. Schuurmans, Y. Bengio, L. Bottou, Advances in NIPS vol. 21 2009
27. J. Yang, K. Yu, Y. Gong, and T. Huang Linear spatial pyramid matching using sparse coding for image classification IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2009
28. E. Amaldi, and V. Kann On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems Theoretical Computer Science 209 1998 237 260 (Pubitemid 128450440)
29. D. Donoho For most large underdetermined systems of linear equations the minimal l1-norm solution is also the sparsest solution Communications on Pure and Applied Mathematics 59 6 2006 797 829 (Pubitemid 43667226)
30. E. Candes, J. Romberg, and T. Tao Stable signal recovery from incomplete and inaccurate measurements, Communications on Pure and Applied Mathematics 59 8 2006 1207 1223 (Pubitemid 43988295)
31. J. Wright, and Y. Ma. Dense error correction via l1-minimization IEEE Transactions on Information Theory 56 7 2010 3540 3560
32. V. Vapnik. The Nature of Statistical Learning Theory 1995 Springer N.Y.
33. S. Chen, D. Donoho, and M. Saunders Atomic decomposition by basis pursuit SIAM Review 43 1 2001 129 159 (Pubitemid 32406896)
34. D. Donoho, Y. Tsaig, Fast Solution of L1 Norm Minimization Problems when the Solution May Be Sparse. 〈http://www-stat.stanford.edu/∼donoho/ reports.html〉, 2006.
35. P. Vincent, and Y. Bengio K-local hyperplane and convex distance nearest neighbor algorithms Advances in Neural Information Processing Systems (NIPS2002) 2002
36. D. Donoho, M. Elad, Maximal Sparsity Representation Via l1 Minimization. 〈http://www-stat.stanford.edu/∼donoho/reports.html〉.
37. D. Donoho, Neighborly Polytopes and Sparse Solution of Underdetermined Linear Equations, Technical Report 2005-4, Department. of Statistics, Stanford University, 2005.
38. P. McMullen, and G.C. Shephard Diagrams for centrally symmetric polytopes Mathematika 15 1968 123 138
39. J. Yang, Z. Lou, Z. Jin, J.-y. Yang, Minimal local reconstruction error measure based discriminant feature extraction and classification, in: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2008), Alaska, June, 2008.
40. S.Z. Li, and Juwei Lu Face recognition using the nearest feature line method IEEE Transactions on Neural Networks 10 2 1999 439 443
41. J.-T. Chien, and C.-C. Wu Discriminant waveletfaces and nearest feature classifiers for face recognition, IEEE Trans. Pattern Anal Machine Intelligence 24 12 2002 1644 1649
42. A.M. Martinez, R. Benavente, The AR Face Database, CVC Technical Report #24, June 1998.
43. E. Candes, J. Romberg, l1-Magic: Recovery of Sparse Signals Via Convex Programming, 〈http://www.acm.caltech.edu/l1magic/〉.
44. S.-J. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky A method for large-scale l1-regularized least squares IEEE Journal on Selected Topics in Signal Processing 1 4 2007 606 617
45. S.X. Liao, and M. Pawlak On image analysis by moments IEEE Transactions on Pattern Analysis and Machine Intelligence 18 3 1996 254 266 (Pubitemid 126773883)
46. Y.H. Tseng, C.C. Kuo, and H.J. Lee Speeding up Chinese character recognition in an automatic document reading system Pattern Recognition 31 11 1998 1601 1612
47. A.S. Georghiades, P.N. Belhumeur, and D.J. Kriegman From few to many: illumination cone models for face recognition under variable lighting and pose IEEE Transactions on Pattern Analysis and Machine Intelligence 23 6 2001 643 660 (Pubitemid 32585276)
48. K.C. Lee, J. Ho, and D. Driegman Acquiring linear subspaces for face recognition under variable lighting IEEE Transactions on Pattern Analysis and Machine Intelligence 27 5 2005 684 698 (Pubitemid 40608402)
49. M. Turk, and A. Pentland Face recognition using eigenfaces, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition 1991 586 591
50. P.N. Belhumeur, J.P. Hespanha, and D.J. Kriengman Eigenfaces vs. Fisherfaces: recognition using class specific linear projection IEEE Transactions on Pattern Analysis and Machine Intelligence 19 7 1997 711 720 (Pubitemid 127762263)
51. X. He, S. Yan, Y. Hu, P. Niyogi, and H.-J. Zhang Face recognition using Laplacianfaces IEEE Transactions on Pattern Analysis and Machine Intelligence 27 3 2005 328 340 (Pubitemid 40377326)
52. T. Sim, S. Baker, and M. Bsat The CMU pose, illumination, and expression database IEEE Transactions on Pattern Analysis and Machine Intelligence 25 12 2003 1615 1618
53. D. Cai, X. He, J. Han, Spectral regression for efficient regularized subspace learning, in: Proceedings of IEEE 11th International Conference on Computer Vision (ICCV 2007).