Sparse subspace clustering: Algorithm, theory, and applications

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 35, Issue 11, 2013, Pages 2765-2781

  Elhamifar, Ehsan ^a   Vidal, Rene ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b Johns Hopkins University   (United States)

Author keywords

( 1) minimization; clustering; convex programming; face clustering; High dimensional data; intrinsic low dimensionality; motion segmentation; principal angles; sparse representation; spectral clustering; subspaces

Indexed keywords

CLUSTERING; FACE CLUSTERING; HIGH DIMENSIONAL DATA; LOW DIMENSIONALITY; MOTION SEGMENTATION; PRINCIPAL ANGLES; SPARSE REPRESENTATION; SPECTRAL CLUSTERING; SUBSPACES;

CONVEX PROGRAMMING; OPTIMIZATION; RELAXATION PROCESSES;

CLUSTERING ALGORITHMS;

ALGORITHM; ANATOMY AND HISTOLOGY; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; BIOMETRY; COMPUTER ASSISTED DIAGNOSIS; FACE; HUMAN; PROCEDURES; SAMPLE SIZE; ARTICLE; HISTOLOGY; METHODOLOGY;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BIOMETRY; FACE; HUMANS; IMAGE INTERPRETATION, COMPUTER-ASSISTED; PATTERN RECOGNITION, AUTOMATED; SAMPLE SIZE;

EID: 84884541998     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2013.57     Document Type: Article

References (68)

1. R.E. Bellman, Dynamic Programming. Princeton Univ. Press, 1957.
2. C. Tomasi and T. Kanade, "Shape and Motion from Image Streams under Orthography," Int'l J. Computer Vision, vol. 9, no. 2, pp. 137- 154, 1992. (Pubitemid 23621943)
3. R. Basri and D. Jacobs, "Lambertian Reflection and Linear Subspaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 3, pp. 218-233, Feb. 2003.
4. T. Hastie and P. Simard, "Metrics and Models for Handwritten Character Recognition," Statistical Science, vol. 13, no. 1, pp. 54-65, 1998. (Pubitemid 128469056)
5. R. Vidal, "Subspace Clustering," Signal Processing Magazine, vol. 28, no. 2, pp. 52-68, 2011.
6. W. Hong, J. Wright, K. Huang, and Y. Ma, "Multi-Scale Hybrid Linear Models for Lossy Image Representation," IEEE Trans. Image Processing, vol. 15, no. 12, pp. 3655-3671, Dec. 2006. (Pubitemid 44811680)
7. A. Yang, J. Wright, Y. Ma, and S. Sastry, "Unsupervised Segmentation of Natural Images via Lossy Data Compression," Computer Vision and Image Understanding, vol. 110, no. 2, pp. 212- 225, 2008. (Pubitemid 351509774)
8. J. Costeira and T. Kanade, "A Multibody Factorization Method for Independently Moving Objects," Int'l J. Computer Vision, vol. 29, no. 3, pp. 159-179, 1998. (Pubitemid 128517909)
9. K. Kanatani, "Motion Segmentation by Subspace Separation and Model Selection," Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 586-591, 2001. (Pubitemid 32795109)
10. R. Vidal, Y. Ma, and S. Sastry, "Generalized Principal Component Analysis (GPCA)," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 12, pp. 1-15, Dec. 2005.
11. R. Duda, P. Hart, and D. Stork, Pattern Classification. Wiley- Interscience, Oct. 2004.
12. P. Tseng, "Nearest q-Flat to m Points," J. Optimization Theory and Applications, vol. 105, no. 1, pp. 249-252, 2000.
13. J. Ho, M.H. Yang, J. Lim, K. Lee, and D. Kriegman, "Clustering Appearances of Objects under Varying Illumination Conditions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
14. T. Zhang, A. Szlam, and G. Lerman, "Median k-Flats for Hybrid Linear Modeling with Many Outliers," Proc. IEEE Int'l Workshop Subspace Methods, 2009.
15. C.W. Gear, "Multibody Grouping from Motion Images," Int'l J. Computer Vision, vol. 29, no. 2, pp. 133-150, 1998. (Pubitemid 128517908)
16. Y. Ma, A. Yang, H. Derksen, and R. Fossum, "Estimation of Subspace Arrangements with Applications in Modeling and Segmenting Mixed Data," SIAM Rev., vol. 50, pp. 413-458, 2008.
17. M. Tipping and C. Bishop, "Mixtures of Probabilistic Principal Component Analyzers," Neural Computation, vol. 11, no. 2, pp. 443-482, 1999.
18. Y. Sugaya and K. Kanatani, "Geometric Structure of Degeneracy for Multi-Body Motion Segmentation," Proc. Workshop Statistical Methods in Video Processing, 2004.
19. A. Gruber and Y. Weiss, "Multibody Factorization with Uncertainty and Missing Data Using the EM Algorithm," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 707-714, 2004.
20. M.A. Fischler and R.C. Bolles, "RANSAC Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," Comm. ACM, vol. 26, pp. 381-395, 1981.
21. S. Rao, R. Tron, R. Vidal, and Y. Ma, "Motion Segmentation in the Presence of Outlying, Incomplete, or Corrupted Trajectories," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 10, pp. 1832-1845, Oct. 2010.
22. J. Yan and M. Pollefeys, "A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-Rigid, Degenerate and Non-Degenerate," Proc. European Conf. Computer Vision, pp. 94-106, 2006.
23. A. Goh and R. Vidal, "Segmenting Motions of Different Types by Unsupervised Manifold Clustering," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
24. T. Zhang, A. Szlam, Y. Wang, and G. Lerman, Hybrid Linear Modeling via Local Best-Fit Flats, Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1927-1934, 2010.
25. L. Zelnik-Manor and M. Irani, "Degeneracies, Dependencies and Their Implications in Multi-Body and Multi-Sequence Factorization," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 287-293, 2003.
26. A. Ng, Y. Weiss, and M. Jordan, "On Spectral Clustering: Analysis and an Algorithm," Proc. Neural Information Processing Systems Conf., pp. 849-856, 2001.
27. U. von Luxburg, "A Tutorial on Spectral Clustering," Statistics and Computing, vol. 17, pp. 395-416, 2007.
28. G. Chen and G. Lerman, "Spectral Curvature Clustering (SCC)," Int'l J. Computer Vision, vol. 81, no. 3, pp. 317-330, 2009.
29. D.L. Donoho, "For Most Large Underdetermined Systems of Linear Equations the Minimal '1-Norm Solution Is Also the Sparsest Solution," Comm. Pure and Applied Math., vol. 59, no. 6, pp. 797-829, 2006. (Pubitemid 43667226)
30. E. Candès and T. Tao, "Decoding by Linear Programming," IEEE Trans. Information Theory, vol. 51, no. 12, pp. 4203-4215, Dec. 2005. (Pubitemid 41800353)
31. R. Tibshirani, "Regression Shrinkage and Selection via the LASSO," J. Royal Statistical Soc. B, vol. 58, no. 1, pp. 267-288, 1996.
32. E. Candès, X. Li, Y. Ma, and J. Wright, "Robust Principal Component Analysis," J. ACM, vol. 58, 2011.
33. E. Candès and B. Recht, "Exact Matrix Completion via Convex Optimization," Foundations of Computational Math., vol. 9, pp. 717- 772, 2008.
34. B. Recht, M. Fazel, and P. Parrilo, "Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization," SIAM Rev., vol. 52, no. 3, pp. 471-501, 2010.
35. E. Elhamifar and R. Vidal, "Sparse Subspace Clustering," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
36. E. Elhamifar and R. Vidal, Clustering Disjoint Subspaces via Sparse Representation, Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, 2010.
37. M. Soltanolkotabi and E.J. Candes, "A Geometric Analysis of Subspace Clustering with Outliers," Annals of Statistics, vol. 40, pp. 2195-2238, 2012.
38. G. Liu, Z. Lin, and Y. Yu, Robust Subspace Segmentation by Low-Rank Representation, Proc. Int'l Conf. Machine Learning, 2010.
39. G. Liu and S. Yan, Latent Low-Rank Representation for Subspace Segmentation and Feature Extraction, Proc. Int'l Conf. Computer Vision, 2011.
40. G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, and Y. Ma, "Robust Recovery of Subspace Structures by Low-Rank Representation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 171-184, Jan. 2013.
41. P. Favaro, R. Vidal, and A. Ravichandran, A Closed Form Solution to Robust Subspace Estimation and Clustering, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
42. E. Elhamifar and R. Vidal, Robust Classification Using Structured Sparse Representation, Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
43. E. Elhamifar and R. Vidal, Block-Sparse Recovery via Convex Optimization, IEEE Trans. Signal Processing, 2012.
44. F. Parvaresh, H. Vikalo, S. Misra, and B. Hassibi, "Recovering Sparse Signals Using Sparse Measurement Matrices in Compressed DNA Microarrays," IEEE J. Selected Topics in Signal Processing, vol. 2, no. 3, pp. 275-285, June 2008.
45. M. Stojnic, F. Parvaresh, and B. Hassibi, "On the Reconstruction of Block-Sparse Signals with and Optimal Number of Measurements," IEEE Trans. Signal Processing, vol. 57, no. 8, pp. 3075-3085, Aug. 2009.
46. Y.C. Eldar and M. Mishali, "Robust Recovery of Signals from a Structured Union of Subspaces," IEEE Trans. Information Theory, vol. 55, no. 11, pp. 5302-5316, Nov. 2009.
47. Y.C. Eldar, P. Kuppinger, and H. Bolcskei, "Compressed Sensing of Block-Sparse Signals: Uncertainty Relations and Efficient Recovery," IEEE Trans. Signal Processing, vol. 58, no. 6, pp. 3042- 3054, June 2010.
48. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge Univ. Press, 2004.
49. S.J. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, "An Interior-Point Method for Large-Scale l1-Regularized Least Squares," IEEE J. Selected Topics in Signal Processing, vol. 1, no. 4, pp. 606-617, Dec. 2007. (Pubitemid 351266626)
50. S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein, "Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers," Foundations and Trends in Machine Learning, vol. 3, no. 1, pp. 1-122, 2010.
51. E. Amaldi and V. Kann, "On the Approximability of Minimizing Nonzero Variables or Unsatisfied Relations in Linear Systems," Theoretical Computer Science, vol. 209, pp. 237-260, 1998. (Pubitemid 128450440)
52. R. Zass and A. Shashua, "Doubly Stochastic Normalization for Spectral Clustering," Neural Information Processing Systems, vol. 19, pp. 1569-1576, 2006.
53. T. Brox and J. Malik, Object Segmentation by Long Term Analysis of Point Trajectories, Proc. European Conf. Computer Vision, 2010.
54. J. Wright, A. Yang, A. Ganesh, S. Sastry, and Y. Ma, "Robust Face Recognition via Sparse Representation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 210-227, Feb. 2009.
55. R. Vidal and R. Hartley, "Motion Segmentation with Missing Data by Power Factorization and Generalized PCA," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 310-316, 2004.
56. D.L. Donoho and M. Elad, "Optimally Sparse Representation in General (Nonorthogonal) Dictionaries via '1 Minimization," Proc. Nat'l Academy of Sciences USA, vol. 100, no. 5, pp. 2197-2202, 2003. (Pubitemid 36297476)
57. R. Gribonval and M. Nielsen, "Sparse Representations in Unions of Bases," IEEE Trans. Information Theory, vol. 49, no. 12, pp. 3320- 3325, Dec. 2003.
58. E. van den Berg and M. Friedlander, "Theoretical and Empirical Results for Recovery from Multiple Measurements," IEEE Trans. Information Theory, vol. 56, no. 5, pp. 2516-2527, May 2010.
59. D.L. Donoho, "Neighborly Polytopes and Sparse Solution of Underdetermined Linear Equations," technical report, Stanford Univ., 2005.
60. B. Nasihatkon and R. Hartley, "Graph Connectivity in Sparse Subspace Clustering," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
61. E. Elhamifar, G. Sapiro, and R. Vidal, "See All by Looking at a Few: Sparse Modeling for Finding Representative Objects," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2012.
62. R. Jenatton, J.Y. Audibert, and F. Bach, "Structured Variable Selection with Sparsity-Inducing Norms," J. Machine Learning Research, vol. 12, pp. 2777-2824, 2011.
63. J.A. Tropp, "Algorithms for Simultaneous Sparse Approximation: Part II: Convex Relaxation," Signal Processing, vol. 86, pp. 589-602, 2006. (Pubitemid 43106573)
64. D. Gabay and B. Mercier, "A Dual Algorithm for the Solution of Nonlinear Variational Problems via Finite-Element Approximations," Computer Math. Applications, vol. 2, pp. 17-40, 1976.
65. F. Lauer and C. Schnorr, "Spectral Clustering of Linear Subspaces for Motion Segmentation," Proc. IEEE Int'l Conf. Computer Vision, 2009.
66. R. Tron and R. Vidal, "A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
67. T. Boult and L. Brown, "Factorization-Based Segmentation of Motions," Proc. IEEE Workshop Motion Understanding, pp. 179-186, 1991. (Pubitemid 23618715)
68. K.-C. Lee, J. Ho, and D. Kriegman, "Acquiring Linear Subspaces for Face Recognition under Variable Lighting," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 684-698, May 2005. (Pubitemid 40608402)