Dictionary learning-based subspace structure identification in spectral clustering

IEEE Transactions on Neural Networks and Learning Systems

Volumn 24, Issue 8, 2013, Pages 1188-1199

  Jing, Liping ^a   Ng, Michael K ^b   Zeng, Tieyong ^b  

^a BEIJING JIAOTONG UNIVERSITY   (China)

^b HONG KONG BAPTIST UNIVERSITY   (Hong Kong)

Author keywords

Dictionary learning (DL); high dimensional data; nonnegative data; proximal optimization; sparsity; spectral clustering (SC); subspace structure

Indexed keywords

DICTIONARY LEARNING; HIGH DIMENSIONAL DATA; NON NEGATIVES; SPARSITY; SPECTRAL CLUSTERING;

CLUSTERING ALGORITHMS; OPTIMIZATION; TESTING;

MICROARRAYS;

EID: 84880916591     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2013.2253123     Document Type: Article

References (52)

1. H. Kriegel, P. Kroger, and A. Zimek, "Clustering high-dimensional data: A survey on subspace clustering, pattern-based clustering and correlation clustering," ACM Trans. Knowl. Discovery Data, vol. 3, no. 1, pp. 1-58, Mar. 2009.
2. U. Von Luxburg, "A tutorial on spectral clustering," J. Stat. Comput., vol. 17, no. 4, pp. 395-416, Dec. 2007.
3. R. Vidal, "A tutorial on subspace clustering," IEEE Signal Process. Mag., vol. 28, no. 2, pp. 52-68, Mar. 2011.
4. D. Niu, J. Dy, and M. Jordan, "Dimensionality reduction for spectral clustering," in Proc. Artif. Intell. Stat., 2011, pp. 552-560.
5. L. Zhong and J. Kwok, "Efficient sparse modeling with automatic feature grouping," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 9, pp. 1436-1447, Sep. 2012.
6. A. Stuhlsatz, J. Lippel, and T. Zielke, "Feature extraction with deep neural networks by a generalized discriminant analysis," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 4, pp. 596-608, Apr. 2012.
7. N. Manukyan, M. Eppstein, and D. Rizzo, "Data-driven cluster reinforcement and visualization in sparsely-matched self-organizing maps," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 5, pp. 846-852, Apr. 2012.
8. S. Roweis and L. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol. 290, no. 5500, pp. 2323-2326, Dec. 2000. (Pubitemid 32041578)
9. J. Tenenbaum, V. Silva, and J. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science, vol. 290, no. 5500, pp. 2319-2323, Dec. 2000. (Pubitemid 32041577)
10. R. Vidal, Y. Ma, and S. Sastry, "Generalized principal component analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 12, pp. 1-15, Dec. 2005.
11. B. Gao, W. Woo, and S. Dlay, "Variational regularized 2-D nonnegative matrix factorization," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 5, pp. 703-716, May 2012.
12. M. Aharon, M. Elad, and A. Bruckstein, "The k-SVD: An algorithm for designing of overcomplete dictionaries for sparse representations," IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, Nov. 2006. (Pubitemid 44637761)
13. E. Elhamifar and R. Vidal, "Sparse subspace clustering," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2009, pp. 2790-2797.
14. G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, and Y. Ma, "Robust recovery of subspace structures by low-rank representation," in Proc. 26th Int. Conf. Mach. Learn., Oct. 2010, pp. 663-670.
15. G. Liu and S. Yan, "Latent low-rank representation for subspace segmentation and feature extraction," in Proc. IEEE Int. Conf. Comput. Vis., Nov. 2011, pp. 1615-1622.
16. D. Luo, F. Nie, C. Ding, and H. Huang, "Multi-subspace representation and discovery," in Proc. Eur. Conf. Mach. Learn. Principles Pract. Knowl. Discovery Databases, Nov. 2011, pp. 405-420.
17. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries," IEEE Trans. Image Process., vol. 15, no. 12, pp. 3736-3745, Dec. 2006. (Pubitemid 44811686)
18. H. Lee, A. Battle, R. Raina, and A. Ng, "Efficient sparse coding algorithms," in Proc. Adv. Neural Inf. Process. Syst., Dec. 2006, pp. 801-808.
19. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Supervised dictionary learning," in Proc. Neural Inf. Process. Syst., Jul. 2009, pp. 1033-1040.
20. R. Rubinstein, A. Bruckstein, and M. Elad, "Dictionaries for sparse representation modeling," IEEE Proc., vol. 98, no. 6, pp. 1045-1057, Jun. 2010.
21. M. Yang, L. Zhang, X. Feng, and D. Zhang, "Fisher discrimination dictionary learning for sparse representation," in Proc. IEEE Int. Conf. Comput. Vis., Nov. 2011, pp. 543-550.
22. P. Favaro, R. Vidal, and A. Ravichandran, "A closed form solution to robust subspace estimation and clustering," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2011, pp. 1801-1807.
23. Z. Liu, R. Liu, and Z. Su, "Linearized alternating direction method with adaptive penalty for low-rank representation," in Proc. NIPS, Sep. 2011, pp. 1-10.
24. B. Olshausen and D. Field, "Sparse coding of sensory inputs," Current Opinion Neurobiol., vol. 14, no. 4, pp. 481-487, Aug. 2004. (Pubitemid 39078497)
25. P. Hoyer, "Nonnegative sparse coding," in Proc. 12th IEEE Workshop Neural Netw. Signal Process., Feb. 2002, pp. 557-565.
26. H. Kim and H. Park, "Sparse nonnegative matrix factorization via alternating nonnegativity constrained least squares for microarray data analysis," Bioinformatics, vol. 23, no. 12, pp. 1495-1502, May 2007. (Pubitemid 47244474)
27. R. Zhi, M. Flierl, Q. Ruan, and W. Kleijn, "Graph-preserving sparse nonnegative matrix factorization with application to facial expression recognition," IEEE Trans. Syst. Man, Cybern. Part B, Cybern., vol. 41, no. 1, pp. 38-52, Feb. 2011.
28. D. Lee and H. Seung, "Learning the parts of objects by nonnegative matrix factorization," Nature, vol. 401, pp. 788-791, Oct. 1999.
29. C. Ding, X. He, and H. Simon, "On the equivalence of nonnegative matrix factorization and spectral clustering," in Proc. SIAM Data Meening Conf., Apr. 2005, pp. 606-610.
30. S. Zafeiriou, A. Tefas, I. Buciu, and I. Pitas, "Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification," IEEE Trans. Neural Netw., vol. 17, no. 3, pp. 683-695, May 2006.
31. Z. Yang and E. Oja, "Unified development of multiplicative algorithms for linear and quadratic nonnegative matrix factorization," IEEE Trans. Neural Netw., vol. 22, no. 12, pp. 1878-1891, Dec. 2011.
32. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, "Online learning for matrix factorization and sparse coding," J. Mach. Learn. Res., vol. 11, pp. 19-60, Sep. 2011.
33. Z. He, S. Xie, R. Zdunek, G. Zhou, and A. Cichocki, "Symmetric nonnegative matrix factorization: Algorithms and applications to probabilistic clustering," IEEE Trans. Neural Netw., vol. 22, no. 12, pp. 2117-2131, Dec. 2011.
34. H. Wang, F. Nie, H. Huang, and F. Makedon, "Fast nonnegative matrix tri-factorization for large-scale data co-clustering," in Proc. 22nd Int. Joint Conf. Artif. Intell., 2011, pp. 1553-1558.
35. N. Guan, D. Tao, Z. Luo, and B. Yuan, "Online nonnegative matrix factorization with robust stochastic approaximation," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 7, pp. 1087-1099, Jul. 2012.
36. D. Bertsekas, Nonlinear Programming. Belmont, MA, USA: Athena Scientific, 1999.
37. C. Lin, "On the convergence of multiplicative update algorithms for nonnegative matrix factorization," IEEE Trans. Neural Netw., vol. 18, no. 6, pp. 1589-1596, Nov. 2007. (Pubitemid 350148414)
38. C. Lin, "Projected gradient methods for nonnegative matrix factorization," Neural Comput., vol. 19, no. 10, pp. 2756-2779, Oct. 2007.
39. S. Ji and J. Ye, "An accelerated gradient method for trace norm minimization," in Proc. 26th Int. Conf. Mach. Learn., Jun. 2009, pp. 1-8.
40. L. Grippo and M. Sciandrone, "On the convergence of the block nonlinear Gauss-Seidel method under convex constraints," Oper. Res. Lett., vol. 26, no. 3, pp. 127-136, Apr. 2000. (Pubitemid 30564332)
41. M. Soltanolkotabi and E. Candes, "A geometric analysis of subspace clustering with outilers," Ann. Stat., vol. 40, no. 6, pp. 1?40, Dec. 2011.
42. A. Strehl and J. Ghosh, "Cluster ensembles-A knowledge reuse framework for combining multiple partitions," J. Mach. Learn. Res., vol. 3, pp. 583-617, Nov. 2002.
43. Y. Zhao and G. Karypis, "Empirical and theoretical comparisons of selected criterion functions for document clustering," Mach. Learn., vol. 55, no. 3, pp. 311-331, Jun. 2004.
44. M. Lan, C. Tan, J. Su, and Y. Lu, "Supervised and traditional term weighting methods for automatic text categorization," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 4, pp. 721-735, Apr. 2009.
45. U. Alon, N. Barkai, D. Notterman, K. Gish, S. Ybarra, D. Mack, and A. Levine, "Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays," Proc. Nat. Acad. Sci., vol. 96, no. 12, pp. 6745-6750, Jun. 1999. (Pubitemid 29274954)
46. T. Golub, "Molecular classification of cancer: Class discovery and class prediction by gene expression monitoring," Science, vol. 286, no. 5439, pp. 531-537, Oct. 1999.
47. M. West, "Predicting the clinical status of human breast cancer by using gene expression profiles," Proc. Nat. Acad. Sci., vol. 98, no. 20, pp. 11462-11467, Sep. 2001. (Pubitemid 32928751)
48. J. Khan, J. Wei, M. Ringner, L. Saal, M. Ladanyi, F. Westermann, F. Berthold, M. Schwab, C. Antonescu, C. Peterson, and P. Meltzer, "Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks," Nature Med., vol. 7, no. 6, pp. 673-679, Jun. 2001. (Pubitemid 32588022)
49. L. Li and L. Fei-Fei, "What, where and who? Classifying event by scene and object recognition," in Proc. IEEE Int. Conf. Comput. Vis., Oct. 2007, pp. 1-8.
50. B. Russell, A. Torralba, K. Murphy, and W. Freeman, "LabelMe: A database and web-based tool for image annotation," Int. J. Comput. Vis., vol. 77. nos. 1-3, pp. 157-173, May 2008.
51. K. Lee, J. Ho, and D. Kriegman, "Acquiring linear subspaces for face recognition under variable lighting," IEEE Trans. IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 5, pp. 684-698, May 2005. (Pubitemid 40608402)
52. R. Sandler and M. Lindenbaum, "Nonnegative matrix factorization with earth mover?s distance metric for image analysis," IEEE Trans. Pattren Anal. Mach. Intell., vol. 33, no. 8, pp. 1590-1602, Aug. 2011.