Double shrinking sparse dimension reduction

IEEE Transactions on Image Processing

Volumn 22, Issue 1, 2013, Pages 244-257

  Zhou, Tianyi ^a   Tao, Dacheng ^a  

^a UNIVERSITY OF TECHNOLOGY SYDNEY   (Australia)

Author keywords

1 regularization; compressed sensing; dimension reduction; image compression; manifold embedding; path following; sparse learning

Indexed keywords

COMPRESSIVE SENSING; DIMENSION REDUCTION; MANIFOLD EMBEDDING; PATH FOLLOWING; SPARSE LEARNING;

ALGORITHMS; CRYPTOGRAPHY; IMAGE COMPRESSION; ITERATIVE METHODS; SHRINKAGE;

DATA REDUCTION;

EID: 84871666572     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2012.2202678     Document Type: Article

References (62)

1. E. J. Candès, J. K. Romberg, and T. Tao, "Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information," IEEE Trans. Inform. Theory, vol. 52, no. 2, pp. 489-509, Feb. 2006. (Pubitemid 43174093)
2. D. L. Donoho, "Compressed sensing," IEEE Trans. Inform. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
3. L. Sun, J. Liu, J. Chen, and J. Ye, "Efficient recovery of jointly sparse vectors," in Proc. Adv. Neural Inform. Process. Syst., 2009, pp. 1-9.
4. J. Lv and Y. Fan, "A unified approach to model selection and sparse recovery using regularized least squares," Ann. Statist., vol. 37, no. 6, pp. 3498-3528, 2009.
5. T. Zhou and D. Tao, "1-bit hamming compressed sensing," in Proc. IEEE Int. Symp. Inform. Theory, 2012, pp. 1-7.
6. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Royal Statist. Soc., vol. 58, no. 1, pp. 267-288, 1996.
7. J. Liu, J. Chen, and J. Ye, "Large-scale sparse logistic regression," in Proc. 15th Int. Conf. Knowl. Discovery Data Mining, 2009, pp. 547-556.
8. J. Liu, S. Ji, and J. Ye. (2009). SLEP: Sparse Learning with Efficient Projections [Online]. Available: http://www.public.asu.edu/~jye02/Software/SLEP
9. J. Mairal, M. Elad, and G. Sapiro, "Sparse representation for color image restoration," IEEE Trans. Image Process., vol. 17, no. 1, pp. 53-69, Jan. 2008.
10. E. Arias-Castro, D. L. Donoho, and X. Huo, "Near-optimal detection of geometric objects by fast multiscale methods," IEEE Trans. Inform. Theory, vol. 51, no. 7, pp. 2402-2425, Jul. 2005. (Pubitemid 41136400)
11. H. Hotelling, "Analysis of a complex of statistical variables into principal components," J. Edu. Phys., vol. 24, no. 6, pp. 417-441, 1936.
12. T. Zhou and D. Tao, "Bilateral random projections," in Proc. IEEE Int. Symp. Inform. Theory, 2012, pp. 1-17.
13. R. A. Fisher, "The use of multiple measurements in taxonomic problems," Ann. Eugenics, vol. 7, no. 2, pp. 179-188, 1936.
14. S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol. 290, no. 5500, pp. 2323-2326, 2000. (Pubitemid 32041578)
15. J. B. Tenenbaum, V. Silva, and J. C. Langford, "A global geometric framework for nonlinear dimensionality reduction," Science, vol. 290, no. 5500, pp. 2319-2323, 2000. (Pubitemid 32041577)
16. T. Zhou, D. Tao, and X. Wu, "Nesvm: A fast gradient method for support vector machines," in Proc. IEEE Int. Conf. Data Mining, 2010, pp. 679-688.
17. J. Fan and Y. Fan, "High dimensional classification using features annealed independence rules," Ann. Statist., vol. 36, no. 6, pp. 2605-2637, 2008.
18. Y. Bengio, J.-F. Paiement, P. Vincent, O. Delalleau, N. L. Roux, and M. Ouimet, "Out-of-sample extensions for lle, isomap, mds, eigen-maps, and spectral clustering," in Proc. Adv. Neural Inform. Process. Syst., 2004, pp. 177-184.
19. D. L. Donoho and C. Grimes, "Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data," Proc. Nat. Academy Sci., vol. 100, no. 10, pp. 5591-5596, 2003. (Pubitemid 36576852)
20. M. Belkin and P. Niyogi, "Laplacian eigenmaps and spectral techniques for embedding and clustering," in Proc. Adv. Neural Inform. Process. Syst., 2001, pp. 585-591.
21. X. He, D. Cai, S. Yan, and H.-J. Zhang, "Neighborhood preserving embedding," in Proc. IEEE Int. Conf. Comput. Vision, 2005, pp. 1208-1213. (Pubitemid 44042318)
22. A. Y. Ng, M. I. Jordan, and Y Weiss, "On spectral clustering: Analysis and an algorithm," in Proc. Adv. Neural Inform. Process. Syst., 2002, pp. 1-8.
23. J. A. Tropp and A. C. Gilbert, "Signal recovery from random measurements via orthogonal matching pursuit," IEEE Trans. Inform. Theory, vol. 53, no. 12, pp. 4655-4666, Dec. 2007.
24. D. Needell and J. A. Tropp, "Cosamp: Iterative signal recovery from incomplete and inaccurate samples," Appl. Comput. Harmonic Anal., vol. 26, no. 3, pp. 301-321, 2008.
25. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAMJ. Sci. Comput, vol. 20, no. 1, pp. 33-61, 1999. (Pubitemid 128689501)
26. S. Becker, J. Bobin, and E. J. Candès, "NESTA: A fast and accurate first-order method for sparse recovery," SIAM J. Imaging Sci., vol. 4, no. 1, pp. 1-39, 2009.
27. Y Nesterov, "Smooth minimization of non-smooth functions," Math. Program., vol. 103, no. 1, pp. 127-152, 2005. (Pubitemid 40537674)
28. S.-J. Kim, K. Koh, M. Lustig, S. Boyd, and D. Gorinevsky, "An interior-point method for large-scale l1-regularized least squares," IEEE J. Sel. Topics Signal Process., vol. 1, no. 4, pp. 606-617, Sep. 2007. (Pubitemid 351266626)
29. P. Tseng and S. Yun, "A coordinate gradient descent method for nonsmooth separable minimization," Math. Program., vol. 117, nos. 1-2, pp. 387-423, 2009.
30. M. A. T. Figueiredo, R. D. Nowak, and S. J. Wright, "Gradient projection for sparse reconstruction: Application to compressed sensing and other inverse problems," IEEE J. Sel. Topics Signal Process., vol. 1, no. 4, pp. 586-597, Dec. 2007. (Pubitemid 351276802)
31. D. L. Donoho, A. Maleki, and A. Montanari, "Message passing algorithms for compressed sensing," in Proc. Nat. Academy Sci., 2009, pp. 1-15.
32. I. Daubechies, M. Defrise, and C. D. Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint," Commun. Pure Appl. Math., vol. 57, no. 11, pp. 1413-1457, 2004. (Pubitemid 39427442)
33. K. Bredies and D. A. Lorenz, "Iterated hard shrinkage for minimization problems with sparsity constraints," SIAM J. Sci. Comput, vol. 30, no. 2, pp. 657-683, 2008.
34. W. Yin, S. Osher, D. Goldfarb, and J. Darbon, "Bregman iterative algorithms for l1-minimization with applications to compressed sensing," SIAM J. Imag. Sci., vol. 1, no. 1, pp. 143-168, 2008.
35. E. T. Hale, W Yin, and Y Zhang, "Fixed-point continuation for t1-minimization: Methodology and convergence," SIAM J. Opt, vol. 19, no. 3, pp. 1107-1130, 2008.
36. B. Rao, K. Engan, S. Cotter, J. Palmer, and K. Kreutz-Delgado, "Subset selection in noise based on diversity measure minimization," IEEE Trans. Signal Process., vol. 51, no. 3, pp. 760-770, Mar. 2003.
37. H. Zou, T. Hastie, and R. Tibshirani, "Sparse principal component analysis," J. Comput. Graphical Statist, vol. 15, no. 2, pp. 262-286, 2006.
38. H. Shen and J. Z. Huang, "Sparse principal component analysis via regularized low rank matrix approximation," J. Multivariate Anal, vol. 99, no. 6, pp. 1015-1034, 2008.
39. A. Aspremont, L. E. Ghaoui, M. I. Jordan, and G. R. G. Lanckriet, "A direct formulation for sparse pca using semidefinite programming," SIAM Rev., vol. 49, no. 3, pp. 434-448, 2007. (Pubitemid 47377693)
40. A. Aspremont, F Bach, and L. E. Ghaoui, "Optimal solutions for sparse principal component analysis," J. Mach. Learn. Res., vol. 9, pp. 1269-1294, Jul. 2008.
41. D. M. Witten, R. Tibshirani, and T. Hastie, "A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis," Biostatistics, vol. 10, no. 3, pp. 515-534, 2009.
42. B. Moghaddam, Y Weiss, and S. Avidan, "Spectral bounds for sparse pca: Exact and greedy algorithms," in Proc. Adv. Neural Inform. Process. Syst., 2006, pp. 915-922.
43. T. Zhou, D. Tao, and X. Wu, "Compressed labeling on distilled labelsets for multi-label learning," Mach. Learn., vol. 88, nos. 1-2, pp. 69-126, 2012.
44. J. Lv and J. S. Liu, "Model selection principles in misspecified models," arXiv:1005.5483v1, 2010.
45. J. Fan and J. Lv, "A selective overview of variable selection in high dimensional feature space," Stat. Sinica, vol. 20, no. 1, pp. 101-148, 2010.
46. S. Boyd and L. Vandenberghe, Convex Optimization. London, U.K: Cambridge Univ. Press, 2004.
47. L. Mackey, "Deflation methods for sparse pca," in Proc. Adv. Neural Inform. Process. Syst., 2008, pp. 1-8.
48. K. B. Petersen and M. S. Pedersen. (2008). The Matrix Cookbook [Online]. Available: http://www2.imm.dtu.dk/pubdb/p.php?3274
49. G. H. Golub and C. F. Van Loan, Matrix Computations. Baltimore, MD: Johns Hopkins Univ. Press, 1996.
50. P. J. Phillips, H. Moon, S. A. Rizvi, and P. J. Rauss, "The feret evaluation methodology for face-recognition algorithms," IEEE Trans. Patt. Anal. Mach. Intell., vol. 22, no. 10, pp. 1090-1104, Oct. 2000.
51. P. N. Belhumeur, J. A. P. Hespanha, and D. J. Kriegman, "Eigenfaces versus fisherfaces: Recognition using class specific linear projection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 12, no. 7, pp. 711-720, Jul. 1997. (Pubitemid 127762263)
52. D. B. Graham and N. M. Allinson, "Characterizing virtual eigensigna-tures for general purpose face recognition," Face Recogn. Appl. Comput. Syst. Sci., vol. 163, pp. 446-456, Jul. 1997.
53. F. S. Samaria, A. Harter, and O. A. Site, "Parameterisation of a stochastic model for human face identification," in Proc. IEEE Workshop Appl. Comput. Vision, Dec. 1994, pp. 138-142.
54. S. Nene, S. K. Nayar, and H. Murase, "Columbia object image library (coil-20)," Dept. Comput. Sci., Columbia Univ., New York, Tech. Rep. CUCS-006-96, 1996.
55. A. Asuncion and D. Newman. (2007). UCI Machine Learning Repository [Online]. Available: http://www.ics.uci.edu/∼mlearn/MLReposi-tory.html
56. N. Trendafilov, I. T. Jolliffe, and M. Uddin, "A modified principal component technique based on the lasso," J. Comput. Graphical Statist., vol. 12, no. 3, pp. 531-547, 2003. (Pubitemid 37239916)
57. U. Alon, N. Barkai, D. A. Notterman, K. Gish, S. Ybarra, D. Mack, and A. J. Levine, "Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays," Cell Biol., vol. 96, pp. 6745-6750, Jun. 1999. (Pubitemid 29274954)
58. A. Alizadeh, M. Eisen, R. Davis, C. Ma, I. Lossos, and A. Rosenwald, "Distinct types of diffuse large b-cell lymphoma identified by gene expression profiling," Nature, vol. 403, pp. 503-511, Jan. 2000. (Pubitemid 30082188)
59. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Second Edition. New York: Springer-Verlag, 2009.
60. W. Xu, X. Liu, and Y. Gong, "Document clustering based on nonnegative matrix factorization," in Proc. ACM SIGIR Conf. Res. Devel. Inform. Retrieval, 2003, pp. 267-273.
61. D. Cai, X. He, and J. Han, "Document clustering using locality preserving indexing," IEEE Trans. Knowl. Data Eng., vol. 17, no. 12, pp. 1624-1637, Dec. 2005. (Pubitemid 43060413)
62. T. Zhou, D. Tao, and X. Wu, "Manifold elastic net: A unified framework for sparse dimension reduction," Data Mining Knowl. Discovery vol. 22, no. 3, pp. 340-371, 2011.