Efficient kernel sparse coding via first-order smooth optimization

IEEE Transactions on Neural Networks and Learning Systems

Volumn 25, Issue 8, 2014, Pages 1447-1459

  Kim, Minyoung ^a  

^a Seoul National University of Science and Technology   (South Korea)

Author keywords

Dictionary learning; first order smooth optimization; kernel representer theorem; sparse coding

Indexed keywords

IMAGE CLASSIFICATION;

DICTIONARY LEARNING; DUAL OPTIMIZATIONS; FIRST-ORDER; LARGE-SCALE DATUM; NON-DIFFERENTIABLE OBJECTIVES; OPTIMIZATION TECHNIQUES; REPRESENTER THEOREM; SPARSE CODING;

CODES (SYMBOLS);

EID: 84904756930     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2013.2294059     Document Type: Article

References (47)

1. K. Balasubramanian, K. Yu, and G. Lebanon, "Smooth sparse coding via marginal regression for learning sparse representations," in Proc. 30th Int. Conf. Mach. Learn., 2013, pp. 1-9.
2. A. Beck and M. Teboulle, "A fast iterative shrinkage-thresholding algorithm for linear inverse problems," SIAM J. Imaging Sci., vol. 2, no. 1, pp. 183-202, 2009.
3. D. Blei, A. Ng, and M. Jordan, "Latent dirichlet allocation," J. Mach. Learn. Res., vol. 3, pp. 993-1022, Jan. 2003.
4. Y. Censor and S. Zenios, Parallel Optimization: Theory, Algorithms, and Applications. London, U.K.: Oxford Univ. Press, 2002.
5. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol. 20, no. 1, pp. 33-61, 1998. (Pubitemid 128689501)
6. CVX Research, Inc., Austin, TX, USA. (2012, Sep.). CVX: Matlab Software for Disciplined Convex Programming, Version 2.0 Beta [Online]. Available: http://cvxr.com/cvx
7. A. d'Aspremont, O. Banerjee, and L. E. Ghaoui, "First-order methods for sparse covariance selection," SIAM J. Matrix Anal. Appl., vol. 30, no. 56, pp. 367-385, 2008.
8. D. L. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
9. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, "Least angle regression," Ann. Statist., vol. 32, no. 2, pp. 407-499, 2004. (Pubitemid 41250302)
10. L. Fei-Fei and P. Perona, "A Bayesian hierarchical model for learning natural scene categories," in Proc. IEEE Int. Conf. CVPR, vol. 2. Jun. 2005, pp. 524-531.
11. J. Feng, X. Yuan, Z. Wang, H. Xu, and S. Yan, "Auto-grouped sparse representation for visual analysis," in Proc. Eur. Conf. Comput. Vis., Oct. 2012, pp. 640-653.
12. S. Gao, I. W.-H. Tsang, and L.-T. Chia, "Kernel sparse representation for image classification and face recognition," in Proc. ECCV, Sep. 2010, pp. 1-14.
13. A. S. Georghiades, P. N. Belhumeur, and D. J. Kriegman, "From few to many: Illumination cone models for face recognition under variable lighting and pose," IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 6, pp. 643-660, Jun. 2001. (Pubitemid 32585276)
14. M. Grant and S. Boyd, "Graph implementations for nonsmooth convex programs," in Recent Advances in Learning and Control (Lecture Notes in Control and Information Sciences). New York, NY, USA: Springer-Verlag, 2008, pp. 95-110.
15. K. Gregor and Y. LeCun, "Learning fast approximations of sparse coding," in Proc. 27th Int. Conf. Mach. Learn., 2010, pp. 1-8.
16. 1-regularized least squares," IEEE J. Sel. Topics Signal Process., vol. 1, no. 4, pp. 606-617, Dec. 2007. (Pubitemid 351266626)
17. G. S. Kimeldorf and G. Wahba, "A correspondence between Bayesian estimation on stochastic processes and smoothing by splines," Ann. Math. Statist., vol. 41, no. 2, pp. 495-502, 1970.
18. H. Lee, A. Battle, R. Raina, and A. Y. Ng, "Efficient sparse coding algorithms," in Proc. NIPS, 2007, pp. 1-8.
19. C. Lemaréchal and C. Sagastizábal, "Practical aspects of the Moreau-Yosida regularization: Theoretical preliminaries," SIAM J. Optim., vol. 7, no. 2, pp. 367-385, 1997.
20. T. Leung and J. Malik, "Representing and recognizing the visual appearance of materials using three-dimensional textons," Int. J. Comput. Vis., vol. 43, no. 1, pp. 29-44, Jan. 2001. (Pubitemid 32720091)
21. M. S. Lewicki and T. J. Sejnowski, "Learning overcomplete representations," Neural Comput., vol. 12, no. 2, pp. 337-365, Feb. 2000.
22. L.-J. Li and L. Fei-Fei, "What, where and who? Classifying event by scene and object recognition," in Proc. IEEE Int. Conf. Comput. Vis., 2007, pp. 1-8.
23. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, 2004.
24. N. Morioka and S. Satoh, "Generalized Lasso based approximation of sparse coding for visual recognition," in Proc. Adv. Neural Inf. Process. Syst., Dec. 2011, pp. 1-9.
25. Y. Nesterov and A. Nemirovsky, "Interior-point polynomial methods in convex programming," in Studies in Applied Mathematics, vol. 13. Philadelphia, PA, USA: SIAM, 1994.
26. Y. Nesterov, "Smooth minimization of non-smooth functions," Math. Program., vol. 103, no. 1, pp. 127-152, 2005. (Pubitemid 40537674)
27. B. A. Olshausen, "Sparse coding of time-varying natural images," J. Vis., vol. 2, no. 7, 2002.
28. B. A. Olshausen and D. J. Field, "Emergence of simple-cell receptive field properties by learning a sparse code for natural images," Nature, vol. 381, no. 6583, pp. 607-609, Jun. 1996. (Pubitemid 26177476)
29. B. A. Olshausen and D. J. Field, "Sparse coding with an overcomplete basis set: A strategy employed by V1?" Vis. Res., vol. 37, no. 23, pp. 3311-3325, Dec. 1997. (Pubitemid 27493805)
30. B. Olshausen and D. Field, "Sparse coding of sensory inputs," Current Opinion Neurobiol., vol. 14, no. 4, pp. 481-487, Aug. 2004. (Pubitemid 39078497)
31. M. R. Osborne, B. Presnell, and B. Turlach, "A new approach to variable selection in least squares problems," IMA J. Numer. Anal., vol. 20, no. 3, pp. 389-403, 2000.
32. S. Perkins and J. Theiler, "Online feature selection using grafting," in Proc. 20th ICML, 2003, pp. 1-8.
33. J. Platt, "Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods," in Advances in Large Margin Classifiers. Cambridge, MA, USA: MIT Press, 1999.
34. B. T. Polyak, Introduction to Optimization. New York, NY, USA: Optimization Software, 1987.
35. S. Rosset, "Tracking curved regularized optimization solution paths," in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2004.
36. B. Russell, A. Torralba, K. Murphy, and W. T. Freeman, "LabelMe: A database and web-based tool for image annotation," Int. J. Comput. Vis., vol. 77, nos. 1-3, pp. 157-173, May 2008.
37. B. Schölkopf, R. Herbrich, and A. Smola, "A Generalized representer theorem," in Computational Learning Theory. Berlin, Germany: Springer-Verlag, 2001, pp. 416-426. (Pubitemid 33312837)
38. B. Schölkopf, A. J. Smola, and K.-R. Muller, "Nonlinear component analysis as a kernel eigenvalue problem," Neural Comput., vol. 10, no. 5, pp. 1299-1319, 1998. (Pubitemid 128463674)
39. B. Schölkopf and A. Smola, Learning with Kernels. Cambridge, MA, USA: MIT Press, 2002.
40. M. Szummer and R. W. Picard, "Temporal texture modeling," in Proc. IEEE ICIP, vol. 3. Sep. 1996, pp. 823-826.
41. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. R. Statist. Soc., Ser. B, Methodol., vol. 58, no. 1, pp. 267-288, Jan. 1994.
42. M. Turk and A. Pentland, "Face recognition using eigenfaces," in Proc. IEEE Conf. CVPR, Jun. 1991, pp. 586-591.
43. J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, "Robust face recognition via sparse representation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 2, pp. 210-227, Feb. 2009.
44. S. Wright, "Primal-dual interior-point methods," in Society for Industrial and Applied Mathematics. Philadelphia, PA, USA: SIAM, 1997.
45. 1-minimization algorithms for robust face recognition," IEEE Trans. Image Process., vol. 22, no. 8, pp. 3234-3246, Aug. 2013.
46. M. Yang, L. Zhang, J. Yang, and D. Zhang, "Robust sparse coding for face recognition," in Proc. CVPR, 2011, pp. 1-22.
47. Y. Ye, Interior Point Algorithms: Theory and Analysis. New York, NY, USA: Wiley, 1997.