Learning deep l0 encoders

30th AAAI Conference on Artificial Intelligence, AAAI 2016

Volumn , Issue , 2016, Pages 2194-2200

  Wang, Zhangyang ^a   Ling, Qing ^b   Huang, Thomas S ^a  

^a UNIVERSITY OF ILLINOIS AT URBANA CHAMPAIGN   (United States)

^b UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA   (China)

Author keywords

[No Author keywords available]

Indexed keywords

ITERATIVE METHODS; SIGNAL ENCODING;

DEEP LEARNING; END TO END; ITERATIVE ALGORITHM; LEARNING CAPACITY; NUMERICAL RESULTS; SPARSE APPROXIMATIONS; SPARSE CODING; TASK-DRIVEN; APPROXIMATION PROBLEMS; FAST INFERENCE; FEED FORWARD NEURAL NET WORKS; NONCONVEX; REGULARISATION;

ARTIFICIAL INTELLIGENCE; DEEP LEARNING;

EID: 85007190243     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (33)

1. Blumensath, T., and Davies, M. E. 2008. Iterative thresholding for sparse approximations. Journal of Fourier Analysis and Applications 14(5-6):629-654.
2. Bottou, L. 2010. Large-scale machine learning with stochastic gradient descent. In Proceedings of COMPSTAT'2010. Springer. 177-186.
3. Candes, E. J.; Wakin, M. B.; and Boyd, S. P. 2008. Enhancing sparsity by reweighted l1 minimization. Journal of Fourier analysis and applications 14(5-6):877-905.
4. Cheng, B.; Yang, J.; Yan, S.; Fu, Y.; and Huang, T. S. 2010. Learning with l1 graph for image analysis. TIP 19(4).
5. Davies, M., and Mitianoudis, N. 2004. Simple mixture model for sparse overcomplete ica. IEE Proceedings-Vision, Image and Signal Processing 151(1):35-43.
6. Donoho, D. L., and Elad, M. 2003. Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization. Proceedings of the National Academy of Sciences 100(5):2197-2202.
7. Donoho, D. L.; Vetterli, M.; DeVore, R. A.; and Daubechies, I. 1998. Data compression and harmonic analysis. Information Theory, IEEE Transactions on 44(6):2435-2476.
8. Donoho, D. L. 1995. De-noising by soft-thresholding. Information Theory, IEEE Transactions on 41(3):613-627.
9. Gregor, K., and LeCun, Y. 2010. Learning fast approximations of sparse coding. In ICML, 399-406.
10. Gulcehre, C.; Cho, K.; Pascanu, R.; and Bengio, Y. 2014. Learned-norm pooling for deep feedforward and recurrent neural networks. In Machine Learning and Knowledge Discovery in Databases. Springer. 530-546.
11. Hershey, J. R.; Roux, J. L.; and Weninger, F. 2014. Deep unfolding: Model-based inspiration of novel deep architectures. arXiv preprint arXiv:1409.2574.
12. Hornik, K.; Stinchcombe, M.; and White, H. 1989. Multilayer feedforward networks are universal approximators. Neural networks 2(5):359-366.
13. Jiang, Z.; Lin, Z.; and Davis, L. S. 2011. Learning a discriminative dictionary for sparse coding via label consistent k-svd. In CVPR, 1697-1704. IEEE.
14. Konda, K.; Memisevic, R.; and Krueger, D. 2014. Zero-bias autoencoders and the benefits of co-adapting features. arXiv preprint arXiv:1402.3337.
15. Krizhevsky, A.; Sutskever, I.; and Hinton, G. E. 2012. Imagenet classification with deep convolutional neural networks. In NIPS, 1097-1105.
16. LeCun, Y. A.; Bottou, L.; Orr, G. B.; and Müller, K.-R. 2012. Efficient backprop. In Neural networks: Tricks of the trade. Springer. 9-48.
17. Mairal, J.; Bach, F.; and Ponce, J. 2012. Task-driven dictionary learning. TPAMI 34(4):791-804.
18. Mhaskar, H. N., and Micchelli, C. A. 1994. How to choose an activation function. In NIPS, 319-326.
19. Rozell, C. J.; Johnson, D. H.; Baraniuk, R. G.; and Olshausen, B. A. 2008. Sparse coding via thresholding and local competition in neural circuits. Neural computation 20(10):2526-2563.
20. Sim, T.; Baker, S.; and Bsat, M. 2002. The cmu pose, illumination, and expression (pie) database. In Automatic Face and Gesture Recognition, 2002. Proceedings. Fifth IEEE International Conference on, 46-51. IEEE.
21. Sprechmann, P.; Litman, R.; Yakar, T. B.; Bronstein, A. M.; and Sapiro, G. 2013. Supervised sparse analysis and synthesis operators. In NIPS, 908-916.
22. Sprechmann, P.; Bronstein, A.; and Sapiro, G. 2015. Learning efficient sparse and low rank models. TPAMI.
23. Sutskever, I.; Martens, J.; Dahl, G.; and Hinton, G. 2013. On the importance of initialization and momentum in deep learning. In ICML, 1139-1147.
24. Wang, Z.; Yang, Y.; Chang, S.; Li, J.; Fong, S.; and Huang, T. S. 2015a. A joint optimization framework of sparse coding and discriminative clustering. In IJCAI.
25. Wang, Z.; Chang, S.; Zhou, J.; Wang, M.; and Huang, T. S. 2015b. Learning a task-specific deep architecture for clustering. In arXiv preprint arXiv:1509.00151.
26. Wang, Z.; Nasrabadi, N. M.; and Huang, T. S. 2015. Semisupervised hyperspectral classification using task-driven dictionary learning with laplacian regularization. TGRS 53(3):1161-1173.
27. Wang, Y.; Wang, Y.-X.; and Singh, A. 2015. Clustering consistent sparse subspace clustering. arXiv preprint arXiv:1504.01046.
28. Wipf, D. P., and Rao, B. D. 2004. l0-norm minimization for basis selection. In NIPS, 1513-1520.
29. Wright, J.; Yang, A. Y.; Ganesh, A.; Sastry, S. S.; and Ma, Y. 2009. Robust face recognition via sparse representation. TPAMI 31(2):210-227.
30. Xu, L.; Lu, C.; Xu, Y.; and Jia, J. 2011. Image smoothing via l 0 gradient minimization. In TOG, volume 30, 174. ACM.
31. Yang, J.; Yu, K.; and Huang, T. 2010. Supervised translation-invariant sparse coding. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, 3517-3524. IEEE.
32. Yuan, G., and Ghanem, B. 2015. Lotv: A new method for image restoration in the presence of impulse noise.
33. Zeiler, M. D.; Taylor, G. W.; and Fergus, R. 2011. Adaptive deconvolutional networks for mid and high level feature learning. In ICCV, 2018-2025. IEEE.