Kernelized supervised dictionary learning

IEEE Transactions on Signal Processing

Volumn 61, Issue 19, 2013, Pages 4753-4767

  Gangeh, Mehrdad J ^a,b,c   Ghodsi, Ali ^a   Kamel, Mohamed S ^a  

^a UNIVERSITY OF WATERLOO   (Canada)

^b UNIVERSITY OF TORONTO   (Canada)

^c SUNNYBROOK RESEARCH INSTITUTE   (Canada)

Author keywords

Classification methods; Dictionary learning; HSIC; Non parametric methods; Pattern recognition and classification; Supervised learning

Indexed keywords

CLASSIFICATION METHODS; DICTIONARY LEARNING; HSIC; NONPARAMETRIC METHODS; PATTERN RECOGNITION AND CLASSIFICATION;

ELECTRONICS ENGINEERING; SIGNAL PROCESSING; SUPERVISED LEARNING;

PATTERN RECOGNITION;

EID: 84883782993     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2013.2274276     Document Type: Article

References (67)

1. S. Mallat, A Wavelet Tour of signal Processing: The Sparse Way, 3rd ed. New York, NY, USA: Academic, 2009.
2. B. Olshausen and D. Field, "Emergence of simple-cell receptive field properties by learning a sparse code for natural images," Nature, vol. 381, no. 6583, pp. 607-609, Mar. 1996. (Pubitemid 26177476)
3. A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis. New York, NY, USA: Wiley , 2001.
4. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Royal Statist. Soc., Ser. B, vol. 58, no. 1, pp. 267-288, 1996.
5. D. Lee and H. Seung, "Learning the parts of objects by non-negative matrix factorization," Nature, vol. 401, pp. 788-791, Oct. 1999.
6. M. Biggs, A. Ghodsi, and S. Vavasis, "Nonnegative matrix factorization via rank-one downdate," in Proc. 25th Int. Conf. Mach. Learn. (ICML), 2008, pp. 64-71.
7. C. Jutten and J. Herault, "Blind separation of sources, Part I: An adaptive algorithm based on neuromimetic architecture," Signal Process., vol. 24, no. 1, pp. 1-10, 1991. (Pubitemid 21679270)
8. D. Donoho, "Compressed sensing," IEEE Trans. Inf. Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.
9. S. Chen, D. Donoho, and M. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Scientif. Comput., vol. 20, no. 1, pp. 33-61, 1998. (Pubitemid 128689501)
10. Y. Pati, R. Rezaiifar, and P. Krishnaprasad, "Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition," in Proc. 27th Asilomar Conf. Signals, Syst. Comput., 1993, pp. 40-44.
11. S. Mallat and Z. Zhang, "Matching pursuitswith time-frequency dictionaries," IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397-3415, Dec. 1993.
12. J. Wright, Y. Ma, J. Mairal, G. Sapiro, T. Huang, and S. Yan, "Sparse representation for computer vision and pattern recognition," Proc. IEEE, vol. 98, no. 6, pp. 1031-1044, Jun. 2010.
13. A. M. Bruckstein, D. L. Donoho, and M. Elad, "From sparse solutions of systems of equations to sparse modeling of signals and images," SIAM Rev., vol. 51, no. 1, pp. 34-81, 2009.
14. M. Elad, Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. NewYork,NY,USA: Springer, 2010.
15. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Supervised dictionary learning," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2008, pp. 1033-1040.
16. M. J. Gangeh, A. Ghodsi, and M. S. Kamel, "Dictionary learning in texture classification," in Proc. 8th Int. Conf. Image Anal. Recog.-Vol. Part I, 2011, pp. 335-343.
17. M. Varma and A. Zisserman, "A statistical approach to material classification using image patch exemplars," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 11, pp. 2032-2047, Nov. 2009.
18. C. Zhong, Z. Sun, and T. Tan, "Robust 3D face recognition using learned visual codebook," in Proc. IEEE Conf. Comput. Vision Pattern Recog. (CVPR), 2007, pp. 1-6.
19. J. Wright, A. Yang, A. Ganesh, S. Sastry, and Y. Ma, "Robust face recognition via sparse representation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 2, pp. 210-227, Feb. 2009.
20. M. Yang, L. Zhang, X. Feng, and D. Zhang, "Fisher discrimination dictionary learning for sparse representation," in Proc. 13th IEEE Int. Conf. Comput. Vision (ICCV), 2011, pp. 543-550.
21. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries," IEEE Trans. Image Process., vol. 15, no. 12, pp. 3736-3745, Dec. 2006. (Pubitemid 44811686)
22. J. Mairal, M. Elad, and G. Sapiro, "Sparse representation for color image restoration," IEEE Trans. Image Process., vol. 17, no. 1, pp. 53-69, Jan. 2008.
23. A. Gretton, O. Bousquet, A. Smola, and B. Schölkopf, "Measuring statistical dependence with Hilbert-Schmidt norms," in Proc. 16th Int. Conf. Algorithmic Learning Theory (ALT), 2005, pp. 63-77.
24. A. Gretton, R. Herbrich, A. Smola, O. Bousquet, and B. Schölkopf, "Kernel methods for measuring independence," J. Mach. Learn. Res., vol. 6, pp. 2075-2129, Dec. 2005. (Pubitemid 41798124)
25. R. Cilibrasi and P. Vitányi, "Clustering by compression," IEEE Trans. Inf. Theory, vol. 51, no. 4, pp. 1523-1545, 2005. (Pubitemid 40573236)
26. M. J. Gangeh, A. Ghodsi, andM. S. Kamel, "Supervised texture classification using a novel compression-based similarity measure," in Proc. Int. Conf. Comput. Vision Graph. (ICCVG), 2012, pp. 379-386.
27. K. Huang and S. Aviyente, "Sparse representation for signal classification," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2007, pp. 609-616.
28. M. Aharon, M. Elad, and A. Bruckstein, "K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation," IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, Nov. 2006. (Pubitemid 44637761)
29. J. Mairal, F. Bach, J. Ponce, andG. Sapiro, "Online learning for matrix factorization and sparse coding," J. Mach. Learn. Res., vol. 11, pp. 19-60, Mar. 2010.
30. J. Friedman, T. Hastie, and R. Tibshirani, "Regularization paths for generalized linear models via coordinate descent," J. Statist. Softw., vol. 33, no. 1, pp. 1-22, Feb. 2010.
31. T. Leung and J. Malik, "Representing and recognizing the visual appearance of materials using three-dimensional textons," Int. J. Comput. Vision, vol. 43, pp. 29-44, Jun. 2001. (Pubitemid 32720091)
32. M. Varma and A. Zisserman, "A statistical approach to texture classification from single images," Int. J. Comput. Vis. (Special Issue on Texture Analysis Synthesis), vol. 62, no. 1-2, pp. 61-81, 2005.
33. M. Yang, L. Zhang, J. Yang, and D. Zhang, "Metaface learning for sparse representation based face recognition," in Proc. 17th IEEE Int. Conf. Image Process. (ICIP), 2010, pp. 1601-1604.
34. I. Ramirez, P. Sprechmann, and G. Sapiro, "Classification and clustering via dictionary learning with structured incoherence and shared features," in Proc. IEEE Conf. Computer Vision Pattern Recogn. (CVPR), 2010, pp. 3501-3508.
35. B. Fulkerson, A. Vedaldi, and S. Soatto, "Localizing objects with smart dictionaries," in Proc. 10th Eur. Conf.Comput. Vision (ECCV): Part I, 2008, pp. 179-192.
36. J. Winn, A. Criminisi, and T. Minka, "Object categorization by learned universal visual dictionary," in Proc. 10th IEEE Int. Conf. Comput. Vision (ICCV), 2005, pp. 1800-1807. (Pubitemid 44042395)
37. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Discriminative learned dictionaries for local image analysis," in Proc. IEEE Conf. Comput. Vision Pattern Recogn. (CVPR), 2008, pp. 1-8.
38. Q. Zhang and B. Li, "Discriminative K-SVD for dictionary learning in face recognition," in Proc. IEEE Conf. Comput. Vision Pattern Recogn. (CVPR), 2010, pp. 2691-2698.
39. D.-S. Pham and S. Venkatesh, "Joint learning and dictionary construction for pattern recognition," in Proc. IEEE Conf. Comput. Vision Pattern Recogn. (CVPR), 2008, pp. 1-8.
40. L. Yang, R. Jin, R. Sukthankar, and F. Jurie, "Unifying discriminative visual codebook generation with classifier training for object category recognition," in Proc. IEEE Conf. Comput. Vision Pattern Recogn. (CVPR), 2008, pp. 1-8.
41. J. Mairal, F. Bach, and J. Ponce, "Task-driven dictionary learning," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 4, pp. 791-804, Apr. 2012.
42. S. Lazebnik andM. Raginsky, "Supervised learning of quantizer codebooks by information loss minimization," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 7, pp. 1294-1309, Jul. 2009.
43. F. Moosmann, B. Triggs, and F. Jurie, "Fast discriminative visual codebooks using randomized clustering forests," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2006, pp. 985-992.
44. F. Rodriguez and G. Sapiro, "Sparse representations for image classification: Learning discriminative and reconstructive non-parametric dictionaries," in Proc. IMA Preprint Series 2213, 2007.
45. R. A. Fisher, "The use of multiple measurements in taxonomic problems," Ann. Eugenics, vol. 7, no. 2, pp. 179-188, 1936.
46. T. M. Cover and J. A. Thomas, Elements of Information Theory, 2nd ed. New York, NY, USA: Wiley, 2006.
47. S. Kullback and R. A. Leibler, "On information and sufficiency," Ann. Math. Statist., vol. 22, no. 1, pp. 79-86, 1951.
48. F. R. Bach andM. I. Jordan, "Kernel independent component analysis," J. Mach. Learn. Res., vol. 3, pp. 1-48, 2002.
49. L. Song, J. Bedo, K. Borgwardt, A. Gretton, and A. Smola, "Gene selection via the basic family of algorithms," Bioinformatics, vol. 23, pp. i490-i498, Jul. 2007. (Pubitemid 47244436)
50. H. Shen, S. Jegelka, and A. Gretton, "Fast kernel-based independent component analysis," IEEE Trans. Signal Process., vol. 57, no. 9, pp. 3498-3511, Sep. 2009.
51. N. Quadrianto, A. Smola, L. Song, and T. Tuytelaars, "Kernelized sorting," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 10, pp. 1809-1821, Oct. 2010.
52. A. Gretton, K. M. Borgwardt, M. Rasch, B. Schölkopf, and A. J. Smola, "A kernel method for the two-sample problem," Max Planck Inst. Biolog. Cybern., Tübingen, Baden-Württemberg, Germany, Tech. Rep. 157, Apr. 2008.
53. S. Jegelka, A. Gretton, B. Schölkopf, B. K. Sriperumbudur, and U. Von Luxburg, "Generalized clustering via kernel embeddings," in Proc. 32nd Ann. German Conf. Adv. Artif. Intell., 2009, pp. 144-152.
54. H. Lütkepohl, Handbook of Matrices. New York, NY, USA: Wiley, 1996.
55. D. L. Donoho and I.M. Johnstone, "Adapting to unknown smoothness via wavelet shrinkage," J. Amer. Statist. Assoc., vol. 90, no. 432, pp. 1200-1224, 1995. (Pubitemid 126093135)
56. J. Friedman, T. Hastie, H. Hofling, and R. Tibshirani, "Pathwise coordinate optimization," Ann. Appl. Statist., vol. 1, no. 2, pp. 302-332, 2007.
57. J. L. Alperin, Local Representation Theory: Modular Representations as an Introduction to the Local Representation Theory of Finite Groups. New York, NY, USA: Cambridge Univ. Press, 1986.
58. R. Raina, A. Battle, H. Lee, B. Packer, and A. Y. Ng, "Self-taught learning: Transfer learning from unlabeled data," in Proc. 24th Int. Conf. Mach. Learn. (ICML), 2007, pp. 759-766.
59. Cambridge Univ. Computer Lab., Olivetti Face Dataset AT&T 1994 [Online]. Available: http://www.cl.cam.ac.uk/research/dtg/attarchive/ facedatabase.html
60. R. Rubinstein, M. Zibulevsky, and M. Elad, "Efficient implementation of the K-SVD algorithm using batch orthogonal matching pursuit," Dept. Comput. Sci., The Technion, Haifa, Israel, Tech. Rep. CS-2008-08.revised, 2008.
61. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, "Handwritten digit recognition with a back-propagation network," in Advances in Neural Information Processing Systems. New York, NY, USA: Morgan Kaufmann, 1990, pp. 396-404.
62. M. Ranzato, F. J. Huang, Y.-L. Boureau, and Y. LeCun, "Unsupervised learning of invariant feature hierarchies with applications to object recognition," in Proc. IEEE Conf. Comput. Vision Pattern Recogn. (CVPR), 2007, pp. 1-8.
63. L. Zelnik-Manor and P. Perona, "Self-tuning spectral clustering," in Proc. Adv. Neural Inf. Process. Syst. (NIPS), 2004, pp. 1601-1608.
64. B. Campana and E. Keogh, "A compression-based distance measure for texture," Statist. Anal. Data Mining, vol. 3, no. 6, pp. 381-398, 2010.
65. M. Blaschko and A. Gretton, "Taxonomy inference using kernel dependence measures,"Max Planck Inst. Biolog. Cybern., Tübingen, Baden- Württemberg, Germany, Tech. Rep. 181, Nov. 2008.
66. L. Song,A. Smola,A. Gretton, andK. Borgwardt, "A dependencemaximization view of clustering," in Proc. 24th Int. Conf. Mach. Learn. (ICML), 2007, pp. 815-822.
67. D. Anguita, A. Ghio, L. Oneto, and S. Ridella, "In-sample and out-ofsample model selection and error estimation for support vector machines," IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 9, pp. 1390-1406, Sep. 2012.