Learning deep hierarchical visual feature coding

IEEE Transactions on Neural Networks and Learning Systems

Volumn 25, Issue 12, 2014, Pages 2212-2225

  Goh, Hanlin ^a,b   Thome, Nicolas ^c   Cord, Matthieu ^c   Lim, Joo Hwee ^a,b  

^a INSTITUTE FOR INFOCOMM RESEARCH   (Singapore)

^b 21 01 Connexis South Tower   (Singapore)

^c UNIVERSITÉ PIERRE ET MARIE CURIE   (France)

Author keywords

Bag of words (BoW) framework; computer vision; deep learning; dictionary learning; hierarchical visual architecture; image categorization; restricted Boltzmann machine (RBM); sparse feature coding; transfer learning.

Indexed keywords

CODES (SYMBOLS); COMPUTER ARCHITECTURE; COMPUTER VISION; IMAGING SYSTEMS;

BAG OF WORDS; DEEP LEARNING; DICTIONARY LEARNING; IMAGE CATEGORIZATION; RESTRICTED BOLTZMANN MACHINE; SPARSE FEATURES; TRANSFER LEARNING;

IMAGE CODING;

EID: 84913540875     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2014.2307532     Document Type: Article

References (77)

1. J. Sivic and A. Zisserman, "Video Google: A text retrieval approach to object matching in videos," in Proc. 9th ICCV, 2003, pp. 1470-1477
2. D. Lowe, "Distinctive image features from scale-invariant keypoints," Int. J. Comput. Vis., vol. 60, no. 2, pp. 91-110, 2004
3. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. CVPR, Jun. 2005, pp. 886-893
4. S. Lazebnik, C. Schmid, and J. Ponce, "Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories," in Proc. CVPR, 2006, pp. 2169-2178
5. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Proc. NIPS, 2012, pp. 1-8
6. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, "Imagenet: A large-scale hierarchical image database," in Proc. CVPR, 2009, pp. 248-255
7. H. Goh, N. Thome, M. Cord, and J.-H. Lim, "Unsupervised and supervised visual codes with restricted boltzmann machines," in Proc. 12th ECCV, 2012, pp. 298-311
8. K. Sohn, D. Y. Jung, H. Lee, and A. Hero, "Efficient learning of sparse, distributed, convolutional feature representations for object recognition," in Proc. ICCV, 2011, pp. 2643-2650
9. H. Lee, R. Grosse, R. Ranganath, and A. Y. Ng, "Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations," in Proc. ICML, 2009, pp. 609-616
10. J. Yang, K. Yu, Y. Gong, and T. Huang, "Linear spatial pyramid matching using sparse coding for image classification," in Proc. CVPR, 2009, pp. 1794-1801
11. Y. Bengio, "Learning deep architectures for AI," Found. Trends Mach. Learn., vol. 2, no. 1, pp. 1-127, 2009
12. G. E. Hinton, S. Osindero, and Y.-W. Teh, "A fast learning algorithm for deep belief networks," Neural Comput., vol. 18, no. 7, pp. 1527-1554, 2006
13. Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, "Greedy layerwise training of deep networks," in Proc. NIPS, 2006, pp. 153-160
14. K. Hornik, "Approximation capabilities of multilayer feedforward networks," Neural Netw., vol. 4, no. 2, pp. 251-257, 1991
15. Y. LeCun et al., "Backpropagation applied to handwritten zip code recognition," Neural Comput., vol. 1, no. 4, pp. 541-551, 1989
16. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998
17. M. Ranzato, F. Huang, Y. Boureau, and Y. LeCun, "Unsupervised learning of invariant feature hierarchies with applications to object recognition," in Proc. CVPR, 2007, pp. 1-8
18. G. E. Hinton, "To recognize shapes, first learn to generate images," in Computational Neuroscience: Theoretical Insights into Brain Function, P. Cisek, T. Drew, and J. Kalaska, Eds. New York, NY, USA: Elsevier, 2007
19. P. Smolensky, "Information processing in dynamical systems: Foundations of harmony theory," in Parallel Distributed Processing: Foundations, vol. 1, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA, USA: MIT Press, 1986, pp. 194-281
20. Y. LeCun, "Une procédure dapprentissage pour réseau a seuil asymmetrique a learning scheme for asymmetric threshold networks," in Proc. Cognitiva, Paris, France, 1985, pp. 599-604
21. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back-propagating errors," Nature, vol. 323, pp. 533-536, Oct. 1986
22. G. E. Hinton and R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, no. 5786, pp. 504-507, 2006
23. P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol, "Extracting and composing robust features with denoising autoencoders," in Proc. 25th ICML, 2008, pp. 1096-1103
24. B. A. Olshausen and D. J. Field, "Emergence of simple-cell receptive field properties by learning a sparse code for natural images," Nature, vol. 381, no. 6583, pp. 607-609, 1996
25. K. Kavukcuoglu, P. Sermanet, Y. Boureau, K. Gregor, M. Mathieu, and Y. LeCun, "Learning convolutional feature hierachies for visual recognition," in Proc. NIPS, 2010, pp. 1-9
26. L. Bo and X. R. D. Fox, "Multipath sparse coding using hierarchical matching pursuit," in Proc. CVPR, 2013, pp. 660-667
27. J. Van Gemert, C. Veenman, A. Smeulders, and J.-M. Geusebroek, "Visual word ambiguity," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 7, pp. 1271-1283, Jul. 2010
28. Y. Boureau, N. Le Roux, F. Bach, J. Ponce, and Y. LeCun, "Ask the locals: Multi-way local pooling for image recognition," in Proc. ICCV, 2011, pp. 1-8
29. H. Lee, C. Ekanadham, and A. Ng, "Sparse deep belief net model for visual area V2," in Proc. NIPS, 2008, pp. 1-5
30. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Supervised dictionary learning," in Proc. NIPS, 2008, pp. 1033-1040
31. Z. Jiang, Z. Lin, and L. S. Davis, "Learning a discriminative dictionary for sparse coding via label consistent K-SVD," in Proc. CVPR, 2011, pp. 1697-1704
32. L. Yang, R. Jin, R. Sukthankar, and F. Jurie, "Unifying discriminative visual codebook generation with classifier training for object category recognition," in Proc. CVPR, 2008, pp. 1-8
33. J. Yang, K. Yu, and T. Huang, "Supervised translation-invariant sparse coding," in Proc. CVPR, 2010, pp. 3517-3524
34. Y. Boureau, F. Bach, Y. LeCun, and J. Ponce, "Learning mid-level features for recognition," in Proc. CVPR, 2010, pp. 2559-2566
35. J. Yang, K. Yu, and T. Huang, "Efficient highly over-complete sparse coding using a mixture model," in Proc. ECCV, 2010, pp. 113-126
36. J. Wang, J. Yang, K. Yu, F. Lv, T. Huang, and Y. Gong, "Localityconstrained linear coding for image classification," in Proc. CVPR, 2010, pp. 3360-3367
37. H. Goh, N. Thome, and M. Cord, "Biasing restricted Boltzmann machines to manipulate latent selectivity and sparsity," in Proc. NIPS Workshop, 2010, pp. 1-10
38. H. Goh, L. Kusmierz, J.-H. Lim, N. Thome, and M. Cord, "Learning invariant color features with sparse topographic restricted Boltzmann machines," in Proc. 18th ICIP, 2011, pp. 1241-1244
39. S. Lazebnik and M. Raginsky, "Supervised learning of quantizer codebooks by information loss minimization," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 7, pp. 1294-1309, Jul. 2009
40. Y. Boureau, J. Ponce, and Y. LeCun, "A theoretical analysis of feature pooling in vision algorithms," in Proc. ICML, 2010, pp. 111-118
41. S. Avila, N. Thome, M. Cord, E. Valle, and A. Araujo, "Pooling in image representation: The visual codeword point of view," Comput. Vis. Image. Und., vol. 117, no. 5, pp. 453-465, 2012
42. F. Perronnin and C. Dance, "Fisher kernels on visual vocabularies for image categorization," in Proc. CVPR, 2007, pp. 1-8
43. H. Jégou, M. Douze, C. Schmid, and P. Pérez, "Aggregating local descriptors into a compact image representation," in Proc. CVPR, 2010, pp. 3304-3311
44. X. Zhou, K. Yu, T. Zhang, and T. Huang, "Image classification using super-vector coding of local image descriptors," in Proc. ECCV, 2010, pp. 141-154
45. S. McCann and D. G. Lowe, "Spatially local coding for object recognition," in Proc. ACCV, 2012, pp. 204-217
46. V. N. Vapnik, The Nature of Statistical Learning Theory. New York, NY, USA: Springer-Verlag, 1995
47. K. Kavukcuoglu, M. Ranzato, R. Fergus, and Y. LeCun, "Learning invariant features through topographic filter maps," in Proc. CVPR, 2009, pp. 1605-1612
48. K. Yu, Y. Lin, and J. D. Lafferty, "Learning image representations from the pixel level via hierarchical sparse coding," in Proc. CVPR, 2011, pp. 1713-1720
49. M. Riesenhuber and T. Poggio, "Hierarchical models of object recognition in cortex," Nat. Neurosci., vol. 2, no. 11, pp. 1019-1025, 1999
50. S. Bileschi, M. Riesenhuber, T. Poggio, T. Serre, and L. Wolf, "Robust object recognition with cortex-like mechanisms," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 3, pp. 411-426, Mar. 2007
51. J. Mutch and D. Lowe, "Object class recognition and localization using sparse features with limited receptive fields," Int. J. Comput. Vis., vol. 80, no. 1, pp. 45-47, 2008
52. C. Theriault, N. Thome, and M. Cord, "Extended coding and pooling in the HMAX model," IEEE Trans. Image Process., vol. 22, no. 2, pp. 764-777, Feb. 2012
53. G. E. Hinton, "Training products of experts by minimizing contrastive divergence," Neural Comput., vol. 14, no. 8, pp. 1771-1800, 2002
54. G. Hinton, "A practical guide to training restricted Boltzmann machines," Dept. Comput. Sci., Univ. Toronto, Toronto, ON, USA, Tech. Rep. UTML TR 2010-003, 2010
55. K. Swersky, B. Chen, B. Marlin, and N. De Freitas, "A tutorial on stochastic approximation algorithms for training restricted Boltzmann machines and deep belief nets," in Proc. ITA Workshop, 2010, pp. 1-10
56. D. Blackwell, "Conditional expectation and unbiased sequential estimation," Ann. Stat., vol. 18, no. 1, pp. 105-110, Mar. 1947
57. T. M. Mitchell, "The need for biases in learning generalizations," Dept. Comput. Sci., Rutgers Univ., New Brunswick, NJ, USA, Tech. Rep. CBM-TR-117, 1980
58. B. Willmore and D. J. Tolhurst, "Characterizing the sparseness of neural codes," Netw., Comput. Neural, vol. 12, no. 3, pp. 255-270, 2001
59. H. B. Barlow, "Unsupervised learning," Neural Comput., vol. 1, no. 3, pp. 295-311, 1989
60. E. T. Rolls and A. Treves, "The relative advantage of sparse versus distributed encoding for associative neuronal networks in the brain," Netw., Comput. Neural, vol. 1, no. 4, pp. 407-421, 1990
61. P. Földiák, "Neural coding: Non-local but explicit and conceptual," Current. Biol., vol. 19, no. 19, pp. R904-R906, 2009
62. V. Nair and G. Hinton, "3D object recognition with deep belief nets," in Proc. NIPS, 2009 pp. 1339-1347
63. M.Welling, G. E. Hinton, and S. Osindero, "Learning sparse topographic representations with products of student-t distributions," in Proc. NIPS, 2003, pp. 1-8
64. A. Hyvärinen and P. O. Hoyer, "A two-layer sparse coding model learns simple and complex cell receptive fields and topography from natural images," Vis. Res., vol. 41, no. 18, pp. 2413-2423, 2001
65. J. Ngiam, P. W. Koh, Z. Chen, S. Bhaskar, and A. Ng, "Sparse filtering," in Proc. NIPS, 2011, pp. 17-20
66. H. Larochelle and Y. Bengio, "Classification using discriminative restricted Boltzmann machines," in Proc. 25th ICML, 2008, pp. 536-543
67. H. Goh, N. Thome, M. Cord, and J.-H. Lim, "Top-down regularization of deep belief networks," in Proc. NIPS, 2013, pp. .
68. L. Fei-Fei, R. Fergus, and P. Perona, "Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories," in Proc. CVPR Workshop, 2004
69. G. Griffin, A. Holub, and P. Perona, "Caltech-256 object category dataset," California Inst. Technol., Pasadena, CA, USA, Tech. Rep. 7694, 2007
70. L. Liu, L. Wang, and X. Liu, "In defense of soft-assignment coding," in Proc. ICCV, 2011, pp. 2486-2493
71. G. L. Oliveira, E. R. Nascimento, A. W. Vieira, and M. F. M. Campos, "Sparse spatial coding: A novel approach for efficient and accurate object recognition," in Proc. ICRA, 2012, pp. 2592-2598
72. M. D. Zeiler, D. Krishnan, G. W. Taylor, and R. Fergus, "Deconvolutional networks," in Proc. CVPR, 2010, pp. 2528-2535
73. O. Duchenne, A. Joulin, and J. Ponce, "A graph-matching kernel for object categorization," in Proc. ICCV, 2011, pp. 1792-1799
74. J. Feng, B. Ni, Q. Tian, and S. Yan, "Geometricp-norm feature pooling for image classification," in Proc. CVPR, 2011, pp. 2609-2704
75. O. Boiman, E. Shechtman, and M. Irani, "In defense of nearestneighbor based image classification," in Proc. CVPR, 2008, pp. 1-8
76. T. Tuytelaars, M. Fritz, K. Saenko, and T. Darrell, "The NBNN kernel," in Proc. ICCV, 2011, pp. 1824-1831
77. R. Raina, A. Battle, H. Lee, B. Packer, and A. Y. Ng, "Self-taught learning: Transfer learning from unlabeled data," in Proc. ICML, 2007, pp. 759-766