Deep hybrid networks with good out-of-sample object recognition

ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

Volumn , Issue , 2014, Pages 5437-5441

  Ghifary, Muhammad ^a   Kleijn, W Bastiaan ^a   Zhang, Mengjie ^a  

^a VICTORIA UNIVERSITY OF WELLINGTON   (New Zealand)

Author keywords

deep hybrid network; noise robustness; object recognition; out of sample; sparse features

Indexed keywords

ACOUSTIC NOISE; LEARNING SYSTEMS; PROBABILITY DISTRIBUTIONS; SIGNAL PROCESSING;

HIERARCHICAL FEATURES; HYBRID NETWORK; NOISE ROBUSTNESS; OUT-OF-SAMPLE; RESTRICTED BOLTZMANN MACHINE; SPARSE FEATURES; TRAINING DATA; TRAINING EXAMPLE;

OBJECT RECOGNITION;

EID: 84905233815     PISSN: 15206149     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICASSP.2014.6854642     Document Type: Conference Paper

References (26)

1. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, "Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion," In Journal of Machine Learning Research, vol. 11, pp. 3371-3408, 2010.
2. D.-C. He and L. Wang, "Texture unit, texture spectrum, and texture analysis," IEEE Transactions on Geoscience and Remote Sensing, vol. 28, pp. 509-512, 1990.
3. N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in IEEE Conference on Computer Vision and Pattern Recognition, 2005, pp. 886-893.
4. G. E. Hinton and S. Osindero, "A fast learning algorithm for deep belief nets," Neural Computation, vol. 18, no. 7, pp. 1527-1554, 2006.
5. B. A. Olshausen and D. J. Field, "Sparse coding of sensory inputs," in Current Opinion in Neurobiology. Science Direct, 2004, vol. 14, no. 4, pp. 481-487.
6. I. J. Goodfellow, D.Warde-Farley, M. Mirza, A. Courville, and Y. Bengio, "Maxout networks," in Proceedings of the 30th International Conference on Machine Learning, 2013, pp. 1319-1327.
7. R. Salakhutdinov and G. E. Hinton, "Deep Boltzmann machines," in the 12th International Conference on Artificial Intelligence and Statistics (AISTATS), 2009, pp. 448-455.
8. W. Y. Zhou, S. Zhu, A. Y. Ng, and K. Yu, "Deep learning of invariant features via simulated fixations in video," in Advances in Neural Information Processing Systems (NIPS), vol. 25, 2012, pp. 3212-3220.
9. L. Deng, J. Li, J.-T. Huang, K. Yao, D. Yu, F. Seide, M. L. Seltzer, G. Zweig, X. He, J. Williams, Y. Gong, and A. Acero, "Recent advances in deep learning for speech research at microsoft," in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2013, pp. 8604-8608.
10. Y. Tang and C. Eliasmith, "Deep networks for robust visual recognition," in ICML, 2010, pp. 1055-1062.
11. A. Mushtaq and C.-H. Lee, "An MCMC approach to joint estimation of clean speech and noise for robust speech recognition," in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, 2013, pp. 7107-7111.
12. M. L. Seltzer, D. Yu, and Y. Wang, "An investigation of deep neural networks for noise robust speech recognition," in Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2013, pp. 7398-7402.
13. Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, "Greedy layer-wise training of deep networks," in Advances in Neural Information Processing Systems (NIPS), vol. 19, 2007, p. 153.
14. M. Ranzato, Y.-L. Boureau, and Y. LeCun, "Sparse feature learning for deep belief networks," in Advances in Neural Information Processing Systems (NIPS), vol. 20, 2008, pp. 1185-1192.
15. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM Review, pp. 129-159, 2001.
16. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries," in IEEE Transactions on Image Processing, vol. 15, 2006, pp. 3736-3745.
17. J. Mairal, M. Elad, and G. Sapiro, "Sparse representation for color image restoration," in IEEE Transactions on Image Processing, vol. 17, no. 1, 2008, pp. 53-69.
18. D. M. Bradley and J. A. Bagnell, "Differentiable sparse coding," in Advances in Neural Information Processing Systems (NIPS), vol. 21, 2009, pp. 113-120.
19. J. Mairal, F. Bach, and J. Ponce, "Task-driven dictionary learning," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 4, 2012.
20. H. Lee, C. Ekanadham, and A. Y. Ng, "Sparse deep belief net model for visual area V2," in Advances in Neural Information Processing Systems (NIPS), vol. 20, 2007, pp. 873-880.
21. J. Goodfellow, Q. V. Le, A. M. Saxe, H. Lee, and A. Y. Ng, "Measuring invariance in deep networks," in Advances in Neural Information Processing Systems (NIPS), 2009, vol. 22, pp. 646-654.
22. A. Coates and A. Y. Ng, "The importance of encoding versus training with sparse coding and vector quantization," in Proceedings of the 28th International Conference on Machine Learning, 2011, pp. 912-928.
23. G. E. Hinton, "Training products of experts by minimizing contrastive divergence," Neural Computation, vol. 14, pp. 1771-1800, 2002.
24. H. Larochelle, Y. Bengio, J. Louradour, and P. Lamblin, "Exploring strategies for training deep neural networks," Journal of Machine Learning Research, vol. 1, pp. 1-40, 2009.
25. Q. V. Le, J. Ngiam, A. C. A. Lahiri, B. Prochnow, and A. Y. Ng, "On optimization methods for deep learning," in Proceedings of the 28th International Conference on Machine Learning (ICML), L. Getoor and T. Scheffer, Eds., 2011, pp. 265-272.
26. E. Fiesler, H. J. Caulfield, and A. Choudry, "Some theoretical upperbounds on the capacity of neural networks," in Proceedings of the First Workshop on Neural Networks, vol. 2, 1990, pp. 51-58.