MDL-CW: A Multimodal Deep Learning Framework with CrossWeights

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn 2016-December, Issue , 2016, Pages 2601-2609

  Rastegar, Sarah ^a   Baghshah, Mahdieh Soleymani ^a   Rabiee, Hamid R ^a   Shojaee, Seyed Mohsen ^a  

^a SHARIF UNIVERSITY OF TECHNOLOGY   (Iran)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; INFORMATION RETRIEVAL; MODAL ANALYSIS;

DEEP LEARNING; HIGH-LEVEL INTERACTIONS; IDEAL MODEL; MULTI-MODAL DATA; MULTIPLE MODALITIES; PRE-TRAINING; SHARED REPRESENTATIONS; STATE OF THE ART;

PATTERN RECOGNITION;

EID: 84986296802     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2016.285     Document Type: Conference Paper

References (26)

1. Y. Bengio, L. Yao, G. Alain, and P. Vincent. Generalized denoising auto-encoders as generative models. In Advances in Neural Information Processing Systems, pages 899-907, 2013.
2. M. M. Bronstein, A. M. Bronstein, F. Michel, and N. Paragios. Data fusion through cross-modality metric learning using similarity-sensitive hashing. In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pages 3594-3601. IEEE, 2010.
3. A. Farhadi, M. Hejrati, M. A. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier, and D. Forsyth. Every picture tells a story: Generating sentences from images. In Computer Vision-ECCV 2010, pages 15-29. Springer, 2010.
4. A. Frome, G. S. Corrado, J. Shlens, S. Bengio, J. Dean, T. Mikolov, et al. Devise: A deep visual-semantic embedding model. In Advances in Neural Information Processing Systems, pages 2121-2129, 2013.
5. A. Gionis, P. Indyk, R. Motwani, et al. Similarity search in high dimensions via hashing. In VLDB, volume 99, pages 518-529, 1999.
6. Y. Gong and S. Lazebnik. Iterative quantization: A procrustean approach to learning binary codes. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 817-824. IEEE, 2011.
7. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012.
8. A. Karpathy, A. Joulin, and F. F. F. Li. Deep fragment embeddings for bidirectional image sentence mapping. In Advances in neural information processing systems, pages 1889-1897, 2014.
9. A. Kraskov, H. Stögbauer, R. G. Andrzejak, and P. Grassberger. Hierarchical clustering using mutual information. EPL (Europhysics Letters), 70(2):278, 2005.
10. P. L. Lai and C. Fyfe. Kernel and nonlinear canonical correlation analysis. International Journal of Neural Systems, 10(05):365-377, 2000.
11. Y. LeCun and C. Cortes. The mnist database of handwritten digits, 1998.
12. D. Lin. An information-theoretic definition of similarity. In ICML, volume 98, pages 296-304, 1998.
13. J. Ngiam, A. Khosla, M. Kim, J. Nam, H. Lee, and A. Y. Ng. Multimodal deep learning. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pages 689-696, 2011.
14. B. Ozdemir and L. S. Davis. A probabilistic framework for multimodal retrieval using integrative indian buffet process. In Advances in Neural Information Processing Systems, pages 2384-2392, 2014.
15. G. Patterson and J. Hays. Sun attribute database: Discovering, annotating, and recognizing scene attributes. In Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pages 2751-2758. IEEE, 2012.
16. M. Rastegari, J. Choi, S. Fakhraei, D. Hal, and L. Davis. Predictable dual-view hashing. In Proceedings of The 30th International Conference on Machine Learning, pages 1328-1336, 2013.
17. R. Socher, A. Karpathy, Q. V. Le, C. D. Manning, and A. Y. Ng. Grounded compositional semantics for finding and describing images with sentences. Transactions of the Association for Computational Linguistics, 2:207-218, 2014.
18. R. Socher, C. C. Lin, C. Manning, and A. Y. Ng. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th international conference on machine learning (ICML-11), pages 129-136, 2011.
19. K. Sohn, W. Shang, and H. Lee. Improved multimodal deep learning with variation of information. In Advances in Neural Information Processing Systems, pages 2141-2149, 2014.
20. N. Srivastava and R. R. Salakhutdinov. Multimodal learning with deep boltzmann machines. In Advances in neural information processing systems, pages 2222-2230, 2012.
21. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. The Journal of Machine Learning Research, 11:3371-3408, 2010.
22. Y. Weiss, A. Torralba, and R. Fergus. Spectral hashing. In Advances in neural information processing systems, pages 1753-1760, 2009.
23. J. Xiao, J. Hays, K. A. Ehinger, A. Oliva, and A. Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In Computer vision and pattern recognition (CVPR), 2010 IEEE conference on, pages 3485-3492. IEEE, 2010.
24. E. P. Xing, R. Yan, and A. G. Hauptmann. Mining associated text and images with dual-wing harmoniums. arXiv preprint arXiv:1207.1423, 2012.
25. Y. Zhen and D.-Y. Yeung. Co-regularized hashing for multimodal data. In Advances in Neural Information Processing Systems, pages 1376-1384, 2012.
26. Y. Zhen and D.-Y. Yeung. A probabilistic model for multimodal hash function learning. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 940-948. ACM, 2012.