Expressing an image stream with a sequence of natural sentences

Advances in Neural Information Processing Systems

Volumn 2015-January, Issue , 2015, Pages 73-81

  Park, Cesc Chunseong ^a   Kim, Gunhee ^a  

^a Seoul National University   (South Korea)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER HARDWARE DESCRIPTION LANGUAGES; CONVOLUTION; INFORMATION SCIENCE; NEURAL NETWORKS;

AMAZON MECHANICAL TURKS; BIDIRECTIONAL RECURRENT NEURAL NETWORKS; CONVOLUTIONAL NETWORKS; CONVOLUTIONAL NEURAL NETWORK; INPUT AND OUTPUTS; MULTIMODAL ARCHITECTURES; QUANTITATIVE MEASURES; SEQUENCE OF IMAGES;

RECURRENT NEURAL NETWORKS;

EID: 84965149840     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (32)

1. R. Barzilay and M. Lapata. Modeling Local Coherence: An Entity-Based Approach. In ACL, 2008.
2. S. Bird, E. Loper, and E. Klein. Natural Language Processing with Python. O'Reilly Media Inc., 2009.
3. X. Chen and C. L. Zitnick. Mind's Eye: A Recurrent Visual Representation for Image Caption Generation. In CVPR, 2015.
4. F. Y. Y. Choi, P. Wiemer-Hastings, and J. Moore. Latent Semantic Analysis for Text Segmentation. In EMNLP, 2001.
5. J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term Recurrent Convolutional Networks for Visual Recognition and Description. In CVPR, 2015.
6. Y. Gong, L. Wang, M. Hodosh, J. Hockenmaier, and S. Lazebnik. Improving Image-Sentence Embeddings Using Large Weakly Annotated Photo Collections. In ECCV, 2014.
7. K. He, X. Zhang, S. Ren, and J. Sun. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. In arXiv, 2015.
8. M. Hodosh, P. Young, and J. Hockenmaier. Framing Image Description as a Ranking Task: Data, Models and Evaluation Metrics. JAIR, 47:853-899, 2013.
9. A. Karpathy and L. Fei-Fei. Deep Visual-Semantic Alignments for Generating Image Descriptions. In CVPR, 2015.
10. G. Kim, S. Moon, and L. Sigal. Joint Photo Stream and Blog Post Summarization and Exploration. In CVPR, 2015.
11. G. Kim, S. Moon, and L. Sigal. Ranking and Retrieval of Image Sequences from Multiple Paragraph Queries. In CVPR, 2015.
12. R. Kiros, R. Salakhutdinov, and R. Zemel. Multimodal Neural Language Models. In ICML, 2014.
13. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet Classification with Deep Convolutional Neural Networks. In NIPS, 2012.
14. G. Kulkarni, V. Premraj, S. Dhar, S. Li, Y. Choi, A. C. Berg, and T. L. Berg. Baby Talk: Understanding and Generating Image Descriptions. In CVPR, 2011.
15. P. Kuznetsova, V. Ordonez, T. L. Berg, and Y. Choi. TreeTalk: Composition and Compression of Trees for Image Descriptions. In TACL, 2014.
16. S. B. A. Lavie. METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. In ACL, 2005.
17. Q. Le and T. Mikolov. Distributed Representations of Sentences and Documents. In ICML, 2014.
18. C. D. Manning, M. Surdeanu, J. Bauer, J. Finkel, S. J. Bethard, and D. McClosky. The Stanford CoreNLP Natural Language Processing Toolkit. In ACL, 2014.
19. J. Mao, W. Xu, Y. Yang, J. Wang, Z. Huang, and A. L. Yuille. Deep Captioning with Multimodal Recurrent Neural Networks (m-RNN). In ICLR, 2015.
20. T. Mikolov. Statistical Language Models based on Neural Networks. In Ph. D. Thesis, Brno University of Technology, 2012.
21. V. Ordonez, G. Kulkarni, and T. L. Berg. Im2Text: Describing Images Using 1 Million Captioned Photographs. In NIPS, 2011.
22. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. BLEU: A Method for Automatic Evaluation of Machine Translation. In ACL, 2002.
23. M. Rohrbach, W. Qiu, I. Titov, S. Thater, M. Pinkal, and B. Schiele. Translating Video Content to Natural Language Descriptions. In ICCV, 2013.
24. M. Schuster and K. K. Paliwal. Bidirectional Recurrent Neural Networks. In IEEE TSP, 1997.
25. K. Simonyan and A. Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. In ICLR, 2015.
26. R. Socher, A. Karpathy, Q. V. Le, C. D. Manning, and A. Y. Ng. Grounded Compositional Semantics for Finding and Describing Images with Sentences. In TACL, 2013.
27. N. Srivastava and R. Salakhutdinov. Multimodal Learning with Deep Boltzmann Machines. In NIPS, 2012.
28. T. Tieleman and G. E. Hinton. Lecture 6.5-RMSProp. In Coursera, 2012.
29. R. Vedantam, C. L. Zitnick, and D. Parikh. CIDEr: Consensus-based Image Description Evaluation. In arXiv:1411.5726, 2014.
30. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and Tell: A Neural Image Caption Generator. In CVPR, 2015.
31. P. J. Werbos. Generalization of Backpropagation with Application to a Recurrent Gas Market Model. Neural Networks, 1:339-356, 1988.
32. R. Xu, C. Xiong, W. Chen, and J. J. Corso. Jointly Modeling Deep Video and Compositional Text to Bridge Vision and Language in a Unified Framework. In AAAI, 2015.