Describing videos by exploiting temporal structure

Proceedings of the IEEE International Conference on Computer Vision

Volumn 2015 International Conference on Computer Vision, ICCV 2015, Issue , 2015, Pages 4507-4515

  Yao, Li ^a   Torabi, Atousa ^a   Cho, Kyunghyun ^a   Ballas, Nicolas ^a   Pal, Christopher ^b   Larochelle, Hugo ^c   Courville, Aaron ^a  

^a UNIVERSITÉ DE MONTRÉAL   (Canada)

^b ÉCOLE POLYTECHNIQUE DE MONTRÉAL   (Canada)

^c UNIVERSITÉ DE SHERBROOKE   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

BEHAVIORAL RESEARCH; COMPUTATIONAL LINGUISTICS; COMPUTER VISION; MODELING LANGUAGES; NEURAL NETWORKS; RECURRENT NEURAL NETWORKS;

ACTION RECOGNITION; ATTENTION MECHANISMS; CONVOLUTIONAL NEURAL NETWORK; IMAGE DESCRIPTIONS; RECURRENT NEURAL NETWORK (RNNS); TEMPORAL MODELING; TEMPORAL SEGMENTS; TEMPORAL STRUCTURES;

MOTION ANALYSIS;

EID: 84973884896     PISSN: 15505499     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICCV.2015.512     Document Type: Conference Paper

References (43)

1. D. Bahdanau, K. Cho, and Y. Bengio. Neural machine translation by jointly learning to align and translate. ICLR, 2015.
2. A. Barbu, A. Bridge, Z. Burchill, D. Coroian, S. Dickinson, S. Fidler, A. Michaux, S. Mussman, S. Narayanaswamy, D. Salvi, et al. Video in sentences out. UAI, 2012.
3. F. Bastien, P. Lamblin, R. Pascanu, J. Bergstra, I. J. Goodfellow, A. Bergeron, N. Bouchard, and Y. Bengio. Theano: new features and speed improvements. Deep Learning and Unsupervised Feature Learning NIPS 2012Workshop, 2012.
4. J. Bergstra, O. Breuleux, F. Bastien, P. Lamblin, R. Pascanu, G. Desjardins, J. Turian, D. Warde-Farley, and Y. Bengio. Theano: A CPU and GPU math expression compiler. In Pro-ceedings of the Python for Scientific Computing Conference (SciPy), 2010.
5. P. Bojanowski, R. Lajugie, F. Bach, I. Laptev, J. Ponce, C. Schmid, and J. Sivic. Weakly supervised action labeling in videos under ordering constraints. In ECCV. 2014.
6. D. L. Chen and W. B. Dolan. Collecting highly parallel data for paraphrase evaluation. In ACL, 2011.
7. X. Chen, H. Fang, T.-Y. Lin, R. Vedantam, S. Gupta, P. Dollar, and C. L. Zitnick. Microsoft coco captions: Data collection and evaluation server. ArXiv 1504. 00325, 2015.
8. K. Cho, B. van Merrienboer, C. Gulcehre, F. Bougares, H. Schwenk, and Y. Bengio. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In EMNLP, Oct. 2014.
9. N. Dalal, B. Triggs, and C. Schmid. Human detection using oriented histograms of flow and appearance. In ECCV. 2006.
10. M. Denkowski and A. Lavie. Meteor universal: Language specific translation evaluation for any target language. In EACL Workshop, 2014.
11. J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venugopalan, K. Saenko, and T. Darrell. Long-term recurrent convolutional networks for visual recognition and description. CVPR, 2015.
12. A. Gaidon, Z. Harchaoui, and C. Schmid. Temporal localization of actions with actoms. PAMI, 2013.
13. S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computation, 1997.
14. M. Hodosh, P. Young, and J. Hockenmaier. Framing image description as a ranking task: Data, models and evaluation metrics. Journal of Artificial Intelligence Research, 2013.
15. S. Ji, W. Xu, M. Yang, and K. Yu. 3d convolutional neural networks for human action recognition. PAMI, 2013.
16. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding. ArXiv:1408. 5093, 2014.
17. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In CVPR, 2014.
18. A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei. Large-scale video classification with convolutional neural networks. In CVPR. IEEE, 2014.
19. R. Kiros, R. Salakhutdinov, and R. S. Zemel. Unifying visual-semantic embeddings with multimodal neural language models. ACL, 2014.
20. A. Kojima, T. Tamura, and K. Fukunaga. Natural language description of human activities from video images based on concept hierarchy of actions. IJCV, 2002.
21. A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In NIPS, 2012.
22. N. Morsillo, G. Mann, and C. Pal. Youtube scale, large vocabulary video annotation. In Video Search and Mining. 2010.
23. K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu. Bleu: A method for automatic evaluation of machine translation. In ACL, 2002.
24. M. Ranzato, A. Szlam, J. Bruna, M. Mathieu, R. Collobert, and S. Chopra. Video (language) modeling: A baseline for generative models of natural videos. ArXiv: 1412. 6604, 2014.
25. A. Rohrbach, M. Rohrbach, N. Tandon, and B. Schiele. A dataset for movie description. CVPR, 2015.
26. M. Rohrbach, W. Qiu, I. Titov, S. Thater, M. Pinkal, and B. Schiele. Translating video content to natural language descriptions. In ICCV, 2013.
27. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun. Overfeat: Integrated recognition, localization and detection using convolutional networks. ICLR, 2014.
28. K. Simonyan and A. Zisserman. Two-stream convolutional networks for action recognition in videos. NIPS, 2014.
29. N. Srivastava, E. Mansimov, and R. Salakhutdinov. Unsupervised learning of video representations using lstms. ArXiv: 1502. 04681, 2015.
30. I. Sutskever, O. Vinyals, and Q. V. V. Le. Sequence to sequence learning with neural networks. In NIPS. 2014.
31. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. CVPR, 2015.
32. K. Tang, L. Fei-Fei, and D. Koller. Learning latent temporal structure for complex event detection. In CVPR. IEEE, 2012.
33. G. W. Taylor, R. Fergus, Y. LeCun, and C. Bregler. Convolutional learning of spatio-temporal features. In Computer Vision-ECCV 2010, pages 140-153. Springer, 2010.
34. J. Thomason, S. Venugopalan, S. Guadarrama, K. Saenko, and R. Mooney. Integrating language and vision to generate natural language descriptions of videos in the wild. In COLING, 2014.
35. A. Torabi, C. Pal, H. Larochelle, and A. Courville. Using descriptive video services to create a large data source for video annotation research. ArXiv: 1503. 01070, 2015.
36. D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri. C3D: Generic features for video analysis. ArXiv:1412. 0767, 2014.
37. R. Vedantam, C. L. Zitnick, and D. Parikh. CIDEr: Consensus-based image description evaluation. CVPR, 2015.
38. S. Venugopalan, H. Xu, J. Donahue, M. Rohrbach, R. Mooney, and K. Saenko. Translating videos to natural language using deep recurrent neural networks. NAACL, 2015.
39. O. Vinyals, A. Toshev, S. Bengio, and D. Erhan. Show and tell: A neural image caption generator. CVPR, 2015.
40. H. Wang, M. M. Ullah, A. Klser, I. Laptev, and C. Schmid. Evaluation of local spatio-temporal features for action recognition. In University of Central Florida, U. S. A, 2009.
41. K. Xu, J. Ba, R. Kiros,, K. Cho, A. Courville, R. Salakhutdinov, R. Zemel, and Y. Bengio. Show, attend and tell: Neural image caption generation with visual attention. ICML, 2015.
42. W. Zaremba, I. Sutskever, and O. Vinyals. Recurrent neural network regularization. ArXiv: 1409. 2329, 2014.
43. M. D. Zeiler. ADADELTA: An adaptive learning rate method. Technical report, 2012.