Diving deep into sentiment: Understanding fine-tuned CNNs for Visual sentiment prediction

ASM 2015 - Proceedings of the 1st International Workshop on Affect and Sentiment in Multimedia, co-located with ACM MM 2015

Volumn , Issue , 2015, Pages 57-62

  Campos, Víctor ^a   Salvador, Amaia ^a   Giró I Nieto, Xavier ^a   Jou, Brendan ^b  

^a UNIVERSITAT POLITÈCNICA DE CATALUNYA   (Spain)

^b Columbia University ^*  (United States)

Author keywords

Convolutional Neural Networks; Fine tuning strategies; Sentiment; Social multimedia

Indexed keywords

CONVOLUTION; DATA MINING; NETWORK ARCHITECTURE; NEURAL NETWORKS;

CONVOLUTIONAL NEURAL NETWORK; DESIGN PATTERNS; FINE TUNING; SENTIMENT; SENTIMENT ANALYSIS; SOCIAL MULTIMEDIA; STATE OF THE ART; VISION PROBLEMS;

FORECASTING;

EID: 84963935991     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2813524.2813530     Document Type: Conference Paper

References (29)

1. D. Borth, R. Ji, T. Chen, T. Breuel, and S.-F. Chang. Large-scale visual sentiment ontology and detectors using adjective noun pairs, In ACM MM, 2013.
2. K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman. Return of the devil in the details: Delving deep into convolutional nets, In British Machine Vision Conference, 2014.
3. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A large-scale hierarchical image database, In Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on, pages 248-255. IEEE, 2009.
4. K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification, arXiv:abs/1502.01852 [cs. CV], 2015.
5. J. Jia, S. Wu, X. Wang, P. Hu, L. Cai, and J. Tang. Can we understand van Gogh's mood?: Learning to infer affects from images in social networks, In ACM MM, 2012.
6. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell. Caffe: Convolutional architecture for fast feature embedding, In ACM MM, 2014.
7. Y.-G. Jiang, B. Xu, and X. Xue. Predicting emotions in user-generated videos, In AAAI, 2014.
8. X. Jin, A. Gallagher, L. Cao, J. Luo, and J. Han. The wisdom of social multimedia: Using Flickr for prediction and forecast, In ACM MM, 2010.
9. B. Jou, S. Bhattacharya, and S.-F. Chang. Predicting viewer perceived emotions in animated GIFs, In ACM MM, 2014.
10. Y. Kim, H. Lee, and E. M. Provost. Deep learning for robust feature generation in audiovisual emotion recognition, In ICASSP, 2013.
11. A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks, In NIPS, 2012.
12. P. Lang, M. Bradley, and B. Cuthbert. International Affective Picture System (IAPS): Technical manual and affective ratings, Technical report, NIMH CSEA, 1997.
13. Y. Le Cun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition, In Proc. of the IEEE, 1998.
14. J. Machajdik and A. Hanbury. Affective image classification using features inspired by psychology and art theory. In ACM MM, 2010.
15. D. McDuff, R. Kaliouby, J. Cohn, and R. Picard. Predicting ad liking and purchase intent: Large-scale analysis of facial responses to ads.
16. M. Oquab, L. Bottou, I. Laptev, and J. Sivic. Learning and transferring mid-level image representations using convolutional neural networks, In Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pages 1717-1724. IEEE, 2014.
17. K.-C. Peng, T. Chen, A. Sadovnik, and A. Gallagher. A mixed bag of emotions: Model, predict, and transfer emotion distributions, In CVPR, 2015.
18. R. Plutchik. Emotion: A Psychoevolutionary Synthesis, Harper & Row, 1980.
19. A. S. Razavian, H. Azizpour, J. Sullivan, and S. Carlsson. CNN features off-the-shelf: An astounding baseline for recognition. In Computer Vision and Pattern Recognition Workshops (CVPRW), 2014 IEEE Conference on, pages 512-519. IEEE, 2014.
20. A. Salvador, M. Zeppelzauer, D. Manchon-Vizuete, A. Calafell, and X. Giro-i Nieto. Cultural event recognition with visual convnets and temporal models, In Computer Vision and Pattern Recognition Workshops (CVPRW), 2015 IEEE Conference on. IEEE, 2015.
21. S. Siersdorfer, E. Minack, F. Deng, and J. Hare. Analyzing and predicting sentiment of images on the social web, In Proceedings of the international conference on Multimedia, pages 715-718. ACM, 2010.
22. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions, arXiv preprint arXiv:1409.4842, 2014.
23. C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus. Intriguing properties of neural networks, In ICLR, 2014.
24. Y. Tang. Deep learning using linear support vector machines, In ICML Workshop on Challenges in Representation Learning, 2013.
25. C. Xu, S. Cetintas, K.-C. Lee, and L.-J. Li. Visual sentiment prediction with deep convolutional neural networks, arXiv preprint arXiv:1411.5731, 2014.
26. V. Yanulevskaya, J. van Gemert, K. Roth, A. Herbold, N. Sebe, and J. M. Geusebroek. Emotional valence categorization using holistic image features, In ICIP, 2008.
27. Q. You, J. Luo, H. Jin, and J. Yang. Robust image sentiment analysis using progressively trained and domain transferred deep networks, In The Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI), 2015.
28. M. D. Zeiler and R. Fergus. Visualizing and understanding convolutional networks, In Computer Vision-ECCV 2014, pages 818-833. Springer, 2014.
29. B. Zhou, A. Khosla, A. Lapedriza, A. Oliva, and A. Torralba. Object detectors emerge in deep scene cnns, 2015.