Multi-Task CNN Model for Attribute Prediction

IEEE Transactions on Multimedia

Volumn 17, Issue 11, 2015, Pages 1949-1959

  Abdulnabi, Abrar H ^a,b   Wang, Gang ^c   Lu, Jiwen ^d,e   Jia, Kui ^f  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^b Singapore Pte Ltd   (Singapore)

^c NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^d ADVANCED DIGITAL SCIENCES CENTER   (Singapore)

^e TSINGHUA UNIVERSITY   (China)

^f UNIVERSITY OF MACAU   (Macao)

Author keywords

Deep CNN; latent tasks matrix; multi task learning; semantic attributes

Indexed keywords

ALGORITHMS; FORECASTING; LEARNING SYSTEMS; NEURAL NETWORKS; SEMANTICS;

BINARY ATTRIBUTES; COMBINATION MATRIX; CONVOLUTIONAL NEURAL NETWORK; DEEP CNN; FEATURE REPRESENTATION; MULTITASK LEARNING; SEMANTIC ATTRIBUTE; VISUAL KNOWLEDGE;

LEARNING ALGORITHMS;

EID: 84946722304     PISSN: 15209210     EISSN: None     Source Type: Journal    
DOI: 10.1109/TMM.2015.2477680     Document Type: Article

References (62)

1. A. Argyriou, T. Evgeniou, and M. Pontil, "Multi-task feature learning," in Proc. NIPS, 2007, pp. 41-48.
2. F. Bach, "Consistency of the group lasso and multiple kernel learning," CoRR, 2007 [Online]. Available: http://arxiv.org/abs/0707.3390
3. Y. Bengio, A. Courville, and P. Vincent, "Representation learning: A review and new perspectives," IEEE Trans. Pattern Anal.Mach. Intell., vol. 35, no. 8, pp. 1798-1828, Aug. 2013.
4. R. Caruana, "Multitask learning," Mach. Learn., vol. 28, no. 1, pp. 41-75, Jul. 1997.
5. S. Chang, G.-J. Qi, J. Tang, Q. Tian, Y. Rui, and T. S. Huang, "Multimedia LEGO: Learning structured model by probabilistic logic ontology tree," in Proc. 13th IEEE Int. Conf. Data Mining, Dec. 2013, pp. 979-984.
6. K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman, "Return of the devil in the details: Delving deep into convolutional nets," in Proc. BMVC, 2014.
7. H. Chen, A. Gallagher, and B. Girod, "Describing clothing by semantic attributes," in Proc. 12th Eur. Conf. Comput. Vis.-Vol. Part III, 2012, pp. 609-623.
8. H. Chen, A. Gallagher, and B. Girod, "Describing clothing by semantic attributes," in Proc. ECCV, 2012, pp. 609-623.
9. X. Chen, Q. Lin, S. Kim, J. G. Carbonell, and E. P. Xing, "Smoothing proximal gradient method for general structured sparse learning," in Proc. 27th Conf. Uncertainty Artificial Intell., 2012, pp. 1-10.
10. M.-M. Cheng, Z. Zhang, W.-Y. Lin, and P. H. S. Torr, "BING: Binarized normed gradients for objectness estimation at 300fps," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2014, pp. 3286-3293.
11. J. Deng, N. Ding, Y. Jia, A. Frome, K. Murphy, S. Bengio, Y. Li, H. Neven, and H. Adam, "Large-scale object classification using label relation graphs," in Computer Vision-ECCV 2014 , ser. Lecture Notes in Comput. Sci., D. Fleet, T. Pajdla, B. Schiele, and T. Tuytelaars, Eds. New York, NY, USA: Springer, 2014, vol. 8689, pp. 48-64.
12. J. Donahue, Y. Jia, O. Vinyals, J. Hoffman, N. Zhang, E. Tzeng, and T. Darrell, "Decaf: A deep convolutional activation feature for generic visual recognition," in Proc. ICML, 2013, pp. 647-655.
13. A. Farhadi, I. Endres, D. Hoiem, and D. Forsyth, "Describing objects by their attributes," in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog., Jun. 2009, pp. 1778-1785.
14. V. Ferrari and A. Zisserman, "Learning visual attributes," in Proc. Adv. Neural Inf. Process. Syst., Dec. 2007, pp. 433-440.
15. A. Gallagher and T. Chen, "Clothing cosegmentation for recognizing people," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2008, pp. 1-8.
16. P. Gong, J. Ye, and C. Zhang, "Robust multi-task feature learning," in Proc. 18th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2012, pp. 895-903.
17. Y. Gong, Y. Jia, T. Leung, A. Toshev, and S. Ioffe, "Deep convolutional ranking for multilabel image annotation," CoRR, 2013 [Online]. Available: http://arxiv.org/abs/1312.4894
18. B. Hariharan, L. Zelnik-manor, S. V. N. Vishwanathan, and M. Varma, "Large scale max-margin multi-label classification with priors," in Proc. Int. Conf. Machine Learn., 2010, pp. 423-430.
19. S. J. Hwang, F. Sha, and K. Grauman, "Sharing features between objects and their attributes," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2011, pp. 1761-1768.
20. D. Jayaraman and K. Grauman, "Zero shot recognition with unreliable attributes," CoRR, 2014 [Online]. Available: http://arxiv.org/abs/1409. 4327
21. D. Jayaraman, F. Sha, and K. Grauman, "Decorrelating semantic visual attributes by resisting the urge to share," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2014, pp. 1629-1636.
22. S. Kim and E. Xing, "Tree-guided group lasso for multi-task regression with structured sparse," in Proc. ICML, 2010, pp. 543-550.
23. S. Kim and E. P. Xing, "Tree-guided group lasso for multi-task regression with structured sparsity," in Proc. ICML, 2010, pp. 543-550.
24. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," in Advances in Neural Information Processing Systems 25, F. Pereira, C. Burges, L. Bottou, and K. Weinberger, Eds. Red Hook, NY, USA: Curran Assoc., 2012, pp. 1097-1105.
25. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," in Proc. NIPS, 2012, pp. 1097-1105.
26. A. Kumar and H. Daume, "Learning task grouping and overlap in multi-task learning," in Proc. 29th Int. Conf. Machine Learn., 2012, pp. 1383-1390.
27. N. Kumar, A. C. Berg, P. N. Belhumeur, and S. K. Nayar, "Attribute and simile classifiers for face verification," in Proc.12th IEEE Int. Conf. Comput. Vis., Sep.-Oct. 2009, pp. 365-372.
28. C. Lampert, H. Nickisch, and S. Harmeling, "Learning to detect unseen object classes by between-class attribute transfer," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2009, pp. 951-958.
29. C. Lampert, H. Nickisch, and S. Harmeling, "Attribute-based classification for zero-shot visual object categorization," IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 3, pp. 453-465, Mar. 2014.
30. C. H. Lampert, H. Nickisch, and S. Harmeling, "Learning to detect unseen object classes by between class attribute transfer," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2009, pp. 951-958.
31. C. Li, J. Zhu, and J. Chen, "Bayesian max-margin multi-task learning with data augmentation," in Proc. 31st Int. Conf. Mach. Learn., 2014, pp. 415-423.
32. S. Ma, S. Sclaroff, and N. Ikizler-Cinbis, "Unsupervised learning of discriminative relative visual attributes," in Proc. ECCV Workshops, 2012, vol. 12, no. 3, pp. 61-70.
33. M. Ozeki and T. Okatani, "Understanding convolutional neural networks in terms of category-level attributes," in Proc. ACCV, 2014, pp. 362-375.
34. D. Parikh and K. Grauman, "Relative attributes," in Proc. IEEE Int. Conf. Comput. Vis., Nov. 2011, pp. 503-510.
35. G.-J. Qi, C. Aggarwal, Q. Tian, H. Ji, and T. Huang, "Exploring context and content links in social media: A latent space method," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 5, pp. 850-862, May 2012.
36. G.-J. Qi, X.-S. Hua, Y. Rui, J. Tang, T. Mei, and H.-J. Zhang, "Correlative multi-label video annotation," in Proc. 15th Int. Conf. Multimedia, 2007, pp. 17-26.
37. G.-J. Qi, X.-S. Hua, Y. Rui, J. Tang, and H.-J. Zhang, "Two-dimensional active learning for image classification," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2008, pp. 1-8.
38. G.-J. Qi, X.-S. Hua, and H.-J. Zhang, "Learning semantic distance from community-tagged media collection," in Proc. ACM Int. Conf. Multimedia, 2009, pp. 243-252.
39. P. Rai and H. D. Iii, "Infinite predictor subspace models for multitask learning," in Proc. 13th Int. Conf. Artificial Intell. Statist., 2010, pp. 613-620.
40. A. S. Razavian, H. Azizpour, J. Sullivan, and S. Carlsson, "CNN features off-the-shelf: An astounding baseline for recognition," in Proc. IEEE Conf. Comput. Vis. Pattern Recog. Workshops, Jun. 2014, pp. 512-519.
41. J. Read, B. Pfahringer, G. Holmes, and E. Frank, "Classifier chains for multi-label classification," in Proc. Eur. Conf. Mach. Learn. Knowl. Discovery Databases: Part II, 2009, pp. 254-269.
42. J. Read, B. Pfahringer, G. Holmes, and E. Frank, "Classifier chains for multi-label classification," in Proc. Eur. Conf. Mach. Learn. Knowledge Discovery Databases: Part II, 2009, pp. 254-269.
43. O. Russakovsky and L. Fei-fei, "Attribute learning in large-scale datasets," in Trends and Topics in Computer Vision, ser. Lecture Notes in Comput. Sci.. Berlin, Germany: Springer-Verlag, pp. 1-14.
44. R. Salakhutdinov, A. Torralba, and J. Tenenbaum, "Learning to share visual appearance for multiclass object detection," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2011, pp. 1481-1488.
45. R. Sandeep, Y. Verma, and C. Jawahar, "Relative parts: Distinctive parts for learning relative attributes," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2014, pp. 3614-3621.
46. W. Scheirer, N. Kumar, P. Belhumeur, and T. Boult, "Multi-attribute spaces: Calibration for attribute fusion and similarity search," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2012, pp. 2933-2940.
47. E. P. X. Seyoung Kim, "Tree-guided group lasso for multi-response regression with structured sparsity, with an application to eQTL mapping," Ann. Appl. Stat., vol. 6, no. 3, pp. 1095-1117, 2012.
48. Y. W. Si Chen, "Convolutional neural network and convex optimization," Dept. of Elect. and Comput. Eng., Univ. of California at San Diego, San Diego, CA, USA, Tech. Rep., 2014 [Online]. Available: http://acsweb.ucsd.edu/~yuw176/report/ECE273.pdf
49. B. Siddiquie, R. Feris, and L. Davis, "Image ranking and retrieval based on multi-attribute queries," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2011, pp. 801-808.
50. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," CoRR, 2014 [Online]. Available: http://arxiv.org/abs/1409.1556
51. Y. Tang, "Deep learning using linear support vector machines," CoRR, 2015 [Online]. Available: http://arxiv.org/abs/1306.0239
52. R. Tibshirani, "Regression shrinkage and selection via the lasso," J. Royal Statist. Soc., vol. 58, pp. 267-288, 1996.
53. P. Tseng, "On accelerated proximal gradient methods for convex-concave optimization," SIAM J. Optimization, submitted for publication.
54. G. Tsoumakas and I. Katakis, "Multi-label classification: An overview," Int. J. DataWarehousing Mining, vol. 2007, pp. 1-13, 2007.
55. A. Vedaldi, S. Mahendran, S. Tsogkas, S. Maji, R. Girshick, J. Kannala, E. Rahtu, I. Kokkinos, M. Blaschko, D. Weiss, B. Taskar, K. Simonyan, N. Saphra, and S. Mohamed, "Understanding objects in detail with finegrained attributes," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2014, pp. 3622-3629.
56. G. Wang and D. Forsyth, "Joint learning of visual attributes, object classes and visual saliency," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Sep.-Oct. 2009, pp. 537-544.
57. Y. Wei, W. Xia, J. Huang, B. Ni, J. Dong, Y. Zhao, and S. Yan, "CNN: Single-label to multi-label," CoRR, 2014 [Online]. Available: http:// arxiv.org/abs/1406.5726
58. M. Yang, Y. Li, and Z. University, "Multi-task learning with Gaussian matrix generalized inverse Gaussian model," in Proc. 30th Int. Conf. Mach. Learn., May 2013, vol. 28, pp. 423-431.
59. M. D. Zeiler and R. Fergus, "Visualizing and understanding convolutional networks," in Proc. ECCV, 2014, pp. 818-833.
60. N. Zhang, M. Paluri, M. Ranzato, T. Darrell, and L. D. Bourdev, "PANDA: Pose aligned networks for deep attribute modeling," CoRR, 2013 [Online]. Available: http://arxiv.org/abs/1311.5591
61. Y. Zhang and J. Schneider, "Learning multiple tasks with a sparse matrix- normal penalty," in Proc. NIPS, 2010, pp. 2550-2558.
62. Q. Zhou, G. Wang, K. Jia, and Q. Zhao, "Learning to share latent tasks for action recognition," in Proc. ICCV, 2013, pp. 2264-2271.