Structured Set Matching Networks for One-Shot Part Labeling

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition

Volumn , Issue , 2018, Pages 3627-3636

  Choi, Jonghyun ^a   Krishnamurthy, Jayant ^a,b   Kembhavi, Aniruddha ^a   Farhadi, Ali ^a,c  

^a Allen Institute for Artificial Intelligence ^*  (United States)

^b Semantic Machines   (United States)

^c UNIVERSITY OF WASHINGTON   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER VISION; GRAPHIC METHODS; NEURAL NETWORKS; TEXTURES;

COLOR AND TEXTURES; CONVOLUTIONAL NEURAL NETWORK; GLOBAL CONSISTENCY; GLOBAL NORMALIZATION; MATCHING CONSTRAINTS; MATCHING PROBLEMS; STRUCTURED PREDICTION; TRAINING DATA SETS;

LARGE DATASET;

EID: 85062858798     PISSN: 10636919     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/CVPR.2018.00382     Document Type: Conference Paper

References (58)

1. D. Andor, C. Alberti, D. Weiss, A. Severyn, A. Presta, K. Ganchev, S. Petrov, and M. Collins. Globally normalized transition-based neural networks. In ACL, 2016. 3, 6
2. H. Azizpour and I. Laptev. Object detection using stronglysupervised deformable part models. In ECCV, 2012. 3
3. A. Bearman, O. Russakovsky, V. Ferrari, and L. Fei-Fei. What's the point: Semantic segmentation with point supervision. In ECCV, 2016. 6
4. J. Bromley, J. W. Bentz, L. Bottou, I. Guyon, Y. LeCun, C. Moore, E. Säckinger, and R. Shah. Signature Verification Using A "Siamese" Time Delay Neural Network. In NIPS, 1993. 3
5. L.-C. Chen, A. Schwing, A. Yuille, and R. Urtasun. Learning deep structured models. In D. Blei and F. Bach, editors, Proceedings of the 32nd International Conference on Machine Learning (ICML-15), pages 1785-1794. JMLR Workshop and Conference Proceedings, 2015. 3
6. X. Chen, R. Mottaghi, X. Liu, S. Fidler, R. Urtasun, and A. Yuille. Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts. In CVPR, 2014. 2, 3, 6
7. S. Chopra, R. Hadsell, and Y. LeCun. Learning a Similarity Metric Discriminatively, with Application to Face Verification. 1: 539-546, 2005. 3
8. C. B. Choy, J. Gwak, S. Savarese, and M. K. Chandraker. Universal correspondence network. In NIPS, 2016. 3, 7
9. M. Collins and B. Roark. Incremental parsing with the perceptron algorithm. In Proceedings of the 42nd Meeting of the Association for Computational Linguistics (ACL'04), Main Volume, pages 111-118, Barcelona, Spain, July 2004. 3
10. H. Daumé, III and D. Marcu. Learning as search optimization: Approximate large margin methods for structured prediction. In Proceedings of the 22Nd International Conference on Machine Learning, ICML '05, pages 169-176, New York, NY, USA, 2005. ACM. 3, 5
11. T.-M.-T. Do and T. Artieres. Neural conditional random fields. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, number EPFLCONF-150585. JMLR: W&CP, 2010. 3
12. J. Eisner. Inside-outside and forward-backward algorithms are just backprop. In Proceedings of the EMNLP Workshop on Structured Prediction for NLP, 2016. 3
13. M. Eitz, J. Hays, and M. Alexa. How Do Humans Sketch Objects? ACM Trans. Graph. (Proc. SIGGRAPH), 31 (4), 2012. 2
14. M. Eitz, R. Richter, T. Boubekeur, K. Hildebrand, and M. Alexa. Sketch-based shape retrieval. ACM Trans. Graph., 31: 31: 1-31: 10, 2012. 2
15. L. Fei-Fei, R. Fergus, and P. Perona. A Bayesian approach to unsupervised one-shot learning of object categories. In ICCV, 2003. 2
16. L. Fei-Fei, R. Fergus, and P. Perona. One-shot learning of object categories. IEEE Trans. Pattern Anal. Mach. Intell., 28: 594-611, 2006. 2
17. P. F. Felzenszwalb, R. B. Girshick, D. McAllester, and D. Ramanan. Object detection with discriminatively trained partbased models. IEEE transactions on pattern analysis and machine intelligence, 32 (9): 1627-1645, 2010. 3
18. A. Ghosh, V. Kulharia, A. Mukerjee, V. P. Namboodiri, and M. Bansal. Contextual rnn-gans for abstract reasoning diagram generation. CoRR, abs/1609. 09444, 2017. 1, 3
19. M. R. Gormley, M. Dredze, and J. Eisner. Approximationaware dependency parsing by belief propagation. Transactions of the Association for Computational Linguistics (TACL), 2015. 3
20. B. Ham, M. Cho, C. Schmid, and J. Ponce. Proposal flow. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. 3
21. K. Han, R. S. Rezende, B. Ham, K.-Y. K. Wong, M. Cho, C. Schmid, and J. Ponce. SCNet: Learning Semantic Correspondence. In ICCV, 2017. 3
22. M. Jaderberg, K. Simonyan, A. Zisserman, and K. Kavukcuoglu. Spatial transformer networks. In NIPS, 2015. 3
23. A. Karpathy and L. Fei-Fei. Deep visual-semantic alignments for generating image descriptions. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2015. 3
24. A. Kembhavi, M. Salvato, E. Kolve, M. J. Seo, H. Hajishirzi, and A. Farhadi. A diagram is worth a dozen images. In ECCV, 2016. 1, 2, 3
25. G. Koch, R. Zemel, and R. Salakhutdinov. Siamese neural networks for one-shot image recognition. In ICML, 2015. 2
26. J. Krishnamurthy, O. Tafjord, and A. Kembhavi. Semantic parsing to probabilistic programs for situated question answering. In EMNLP, 2016. 1, 3
27. J. Lafferty, A. McCallum, F. Pereira, et al. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of the eighteenth international conference on machine learning, ICML, volume 1, pages 282-289, 2001. 2, 3, 4, 5
28. C. Liu, A. G. Schwing, K. Kundu, R. Urtasun, and S. Fidler. Rent3d: Floor-plan priors for monocular layout estimation. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2015. 1
29. C. Liu, J. Yuen, and A. Torralba. Nonparametric scene parsing via label transfer. IEEE Trans. Pattern Anal. Mach. Intell., 33: 2368-2382, 2011. 2
30. K. W. Murray and J. Krishnamurthy. Probabilistic neural programs. CoRR, abs/1612. 00712, 2016. 6
31. S. Ouyang, T. M. Hospedales, Y.-Z. Song, and X. Li. Forgetmenot: Memory-aware forensic facial sketch matching. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 1
32. J. Peng, L. Bo, and J. Xu. Conditional neural fields. In Advances in neural information processing systems, pages 1419-1427, 2009. 3
33. P. Sangkloy, N. Burnell, C. Ham, and J. Hays. The sketchy database: Learning to retrieve badly drawn bunnies. ACM Transactions on Graphics (proceedings of SIGGRAPH), 2016. 1, 2
34. A. G. Schwing and R. Urtasun. Fully connected deep structured networks. arXiv preprint arXiv: 1503. 02351, 2015. 3
35. M. J. Seo, H. Hajishirzi, A. Farhadi, and O. Etzioni. Diagram understanding in geometry questions. In AAAI, 2014. 1, 3
36. M. J. Seo, H. Hajishirzi, A. Farhadi, O. Etzioni, and C. Malcolm. Solving geometry problems: Combining text and diagram interpretation. In EMNLP, 2015. 1, 3
37. A. Shrivastava and A. Gupta. Building part-based object detectors via 3D geometry. In 2013 IEEE International Conference on Computer Vision, 2013. 3
38. K. Simonyan and A. Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR, abs/1409. 1556, 2014. 4
39. H. O. Song, Y. Xiang, S. Jegelka, and S. Savarese. Deep Metric Learning via Lifted Structured Feature Embedding. In CVPR, 2016. 3
40. M. Sun and S. Savarese. Articulated part-based model for joint object detection and pose estimation. In 2011 International Conference on Computer Vision, 2011. 3
41. B. Taskar, C. Guestrin, and D. Koller. Max-margin markov networks. In NIPS, 2003. 5
42. J. J. Tompson, A. Jain, Y. LeCun, and C. Bregler. Joint training of a convolutional network and a graphical model for human pose estimation. In Advances in neural information processing systems, pages 1799-1807, 2014. 3
43. I. Tsochantaridis, T. Hofmann, T. Joachims, and Y. Altun. Support vector machine learning for interdependent and structured output spaces. In ICML, 2004. 5
44. R. Vedantam, X. Lin, T. Batra, C. L. Zitnick, and D. Parikh. Learning common sense through visual abstraction. In 2015 IEEE International Conference on Computer Vision (ICCV), 2015. 3
45. O. Vinyals, S. Bengio, and M. Kudlur. Order matters: Sequence to sequence for sets. In ICLR, 2016. 5
46. O. Vinyals, C. Blundell, T. Lillicrap, K. Kavukcuoglu, and D. Wierstra. Matching Networks for One Shot Learning. In NIPS, 2016. 2, 5, 7, 8
47. F. Wang, L. Kang, and Y. Li. Sketch-based 3D shape retrieval using convolutional neural networks. CoRR, abs/1504. 03504, 2015. 1, 2
48. S. Wang, S. Fidler, and R. Urtasun. Proximal deep structured models. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 865-873. Curran Associates, Inc., 2016. 3
49. S. Wiseman and A. M. Rush. Sequence-to-sequence learning as beam-search optimization. In EMNLP, 2016. 3, 6
50. M. Yatskar, L. Zettlemoyer, and A. Farhadi. Situation recognition: Visual semantic role labeling for image understanding. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016. 3
51. Q. Yu, F. Liu, Y.-Z. Song, T. Xiang, T. M. Hospedales, and C. C. Loy. Sketch me that shoe. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 1, 2
52. Q. Yu, Y. Yang, F. Liu, Y.-Z. Song, T. Xiang, and T. M. Hospedales. Sketch-a-net: A deep neural network that beats humans. In International Journal of Computer Vision, 2016. 1, 2, 4, 8
53. S. Zagoruyko and N. Komodakis. Learning to compare image patches via convolutional neural networks. CVPR, 2015. 3
54. J. Zbontar and Y. LeCun. Computing the stereo matching cost with a convolutional neural network. CVPR, 2015. 3
55. W. Zhang, X. Wang, and X. Tang. Coupled informationtheoretic encoding for face photo-sketch recognition. In CVPR, 2011. 1, 2
56. S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang, and P. H. S. Torr. Conditional random fields as recurrent neural networks. In Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), ICCV '15, pages 1529-1537, Washington, DC, USA, 2015. IEEE Computer Society. 3
57. L. Zhu, Y. Chen, A. L. Yuille, and W. T. Freeman. Latent hierarchical structural learning for object detection. In CVPR, 2010. 3
58. C. L. Zitnick and D. Parikh. Bringing semantics into focus using visual abstraction. In 2013 IEEE Conference on Computer Vision and Pattern Recognition, 2013. 3