Structured attention networks

5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings

Volumn , Issue , 2017, Pages

  Kim, Yoon ^a   Denton, Carl ^a   Hoang, Luong ^a   Rush, Alexander M ^a  

^a HARVARD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DEEP NEURAL NETWORKS; GRAPHIC METHODS; NATURAL LANGUAGE PROCESSING SYSTEMS; NETWORK LAYERS;

ATTENTION MODEL; CONDITIONAL RANDOM FIELD; DIFFERENT CLASS; EFFECTIVE APPROACHES; GRAPHICAL MODEL; MACHINE TRANSLATIONS; NATURAL LANGUAGES; QUESTION ANSWERING;

MULTILAYER NEURAL NETWORKS;

EID: 85088226886     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (57)

1. Daniel Andor, Chris Alberti, David Weiss, Aliaksei Severyn, Alessandro Presta, Kuzman Ganchev, Slav Petrov, and Michael Collins. Globally Normalized Transition-Based Neural Networks. In Proceedings of ACL, 2016.
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural Machine Translation by Jointly Learning to Align and Translate. In Proceedings of ICLR, 2015.
3. James K. Baker. Trainable Grammars for Speech Recognition. Speech Communication Papers for the 97th Meeting of the Acoustical Society, 1979.
4. David Belanger and Andrew McCallum. Structured Prediction Energy Networks. In Proceedings of ICML, 2016.
5. Samuel R. Bowman, Christopher D. Manning, and Christopher Potts. Tree-Structured Composition in Neural Networks without Tree-Structured Architectures. In Proceedings of the NIPS workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches, 2015.
6. Samuel R. Bowman, Jon Gauthier, Abhinav Rastogi, Raghav Gupta, Christopher D. Manning, and Christopher Potts. A Fast Unified Model for Parsing and Sentence Understanding. In Proceedings of ACL, 2016.
7. William Chan, Navdeep Jaitly, Quoc Le, and Oriol Vinyals. Listen, Attend and Spell. arXiv:1508.01211, 2015.
8. Liang-Chieh Chen, Alexander G. Schwing, Alan L. Yuille, and Raquel Urtasun. Learning Deep Structured Models. In Proceedings of ICML, 2015.
9. Qian Chen, Xiaodan Zhu, Zhenhua Ling, Si Wei, and Hui Jiang. Enhancing and Combining Sequential and Tree LSTM for Natural Language Inference. arXiv:1609.06038, 2016.
10. Kyunghyun Cho, Aaron Courville, and Yoshua Bengio. Describing Multimedia Content using Attention-based Encoder-Decoder Networks. In IEEE Transactions on Multimedia, 2015.
11. Jan Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio. Attention-Based Models for Speech Recognition. In Proceedings of NIPS, 2015.
12. Ronan Collobert, Jason Weston, Leon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. Natural Language Processing (almost) from Scratch. Journal of Machine Learning Research, 12:2493-2537, 2011.
13. Trinh-Minh-Tri Do and Thierry Artiéres. Neural Conditional Random Fields. In Proceedings of AISTATS, 2010.
14. Justin Domke. Parameter Learning with Truncated Message-Passing. In Proceedings of CVPR, 2011.
15. Justin Domke. Generic methods for optimization-based modeling. In AISTATS, pp. 318-326, 2012.
16. John Duchi, Elad Hazan, and Yoram Singer. Adaptive Subgradient Methods for Online Learning and Stochastic Optimization. Journal of Machine Learning Research, 12:2021-2159, 2011.
17. Greg Durrett and Dan Klein. Neural CRF Parsing. In Proceedings of ACL, 2015.
18. Jason M. Eisner. Three New Probabilistic Models for Dependency Parsing: An Exploration. In Proceedings of ACL, 1996.
19. Jason M. Eisner. Inside-Outside and Forward-Backward Algorithms are just Backprop. In Proceedings of Structured Prediction Workshop at EMNLP, 2016.
20. Matthew R. Gormley, Mark Dredze, and Jason Eisner. Approximation-Aware Dependency Parsing by Belief Propagation. In Proceedings of TACL, 2015.
21. Alex Graves, Greg Wayne, and Ivo Danihelka. Neural Turing Machines. arXiv:1410.5401, 2014.
22. Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwinska, Sergio Gomez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, Adria Puigdomenech Badia, Karl Moritz Hermann, Yori Zwols, Georg Ostrovski, Adam Cain, Helen King, Christopher Summerfield, Phil Blunsom, Koray Kavukcuoglu, and Demis Hassabis. Hybrid Computing Using a Neural Network with Dynamic External Memory. Nature, October 2016.
23. Edward Grefenstette, Karl Moritz Hermann, Mustafa Suleyman, and Phil Blunsom. Learning to Transduce with Unbounded Memory. In Proceedings of NIPS, 2015.
24. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. Teaching Machines to Read and Comprehend. In Proceedings of NIPS, 2015.
25. Max Jaderberg, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep Structured Output Learning for Unconstrained Text Recognition. In Proceedings of ICLR, 2014.
26. Diederik Kingma and Jimmy Ba. Adam: A Method for Stochastic Optimization. In Proceedings of ICLR, 2015.
27. Eliyahu Kipperwasser and Yoav Goldberg. Simple and Accurate Dependency Parsing using Bidirectional LSTM Feature Representations. In TACL, 2016.
28. Lingpeng Kong, Chris Dyer, and Noah A. Smith. Segmental Recurrent Neural Networks. In Proceedings of ICLR, 2016.
29. John Lafferty, Andrew McCallum, and Fernando Pereira. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In Proceedings of ICML, 2001.
30. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. Neural Architectures for Named Entity Recognition. In Proceedings of NAACL, 2016.
31. Yann LeCun, Leon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based Learning Applied to Document Recognition. In Proceedings of IEEE, 1998.
32. Zhifei Li and Jason Eisner. First- and Second-Order Expectation Semirings with Applications to Minimum-Risk Training on Translation Forests. In Proceedings of EMNLP 2009, 2009.
33. Liang Lu, Lingpeng Kong, Chris Dyer, Noah A. Smith, and Steve Renals. Segmental Recurrent Neural Networks for End-to-End Speech Recognition. In Proceedings of INTERSPEECH, 2016.
34. Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. Effective Approaches to Attention-based Neural Machine Translation. In Proceedings of EMNLP, 2015.
35. Dougal Maclaurin, David Duvenaud, and Ryan P. Adams. Gradient-based Hyperparameter Optimization through Reversible Learning. In Proceedings of ICML, 2015.
36. Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, and Zhi Jin. Natural language inference by tree-based convolution and heuristic matching. In Proceedings of ACL, 2016.
37. Tsendsuren Munkhdalai and Hong Yu. Neural Tree Indexers for Text Understanding. arxiv:1607.04492, 2016.
38. Toshiaki Nakazawa, Manabu Yaguchi, Kiyotaka Uchimoto, Masao Utiyama, Eiichiro Sumita, Sadao Kurohashi, and Hitoshi Isahara. Aspec: Asian scientific paper excerpt corpus. In Nicoletta Calzo-lari (Conference Chair), Khalid Choukri, Thierry Declerck, Marko Grobelnik, Bente Maegaard, Joseph Mariani, Asuncion Moreno, Jan Odijk, and Stelios Piperidis (eds.), Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC 2016), pp. 2204-2208, Portoro, Slovenia, may 2016. European Language Resources Association (ELRA). ISBN 978-2-9517408-9-1.
39. Graham Neubig, Yosuke Nakata, and Shinsuke Mori. Pointwise Prediction for Robust, Adaptable Japanese Morphological Analysis. In Proceedings of ACL, 2011.
40. Ankur P. Parikh, Oscar Tackstrom, Dipanjan Das, and Jakob Uszkoreit. A Decomposable Attention Model for Natural Language Inference. In Proceedings of EMNLP, 2016.
41. Jian Peng, Liefeng Bo, and Jinbo Xu. Conditional Neural Fields. In Proceedings of NIPS, 2009.
42. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. GloVe: Global Vectors for Word Representation. In Proceedings of EMNLP, 2014.
43. Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomas Kocisky, and Phil Blunsom. Reasoning about Entailment with Neural Attention. In Proceedings of ICLR, 2016.
44. John Schulman, Nicolas Heess, Theophane Weber, and Pieter Abbeel. Gradient estimation using stochastic computation graphs. In Advances in Neural Information Processing Systems, pp. 3528-3536, 2015.
45. David A. Smith and Jason Eisner. Dependency Parsing as Belief Propagation. In Proceedings of EMNLP, 2008.
46. Veselin Stoyanov and Jason Eisner. Minimum-Risk Training of Approximate CRF-based NLP Systems. In Proceedings of NAACL, 2012.
47. Veselin Stoyanov, Alexander Ropson, and Jason Eisner. Empirical Risk Minimization of Graphical Model Parameters Given Approximate Inference, Decoding, and Model Structure. In Proceedings of AISTATS, 2011.
48. Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, and Rob Fergus. End-To-End Memory Networks. In Proceedings of NIPS, 2015.
49. Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer Networks. In Proceedings of NIPS, 2015.
50. Shenlong Wang, Sanja Fidler, and Raquel Urtasun. Proximal Deep Structured Models. In Proceedings of NIPS, 2016.
51. Shuohang Wang and Jing Jiang. Learning Natural Language Inference with LSTM. In Proceedings of NAACL, 2016.
52. Jason Weston, Sumit Chopra, and Antoine Bordes. Memory Networks. arXiv:1410.3916, 2014.
53. Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart van Merriënboer, Armand Joulin, and Tomas Mikolov. Towards Ai-complete Question Answering: A Set of Prerequisite Toy Tasks. arXiv preprint arXiv:1502.05698, 2015.
54. Kelvin Xu, Jimma Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, and Yoshua Bengio. Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. In Proceedings of ICML, 2015.
55. Lei Yu, Jan Buys, and Phil Blunsom. Online Segment to Segment Neural Transduction. In Proceedings of EMNLP, 2016.
56. Lei Yu, Phil Blunsom, Chris Dyer, Edward Grefenstette, and Tomas Kocisky. The Neural Noisy Channel. In Proceedings of ICLR, 2017.
57. Kai Zhao, Liang Huang, and Minbo Ma. Textual Entailment with Structured Attentions and Composition. In Proceedings of COLING, 2016.