A hybrid convolutional variational autoencoder for text generation

EMNLP 2017 - Conference on Empirical Methods in Natural Language Processing, Proceedings

Volumn , Issue , 2017, Pages 627-637

  Semeniuta, Stanislau ^a   Severyn, Aliaksei ^b   Barth, Erhardt ^a  

^a UNIVERSITY OF LÜBECK   (Germany)

^b GOOGLE   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

CONVOLUTION; LEARNING SYSTEMS;

AUTO ENCODERS; DETERMINISTIC MODELING; FEED FORWARD; HYBRID ARCHITECTURES; LANGUAGE MODEL; LONG SEQUENCES; RUNTIMES; TEXT GENERATIONS;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 85073165882     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.18653/v1/d17-1066     Document Type: Conference Paper

References (43)

1. Lei Jimmy Ba, Ryan Kiros, and Geoffrey E. Hinton. 2016. Layer normalization. CoRR, abs/1607.06450.
2. Philip Bachman. 2016. An architecture for deep, hierarchical generative models. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Gar-nett, editors, NIPS, pages 4826–4834.
3. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. CoRR, abs/1409.0473.
4. Olivier Bousquet, Sylvain Gelly, Ilya Tolstikhin, Carl-Johann Simon-Gabriel, and Bernhard Schoelkopf. 2017. From optimal transport to generative modeling: the vegan cookbook. CoRR, abs/1705.07642.
5. Samuel R. Bowman, Luke Vilnis, Oriol Vinyals, Andrew M. Dai, Rafal Józefowicz, and Samy Bengio. 2016. Generating sentences from a continuous space. In CONLL, pages 10–21.
6. Xi Chen, Diederik P. Kingma, Tim Salimans, Yan Duan, Prafulla Dhariwal, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016. Variational lossy autoencoder. CoRR, abs/1611.02731.
7. Yann N. Dauphin, Angela Fan, Michael Auli, and David Grangier. 2016. Language modeling with gated convolutional networks. CoRR, abs/1612.08083.
8. Marco Fraccaro, Søren Kaae Sø nderby, Ulrich Paquet, and Ole Winther. 2016. Sequential neural models with stochastic layers. In NIPS, pages 2199–2207.
9. Ishaan Gulrajani, Kundan Kumar, Faruk Ahmed, Adrien Ali Taiga, Francesco Visin, David Vázquez, and Aaron C. Courville. 2016. Pixelvae: A latent variable model for natural images. CoRR, abs/1611.05013.
10. David Ha, Andrew M. Dai, and Quoc V. Le. 2016. Hypernetworks. CoRR, abs/1609.09106.
11. Geoffrey E. Hinton and Drew van Camp. 1993. Keeping the neural networks simple by minimizing the description length of the weights. In Proceedings of the Sixth Annual ACM Conference on Computational Learning Theory, COLT 1993, Santa Cruz, CA, USA, July 26-28, 1993., pages 5–13.
12. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, pages 1735–1780.
13. Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. Controllable text generation. CoRR, abs/1703.00955.
14. Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML, pages 448–456.
15. Rafal Józefowicz, Oriol Vinyals, Mike Schuster, Noam Shazeer, and Yonghui Wu. 2016. Exploring the limits of language modeling. CoRR, abs/1602.02410.
16. Nal Kalchbrenner, Lasse Espeholt, Karen Simonyan, Aäron van den Oord, Alex Graves, and Koray Kavukcuoglu. 2016. Neural machine translation in linear time. CoRR, abs/1610.10099.
17. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. CoRR, abs/1412.6980.
18. Diederik P. Kingma, Tim Salimans, and Max Welling. 2016. Improving variational inference with inverse autoregressive flow. CoRR, abs/1606.04934.
19. Diederik P. Kingma and Max Welling. 2013. Autoencoding variational bayes. CoRR, abs/1312.6114.
20. Hugo Larochelle and Iain Murray. 2011. The neural autoregressive distribution estimator. In AISTATS, pages 29–37.
21. Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, and Ole Winther. 2015. Autoencoding beyond pixels using a learned similarity metric. CoRR, abs/1512.09300.
22. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of english: The penn treebank. Computational Linguistics, 19(2):313–330.
23. Lars M. Mescheder, Sebastian Nowozin, and Andreas Geiger. 2017. Adversarial variational bayes: Unifying variational autoencoders and generative adversarial networks. CoRR, abs/1701.04722.
24. Yishu Miao, Lei Yu, and Phil Blunsom. 2015. Neural variational inference for text processing. CoRR, abs/1511.06038.
25. Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. 2015. Learning deconvolution network for semantic segmentation. CoRR, abs/1505.04366.
26. Aäron van den Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. 2016. Pixel recurrent neural networks. CoRR, abs/1601.06759.
27. Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the difficulty of training recurrent neural networks. In ICML, pages 1310–1318.
28. Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. CoRR, abs/1511.06434.
29. Tapani Raiko, Mathias Berglund, Guillaume Alain, and Laurent Dinh. 2014. Techniques for learning binary stochastic feedforward neural networks. CoRR, abs/1406.2989.
30. Danilo Jimenez Rezende and Shakir Mohamed. 2015. Variational inference with normalizing flows. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, pages 1530–1538.
31. Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. In ICML, pages 1278–1286.
32. Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. CoRR, abs/1509.00685.
33. Iulian Vlad Serban, Alessandro Sordoni, Ryan Lowe, Laurent Charlin, Joelle Pineau, Aaron C. Courville, and Yoshua Bengio. 2016. A hierarchical latent variable encoder-decoder model for generating dialogues. CoRR, abs/1605.06069.
34. Casper Kaae Sønderby, Tapani Raiko, Lars Maaløe, Søren Kaae Sønderby, and Ole Winther. 2016. Ladder variational autoencoders. CoRR, abs/1602.02282.
35. Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2014. Show and tell: A neural image caption generator. CoRR, abs/1411.4555.
36. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian, Nishant Patil, Wei Wang, Cliff Young, Jason Smith, Jason Riesa, Alex Rudnick, Oriol Vinyals, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. CoRR, abs/1609.08144.
37. Xinchen Yan, Jimei Yang, Kihyuk Sohn, and Honglak Lee. 2015. Attribute2image: Conditional image generation from visual attributes. CoRR, abs/1512.00570.
38. Zichao Yang, Zhiting Hu, Ruslan Salakhutdinov, and Taylor Berg-Kirkpatrick. 2017. Improved variational autoencoders for text modeling using dilated convolutions. CoRR, abs/1702.08139.
39. Serena Yeung, Anitha Kannan, Yann Dauphin, and Li Fei-Fei. 2016. Epitomic variational autoencoder. In submission to ICLR 2017.
40. Matthew D. Zeiler, Dilip Krishnan, Graham W. Taylor, and Robert Fergus. 2010. Deconvolutional networks. In CVPR, pages 2528–2535.
41. Biao Zhang, Deyi Xiong, and Jinsong Su. 2016. Variational neural machine translation. CoRR, abs/1605.07869.
42. Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. CoRR, abs/1703.10960.
43. Barret Zoph and Quoc V. Le. 2016. Neural architecture search with reinforcement learning. CoRR, abs/1611.01578.