Recent trends in deep learning based natural language processing [Review Article]

IEEE Computational Intelligence Magazine

Volumn 13, Issue 3, 2018, Pages 55-75

  Young, Tom ^a   Hazarika, Devamanyu ^b   Poria, Soujanya ^c   Cambria, Erik ^d  

^a BEIJING INSTITUTE OF TECHNOLOGY   (China)

^b NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^c NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^d NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

NATURAL LANGUAGE PROCESSING SYSTEMS;

HIERARCHICAL REPRESENTATION; LEARNING METHODS; MODEL DESIGN; MULTIPLE PROCESSING; RECENT TRENDS; STATE OF THE ART;

DEEP LEARNING;

EID: 85051085274     PISSN: 1556603X     EISSN: 15566048     Source Type: Journal    
DOI: 10.1109/MCI.2018.2840738     Document Type: Review

References (164)

1. E. Cambria and B. White, "Jumping NLP curves: A review of natural language processing research, " IEEE Comput. Intell. Mag., vol. 9, no. 2, pp. 48-57, May 2014.
2. T. Mikolov, M. Karafiát, L. Burget, J. Cernocký, S. Khudanpur, "Recurrent neural network based language model, " in Proc. Interspeech, vol. 2, p. 3, 2010.
3. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, J. Dean, "Distributed representations of words and phrases and their compositionality, " in Proc. Advances Neural Information Processing Systems, 2013, pp. 3111-3119.
4. R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, C. Potts, "Recursive deep models for semantic compositionality over a sentiment treebank, " in Proc. Conf. Empirical Methods Natural Language Processing, 2013, pp. 1631-1642.
5. R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, P. Kuksa, "Natural language processing (almost) from scratch, " J. Mach. Learn. Res., vol. 12, pp. 2493-2537, Aug. 2011.
6. Y. Goldberg, "A primer on neural network models for natural language processing, " J. Artif. Intell. Res., vol. 57, pp. 345-420, Nov. 2016.
7. Y. Bengio, R. Ducharme, P. Vincent, C. Jauvin, "A neural probabilistic language model, " J. Mach. Learn. Res., vol. 3, pp. 1137-1155, Feb. 2003.
8. T. Mikolov, K. Chen, G. Corrado, J. Dean, "Efficient estimation of word representations in vector space, " arXiv Preprint, arXiv:1301.3781, 2013.
9. J. Weston, S. Bengio, N. Usunier, "Wsabie: Scaling up to large vocabulary image annotation, " Proc. Int. Joint Conf. Artificial Intelligence., 2011, vol. 11, pp. 2764-2770.
10. R. Socher, C. C. Lin, C. Manning, A. Y. Ng, "Parsing natural scenes and natural language with recursive neural networks, " in Proc. 28th Int. Joint Conf. Machine Learning, 2011, pp. 129-136.
11. P. D. Turney and P. Pantel, "From frequency to meaning: Vector space models of semantics, " J. Artif. Intell. Res., vol. 37, pp. 141-188, Nov. 2010.
12. E. Cambria, S. Poria, A. Gelbukh, M. Thelwall, "Sentiment analysis is a big suitcase, " IEEE Intell. Syst., vol. 32, no. 6, pp. 74-80, Nov. 2017.
13. X. Glorot, A. Bordes, Y. Bengio, "Domain adaptation for large-scale sentiment classification: A deep learning approach, " in Proc. 28th Int. Conf. Machine Learning, 2011, pp. 513-520.
14. K. M. Hermann and P. Blunsom, "The role of syntax in vector space models of compositional semantics, " in Proc. 51st Annu. Meeting Association Computational Linguistics, 2013, vol. 1, pp. 894-904.
15. J. L. Elman, "Distributed representations, simple recurrent networks, grammatical structure, " Mach. Learn., vol. 7, no. 2-3, pp. 195-225, 1991.
16. A. M. Glenberg and D. A. Robertson, "Symbol grounding and meaning: A comparison of high-dimensional and embodied theories of meaning, " J. Memory Lang., vol. 43, no. 3, pp. 379-401, Oct. 2000.
17. S. T. Dumais, "Latent semantic analysis, " Annu. Rev. Inf. Sci. Tech., vol. 38, no. 1, pp. 188-230, Nov. 2004.
18. D. M. Blei, A. Y. Ng, M. I. Jordan, "Latent dirichlet allocation, " J. Mach. Learn. Res., vol. 3, pp. 993-1022, 2003.
19. R. Collobert and J. Weston, "A unified architecture for natural language processing: Deep neural networks with multitask learning, " in Proc. 25th Int. Conf. Machine Learning, 2008, pp. 160-167.
20. A. Gittens, D. Achlioptas, M. W. Mahoney, "Skip-gram-zipf + uniform = vector additivity, " in Proc. 55th Annu. Meeting Association Computational Linguistics, 2017, vol. 1, pp. 69-76.
21. J. Pennington, R. Socher, C. D. Manning, "Glove: Global vectors for word representation, " in Proc. Conf. Empirical Methods Natural Language Processing, 2014, vol. 14, pp. 1532-1543.
22. R. Johnson and T. Zhang, "Semi-supervised convolutional neural networks for text categorization via region embedding, " in Proc. Advances Neural Information Processing Systems, 2015, pp. 919-927.
23. R. Socher, J. Pennington, E. H. Huang, A. Y. Ng, C. D. Manning, "Semi-supervised recursive autoencoders for predicting sentiment distributions, " in Proc. Conf. Empirical Methods Natural Language Processing, 2011, pp. 151-161.
24. X. Wang, Y. Liu, C. Sun, B. Wang, X. Wang, "Predicting polarities of tweets by composing word embeddings with long short-term memory, " in Proc. Annu. Meeting Association Computational Linguistics, 2015, pp. 1343-1353.
25. D. Tang, F. Wei, N. Yang, M. Zhou, T. Liu, B. Qin, "Learning sentiment-specific word embedding for twitter sentiment classification, " in Proc. Annu. Meeting Association Computational Linguistics, 2014, pp. 1555-1565.
26. I. Labutov and H. Lipson, "Re-embedding words, " in Proc. Annu. Meeting Association Computational Linguistics, 2013, pp. 489-493.
27. S. Upadhyay, K. Chang, M. Taddy, A. Kalai, J. Zou, "Beyond bilingual: Multi-sense word embeddings using multilingual context, " arXiv Preprint, arXiv: 1706.08160, 2017.
28. Y. Kim, Y. Jernite, D. Sontag, A. M. Rush, "Character-aware neural language models, " in Proc. Association Advancement Artificial Intelligence Conf., 2016, pp. 2741-2749.
29. C. N. Dos Santos and M. Gatti, "Deep convolutional neural networks for sentiment analysis of short texts, " in Proc. Int. Conf. Computational Linguistics, 2014, pp. 69-78.
30. C. N d Santos and V. Guimaraes, "Boosting named entity recognition with neural character embeddings, " arXiv Preprint, arXiv:1505.05008, 2015.
31. C. D. Santos and B. Zadrozny, "Learning characterlevel representations for part-of-speech tagging, " in Proc. 31st Int. Conf. Machine Learning, 2014, pp. 1818-1826.
32. Y. Ma, E. Cambria, S. Gao, "Label embedding for zero-shot fine-grained named entity typing, " in Proc. Int. Conf. Computational Linguistics, Osaka, 2016, pp. 171-180.
33. X. Chen, L. Xu, Z. Liu, M. Sun, H. Luan, "Joint learning of character and word embeddings, " in Proc. Int. Joint Conf. Artificial Intelligence, 2015, pp. 1236-1242.
34. X. Zheng, H. Chen, T. Xu, "Deep learning for chinese word segmentation and pos tagging, " in Proc. Conf. Empirical Methods Natural Language Processing, 2013, pp. 647-657.
35. H. Peng, E. Cambria, X. Zou, "Radical-based hierarchical embeddings for chinese sentiment analysis at sentence level, " in Proc. Int. Florida Artificial Intelligence Research Society Conf., 2017, pp. 347-352.
36. P. Bojanowski, E. Grave, A. Joulin, T. Mikolov, "Enriching word vectors with subword information, " arXiv Preprint, arXiv:1607.04606, 2016.
37. A. Herbelot and M. Baroni, "High-risk learning: Acquiring new word vectors from tiny data, " arXiv Preprint, arXiv:1707.06556, 2017.
38. Y. Pinter, R. Guthrie, J. Eisenstein, "Mimicking word embeddings using subword rnns, " arXiv Preprint, arXiv:1707.06961, 2017.
39. L. Lucy and J. Gauthier, "Are distributional representations ready for the real world? Evaluating word vectors for grounded perceptual meaning, " arXiv Preprint, arXiv:1705.11168, 2017.
40. A. Krizhevsky, I. Sutskever, G. E. Hinton, "Imagenet classification with deep convolutional neural networks, " in Proc. Advances Neural Information Processing Systems, 2012, pp. 1097-1105.
41. A. Sharif Razavian, H. Azizpour, J. Sullivan, S. Carlsson, "CNN features off-the-shelf: An astounding baseline for recognition, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, 2014, pp. 806-813.
42. Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, T. Darrell, "Caffe: Convolutional architecture for fast feature embedding, " in Proc. 22nd ACM Int. Conf. Multimedia, 2014, pp. 675-678.
43. N. Kalchbrenner, E. Grefenstette, P. Blunsom, "A convolutional neural network for modelling sentences, " in Proc. 52nd Annu. Meeting Association Computational Linguistics, 2014, vol. 1, pp. 655-665.
44. Y. Kim, "Convolutional neural networks for sentence classification, " arXiv Preprint, arXiv:1408.5882, 2014.
45. S. Poria, E. Cambria, A. Gelbukh, "Aspect extraction for opinion mining with a deep convolutional neural network, " Knowl.-Based Syst., vol. 108, pp. 42-49, June 2016.
46. A. Kirillov, D. Schlesinger, W. Forkel, A. Zelenin, S. Zheng, P. Torr, C. Rother, "Efficient likelihood learning of a generic CNN-CRF model for semantic segmentation, " arXiv Preprint, arXiv:1511.05067, 2015.
47. A. Waibel, T. Hanazawa, G. Hinton, K. Shikano, K. J. Lang, "Phoneme recognition using time-delay neural networks, " IEEE Trans. Acoust., Speech, Signal Process., vol. 37, no. 3, pp. 328-339, Mar. 1989.
48. A. Mukherjee and B. Liu, "Aspect extraction through semi-supervised modeling, " in Proc. 50th Annu. Meeting Association Computational Linguistics, 2012, pp. 339-348.
49. S. Ruder, P. Ghaffari, J. G. Breslin, "Insight-1 at semeval-2016 task 5: Deep learning for multilingual aspect-based sentiment analysis, " arXiv Preprint, arXiv: 1609.02748, 2016.
50. P. Wang, J. Xu, B. Xu, C. Liu, H. Zhang, F. Wang, H. Hao, "Semantic clustering and convolutional neural network for short text categorization, " in Proc. Annu. Meeting Association Computational Linguistics, 2015, pp. 352-357.
51. S. Poria, E. Cambria, D. Hazarika, P. Vij, "A deeper look into sarcastic tweets using deep convolutional neural networks, " in Proc. Int. Conf. Computational Linguistics, 2016, pp. 1601-1612.
52. M. Denil, A. Demiraj, N. Kalchbrenner, P. Blunsom, N. de Freitas, "Modelling, visualising and summarising documents with a single convolutional neural network, " 26th Int. Conf. Computational Linguistics, 2014, pp. 1601-1612.
53. B. Hu, Z. Lu, H. Li, Q. Chen, "Convolutional neural network architectures for matching natural language sentences, " in Proc. Advances Neural Information Processing Systems, 2014, pp. 2042-2050.
54. Y. Shen, X. He, J. Gao, L. Deng, G. Mesnil, "A latent semantic model with convolutional-pooling structure for information retrieval, " in Proc. 23rd ACM Int. Conf. Information and Knowledge Management, 2014, pp. 101-110.
55. W. Yih, X. He, C. Meek, "Semantic parsing for single-relation question answering, " in Proc. Annu. Meeting Association Computational Linguistics, 2014, pp. 643-648.
56. L. Dong, F. Wei, M. Zhou, K. Xu, "Question answering over freebase with multi-column convolutional neural networks, " in Proc. Annu. Meeting Association Computational Linguistics, 2015, pp. 260-269.
57. A. Severyn and A. Moschitti, "Modeling relational information in question-answer pairs with convolutional neural networks, " arXiv Preprint, arXiv:1604.01178, 2016.
58. Y. Chen, L. Xu, K. Liu, D. Zeng, J. Zhao, "Event extraction via dynamic multi-pooling convolutional neural networks, " in Proc. Annu. Meeting Association Computational Linguistics, 2015, pp. 167-176.
59. O. Abdel-Hamid, A.-R. Mohamed, H. Jiang, L. Deng, G. Penn, D. Yu, "Convolutional neural networks for speech recognition, " IEEE Trans. Acoust., Speech, Signal Process., vol. 22, no. 10, pp. 1533-1545, Oct. 2014.
60. D. Palaz, M. Magimai-Doss, R. Collobert, "Analysis of CNN-based speech recognition system using raw speech as input, " Idiap, Tech. Rep, 2015.
61. Z. Tu, B. Hu, Z. Lu, H. Li, "Context-dependent translation selection using convolutional neural network, " arXiv Preprint, arXiv:1503.02357, 2015.
62. J. L. Elman, "Finding structure in time, " Cogn. Sci., vol. 14, no. 2, pp. 179-211, 1990.
63. T. Mikolov, S. Kombrink, L. Burget, J. C? ernocký, S. Khudanpur, "Extensions of recurrent neural network language model, " in Proc. Int. Conf. Acoustics, Speech and Signal Processing, 2011, pp. 5528-5531.
64. I. Sutskever, J. Martens, G. E. Hinton, "Generating text with recurrent neural networks, " in Proc. 28th Int. Conf. Machine Learning, 2011, pp. 1017-1024.
65. S. Liu, N. Yang, M. Li, M. Zhou, "A recursive recurrent neural network for statistical machine translation, " in Proc. 52nd Annu. Meeting Association Computational Linguistics, 2014, pp. 1491-1500.
66. M. Auli, M. Galley, C. Quirk, G. Zweig, "Joint language and translation modeling with recurrent neural networks, " in Proc. Conf. Empirical Methods Natural Language Processing, 2013, pp. 1044-1054.
67. I. Sutskever, O. Vinyals, Q. V. Le, "Sequence to sequence learning with neural networks, " in Proc. Advances Neural Information Processing Systems, 2014, pp. 3104-3112.
68. T. Robinson, M. Hochberg, S. Renals, "The use of recurrent neural networks in continuous speech recognition, " in Proc. Automatic Speech and Speaker Recognition, 1996, pp. 233-258.
69. A. Graves, A.-r. Mohamed, G. Hinton, "Speech recognition with deep recurrent neural networks, " in Proc. Int. Conf. Acoustics, Speech and Signal Processing, 2013, pp. 6645-6649.
70. A. Graves and N. Jaitly, "Towards end-to-end speech recognition with recurrent neural networks, " in Proc. 31st Int. Conf. Machine Learning, 2014, pp. 1764-1772.
71. H. Sak, A. Senior, F. Beaufays, "Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition, " arXiv Preprint, arXiv:1402.1128, 2014.
72. A. Karpathy and L. Fei-Fei, "Deep visual-semantic alignments for generating image descriptions, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2015, pp. 3128-3137.
73. D. Tang, B. Qin, T. Liu, "Document modeling with gated recurrent neural network for sentiment classification, " in Proc. Conf. Empirical Methods Natural Language Processing, 2015, pp. 1422-1432.
74. J. Chung, C. Gulcehre, K. Cho, Y. Bengio, "Empirical evaluation of gated recurrent neural networks on sequence modeling, " arXiv Preprint, arXiv:1412.3555, 2014.
75. K. Cho, B. Van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y. Bengio, "Learning phrase representations using RNN encoder-decoder for statistical machine translation, " arXiv Preprint, arXiv: 1406.1078, 2014.
76. G. Chen, D. Ye, E. Cambria, J. Chen, Z. Xing, "Ensemble application of convolutional and recurrent neural networks for multi-label text categorization, " in Proc. Int. Joint Conf. Neural Networks, 2017, pp. 2377-2383.
77. S. Poria, E. Cambria, D. Hazarika, N. Mazumder, A. Zadeh, L. Morency, "Context-dependent sentiment analysis in user-generated videos, " in Proc. Annu. Meeting Association Computational Linguistics, 2017, pp. 873-883.
78. A. Zadeh, M. Chen, S. Poria, E. Cambria, L. Morency, "Tensor fusion network for multimodal sentiment analysis, " in Proc. Conf. Empirical Methods Natural Language Processing, 2017, pp. 1114-1125.
79. E. Tong, A. Zadeh, C. Jones, L.-P. Morency, "Combating human trafficking with deep multimodal models, " arXiv Preprint, arXiv:1705.02735, 2017.
80. I. Chaturvedi, E. Ragusa, P. Gastaldo, R. Zunino, E. Cambria, "Bayesian network based extreme learning machine for subjectivity detection, " J. Franklin Inst., vol. 355, no. 4, pp. 1780-1797, July 2018.
81. Y. N. Dauphin, A. Fan, M. Auli, D. Grangier, "Language modeling with gated convolutional networks, " arXiv Preprint, arXiv:1612.08083, 2016.
82. W. Yin, K. Kann, M. Yu, H. Schütze, "Comparative study of CNN and RNN for natural language processing, " arXiv Preprint, arXiv:1702.01923, 2017.
83. S. Hochreiter and J. Schmidhuber, "Long short-term memory, " Neural Comput., vol. 9, no. 8, pp. 1735-1780, 1997.
84. F. A. Gers, J. Schmidhuber, F. Cummins, "Learning to forget: Continual prediction with LSTM, " in Proc. 9th Int. Conf. Artificial Neural Networks, pp. 850-855, 1999.
85. G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, C. Dyer, "Neural architectures for named entity recognition, " arXiv Preprint, arXiv:1603.01360, 2016.
86. A. Graves, "Generating sequences with recurrent neural networks, " arXiv Preprint, arXiv:1308.0850, 2013.
87. M. Sundermeyer, H. Ney, R. Schlüter, "From feedforward to recurrent LSTM neural networks for language modeling, " IEEE Trans. Audio, Speech, Language Process., vol. 23, no. 3, pp. 517-529, Mar. 2015.
88. M. Sundermeyer, T. Alkhouli, J. Wuebker, H. Ney, "Translation modeling with bidirectional recurrent neural networks, " in Proc. Conf. Empirical Methods Natural Language Processing, 2014, pp. 14-25.
89. R. Lowe, N. Pow, I. Serban, J. Pineau, "The ubuntu dialogue corpus: A large dataset for research in unstructured multi-turn dialogue systems, " arXiv Preprint, arXiv:1506.08909, 2015.
90. O. Vinyals, A. Toshev, S. Bengio, D. Erhan, "Show and tell: A neural image caption generator, " in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2015, pp. 3156-3164.
91. O. Vinyals and Q. Le, "A neural conversational model, " arXiv Preprint arXiv:1506.05869, 2015.
92. J. Li, M. Galley, C. Brockett, G. P. Spithourakis, J. Gao, B. Dolan, "A persona-based neural conversation model, " arXiv Preprint, arXiv:1603.06155, 2016.
93. M. Malinowski, M. Rohrbach, M. Fritz, "Ask your neurons: A neural-based approach to answering questions about images, " in Proc. IEEE Int. Conf. Computer Vision, 2015, pp. 1-9.
94. D. Bahdanau, K. Cho, Y. Bengio, "Neural machine translation by jointly learning to align and translate, " arXiv Preprint, arXiv:1409.0473, 2014.
95. A. M. Rush, S. Chopra, J. Weston, "A neural attention model for abstractive sentence summarization, " arXiv Preprint, arXiv:1509.00685, 2015.
96. K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhudinov, R. Zemel, Y. Bengio, "Show, attend and tell: Neural image caption generation with visual attention, " in Proc. Int. Conf. Machine Learning, 2015, pp. 2048-2057.
97. O. Vinyals, ?. Kaiser, T. Koo, S. Petrov, I. Sutskever, G. Hinton, "Grammar as a foreign language, " in Proc. Advances Neural Information Processing Systems, 2015, pp. 2773-2781.
98. O. Vinyals, M. Fortunato, N. Jaitly, "Pointer networks, " in Proc. Advances Neural Information Processing Systems, 2015, pp. 2692-2700.
99. R. Paulus, C. Xiong, R. Socher, "A deep reinforced model for abstractive summarization, " arXiv Preprint, arXiv:1705.04304, 2017.
100. Y. Wang, M. Huang, X. Zhu, L. Zhao, "Attention-based LSTM for aspect-level sentiment classification, " in Proc. Conf. Empirical Methods Natural Language Processing, 2016, pp. 606-615.
101. Y. Ma, H. Peng, E. Cambria, "Targeted aspectbased sentiment analysis via embedding commonsense knowledge into an attentive LSTM, " in Proc. Association Advancement Artificial Intelligence Conf., 2018, pp. 5876-5883.
102. R. Socher, B. Huval, C. D. Manning, A. Y. Ng, "Semantic compositionality through recursive matrixvector spaces, " in Proc. Joint Conf. Empirical Methods Natural Language Processing and Computational Natural Language Learning, 2012, pp. 1201-1211.
103. B. Taskar, C. Guestrin, D. Koller, "Max-margin Markov networks, " in Proc. Advances Neural Information Processing Systems, 2004, pp. 25-32.
104. S. R. Bowman, C. Potts, C. D. Manning, "Recursive neural networks can learn logical semantics, " arXiv Preprint, arXiv:1406.1827, 2014.
105. K. S. Tai, R. Socher, C. D. Manning, "Improved semantic representations from tree-structured long short-term memory networks, " arXiv Preprint, arXiv:1503.00075, 2015.
106. S. Bengio, O. Vinyals, N. Jaitly, N. Shazeer, "Scheduled sampling for sequence prediction with recurrent neural networks, " in Proc. Advances Neural Information Processing Systems, 2015, pp. 1171-1179.
107. M. Ranzato, S. Chopra, M. Auli, W. Zaremba, "Sequence level training with recurrent neural networks, " arXiv Preprint, arXiv:1511.06732, 2015.
108. J. Li, W. Monroe, A. Ritter, M. Galley, J. Gao, D. Jurafsky, "Deep reinforcement learning for dialogue generation, " arXiv Preprint, arXiv:1606.01541, 2016.
109. R. J. Williams, "Simple statistical gradient-following algorithms for connectionist reinforcement learning, " Mach. Learn., vol. 8, no. 3-4, pp. 229-256, 1992.
110. S. Young, M. Gašic?, S. Keizer, F. Mairesse, J. Schatzmann, B. Thomson, K. Yu, "The hidden information state model: A practical framework for POMDP-based spoken dialogue management, " Comput. Speech Lang., vol. 24, no. 2, pp. 150-174, June 2010.
111. S. Young, M. Gašic?, B. Thomson, J. D. Williams, "POMDP-based statistical spoken dialog systems: A review, " Proc. IEEE, vol. 101, no. 5, pp. 1160-1179, 2013.
112. P.-h. Su, V. David, D. Kim, T.-h. Wen, S. Young, "Learning from real users: Rating dialogue success with neural networks for reinforcement learning in spoken dialogue systems, " in Proc. Interspeech Conf., 2015, pp. 2007-2011.
113. P.-H. Su, M. Gasic, N. Mrksic, L. Rojas-Barahona, S. Ultes, D. Vandyke, T. Wen, S. Young, "On-line active reward learning for policy optimisation in spoken dialogue systems, " arXiv Preprint, arXiv:1605.07669, 2016.
114. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, "Generative adversarial nets, " in Proc. Advances Neural Information Processing Systems, 2014, pp. 2672-2680.
115. J. Li, W. Monroe, T. Shi, A. Ritter, D. Jurafsky, "Adversarial learning for neural dialogue generation, " arXiv Preprint, arXiv:1701.06547, 2017.
116. R. Kiros, Y. Zhu, R. R. Salakhutdinov, R. Zemel, R. Urtasun, A. Torralba, S. Fidler, "Skip-thought vectors, " in Proc. Advances Neural Information Processing Systems, 2015, pp. 3294-3302.
117. A. M. Dai and Q. V. Le, "Semi-supervised sequence learning, " in Proc. Advances Neural Information Processing Systems, 2015, pp. 3079-3087.
118. D. E. Rumelhart, G. E. Hinton, R. J. Williams, "Learning internal representations by error propagation, " DTIC Document, Tech. Rep, 1985.
119. D. P. Kingma and M. Welling, "Auto-encoding variational bayes, " arXiv Preprint, arXiv:1312.6114, 2013.
120. S. R. Bowman, L. Vilnis, O. Vinyals, A. M. Dai, R. Jozefowicz, S. Bengio, "Generating sentences from a continuous space, " arXiv Preprint, arXiv:1511.06349, 2015.
121. Y. Zhang, Z. Gan, L. Carin, "Generating text via adversarial training, " in Proc. Neural Information Processing Systems Workshop Adversarial Training, 2016.
122. Z. Hu, Z. Yang, X. Liang, R. Salakhutdinov, E. P. Xing, "Controllable text generation, " arXiv Preprint, arXiv:1703.00955, 2017.
123. L. Yu, W. Zhang, J. Wang, Y. Yu, "Seqgan: Sequence generative adversarial nets with policy gradient, " in Proc. Association Advancement Artificial Intelligence Conf., 2017, pp. 2852-2858.
124. S. Rajeswar, S. Subramanian, F. Dutil, C. Pal, A. Courville, "Adversarial generation of natural language, " arXiv Preprint, arXiv:1705.10929, 2017.
125. J. Giménez and L. Marquez, "Fast and accurate part-of-speech tagging: The SVM approach revisited, " Recent Adv. Natural Lang. Process., pp. 153-162, 2004.
126. Z. Huang, W. Xu, K. Yu, "Bidirectional LSTM-CRF models for sequence tagging, " arXiv Preprint, arXiv:1508.01991, 2015.
127. D. Andor, C. Alberti, D. Weiss, A. Severyn, A. Presta, K. Ganchev, S. Petrov, M. Collins, "Globally normalized transition-based neural networks, " arXiv Preprint, arXiv:1603.06042, 2016.
128. A. Kumar, O. Irsoy, P. Ondruska, M. Iyyer, J. Bradbury, I. Gulrajani, V. Zhong, R. Paulus, R. Socher, "Ask me anything: Dynamic memory networks for natural language processing, " in Proc. Int. Conf. Machine Learning, 2016, pp. 1378-1387.
129. D. Chen and C. D. Manning, "A fast and accurate dependency parser using neural networks, " in Proc. Conf. Empirical Methods Natural Language Processing, 2014, pp. 740-750.
130. D. Weiss, C. Alberti, M. Collins, S. Petrov, "Structured training for neural network transition-based parsing, " arXiv Preprint, arXiv:1506.06158, 2015.
131. C. Dyer, M. Ballesteros, W. Ling, A. Matthews, N. A. Smith, "Transition-based dependency parsing with stack long short-term memory, " arXiv Preprint, arXiv:1505.08075, 2015.
132. H. Zhou, Y. Zhang, C. Cheng, S. Huang, X. Dai, J. Chen, "A neural probabilistic structured-prediction method for transition-based natural language processing, " J. Artif. Intell. Res., vol. 58, pp. 703-729, Mar. 2017.
133. S. Petrov, L. Barrett, R. Thibaux, D. Klein, "Learning accurate, compact, interpretable tree annotation, " in Proc. 21st Int. Conf. Computational Linguistics, 2006, pp. 433-440.
134. M. Zhu, Y. Zhang, W. Chen, M. Zhang, J. Zhu, "Fast and accurate shift-reduce constituent parsing, " in Proc. Annu. Meeting Association Computational Linguistics, 2013, pp. 434-443.
135. J. Weston, S. Chopra, A. Bordes, "Memory networks, " arXiv Preprint, arXiv:1410.3916, 2014.
136. S. Sukhbaatar, A. Szlam, J. Weston, R. Fergus, "End-to-end memory networks, " in Proc. Advances Neural Information Processing Systems, 2015, pp. 2440-2448.
137. C. Xiong, S. Merity, R. Socher, "Dynamic memory networks for visual and textual question answering, " in Proc. Int. Conf. Machine Learning, 2016, pp. 2397-2406.
138. A. Passos, V. Kumar, A. McCallum, "Lexicon infused phrase embeddings for named entity resolution, " arXiv Preprint, arXiv:1404.5367, 2014.
139. J. P. Chiu and E. Nichols, "Named entity recognition with bidirectional LSTM-CNNs, " arXiv Preprint, arXiv:1511.08308, 2015.
140. G. Luo, X. Huang, C. Lin, Z. Nie, "Joint named entity recognition and disambiguation, " in Proc. Conf. Empirical Methods Natural Language Processing, 2015, pp. 879-880.
141. E. Strubell, P. Verga, D. Belanger, A. McCallum, "Fast and accurate sequence labeling with iterated dilated convolutions, " arXiv Preprint, arXiv:1702.02098, 2017.
142. O. Täckström, K. Ganchev, D. Das, "Efficient inference and structured learning for semantic role labeling, " Trans. Assoc. Comput. Linguistics, vol. 3, pp. 29-41, Jan. 2015.
143. J. Zhou and W. Xu, "End-to-end learning of semantic role labeling using recurrent neural networks, " in Proc. Annu. Meeting Association Computational Linguistics, 2015, pp. 1127-1137.
144. L. He, K. Lee, M. Lewis, L. Zettlemoyer, "Deep semantic role labeling: What works and what's next, " in Proc. Annu. Meeting Association for Computational Linguistics, 2017, pp. 473-483.
145. Q. Le and T. Mikolov, "Distributed representations of sentences and documents, " in Proc. 31st Int. Conf. Machine Learning, 2014, pp. 1188-1196.
146. L. Yu, J. Wang, K. R. Lai, X. Zhang, "Refining word embeddings for sentiment analysis, " in Proc. Conf. Empirical Methods Natural Language Processing, 2017, pp. 545-550.
147. Y. Wu, M. Schuster, Z. Chen, Q. V. Le, M. Norouzi, W. Macherey, M. Krikun, Y. Cao, Q. Gao, K. Macherey, et al., "Google's neural machine translation system: Bridging the gap between human and machine translation, " arXiv Preprint, arXiv:1609.08144, 2016.
148. J. Gehring, M. Auli, D. Grangier, D. Yarats, Y. N. Dauphin, "Convolutional sequence to sequence learning, " arXiv Preprint, arXiv:1705.03122, 2017.
149. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, "Attention is all you need, " arXiv Preprint, arXiv: 1706.03762, 2017.
150. R. K. Srivastava, K. Greff, J. Schmidhuber, "Training very deep networks, "in Proc. Advances Neural Information Processing Systems, 2015, pp. 2377-2385.
151. B. Pang and L. Lee, "Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales, " in Proc. 43rd Annu. Meeting Association Computational Linguistics, 2005, pp. 115-124.
152. A. Zadeh, R. Zellers, E. Pincus, L. Morency, "Multimodal sentiment intensity analysis in videos: Facial gestures and verbal messages, " IEEE Intell. Syst., vol. 31, no. 6, pp. 82-88, Nov. 2016.
153. A. Fader, L. S. Zettlemoyer, O. Etzioni, "Paraphrase-driven learning for open question answering, " in Proc. Annu. Meeting Association Computational Linguistics, 2013, pp. 1608-1618.
154. A. Bordes, J. Weston, N. Usunier, "Open question answering with weakly supervised embedding models, " in Proc. Joint European Conf. Machine Learning and Knowledge Discovery Databases, 2014, pp. 165-180.
155. A. Ritter, C. Cherry, W. B. Dolan, "Data-driven response generation in social media, " in Proc. Conf. Empirical Methods Natural Language Processing, 2011, pp. 583-593.
156. A. Sordoni, M. Galley, M. Auli, C. Brockett, Y. Ji, M. Mitchell, J.-Y. Nie, J. Gao, B. Dolan, "A neural network approach to context-sensitive generation of conversational responses, " arXiv Preprint, arXiv: 1506.06714, 2015.
157. J. Li, M. Galley, C. Brockett, J. Gao, B. Dolan, "A diversity-promoting objective function for neural conversation models, " arXiv Preprint, arXiv:1510.03055, 2015.
158. J. Dodge, A. Gane, X. Zhang, A. Bordes, S. Chopra, A. Miller, A. Szlam, J. Weston, "Evaluating prerequisite qualities for learning end-to-end dialog systems, " arXiv Preprint, arXiv:1511.06931, 2015.
159. X. Zhou, D. Dong, H. Wu, S. Zhao, D. Yu, H. Tian, X. Liu, R. Yan, "Multi-view response selection for human-computer conversation, " in Proc. Conf. Empirical Methods Natural Language Processing, 2016, pp. 372-381.
160. P. Koehn, F. J. Och, D. Marcu, "Statistical phrase-based translation, " in Proc. Conf. North American Chapter Association Computational Linguistics, 2003, pp. 48-54.
161. K. Papineni, S. Roukos, T. Ward, W. Zhu, "Bleu: A method for automatic evaluation of machine translation, " in Proc. 40th Annu. Meeting Association Computational Linguistics, 2002, pp. 311-318.
162. I. V. Serban, A. Sordoni, Y. Bengio, A. C. Courville, J. Pineau, "Building end-to-end dialogue systems using generative hierarchical neural network models, " in Proc. Association Advancement Artificial Intelligence Conf., 2016, pp. 3776-3784.
163. Y. LeCun, Y. Bengio, G. Hinton, "Deep learning, " Nature, vol. 521, no. 7553, pp. 436-444, May 2015.
164. T. Baltrušaitis, C. Ahuja, L. Morency, "Multimodal machine learning: A survey and taxonomy, " arXiv Preprint, arXiv:1705.09406, 2017.