How to generate a good word embedding

IEEE Intelligent Systems

Volumn 31, Issue 6, 2016, Pages 5-14

  Lai, Siwei ^a   Liu, Kang ^a   He, Shizhu ^a   Zhao, Jun ^a  

^a INSTITUTE OF AUTOMATION   (China)

Author keywords

distributed representation; intelligent systems; neural network; word embedding

Indexed keywords

INTELLIGENT SYSTEMS; NEURAL NETWORKS;

CRITICAL COMPONENT; DISTRIBUTED REPRESENTATION; EMBEDDING METHOD; EMBEDDINGS; SEMANTIC PROPERTIES; TRAINING PARAMETERS; WORD EMBEDDING;

SEMANTICS;

EID: 85012088618     PISSN: 15411672     EISSN: None     Source Type: Journal    
DOI: 10.1109/MIS.2016.45     Document Type: Article

References (22)

1. J. Turian, L. Ratinov, and Y. Bengio, "Word Representations: A Simple and General Method for Semi-supervised Learning," Proc. Assoc. Computational Linguistics, 2010, pp. 384-394.
2. M. Baroni, G. Dinu, and G. Kruszewski, "Don't Count, Predict! A Systematic Comparison of Context-Counting vs. Context-Predicting Semantic Vectors," Proc. Assoc. Computational Linguistics, 2014, pp. 238-247.
3. J. Pennington, R. Socher, and C.D. Manning, "GloVe: Global Vectors for Word Representation," Proc. Empirical Methods Natural Language Processing, 2014, pp. 1532-1543.
4. T. Mikolov et al., "Efficient Estimation of Word Representations in Vector Space," Proc. Int'l Conf. Learning Representation, 2013; http://arxiv.org/pdf/1301.3781.pdf.
5. Y. Bengio et al., "A Neural Probabilistic Language Model," J. Machine Learning Research, vol. 3, 2003, pp. 1137-1155.
6. R. Collobert and J. Weston, "A Unified Architecture for Natural Language Processing: Deep Neural Networks with Multitask Learning," Proc. Int'l Conf. Machine Learning, 2008, pp. 160-167.
7. J. Turian, L. Ratinov, and Y. Bengio, "Word Representations: A Simple and General Method for Semi-supervised Learning," Proc. Assoc. Computational Linguistics, 2010, pp. 384-394.
8. T. Mikolov et al., "Efficient Estimation of Word Representations in Vector Space," Proc. Int'l Conf. Learning Representation, 2013, arXiv:1301.3781.
9. A. Mnih and G. Hinton, "Three New Graphical Models for Statistical Language Modelling," Proc. Int'l Conf. Machine Learning, 2007, pp. 641-648.
10. Z.S. Harris, "Distributional Structure," Word, vol. 10, no. 2, 1954, pp. 146-162.
11. T.K. Landauer, "On the Computational Basis of Learning and Cognition: Arguments from LSA," Psychology of Learning and Motivation, vol. 41, 2002, pp. 43-84.
12. M. Baroni, G. Dinu, and G. Kruszewski, "Don't Count, Predict! A Systematic Comparison of Context-Counting vs. Context-Predicting Semantic Vectors," Proc. Assoc. Computational Linguistics, 2014, pp. 238-247.
13. J. Pennington, R. Socher, and C.D. Manning, "GloVe: Global Vectors for Word Representation," Proc. Empirical Methods in Natural Language Processing, 2014, pp. 1532-1543.
14. L. Finkelstein et al., "Placing Search in Context: The Concept Revisited," ACM Trans. Information Systems, vol. 20, no. 1, 2002, pp. 116-131.
15. T.K. Landauer and S.T. Dumais, "A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge," Psychological Rev., vol. 104, no. 2, 1997, p. 211.
16. A.L. Maas et al., "Learning Word Vectors for Sentiment Analysis," Proc. Assoc. Computational Linguistics, 2011, pp. 142-150.
17. D. Erhan et al., "Why Does Unsupervised Pre-training Help Deep Learning?," J. Machine Learning Research, vol. 11, 2010, pp. 625-660.
18. Y. Kim, "Convolutional Neural Networks for Sentence Classification," Proc. Empirical Methods Natural Language Processing, 2014, pp. 1746-1751.
19. R. Socher et al., "Recursive Deep Models for Semantic Compositionality over a Sentiment Treebank," Proc. Empirical Methods in Natural Language Processing, 2013, pp. 1631-1642.
20. R. Collobert et al., "Natural Language Processing (almost) from Scratch," J. Machine Learning Research, vol. 12, no. 8, 2011, pp. 2493-2537.
21. K. Toutanova et al., "Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network," Proc. Annual Conf. North Am. Chapter Assoc. Computational Linguistics: Human Language Technologies, 2003, pp. 173-180.
22. T. Mikolov et al., "Distributed Representations of Words and Phrases and Their Compositionality," Proc. Neural Information Processing Systems, 2013, pp. 3111-3119.