Sentiment Embeddings with Applications to Sentiment Analysis

IEEE Transactions on Knowledge and Data Engineering

Volumn 28, Issue 2, 2016, Pages 496-509

  Tang, Duyu ^a   Wei, Furu ^b   Qin, Bing ^a   Yang, Nan ^b   Liu, Ting ^a   Zhou, Ming ^b  

^a HARBIN INSTITUTE OF TECHNOLOGY   (China)

^b MICROSOFT RESEARCH   (United States)

Author keywords

Natural language processing; neural networks; sentiment analysis; word embeddings

Indexed keywords

ALGORITHMS; COMPUTATIONAL LINGUISTICS; DATA MINING; LEARNING ALGORITHMS; NEURAL NETWORKS;

BENCHMARK DATASETS; EMBEDDINGS; FEATURE ENGINEERINGS; NATURAL LANGUAGE PROCESSING; NEAREST NEIGHBORS; SENTIMENT ANALYSIS; SENTIMENT CLASSIFICATION; SENTIMENT LEXICONS;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 84962367256     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2015.2489653     Document Type: Conference Paper

References (71)

1. D. Tang, F. Wei, N. Yang, M. Zhou, T. Liu, and B. Qin, "Learning Sentiment-specific word embedding for twitter sentiment classification, " in Proc. 52th Annu. Meeting Assoc. Comput. Linguistics., 2014, pp. 1555-1565.
2. D. Tang, F. Wei, B. Qin, M. Zhou, and T. Liu, "Building large-scale twitter-specific sentiment lexicon:A representation learning approach, " in Proc. 25th Int. Conf. Comput. Linguistics, 2014, pp. 172-182
3. D. Tang, F. Wei, B. Qin, T. Liu, and M. Zhou, "Coooolll:A deep learning system for twitter sentiment classification, " in Proc. 8th Int. Workshop Semantic Eval., 2014, pp. 208-212.
4. C. D. Manning and H. Schütze, Foundations of Statistical Natural Language Processing. Cambridge, MA, USA:MIT Press, 1999.
5. D. Jurafsky and H. James, Speech and Language Processing:An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Englewood Cliffs, NJ, USA:Prentice-Hall, 2000.
6. Y. Bengio, R. Ducharme, P. Vincent, and C. Janvin, "A neural probabilistic language model, " J. Mach. Learning Res., vol. 3, pp. 1137-1155, 2003.
7. T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean, "Distributed representations of words and phrases and their compositionality, " in Proc. Conf. Neural Inf. Process. Syst., 2013, pp. 3111-3119.
8. J. Pennington, R. Socher, and C. Manning, "Glove:Global vectors for word representation, " in Proc. Conf. Empirical Methods Natural Lang. Process., 2014, pp. 1532-1543.
9. Z. S. Harris, "Distributional structure, " Word, vol. 10, pp. 146-162, 1954.
10. N. Yang, S. Liu, M. Li, M. Zhou, and N. Yu, "Word alignment modeling with context dependent deep neural network, " in Proc. 51st Annu. Meeting Assoc. Comput. Linguistics, 2013, pp. 166-175.
11. R. Socher, J. Bauer, C. D. Manning, and A. Y. Ng, "Parsing with compositional vector grammars, " in Proc. Annu. Meeting Assoc. Comput. Linguistics, 2013, pp. 455-465.
12. M. Iyyer, J. Boyd-Graber, L. Claudino, R. Socher, and H. Daume III, "A neural network for factoid question answering over paragraphs, " in Proc. Conf. Empirical Methods Natural Lang. Process., 2014, pp. 633-644.
13. J. Li, R. Li, and E. Hovy, "Recursive deep models for discourse parsing, " in Proc. Conf. Empirical Methods Natural Lang. Process., 2014, pp. 2061-2069.
14. R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa, "Natural language processing (almost) from scratch, " J. Mach. Learning Res., vol. 12, pp. 2493-2537, 2011.
15. B. Pang and L. Lee, "Opinion mining and sentiment analysis, " Found. Trends Inf. Retrieval, vol. 2, nos. 1/2, pp. 1-135, 2008.
16. B. Liu, "Sentiment analysis and opinion mining, " Synthesis Lectures Human Lang. Technol., vol. 5, no. 1, pp. 1-167, 2012.
17. R. Feldman, "Techniques and applications for sentiment analysis, " Commun. ACM, vol. 56, no. 4, pp. 82-89, 2013.
18. J. Ma, Y. Zhang, and J. Zhu, "Tagging the web:Building a robust web tagger with neural network, " in Proc. 52nd Annu. Meeting Assoc. Comput. Linguistics, 2014, pp. 144-154.
19. M. Hu and B. Liu, "Mining and summarizing customer reviews, " in Proc. 10th ACM SIGKDD Conf. Knowl. Discovery Data Mining, 2004, pp. 168-177.
20. T. Wilson, J. Wiebe, and P. Hoffmann, "Recognizing contextual polarity in phrase-level sentiment analysis, " in Proc. Conf. Empirical Methods Natural Lang. Process., 2005, pp. 347-354.
21. P. Nakov, S. Rosenthal, Z. Kozareva, V. Stoyanov, A. Ritter, and T. Wilson, "Semeval-2013 task 2:Sentiment analysis in twitter, " in Proc. Int. Workshop Semantic Eval., 2013, vol. 13, pp. 312-320.
22. A. Severyn and A. Moschitti, "Twitter sentiment analysis with deep convolutional neural networks, " in Proc. Special Interest Group Inf. Retrieval, 2015, pp. 959-962.
23. B. Pang and L. Lee, "Seeing stars:Exploiting class relationships for sentiment categorization with respect to rating scales, " in Proc. 43rd Annu. Meeting Assoc. Comput. Linguistics, 2005, pp. 115-124.
24. A. Severyn and A. Moschitti, "On the automatic learning of sentiment lexicons, " in Proc. Conf. North Amer. Ch. Assoc. Comput. Linguistics, 2015, pp. 1397-1402.
25. P. F. Brown, P. V. Desouza, R. L. Mercer, V. J. D. Pietra, and J. C. Lai, "Class-based N-gram models of natural language, " Comput. Linguistics, vol. 18, no. 4, pp. 467-479, 1992.
26. L. D. Baker and A. K. McCallum, "Distributional clustering of words for text classification, " in Proc. 21st Annu. Int. ACM SIGIR Conf. Res. Develop. Inf. Retrieval, 1998, pp. 96-103.
27. S. C. Deerwester, S. T. Dumais, T. K. Landauer, G. W. Furnas, and R. A. Harshman, "Indexing by latent semantic analysis, " J. Amer. Soc. Inf. Sci., vol. 41, no. 6, pp. 391-407, 1990.
28. K. Lund and C. Burgess, "Producing high-dimensional semantic spaces from lexical Co-occurrence, " Behavior Res. Methods, Instrum. Comput., vol. 28, no. 2, pp. 203-208, 1996.
29. R. Lebret, J. Legrand, and R. Collobert, "Is deep learning really necessary for word embeddings?" in Proc. NIPS Workshop, 2013.
30. G. Hinton and R. Salakhutdinov, "Reducing the dimensionality of data with neural networks, " Science, vol. 313, no. 5786, pp. 504-507, 2006.
31. N. Jones, "Computer science:The learning machines, " Nature, vol. 505, no. 7482, pp. 146, 2014.
32. Y. Bengio, I. J. Goodfellow, and A. Courville, Deep Learning. Cambridge, MA, USA:MIT Press, 2015.
33. Y. Bengio and J.-S. Senecal, "Quick training of probabilistic neural nets by importance sampling, " in Proc. AISTATS Conf., 2003.
34. F. Morin and Y. Bengio, "Hierarchical probabilistic neural network language model, " in Proc. Int. Workshop Artif. Intell. Statist., 2005, pp. 246-252.
35. A. Mnih and G. Hinton, "Three new graphical models for statistical language modelling, " in Proc. 24th Int. Conf. Mach. Learning, 2007, pp. 641-648.
36. R. Collobert and J. Weston, "A unified architecture for natural language processing:Deep neural networks with multitask learning, " in Proc. 25th Int. Conf. Mach. Learning, 2008, pp. 160-167.
37. N. A. Smith and J. Eisner, "Contrastive estimation:Training loglinear models on unlabeled data, " in Proc. 43rd Annu. Meeting Assoc. Comput. Linguistics, 2005, pp. 354-362.
38. T. Mikolov, K. Chen, G. Corrado, and J. Dean, "Efficient estimation of word representations in vector space, " in Proc. Int. Conf. Learning Representations, 2013.
39. A. Mnih and K. Kavukcuoglu, "Learning word embeddings efficiently with noise-contrastive estimation, " in Proc. Adv. Neural Inf. Process. Syst. Conf., 2013, pp. 2265-2273.
40. M. U. Gutmann and A. Hyvärinen, "Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics, " J. Mach. Learning Res., vol. 13, no. 1, pp. 307-361, 2012.
41. E. H. Huang, R. Socher, C. D. Manning, and A. Y. Ng, "Improving word representations via global context and multiple word prototypes, " in Proc. 50th Annu. Meeting Assoc. Comput. Linguistics:Long Papers, 2012, pp. 873-882.
42. Y. Sun, L. Lin, N. Yang, Z. Ji, and X. Wang, "Radical-enhanced chinese character embedding, " Proc. 21st Int. Conf. Neural Inf. Process., 2014, pp. 279-286.
43. O. Levy and Y. Goldberg, "Dependency-based word embeddings, " in Proc. 52nd Annu. Meeting Assoc. Comput. Linguistics, 2014, pp. 302-308.
44. J. Li and D. Jurafsky, "Do multi-sense embeddings improve natural language understanding?" in Proc. Conf. Empirical Methods Natural Lang. Process., 2015, pp. 1722-1732.
45. M. Faruqui, J. Dodge, S. K. Jauhar, C. Dyer, E. Hovy, and N. A. Smith, "Retrofitting word vectors to semantic lexicons, " in Proc. Conf. NorthAmer. Ch. Assoc. Comput. Linguistics, 2015, pp. 1606-1615.
46. A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y. Ng, and C. Potts, "Learning word vectors for sentiment analysis, " in Proc. 49th Annu. Meeting Assoc. Comput. Linguistics, 2011, pp. 142-150.
47. I. Labutov and H. Lipson, "Re-embedding words, " in Proc. Annu. Meeting Assoc. Comput. Linguistics, 2013, pp. 489-493.
48. M. Baroni, G. Dinu, and G. Kruszewski, "Don't count, predict! A systematic comparison of Context-counting vs. Context-predicting semantic vectors, " in Proc. 52nd Annu. Meeting Assoc. Comput. Linguistics, 2014, pp. 238-247.
49. J. S. Bridle, "Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition, " in Neurocomputing. New York, NY, USA:Springer, 1990, pp. 227-236.
50. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning representations by back-propagating errors, " Cognitive Modeling, vol. 5, 1988.
51. K. Hashimoto, P. Stenetorp, M. Miwa, and Y. Tsuruoka, "Task-oriented learning of word embeddings for semantic relation classification, " in Proc. 19th Conf. Comput. Natural Lang. Learning, 2015, pp. 268-278.
52. S. Gouws and A. Søgaard, "Simple task-specific bilingual word embeddings, " in Proc. Conf. North Amer. Ch. Assoc. Comput. Linguistics, Human Lang. Technol., 2015, pp. 1386-1390.
53. Z. Chen, W. Lin, Q. Chen, X. Chen, S. Wei, H. Jiang, and X. Zhu, "Revisiting word embedding for contrasting meaning, " in Proc. 53rd Annu. Meeting Assoc. Comput. Linguistics, 2015, pp. 106-115.
54. X. Hu, J. Tang, H. Gao, and H. Liu, "Unsupervised sentiment analysis with emotional signals, " in Proc. Int. World Wide Web Conf., 2013, pp. 607-618.
55. A. Pak and P. Paroubek, "Twitter as a corpus for sentiment analysis and opinion mining, " in Proc. 7th Int. Conf. Lang. Resources Evaluation, vol. 2010, 2010.
56. K.-L. Liu, W.-J. Li, and M. Guo, Emoticon Smoothed Language Models for Twitter Sentiment Analysis. Palo Alto, CA, USA:AAAI Press, 2012.
57. K. Gimpel, N. Schneider, B. O'Connor, D. Das, D. Mills, J. Eisenstein, M. Heilman, D. Yogatama, J. Flanigan, and N. A. Smith, "Part-of-speech tagging for twitter:Annotation, features, and experiments, " in Proc. Annu. Meeting Assoc. Comput. Linguistics, 2011, pp. 42-47.
58. J. Duchi, E. Hazan, and Y. Singer, "Adaptive subgradient methods for online learning and stochastic optimization, " J. Mach. Learning Res., vol. 12, pp. 2121-2159, 2011.
59. R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, and C. Potts, "Recursive deep models for semantic compositionality over a sentiment treebank, " in Proc. Conf. Empirical Methods Natural Lang. Process., 2013, pp. 1631-1642.
60. J. Li and E. Hovy, "A model of coherence based on distributed sentence representation, " in Proc. Empirical Methods Natural Lang. Process., 2014, pp. 2039-2048.
61. J. Li, T. Luong, D. Jurafsky, and E. Hovy, "When are tree structures necessary for deep learning of representations?" in Proc. Conf. Empirical Methods Natural Lang. Process., 2015, pp. 2304-2314.
62. S. M. Mohammad and P. D. Turney, "Crowdsourcing a word-emotion association lexicon, " Comput. Intell., vol. 29, no. 3, pp. 436-465, 2013.
63. B. Pang, L. Lee, and S. Vaithyanathan, "Thumbs up?:Sentiment classification using machine learning techniques, " in Proc. Conf. Empirical Methods Natural Lang. Process., 2002, pp. 79-86.
64. S. M. Mohammad, S. Kiritchenko, and X. Zhu, "NRC-canada: Building the state-of-the-art in sentiment analysis of tweets, " Proc. Int. Workshop Semantic Evaluation, 2013, pp. 321-327.
65. G. Frege, "On sense and reference, " Ludlow, vol. 1997, pp. 563-584, 1892.
66. J. Mitchell and M. Lapata, "Composition in distributional models of semantics, " Cognitive Sci., vol. 34, no. 8, pp. 1388-1429, 2010.
67. S. Kiritchenko, X. Zhu, and S. M. Mohammad, "Sentiment analysis of short informal texts, " J. Artif. Intell. Res., vol. 50, pp. 723-762, 2014.
68. S. Rosenthal, A. Ritter, P. Nakov, and V. Stoyanov, "Semeval-2014 task 9:Sentiment analysis in twitter, " in Proc. 8th Int. Workshop Semantic Evaluation, 2014, pp. 73-80.
69. A. Go, R. Bhayani, and L. Huang, "Twitter sentiment classification using distant supervision, " CS224N Project Report, Stanford, CA, USA, 2009, pp. 1-12.
70. R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin, "Liblinear:A library for large linear classification, " J. Mach. Learning Res., vol. 9, pp. 1871-1874, 2008.
71. D. Tang, B. Qin, T. Liu, and Y. Yang, "User modeling with neural network for review rating prediction, " in Proc. 24th Int. Conf. Artif. Intell., 2015, pp. 1340-1346.