Deep learning for sentiment analysis: Successful approaches and future challenges

Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery

Volumn 5, Issue 6, 2015, Pages 292-303

  Tang, Duyu ^a   Qin, Bing ^a   Liu, Ting ^a  

^a HARBIN INSTITUTE OF TECHNOLOGY   (China)

Author keywords

[No Author keywords available]

Indexed keywords

DATA MINING; LEARNING SYSTEMS; SEMANTICS; SENTIMENT ANALYSIS;

COMPUTATIONAL MODEL; DATA REPRESENTATIONS; FEATURE ENGINEERINGS; MACHINE LEARNING APPROACHES; NATURAL LANGUAGE PROCESSING; SEMANTIC REPRESENTATION; SENTIMENT CLASSIFICATION; SENTIMENT LEXICONS;

DEEP LEARNING;

EID: 84945297056     PISSN: 19424787     EISSN: 19424795     Source Type: Journal    
DOI: 10.1002/widm.1171     Document Type: Article

References (108)

1. Li J, Li R, Hovy E. Recursive deep models for discourse parsing. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2014, 2061-2069.
2. Pang B, Lee L. Opinion mining and sentiment analysis. Found Trends Inf Retr 2008, 2:1-135.
3. Liu B. Sentiment analysis and opinion mining. Synth Lect Human Lang Technol 2012, 5:1-167.
4. Feldman R. Techniques and applications for sentiment analysis. Commun ACM 2013, 56:82-89.
5. Manning CD, Schütze H. Foundations of Statistical Natural Language Processing. Cambridge, MA: MIT Press; 1999.
6. Martin JH, Jurafsky D. Speech and Language Processing. 2nd ed. New Jersey: Prentice Hall; 2000.
7. Hu M, Liu B. Mining and summarizing customer reviews. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, WA, August 2004, 168-177. ACM.
8. Deng L, Wiebe J. MPQA 3.0: An Entity/Event-Level Sentiment Corpus. NAACL; 2015.
9. Pang B, Lee L, Vaithyanathan S. Thumbs up?: sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing, vol 10, Morristown, NJ, USA, July 2002, 79-86. Association for Computational Linguistics.
10. Yang B, Cardie C. Joint inference for fine-grained opinion extraction. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, vol 1, Sofia, Bulgaria, 2013, 1640-1649. Association for Computational Linguistics.
11. Bengio Y, Goodfellow IJ, Courville A. Deep Learning. Cambridge, MA: MIT Press (in preparation); 2015.
12. Mohammad SM, Kiritchenko S, Zhu X. NRC-Canada: building the state-of-the-art in sentiment analysis of tweets. In: Second Joint Conference on Lexical and Computational Semantics (*SEM), vol 2, Atlanta, Georgia, USA, June 2013, 321-327.
13. Yang B, Cardie C. Extracting opinion expressions with semi-markov conditional random fields. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Jeju Island, Korea, July 2012, 1335-1345. Association for Computational Linguistics.
14. Bengio Y, Courville A, Vincent P. Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 2013, 35:1798-1828.
15. LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015, 521:436-444.
16. Socher R, Perelygin A, Wu JY, Chuang J, Manning CD, Ng AY, Potts C. Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), Seattle, October 2013, 1631-1642.
17. Kalchbrenner N, Grefenstette E, Blunsom P. A convolutional neural network for modelling sentences. 2014, arXiv preprint arXiv:1404.2188
18. Irsoy O, Cardie C. Opinion mining with deep recurrent neural networks. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, 2014, 720-728.
19. Tang D, Wei F, Qin B, Zhou M, Liu T. Building large-scale twitter-specific sentiment lexicon: a representation learning approach. In: Proceedings of COLING, Dublin, Ireland, 2014, 172-182.
20. Brown PF, Desouza PV, Mercer RL, Pietra VJD, Lai JC. Class-based n-gram models of natural language. Comput Linguist 1992, 18:467-479.
21. Baker LD, McCallum AK. Distributional clustering of words for text classification. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Melbourne, Australia, August 1998, 96-103. ACM.
22. Harris ZS. Distributional structure. Word 1954.
23. Deerwester SC, Dumais ST, Landauer TK, Furnas GW, Harshman RA. Indexing by latent semantic analysis. J Am Soc Inf Sci 1990, 41:391-407.
24. Lund K, Burgess C. Producing high-dimensional semantic spaces from lexical co-occurrence. Behav Res Methods Instrum Comput 1996, 28:203-208.
25. Lebret R, Legrand J, Collobert R. Is deep learning really necessary for word embeddings? EPFL Report No. 196986, Idiap Research Institute, 2013.
26. Bai B, Weston J, Grangier D, Collobert R, Sadamasa K, Qi Y, Chapelle O, Weinberger K. Learning to rank with (a lot of) word features. Inf Retr 2010, 13:291-314.
27. Hinton GE, Salakhutdinov RR. Reducing the dimensionality of data with neural networks. Science 2006, 313:504-507.
28. Bengio Y, Ducharme R, Vincent P, Janvin C. A neural probabilistic language model. J Mach Learn Res 2003, 3:1137-1155.
29. Morin F, Bengio Y. Hierarchical probabilistic neural network language model. In: Proceedings of the International Workshop on Artificial Intelligence and Statistics, January 2005, 246-252.
30. Mnih A, Hinton G. Three new graphical models for statistical language modelling. In: Proceedings of the 24th International Conference on Machine Learning, Corvalis, Oregon, USA, June 2007, 641-648. ACM.
31. Collobert R, Weston J. A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, Helsinki, Fabianinkatu, July 2008, 160-167. ACM.
32. Smith NA, Eisner J. Contrastive estimation: training log-linear models on unlabeled data. In: Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, Ann Arbor, Michigan, June 2005, 354-362. Association for Computational Linguistics.
33. Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. 2013, arXiv preprint arXiv:1301.3781.
34. Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J. Distributed representations of words and phrases and their compositionality. In: Conference on Advances in Neural Information Processing Systems, 2013, 3111-3119.
35. Mnih A, Kavukcuoglu K. Learning word embeddings efficiently with noise-contrastive estimation. In: Conference on Advances in Neural Information Processing Systems, 2013, 2265-2273.
36. Gutmann MU, Hyvärinen A. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. J Mach Learn Res 2012, 13:307-361.
37. Huang EH, Socher R, Manning CD, Ng AY. Improving word representations via global context and multiple word prototypes. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, vol 1, Jeju, Republic of Korea, July 2012, 873-882. Association for Computational Linguistics.
38. Qiu S, Cui Q, Bian J, Gao B, Liu TY. Co-Learning of Word Representations and Morpheme Representations. In: Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers, pages 141-150, Dublin, Ireland, August 23-29, 2014.
39. Levy O, Goldberg Y. Dependency-based word embeddings. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, vol 2, Baltimore, Maryland, USA, 2014, 302-308.
40. Pennington J, Socher R, Manning CD. Glove: global vectors for word representation. In: Proceedings of the Empiricial Methods in Natural Language Processing (EMNLP 2014), vol 12, Doha, Qatar, 2014, 1532-1543.
41. Li J, Jurafsky D Do multi-sense embeddings improve natural language understanding? 2015, arXiv preprint arXiv:1506.01070.
42. Faruqui M, Dodge J, Jauhar SK, Dyer C, Hovy E, Smith NA. Retrofitting word vectors to semantic lexicons. 2014, arXiv preprint arXiv:1411.4166.
43. Yogatama D, Faruqui M, Dyer C, Smith NALearning word representations with hierarchical sparse coding. 2014, arXiv preprint arXiv:1406.2035.
44. Ma J, Zhang Y, Zhu J. Tagging the web: building a robust web tagger with neural network. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, vol 1, Baltimore, Maryland, 2014, 144-154.
45. Maas AL, Daly RE, Pham PT, Huang D, Ng AY, Potts C. Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol 1, June 2011, 142-150. Association for Computational Linguistics.
46. Labutov I, Lipson H. Re-embedding words. In: Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, Sofia, Bulgaria, August 2013, 489-493.
47. Tang D, Wei F, Yang N, Zhou M, Liu T, Qin B. Learning sentiment-specific word embedding for twitter sentiment classification. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, vol 1, Baltimore, Maryland, USA, 2014, 1555-1565.
48. Tang D, Wei F, Qin B, Liu T, Zhou M. Coooolll: a deep learning system for twitter sentiment classification. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), Dublin, Ireland, August 2014, 208-212.
49. Turney PD. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, Philadelphia, PA, USA, July 2002, 417-424. Association for Computational Linguistics.
50. Taboada M, Brooke J, Tofiloski M, Voll K, Stede M. Lexicon-based methods for sentiment analysis. Comput Linguist 2011, 37:267-307.
51. Ding X, Liu B, Yu PS. A holistic lexicon-based approach to opinion mining. In: Proceedings of the 2008 International Conference on Web Search and Data Mining, Palo Alto, California, USA, February 2008, 231-240. ACM.
52. Thelwall M, Buckley K, Paltoglou G. Sentiment strength detection for the social web. J Am Soc Inf Sci Technol 2012, 63:163-173.
53. Reckman H, Baird C, Crawford J, Crowell R, Micciulla L, Sethi S, Veress F. teragram: rule-based detection of sentiment phrases using SAS sentiment analysis. In: Second Joint Conference on Lexical and Computational Semantics (*SEM), Atlanta, GA, 2013, 513-519.
54. Zhao J, Dong L, Wu J, Xu K. Moodlens: an emoticon-based sentiment analysis system for Chinese tweets. In: Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 2012, 1528-1531. ACM.
55. Wang S, Manning CD. Baselines and bigrams: simple, good sentiment and topic classification. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, vol 2, Jeju, Republic of Korea, July 2012, 90-94. Association for Computational Linguistics.
56. Paltoglou G, Thelwall M. A study of information retrieval weighting schemes for sentiment analysis. In: Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, Uppsala, Sweden, July 2010, 1386-1395. Association for Computational Linguistics.
57. Nakagawa T, Inui K, Kurohashi S. Dependency tree-based sentiment classification using CRFs with hidden variables. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, June 2010, 786-794. Association for Computational Linguistics.
58. Jiang L, Yu M, Zhou M, Liu X, Zhao T. Target-dependent twitter sentiment classification. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol 1, Portland, Oregon, USA, June 2011, 151-160. Association for Computational Linguistics.
59. Mitchell J, Lapata M. Composition in distributional models of semantics. Cognit Sci 2010, 34:1388-1429.
60. Frege G. On sense and nominatum. In: Feighl H, Sellars W, eds. Readings in Philosophical Aristotelian Society. New York: Appleton-Century-Crofts; 1949, 85-102.
61. Bespalov D, Bai B, Qi Y, Shokoufandeh A. Sentiment classification based on supervised latent n-gram analysis. In: Proceedings of the 20th ACM International Conference on Information and Knowledge Management, Glasgow, United Kingdom, October 2011, 375-382. ACM.
62. Glorot X, Bordes A, Bengio Y. Domain adaptation for large-scale sentiment classification: a deep learning approach. In: Proceedings of the 28th International Conference on Machine Learning (ICML-11), 2011, 513-520.
63. Socher R, Pennington J, Huang EH, Ng AY, Manning CD. Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Edinburgh, Scotland, UK, July 2011, 151-161. Association for Computational Linguistics.
64. Socher R, Huval B, Manning CD, Ng AY. Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Jeju Island, Korea, July 2012, 1201-1211. Association for Computational Linguistics.
65. Paulus R, Socher R, Manning CD. Global belief recursive neural networks. In: Advances in Neural Information Processing Systems, 2014, 2888-2896.
66. Li J. Feature weight tuning for recursive neural networks. 2014, arXiv preprint arXiv:1412.3714.
67. Irsoy O, Cardie C. Deep recursive neural networks for compositionality in language. In: Advances in Neural Information Processing Systems, 2014, 2096-2104.
68. Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K. Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, 2014, 49-54.
69. Hermann KM, Blunsom P. The role of syntax in vector space models of compositional semantics. In: ACL (1), August 2013, 894-904.
70. Xu L, Liu K, Lai S, Chen Y, Zhao J. Mining opinion words and opinion targets in a two-stage framework. In: ACL (1), 2013, 1764-1773.
71. Severyn A, Moschitti A. Twitter sentiment analysis with deep convolutional neural networks. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, August 2015, 959-962. ACM.
72. Kim Y. Convolutional neural networks for sentence classification. 2014, arXiv preprint arXiv:1408.5882.
73. Johnson R., Zhang T. Effective use of word order for text categorization with convolutional neural networks. 2014, arXiv preprint arXiv:1412.1058.
74. Li J, Jurafsky D, Hovy E. When are tree structures necessary for deep learning of representations? 2015, arXiv preprint arXiv:1503.00185.
75. Tai KS, Socher R, Manning CD. Improved semantic representations from tree-structured long short-term memory networks. 2015, arXiv preprint arXiv:1503.00075.
76. Zhu X, Sobhani P, Guo H. Long short-term memory over tree structures. 2015, arXiv preprint arXiv:1503.04881.
77. Le QV, Mikolov T. Distributed representations of sentences and documents. 2014, arXiv preprint arXiv:1405.4053.
78. Li J, Luong MT, Jurafsky D. A hierarchical neural autoencoder for paragraphs and documents. 2015, arXiv preprint arXiv:1506.01057.
79. Tang D, Qin B, Liu T. Document modeling with gated recurrent neural network for sentiment classification. In: EMNLP, 2015, 1422-1432.
80. Bhatia P, Ji Y, Eisenstein J. Better document-level sentiment analysis from RST discourse parsing. 2015, arXiv preprint arXiv:1509.01599.
81. Choi Y, Cardie C, Riloff E, Patwardhan S. Identifying sources of opinions with conditional random fields and extraction patterns. In: Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, Vancouver, British Columbia, Canada, October 2005, 355-362. Association for Computational Linguistics.
82. Breck E, Choi Y, Cardie C. Identifying expressions of opinion in context. In: Proceedings of the Twentieth International Joint Conference on Artificial Intelligence (IJCAI-2007), vol 7, Hyderabad, India, January 2007, 2683-2688.
83. Vo DT, Zhang Y. Target-dependent twitter sentiment classification with rich automatic features. In: Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015), Buenos Aires, Argentina, 2015, 1347-1353.
84. Zhang M, Zhang Y, Vo DT. Neural networks for open domain targeted sentiment. In: Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), Lisbon, Portugal, 2015.
85. Liu P, Joty S, Meng H. Fine-grained opinion mining with recurrent neural networks and word embeddings. In: Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), Lisbon, Portugal, 2015.
86. Wiebe J, Mihalcea R. Word sense and subjectivity. In: Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, Sydney, Australia, July 2006, 1065-1072. Association for Computational Linguistics.
87. Kim SM, Hovy E. Determining the sentiment of opinions. In: Proceedings of the 20th International Conference on Computational Linguistics, August 2004, 1367. Association for Computational Linguistics.
88. Esuli A, Sebastiani F. Determining the semantic orientation of terms through gloss classification. In: Proceedings of the 14th ACM International Conference on Information and Knowledge Management, October 2005, 617-624. ACM.
89. Rao D, Ravichandran D. Semi-supervised polarity lexicon induction. In: Proceedings of the 12th Conference of the European Chapter of the Association for Computational Linguistics, Athens, Greece, March 2009, 675-682. Association for Computational Linguistics.
90. Feng S, Bose R, Choi Y. Learning general connotation of words using graph-based algorithms. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Edinburgh, Scotland, UK, July 2011, 1092-1103. Association for Computational Linguistics.
91. Feng S, Kang JS, Kuznetsova P, Choi Y. Connotation Lexicon: a dash of sentiment beneath the surface meaning. In: ACL (1), 2013, 1774-1784.
92. Baccianella S, Esuli A, Sebastiani F. SentiWordNet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. In: LREC, vol 10, May 2010, 2200-2204.
93. Esuli A, Sebastiani F. Pageranking wordnet synsets: an application to opinion mining. In: ACL, vol 7, June 2007, 442-431.
94. Su F, Markert K. Subjectivity recognition on word senses via semi-supervised mincuts. In: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, May 2009, 1-9. Association for Computational Linguistics.
95. Hatzivassiloglou V, McKeown KR. Predicting the semantic orientation of adjectives. In: Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and Eighth Conference of the European Chapter of the Association for Computational Linguistics, Madrid, Spain, July 1997, 174-181. Association for Computational Linguistics.
96. Qiu G, Liu B, Bu J, Chen C. Opinion word expansion and target extraction through double propagation. Computat Linguist 2011, 37:9-27.
97. Liu Q, Gao Z, Liu B, Zhang Y. Automated rule selection for aspect extraction in opinion mining. In: International Joint Conference on Artificial Intelligence (IJCAI), 2015.
98. Velikovich L, Blair-Goldensohn S, Hannan K, McDonald R. The viability of web-derived polarity lexicons. In: Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Los Angeles, California, USA, June 2010, 777-785. Association for Computational Linguistics.
99. Chen L, Wang W, Nagarajan M, Wang S, Sheth AP. Extracting diverse sentiment expressions with target-dependent polarity from Twitter. In: ICWSM, June 2012.
100. Severyn A, Moschitti A. On the automatic learning of sentiment lexicons. In: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics (NAACL HLT 2015), Denver, Colorado, USA, 2015.
101. Wilson T, Wiebe J, Hoffmann P. Recognizing contextual polarity in phrase-level sentiment analysis. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing, Vancouver, B.C., Canada, October 2005, 347-354. Association for Computational Linguistics.
102. Tang D, Qin B, Liu T. Learning semantic representations of users and products for document level sentiment classification. In: Proceedings of the ACL, 2015.
103. Tang D, Qin B, Liu T, Yang Y. User modeling with neural network for review rating prediction. In: Proceedings of the IJCAI, Buenos Aires, Argentina, 2015.
104. Tang D, Wei F, Qin B, Dong L, Liu T, Zhou M. A joint segmentation and classification framework for sentiment analysis. In: Proceedings of EMNLP, Doha, Qatar, 2014.
105. Tang D, Qin B, Wei F, Dong L, Liu T, Zhou M. A joint segmentation and classification framework for sentence level sentiment classification. IEEE Trans Audio Speech Lang Process 2015, 23:1750-1761.
106. Dong L, Wei F, Liu S, Zhou M, Xu K. A statistical parsing framework for sentiment classification. Comput Linguist 2015, 41:293-336.
107. Smith NA. Linguistic structure prediction. Synth Lect Human Lang Technol 2011, 4:1-274.
108. Li J, Chen X, Luong MT, Jurafsky D. Visualizing and understanding neural models in NLP. 2015, arXiv preprint arXiv:1506.01066.