Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN

Expert Systems with Applications

Volumn 72, Issue , 2017, Pages 221-230

  Chen, Tao ^a   Xu, Ruifeng ^a,b   He, Yulan ^c   Wang, Xuan ^a  

^a HARBIN INSTITUTE OF TECHNOLOGY   (China)

^b Guangdong Provincial Pet Engineering Technology Research Center   (China)

^c ASTON UNIVERSITY   (United Kingdom)

Author keywords

Deep neural network; Natural language processing; Sentiment analysis

Indexed keywords

BENCHMARKING; COMPLEX NETWORKS; DATA MINING; NATURAL LANGUAGE PROCESSING SYSTEMS; NEURAL NETWORKS;

CONVOLUTIONAL NEURAL NETWORK; DEEP NEURAL NETWORKS; DIVIDE-AND-CONQUER APPROACH; NATURAL LANGUAGE PROCESSING; SENTIMENT ANALYSIS; SENTIMENT CLASSIFICATION; SEQUENCE MODELING; TYPE CLASSIFICATIONS;

CLASSIFICATION (OF INFORMATION);

EID: 85006717242     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2016.10.065     Document Type: Article

References (80)

1. Álvarez López, T., Juncal-Martínez, J., Fernández-Gavilanes, M., Costa-Montenegro, E., González-Castaño, F.J., GTI at SemEval-2016 task 5: Svm and crf for aspect detection and unsupervised aspect-based sentiment analysis. Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), 2016, Association for Computational Linguistics, San Diego, California, 306–311 URL: http://www.aclweb.org/anthology/S16-1049.
2. Appel, O., Chiclana, F., Carter, J., Fujita, H., A hybrid approach to the sentiment analysis problem at the sentence level. Knowledge-Based Systems 108 (2016), 110–124.
3. Baccianella, S., Esuli, A., Sebastiani, F., SentiWordNet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. Proceedings of the international conference on language resources and evaluation (LREC), vol. 10, 2010, 2200–2204.
4. Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., A neural probabilistic language model. Journal of Machine Learning Research 3 (2003), 1137–1155.
5. Brun, C., Perez, J., Roux, C., XRCE at SemEval-2016 task 5: Feedbacked ensemble modeling on syntactico-semantic knowledge for aspect based sentiment analysis. Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), 2016, Association for Computational Linguistics, San Diego, California, 277–281 URL: http://www.aclweb.org/anthology/S16-1044.
6. Carrillo-de Albornoz, J., Plaza, L., An emotion-based model of negation, intensifiers, and modality for polarity and intensity classification. Journal of the American Society for Information Science and Technology 64:8 (2013), 1618–1633.
7. Çetin, F.S., Yıldırım, E., Özbey, C., Eryiğit, G., TGB at SemEval-2016 task 5: Multi-lingual constraint system for aspect based sentiment analysis. Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), 2016, Association for Computational Linguistics, San Diego, California, 337–341 URL: http://www.aclweb.org/anthology/S16-1054.
8. Chaturvedi, I., Ong, Y.-S., Tsang, I.W., Welsch, R.E., Cambria, E., Learning word dependencies in text by means of a deep recurrent belief network. Knowledge-Based Systems 108 (2016), 144–154.
9. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P., Natural language processing (almost) from scratch. The Journal of Machine Learning Research 12 (2011), 2493–2537.
10. Crystal, D., Dictionary of linguistics and phonetics. vol. 30, 2011, John Wiley & Sons.
11. Dong, L., Wei, F., Tan, C., Tang, D., Zhou, M., Xu, K., Adaptive recursive neural network for target-dependent twitter sentiment classification. Proceedings of the 52nd annual meeting of the association for computational linguistics (ACL), 2014, Association for Computational Linguistics, Baltimore, Maryland, 49–54.
12. Elman, J.L., Finding structure in time. Cognitive Science 14:2 (1990), 179–211.
13. Fernández-Gavilanes, M., Álvarez-López, T., Juncal-Martínez, J., Costa-Montenegro, E., González-Castaño, F.J., Unsupervised method for sentiment analysis in online texts. Expert Systems with Applications 58 (2016), 57–75.
14. Ganapathibhotla, M., Liu, B., Mining opinions in comparative sentences. Proceedings of the 22nd international conference on computational linguistics (COLING), vol. 1, 2008, Association for Computational Linguistics, 241–248.
15. González-Ibánez, R., Muresan, S., Wacholder, N., Identifying sarcasm in twitter: a closer look. Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies (ACL), vol. 2, 2011, Association for Computational Linguistics, 581–586.
16. Graves, A., Mohamed, A.-r., Hinton, G., Speech recognition with deep recurrent neural networks. IEEE International conference on acoustics, speech and signal processing (ICASSP), 2013, IEEE, 6645–6649.
17. Hatzivassiloglou, V., Wiebe, J.M., Effects of adjective orientation and gradability on sentence subjectivity. Proceedings of the 18th conference on computational linguistics (COLING), vol. 1, 2000, Association for Computational Linguistics, 299–305.
18. Hercig, T., Brychcín, T., Svoboda, L., Konkol, M., UWB at SemEval-2016 task 5: Aspect based sentiment analysis. Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), 2016, Association for Computational Linguistics, San Diego, California, 342–349 URL: http://www.aclweb.org/anthology/S16-1055.
19. Hinton, G.E., Srivastava, N., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.R., Improving neural networks by preventing co-adaptation of feature detectors. Computing Research Repository (CoRR), abs/1207.0580, 2012 URL: http://arxiv.org/abs/1207.0580.
20. Hochreiter, S., Schmidhuber, J., Long short-term memory. Neural Computation 9:8 (1997), 1735–1780.
21. Hu, M., Liu, B., Mining and summarizing customer reviews. Proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining (KDD), 2004, ACM, 168–177.
22. Huang, Z., Xu, W., & Yu, K. (2015). Bidirectional lstm-crf models for sequence tagging. arXiv preprint arXiv: 1508.01991.
23. Irsoy, O., Cardie, C., Deep recursive neural networks for compositionality in language. Advances in neural information processing systems 27: annual conference on neural information processing systems (NIPS), 2014, 2096–2104.
24. Irsoy, O., Cardie, C., Opinion mining with deep recurrent neural networks. Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, 720–728.
25. Jia, L., Yu, C., Meng, W., The effect of negation on sentiment analysis and retrieval effectiveness. Proceedings of the 18th ACM conference on information and knowledge management (CIKM), 2009, ACM, 1827–1830.
26. Jiang, L., Yu, M., Zhou, M., Liu, X., Zhao, T., Target-dependent twitter sentiment classification. Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies (ACL), vol. 1, 2011, Association for Computational Linguistics, 151–160.
27. Jindal, N., Liu, B., Identifying comparative sentences in text documents. Proceedings of the 29th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR), 2006, ACM, 244–251.
28. Jindal, N., Liu, B., Mining comparative sentences and relations. Proceedings, the 21st national conference on artificial intelligence and the 18th innovative applications of artificial intelligence conference (AAAI), vol. 22, 2006, 1331–1336.
29. Kalchbrenner, N., Grefenstette, E., Blunsom, P., A convolutional neural network for modelling sentences. Proceedings of the 52nd annual meeting of the association for computational linguistics (ACL), 2014, Association for Computational Linguistics, Baltimore, Maryland, 655–665.
30. Kessler, W., Kuhn, J., A corpus of comparisons in product reviews. In proceedings of the 9th language resources and evaluation conference (LREC), Reykjavik, Iceland, 2014, 2242–2248 URL: http://www.lrec-conf.org/proceedings/lrec2014/pdf/1001_Paper.pdf.
31. Kim, Y., Convolutional neural networks for sentence classification. Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, 1746–1751.
32. Kumar, A., Kohail, S., Kumar, A., Ekbal, A., Biemann, C., IIT-TUDA at SemEval-2016 task 5: Beyond sentiment lexicon: Combining domain dependency and distributional semantics features for aspect based sentiment analysis. Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), 2016, Association for Computational Linguistics, San Diego, California, 1129–1135 URL: http://www.aclweb.org/anthology/S16-1174.
33. Lafferty, J., McCallum, A., Pereira, F.C., Conditional random fields: Probabilistic models for segmenting and labeling sequence data. Proceedings of the eighteenth international conference on machine learning (ICML), Williams College, Williamstown, MA, USA, June 28, - July 1, 2001, 2001, 282–289.
34. Lample, G., Ballesteros, M., Subramanian, S., Kawakami, K., & Dyer, C. (2016). Neural architectures for named entity recognition. arXiv preprint arXiv: 1603.01360.
35. Le, Q., Mikolov, T., Distributed representations of sentences and documents. Proceedings of the 31th international conference on machine learning (ICML), 2014, 1188–1196.
36. Liu, B., Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies, 2012, Morgan & Claypool Publishers, 10.2200/S00416ED1V01Y201204HLT016.
37. Liu, B., Sentiment analysis: Mining opinions, sentiments, and emotions. 2015, Cambridge University Press.
38. Liu, P., Qiu, X., & Huang, X. (2016). Recurrent neural network for text classification with multi-task learning. arXiv preprint arXiv: 1605.05101.
39. Liu, K., Xu, L., Zhao, J., Extracting opinion targets and opinion words from online reviews with graph co-ranking. Proceedings of the 52nd annual meeting of the association for computational linguistics (ACL), 2014, Association for Computational Linguistics, Baltimore, Maryland, 314–324.
40. Liu, Y., Yu, X., Chen, Z., Liu, B., Sentiment analysis of sentences with modalities. Proceedings of the 2013 international workshop on mining unstructured big data using natural language processing (UnstructureNLP@CIKM), 2013, ACM, 39–44.
41. McCallum, A., Efficiently inducing features of conditional random fields. Proceedings of the 19th conference on uncertainty in artificial intelligence (UAI), 2003, Morgan Kaufmann, 403–410.
42. Mitchell, M., Aguilar, J., Wilson, T., Durme, B.V., Open domain targeted sentiment. Proceedings of the 2013 conference on empirical methods in natural language processing, EMNLP 2013, 18–21 October 2013, Grand Hyatt Seattle, Seattle, Washington, USA, A meeting of SIGDAT, a Special Interest Group of the ACL, 2013, 1643–1654.
43. Muhammad, A., Wiratunga, N., Lothian, R., Contextual sentiment analysis for social media genres. Knowledge-Based Systems 108 (2016), 92–101.
44. Nakagawa, T., Inui, K., Kurohashi, S., Dependency tree-based sentiment classification using CRFs with hidden variables. Human language technologies: The 2010 annual conference of the North American chapter of the association for computational linguistics (NAACL), 2010, Association for Computational Linguistics, 786–794.
45. Narayanan, R., Liu, B., Choudhary, A., Sentiment analysis of conditional sentences. Proceedings of the 2009 conference on empirical methods in natural language processing (EMNLP), vol. 1, 2009, Association for Computational Linguistics, 180–189.
46. Okazaki, N. (2007). Crfsuite: a fast implementation of conditional random fields (CRFs).
47. Pang, B., Lee, L., A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. Proceedings of the 42nd annual meeting on association for computational linguistics (ACL), 2004, Association for Computational Linguistics, 271–278.
48. Pang, B., Lee, L., Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. Proceedings of the 43rd annual meeting on association for computational linguistics (ACL), 2005, Association for Computational Linguistics, 115–124.
49. Pang, B., Lee, L., Vaithyanathan, S., Thumbs up?: sentiment classification using machine learning techniques. Proceedings of the 2002 conference on empirical methods in natural language processing (EMNLP), 2002, 79–86.
50. Park, M., Yuan, Y., Linguistic knowledge-driven approach to chinese comparative elements extraction. Proceedings of the 8th SIGHAN workshop on chinese language processing (SIGHAN-8), 2015, Association for Computational Linguistics, 79–85.
51. Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., AL-Smadi, M., Al-Ayyoub, M., Zhao, Y., Qin, B., Clercq, O.D., Hoste, V., Apidianaki, M., Tannier, X., Loukachevitch, N., Kotelnikov, E., Bel, N., Jimnez-Zafra, S.M., Eryiit, G., SemEval-2016 task 5: Aspect based sentiment analysis. Proceedings of the 10th international workshop on semantic evaluation (SemEval) SemEval ’16, 2016, Association for Computational Linguistics, San Diego, California, 19–30.
52. Popescu, A., Etzioni, O., Extracting product features and opinions from reviews. Proceedings of the 2005, human language technology conference and conference on empirical methods in natural language processing (HLT/EMNLP), 2005, Springer, 9–28.
53. Poria, S., Cambria, E., Gelbukh, A., Aspect extraction for opinion mining with a deep convolutional neural network. Knowledge-Based Systems 108 (2016), 42–49.
54. Poria, S., Cambria, E., Winterstein, G., Huang, G.-B., Sentic patterns: Dependency-based rules for concept-level sentiment analysis. Knowledge-Based Systems 69 (2014), 45–63.
55. Qiu, G., Liu, B., Bu, J., Chen, C., Opinion word expansion and target extraction through double propagation. Computational Linguistics 37:1 (2011), 9–27, 10.1162/coli_a_00034.
56. Ravi, K., Ravi, V., A survey on opinion mining and sentiment analysis: tasks, approaches and applications. Knowledge-Based Systems 89 (2015), 14–46.
57. Rill, S., Reinel, D., Scheidt, J., Zicari, R.V., Politwi: Early detection of emerging political topics on twitter and the impact on concept-level sentiment analysis. Knowledge-Based Systems 69 (2014), 24–33.
58. Riloff, E., Qadir, A., Surve, P., De Silva, L., Gilbert, N., Huang, R., Sarcasm as contrast between a positive sentiment and negative situation. Proceedings of the 2013 conference on empirical methods on natural language processing (EMNLP), 2013, Association for Computational Linguistics, 704–714.
59. Riloff, E., Wiebe, J., Learning extraction patterns for subjective expressions. Proceedings of the 2003 conference on empirical methods in natural language processing (emnlp), 2003, Association for Computational Linguistics, 105–112.
60. Schuster, M., Paliwal, K.K., Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing 45:11 (1997), 2673–2681.
61. Socher, R., Huval, B., Manning, C.D., Ng, A.Y., Semantic compositionality through recursive matrix-vector spaces. Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL), 2012, ACL, 1201–1211.
62. Socher, R., Pennington, J., Huang, E.H., Ng, A.Y., Manning, C.D., Semi-supervised recursive autoencoders for predicting sentiment distributions. Proceedings of the conference on empirical methods in natural language processing, 2011, Association for Computational Linguistics, 151–161.
63. Socher, R., Perelygin, A., Wu, J.Y., Chuang, J., Manning, C.D., Ng, A.Y., Potts, C., Recursive deep models for semantic compositionality over a sentiment treebank. Proceedings of the 2013 conference on empirical methods in natural language processing (EMNLP), 2013, Citeseer, 1631–1642.
64. Stoyanov, V., Cardie, C., Topic identification for fine-grained opinion analysis. Proceedings of the 22nd international conference on computational linguistics (COLING), vol. 1, 2008, Association for Computational Linguistics, 817–824.
65. Tai, K. S., Socher, R., & Manning, C. D. (2015). Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv: 1503.00075.
66. Tang, D., Qin, B., Feng, X., & Liu, T. (2015a). Target-dependent sentiment classification with long short term memory. arXiv preprint arXiv: 1512.01100.
67. Tang, D., Qin, B., Liu, T., Deep learning for sentiment analysis: Successful approaches and future challenges. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 5:6 (2015), 292–303, 10.1002/widm.1171 URL: http://dx.doi.org/10.1002/widm.1171.
68. Tang, D., Qin, B., Wei, F., Dong, L., Liu, T., Zhou, M., A joint segmentation and classification framework for sentence level sentiment classification. IEEE/ACM Transactions on Audio, Speech, and Language Processing 23:11 (2015), 1750–1761, 10.1109/TASLP.2015.2449071.
69. Toh, Z., Su, J., NLANGP at SemEval-2016 task 5: Improving aspect based sentiment analysis using neural network features. Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), 2016, Association for Computational Linguistics, San Diego, California, 282–288 URL: http://www.aclweb.org/anthology/S16-1045.
70. Tsur, O., Davidov, D., Rappoport, A., Icwsm-a great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews. Proceedings of the 4th international conference on weblogs and social media (ICWSM), 2010, 162–169.
71. Turney, P.D., Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews. Proceedings of the 40th annual meeting on association for computational linguistics (acl), 2002, Association for Computational Linguistics, 417–424.
72. Vo, D.-T., Zhang, Y., Target-dependent twitter sentiment classification with rich automatic features. Proceedings of the twenty-fourth international joint conference on artificial intelligence (IJCAI), 2015, 1347–1353.
73. Wang, S., Manning, C.D., Baselines and bigrams: Simple, good sentiment and topic classification. Proceedings of the 50th annual meeting of the association for computational linguistics (ACL), vol. 2, 2012, Association for Computational Linguistics, 90–94.
74. Wiebe, J.M., Bruce, R.F., O'Hara, T.P., Development and use of a gold-standard data set for subjectivity classifications. Proceedings of the 37th annual meeting of the association for computational linguistics (ACL), 1999, Association for Computational Linguistics, 246–253.
75. Wiebe, J., Wilson, T., Learning to disambiguate potentially subjective expressions. Proceedings of the 6th conference on natural language learning (CoNLL), 2002, Association for Computational Linguistics, 1–7.
76. Wiebe, J., Wilson, T., Cardie, C., Annotating expressions of opinions and emotions in language. Language Resources and Evaluation 39:2–3 (2005), 165–210.
77. Yang, S., Ko, Y., Extracting comparative entities and predicates from texts using comparative type classification. Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies (ACL), vol. 1, 2011, Association for Computational Linguistics, 1636–1644.
78. Yu, H., Hatzivassiloglou, V., Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences. Proceedings of the 2003 conference on empirical methods in natural language processing (EMNLP), 2003, Association for Computational Linguistics, 129–136.
79. Zhang, L., Ferrari, S., Enjalbert, P., Opinion analysis: the effect of negation on polarity and intensity. KONVENS workhop PATHOS-1st workshop on practice and theory of opinion mining and sentiment analysis, 2012, 282–290.
80. Zhang, M., Zhang, Y., Vo, D.-T., Neural networks for open domain targeted sentiment. Proceedings of the 2015 conference on empirical methods in natural language processing (EMNLP), 2015, 612–621.