Learning distributed word representation with multi-contextual mixed embedding

Knowledge-Based Systems

Volumn 106, Issue , 2016, Pages 220-230

  Li, Jianqiang ^a   Li, Jing ^a   Fu, Xianghua ^a   Masud, M A ^a   Huang, Joshua Zhexue ^a  

^a SHENZHEN UNIVERSITY   (China)

Author keywords

Distributed word representation; Natural language processing; Word embedding; Word2vec

Indexed keywords

CLASSIFICATION (OF INFORMATION); COMPUTATIONAL LINGUISTICS; DATA MINING; INFORMATION RETRIEVAL SYSTEMS; LINGUISTICS; SEMANTICS;

DISTRIBUTIONAL MODELS; DOCUMENT CLASSIFICATION; NATURAL LANGUAGE PROCESSING; NATURAL LANGUAGE PROCESSING APPLICATIONS; SEMANTIC INFORMATION; WORD EMBEDDING; WORD REPRESENTATIONS; WORD2VEC;

NATURAL LANGUAGE PROCESSING SYSTEMS;

EID: 84973917891     PISSN: 09507051     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.knosys.2016.05.045     Document Type: Article

References (69)

1. E. Cambria Affective computing and sentiment analysis IEEE Intell. Syst. 31 2016 102 107
2. E. Cambria, J. Fu, F. Bisio, S. Poria AffectiveSpace2: enabling affective intuition for concept-level sentiment analysis The Proceedings of 29th AAAI Conference 2015 509 514
3. E. Cambria, B. White Jumping NLP curves: a review of natural language processing research [review article] IEEE Comput. Intell. Mag. 9 2014 48 57
4. Pan W. A survey of transfer learning for collaborative recommendation with auxiliary data Neurocomputing 177 2016 447 453
5. Pan W.; Liu Z.; Ming Z.; Zhong H.; Wang X.; Xu C. Compressed knowledge transfer via factorization machine for heterogeneous collaborative recommendation Knowl. Based Syst. 85 2015 234 244
6. Pan W.; Zhong H.; Xu C.; Ming Z. Adaptive Bayesian personalized ranking for heterogeneous implicit feedbacks Knowl. Based Syst. 73 2015 173 180
7. Wu W.; Li H.; Wang H.; Zhu K.Q. Probase: a probabilistic taxonomy for text understanding Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data Scottsdale, Arizona, USA 2012 481 492
8. H.P. Martinez, Y. Bengio, G.N. Yannakakis Learning deep physiological models of affect IEEE Comput. Intell. Mag. 8 2013 20 33
9. Wang J.; Huang J.Z.; Guo J.; Lan Y. Recommending high-utility search engine queries via a query-recommending model Neurocomputing 167 2015 195 208
10. D.M. Blei, Ng A.Y.; M.I. Jordan Latent dirichlet allocation J. Mach. Learn. Res. 3 2003 993 1022
11. Fu X.; Li J.; Yang K.; Cui L.; Yang L. Dynamic Online HDP model for discovering evolutionary topics from Chinese social texts Neurocomputing 171 2016 412 424
12. Fu X.; Liu G.; Guo Y.; Wang Z. Multi-aspect sentiment analysis for Chinese online social reviews based on topic modeling and HowNet lexicon Know. Based Syst. 37 2013 186 195
13. Fu X.; Yang K.; Huang J.Z.; Cui L. Dynamic non-parametric joint sentiment topic mixture model Know. Based Syst. 82 2015 102 114
14. J. Turian, L. Ratinov, Y. Bengio Word representations: a simple and general method for semi-supervised learning Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics Uppsala, Sweden 2010 384 394
15. Z. Harris Distributional Structure Word 10 1954 146 162
16. P.F. Brown, P.V. deSouza, R.L. Mercer, V.J.D. Pietra, J.C. Lai Class-based n-gram models of natural language Comput. Linguist. 18 1992 467 479
17. F. Huang, A. Yates Distributional representations for handling sparsity in supervised sequence-labeling Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP Suntec, Singapore vol. 1 2009 495 503
18. M. Baroni, A. Lenci Distributional memory: a general framework for corpus-based semantics Comput. Linguist. 36 2010 673 721
19. P.D. Turney, P. Pantel From frequency to meaning: vector space models of semantics J. Artif. Int. Res. 37 2010 141 188
20. C.D. Santos, B. Zadrozny Learning character-level representations for part-of-speech tagging Proceedings of the 31st International Conference on Machine Learning (ICML-14) 2014 1818 1826
21. R. Socher, Lin C.C.; C. Manning, Ng A.Y. Parsing natural scenes and natural language with recursive neural networks Proceedings of the 28th International Conference on Machine Learning (ICML-11) 2011 129 136
22. R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, P. Kuksa Natural language processing (almost) from scratch J. Mach. Learn. Res. 12 2011 2493 2537
23. R. Socher, J. Pennington, Huang E.H.; Ng A.Y.; C.D. Manning Semi-supervised recursive autoencoders for predicting sentiment distributions Proceedings of the Conference on Empirical Methods in Natural Language Processing 2011 151 161
24. R. Socher, Huang E.H.; J. Pennin, C.D. Manning, Ng A.Y. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection Advances in Neural Information Processing Systems 2011 801 809
25. Kyunghyun Cho, B.v. Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, Y. Bengio Learning phrase representations using RNN encoder-decoder for statistical machine translation EMNLP 2014 1724 1734
26. Y. Bengio, R. Ducharme, P. Vincent, C. Janvin A neural probabilistic language model J. Mach. Learn. Res. 3 2003 1137 1155
27. R. Collobert, J. Weston A unified architecture for natural language processing: deep neural networks with multitask learning Proceedings of the 25th international conference on Machine learning Helsinki, Finland 2008 160 167
28. T. Mikolov, S. Kombrink, L. Burget, J. Cernocky, S. Khudanpur Extensions of recurrent neural network language model Proceedings of ICASSP 2011 2011 5528 5531
29. A. Mnih, G.E. Hinton A scalable hierarchical distributed language model Advances in Neural Information Processing Systems 2009 1081 1088
30. Dahl, G.E.; Adams, R.P.; & Larochelle, H. 2012. Training restricted boltzmann machines on word observations. arXiv preprint arXiv:1202.5695.
31. F. Morin, Y. Bengio Hierarchical probabilistic neural network language model AISTATS'05 2005 246 252
32. A. Mnih, Y.W. Teh A fast and simple algorithm for training neural probabilistic language models J. Langford J. Pineau Proceedings of the 29th International Conference on Machine Learning New York, NY, USA 2012 1751 1758
33. J. Dean, G.S. Corrado, R. Monga, Chen K.; M. Devin, Le Q.V.; Mao M.Z.; A. Senior, P. Tucker, Yang K.; Ng A.Y. Large scale distributed deep networks NIPS 2012 1 11
34. Mikolov, T.; Chen, K.; Corrado, G.; & Dean, J. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781.
35. T. Mikolov, I. Sutskever, Chen K.; G.S. Corrado, J. Dean Distributed representations of words and phrases and their compositionality Advances in Neural Information Processing Systems 2013 3111 3119
36. Huang E.H.; R. Socher, C.D. Manning, Ng A.Y. Improving word representations via global context and multiple word prototypes Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers Jeju Island, Korea vol. 1 2012 873 882
37. J. Pennington, R. Socher, C.D. Manning Glove: global vectors for word representation Proceedings of the Empiricial Methods in Natural Language Processing (EMNLP 2014) 2014 1 12
38. Y. Bengio, H. Schwenk, J.-S. Senécal, F. Morin, J.-L. Gauvain Neural probabilistic language models Innovations in Machine Learning 2006 137 186
39. A. Mnih, G. Hinton Three new graphical models for statistical language modelling Proceedings of the 24th International Conference on Machine Learning Corvalis, Oregon 2007 641 648
40. L. De Vine, G. Zuccon, B. Koopman, L. Sitbon, P. Bruza Medical semantic similarity with a neural language model Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management 2014 1819 1822
41. A.-C. Liang Domain adaptation on personalized neural netowrk based language models Dissertation of National Taiwan University 2014 1 30
42. Mikolov, T.; Le, Q.V.; & Sutskever, I. 2013. Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168.
43. Le, Q.V.; & Mikolov, T. 2014. Distributed representations of sentences and documents. arXiv preprint arXiv:1405.4053.
44. Lai S.; Liu K.; Xu L.; Zhao J. How to generate a good word embedding? arXiv:1507.05523 [cs.CL] Computation and Language 2015
45. B.T. McInnes, T. Pedersen, Liu Y.; G.B. Melton, S.V. Pakhomov U-path: an undirected path-based measure of semantic similarity AMIA Annual Symposium Proceedings 2014 882 891
46. T. Mikolov, Yih W.-t.; G. Zweig Linguistic regularities in continuous space word representations HLT-NAACL 2013 746 751
47. Chen, Y.; Perozzi, B.; Al-Rfou, R.; & Skiena, S. 2013. The expressive power of word embeddings. arXiv preprint arXiv:1301.3226.
48. Fu G.; Wang X. Chinese sentence-level sentiment classification based on fuzzy sets Proceedings of the 23rd International Conference on Computational Linguistics: Posters Beijing, China 2010 312 319
49. Yu M.; M Dredze Improving lexical embeddings with semantic knowledge ACL 2014 1 6
50. E. Arisoy, T.N. Sainath, B. Kingsbury, B. Ramabhadran Deep neural network language models Proceedings of the NAACL-HLT 2012 Workshop: Will We Ever Really Replace the N-gram Model? On the Future of Language Modeling for HLT Montreal, Canada 2012 20 28
51. A. Mnih, G. Hinton A scalable hierarchical distributed language model NIPS'08 2008 1081 1088
52. T. Mikolov, M. Karafiat, J. Cernocky, S. Khudanpur Recurrent neural network based language model Proceedings of the 12th Annual Conference of the International Speech Communication Association 2010 1045 1048
53. Qiu L.; Zhang Y.; Lu Y Syntactic dependencies and distributed word representations for chinese analogy detection and mining EMNLP 2015 2441 2450
54. Jin P.; Wu Y. Semeval-2012 task 4: evaluating chinese word similarity Proceedings of the First Joint Conference on Lexical and Computational Semantics-Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation 2012 374 377
55. K. Dave, S. Lawrence, D.M. Pennock Mining the peanut gallery: opinion extraction and semantic classification of product reviews Proceedings of the 12th international conference on World Wide Web 2003 519 528
56. P.D. Turney Thumbs up or thumbs down?: semantic orientation applied to unsupervised classification of reviews Proceedings of the 40th Annual Meeting on Association for Computational Linguistics 2002 417 424
57. Wu Q.; Tan S.; Cheng X. Graph ranking for sentiment transfer Proceedings of the ACL-IJCNLP 2009 Conference Short Papers 2009 317 320
58. Wu Q.; Tan S.; Duan M.; Cheng X. A two-stage algorithm for domain adaptation with application to sentiment transfer problems Information Retrieval Technology 2010 443 453
59. Zhang X.; Wang S.; Xu M.; Yin Y. Chinese text sentiment classification based on granule network Granular Computing, 2009, GRC'09. IEEE International Conference on 2009 775 778
60. A.L. Maas, R.E. Daly, P.T. Pham, Huang D.; Ng A.Y.; C. Potts Learning word vectors for sentiment analysis Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies vol. 1 2011 142 150
61. Le Q.; T. Mikolov Distributed representation of sentences and documents Proceedings of the 31 st International Conference on Machine Learning Beijing, China 2014
62. Tang D. Sentiment-specific representation learning for document-level sentiment analysis Proceedings of the Eighth ACM International Conference on Web Search and Data Mining Shanghai, China 2015 447 452
63. J. Bullinaria, J. Levy Extracting semantic representations from word co-occurrence statistics: A computational study Behavior Research Methods 39 2007 510 526
64. R. Cohen, Y. Goldberg, M. Elhadad Domain adaptation of a dependency parser with a class-class selectional preference model Proceedings of ACL 2012 Student Research Workshop. Publishing Jeju Island, Korea 2012 43 48
65. S. Miller, J. Guinness, A. Zamanian Name Tagging with Word Clusters and Discriminative Training HLT-NAACL Boston, Massachusetts 2004 337 342
66. P. Liang Semi-supervised learning for natural language Disserstation Massachusetts Institute of Technology 2005
67. T. Koo, X. Carreras Pérez, M. Collins Simple semi-supervised dependency parsing Proceedings of 46th Annual Meeting of the ACL Columbus, Ohio 2008 595 603
68. L. Ratinov, D. Roth Design challenges and misconceptions in named entity recognition Proceedings of the 13th Conference on Computational Natural Language Learning of ACL Suntec, Singapore 2009 147 155
69. T. Mikolov, I. Sutskever, K. Chen, G.S. Corrado, J. Dean Distributed representations of words and phrases and their compositionality Advances in Neural Information Processing Systems, Publishing 2013 3111 3119