Improved neural network-based multi-label classification with better initialization leveraging label co-occurrence

2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2016 - Proceedings of the Conference

Volumn , Issue , 2016, Pages 521-526

  Kurata, Gakuto ^a   Xiang, Bing ^b   Zhou, Bowen ^b  

^a IBM RESEARCH   (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL LINGUISTICS; MULTILAYER NEURAL NETWORKS; NEURONS; TEXT PROCESSING;

CLASSIFICATION ACCURACY; COMPUTATIONAL OVERHEADS; INITIALIZATION METHODS; MULTI LABEL CLASSIFICATION; MULTI-LABEL TEXT CLASSIFICATION; MULTIPLE LABELS; NATURAL LANGUAGE QUERIES; NOVEL NEURAL NETWORK;

CLASSIFICATION (OF INFORMATION);

EID: 84994108543     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.18653/v1/n16-1063     Document Type: Conference Paper

References (20)

1. Chih-Chung Chang and Chih-Jen Lin. 2011. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology, 2(3):27:1-27:27.
2. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proc. ICML, pages 160-167. ACM.
3. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493-2537.
4. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. The Journal of Machine Learning Research, 12:2121-2159.
5. Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proc. AISTATS, pages 249-256.
6. Rie Johnson and Tong Zhang. 2015. Effective use of word order for text categorization with convolutional neural networks. In Proc. NAACL-HLT, pages 103-112.
7. Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proc. EMNLP, pages 1746-1751.
8. Quoc V Le, Navdeep Jaitly, and Geoffrey E Hinton. 2015. A simple way to initialize recurrent networks of rectified linear units. arXiv preprint arXiv:1504.00941.
9. David D Lewis, Yiming Yang, Tony G Rose, and Fan Li. 2004. RCV1: A new benchmark collection for text categorization research. The Journal of Machine Learning Research, 5:361-397.
10. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In Proc. ICLR.
11. Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proc. ICML, pages 807-814.
12. Jinseok Nam, Jungi Kim, Eneldo Loza Mencía, Iryna Gurevych, and Johannes Fürnkranz. 2014. Large-scale multi-label text classification - revisiting neural networks. In Proc. ECML-PKDD, pages 437-452.
13. Jesse Read. 2008. A pruned problem transformation method for multi-label classification. In Proc. NZC-SRS, pages 143-150.
14. Timothy N Rubin, America Chambers, Padhraic Smyth, and Mark Steyvers. 2012. Statistical topic models for multi-label document classification. Machine learning, 88(1-2):157-208.
15. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929-1958.
16. Ilya Sutskever, James Martens, George Dahl, and Geoffrey Hinton. 2013. On the importance of initialization and momentum in deep learning. In Proc. ICML, pages 1139-1147.
17. Grigorios Tsoumakas, Ioannis Katakis, and Ioannis Vlahavas. 2010. Mining multi-label data. In Data mining and knowledge discovery handbook, pages 667-685. Springer.
18. Wen-tau Yih, Xiaodong He, and Christopher Meek. 2014. Semantic parsing for single-relation question answering. In Proc. ACL, pages 643-648.
19. Min-Ling Zhang and Zhi-Hua Zhou. 2006. Multilabel neural networks with applications to functional genomics and text categorization. IEEE Transactions on Knowledge and Data Engineering, 18(10):1338-1351.
20. Min-Ling Zhang and Zhi-Hua Zhou. 2014. A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, 26(8):1819-1837.