Learning word dependencies in text by means of a deep recurrent belief network

Knowledge-Based Systems

Volumn 108, Issue , 2016, Pages 144-154

  Chaturvedi, Iti ^a   Ong, Yew Soon ^a   Tsang, Ivor W ^b   Welsch, Roy E ^c   Cambria, Erik ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

^b UNIVERSITY OF TECHNOLOGY SYDNEY   (Australia)

^c MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Deep belief networks; Gaussian networks; Markov Chain Monte Carlo; Time delays; Variable order

Indexed keywords

CHAINS; GAUSSIAN DISTRIBUTION; MARKOV PROCESSES; MONTE CARLO METHODS; TIME DELAY;

DEEP BELIEF NETWORKS; DISTRIBUTED TIME DELAYS; DYNAMIC MOVEMENTS; GAUSSIAN BAYESIAN NETWORKS; GAUSSIAN NETWORKS; MARKOV CHAIN MONTE-CARLO; REAL-WORLD SCENARIO; VARIABLE ORDER;

BAYESIAN NETWORKS;

EID: 84981722706     PISSN: 09507051     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.knosys.2016.07.019     Document Type: Article

References (40)

1. [1] Friedman, N., Nachman, I., Gaussian process networks. Proceedings of the Sixteenth Conference Annual Conference on Uncertainty in Artificial Intelligence (UAI-00), 2000, 211–219.
2. [2] Heckerman, D., Geiger, D., Chickering, D.M., Learning bayesian networks: the combination of knowledge and statistical data. Mach. Learn. 20:3 (1995), 197–243.
3. [3] Wang, Z., Liu, Y., Liu, X., State estimation for jumping recurrent neural networks with discrete and distributed delays. Neural Networks 22:1 (2009), 41–48.
4. [4] Cambria, E., Huang, G.-B., et al. Extreme learning machines. IEEE Intell. Syst. 28:6 (2013), 30–59.
5. [5] Huang, G.-B., Cambria, E., Toh, K.-A., Widrow, B., Xu, Z., New trends of learning in computational intelligence. IEEE Comput. Intell. Mag. 10:2 (2015), 16–17.
6. [6] Galley, M., A skip-chain conditional random field for ranking meeting utterances by importance. Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing (EMNLP 2006), 2006, 364–372.
7. [7] Bock, S., Schedl, M., Polyphonic piano note transcription with recurrent neural networks. Acoustics, Speech and Signal Processing (ICASSP), 2012 IEEE International Conference on, 2012, 121–124.
8. [8] Brueckner, R., Schulter, B., Social signal classification using deep blstm recurrent neural networks. Acoustics, Speech and Signal Processing (ICASSP), 2014 IEEE International Conference on, 2014, 4823–4827.
9. [9] Sutskever, I., Martens, J., Hinton, G., Generating text with recurrent neural networks. Proceedings of the 28th International Conference on Machine Learning (ICML-11) ICML ’11, 2011, 1017–1024.
10. [10] Chen, M., Xu, Z., Weinberger, K., Sha, F., Marginalized denoising autoencoders for domain adaptation. Proceedings of the 29th International Conference on Machine Learning (ICML-12) ICML ’12, 2012, 767–774.
11. [11] Glorot, X., Bordes, A., Bengio, Y., Domain adaptation for large-scale sentiment classification: a deep learning approach. In Proceedings of the Twenty-eight International Conference on Machine Learning, ICML, 2011.
12. [12] Schmidhuber, J., Deep learning in neural networks: An overview. Neural Networks 61:0 (2015), 85–117.
13. [13] Scherer, D., Müller, A., Behnke, S., Evaluation of pooling operations in convolutional architectures for object recognition. Proceedings of the 20th International Conference on Artificial Neural Networks: Part III ICANN’10, 2010, 92–101.
14. [14] Taylor, G.W., Hinton, G.E., Roweis, S.T., Modeling human motion using binary latent variables. Schölkopf, B., Platt, J., Hoffman, T., (eds.) Advances in Neural Information Processing Systems 19, 2007, MIT Press, Cambridge, MA, 1345–1352.
15. [15] Hinton, G.E., Salakhutdinov, R.R., Reducing the dimensionality of data with neural networks. Science 313:5786 (2006), 504–507, 10.1126/science.1127647.
16. [16] Hinton, G.E., Training products of experts by minimizing contrastive divergence. Neural Comput. 14:8 (2002), 1771–1800.
17. [17] Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., Kingsbury, B., Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. Signal Processing Mag. IEEE 29:6 (2012), 82–97.
18. [18] Mohamed, A., Dahl, G., Hinton, G., Acoustic modeling using deep belief networks. Audio Speech Lang. Process. IEEE Trans. 20:1 (2012), 14–22.
19. [19] Lee, H., Grosse, R., Ranganath, R., Ng, A.Y., Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. Proceedings of the 26th Annual International Conference on Machine Learning ICML ’09, 2009, 609–616.
20. [20] Cambria, E., Hussain, A., Havasi, C., Eckl, C., Sentic computing: exploitation of common sense for the development of emotion-sensitive systems. Esposito, A., Campbell, N., Vogel, C., Hussain, A., Nijholt, A., (eds.) Development of Multimodal Interfaces: Active Listening and Synchrony Lecture Notes in Computer Science, 2010, Springer, Berlin, 148–156.
21. [21] Glorot, X., Bordes, A., Bengio, Y., Domain adaptation for large-scale sentiment classification: a deep learning approach. ICML, vol. 27, 2011, 97–110.
22. [22] Cambria, E., Mazzocco, T., Hussain, A., Eckl, C., Sentic medoids: organizing affective common sense knowledge in a multi-dimensional vector space. Liu, D., Zhang, H., Polycarpou, M., Alippi, C., He, H., (eds.) Advances in Neural Networks Lecture Notes in Computer Science, vol. 6677, 2011, Springer-Verlag, Berlin, 601–610.
23. [23] Rajapakse, J., Chaturvedi, I., Gene regulatory networks with variable-order dynamic bayesian networks. Neural Networks (IJCNN), The 2010 International Joint Conference on, 2010, 1–5.
24. [24] Bee, M., Modelling credit default swap spreads by means of normal mixtures and copulas. Appl. Math. Finance 11:2 (2004), 125–146.
25. [25] Wikipedia, Stanford classifier, 20 newsgroup dataset @ONLINE, 2012. URL http://nlp.stanford.edu/wiki/Software/Classifier.
26. [26] Cambria, E., Affective computing and sentiment analysis. IEEE Intell. Syst. 31:2 (2016), 102–107.
27. [27] Poria, S., Gelbukh, A., Cambria, E., Hussain, A., Huang, G.-B., EmoSenticSpace: a novel framework for affective common-sense reasoning. Knowledge Based Syst. 69 (2014), 108–123.
28. [28] Cambria, E., Hussain, A., Durrani, T., Havasi, C., Eckl, C., Munro, J., Sentic computing for patient centered application. IEEE ICSP, Beijing, 2010, 1279–1282.
29. [29] Cambria, E., Schuller, B., Liu, B., Wang, H., Havasi, C., Statistical approaches to concept-level sentiment analysis. IEEE Intell. Syst. 28:3 (2013), 6–9.
30. [30] Prinzie, A., Van den Poel, D., Dynamic Bayesian Networks for Acquisition Pattern Analysis: A Financial-Services Cross-Sell Application New Frontiers in Applied Data Mining. Lecture Notes in Computer Science, vol. 5433, 2009, Springer Berlin / Heidelberg, 123–133.
31. [31] Rencher, A.C., Methods of Multivariate Analysis. 2nd, 2002, Wiley.
32. [32] Castillo, E., Gutierrez, J.M., Hadi, A.S., Expert Systems and Probabilistic Network Models. 1996, Springer-Verlag New York, Inc.
33. [33] Wagner, J., Stolovitzky, G., Stability and time-delay modeling of negative feedback loops. Proc. IEEE 96:8 (2008), 1398–1410.
34. [34] Friedman, N., Murphy, K., Russell, S., Learning the structure of dynamic probabilistic networks. Proceedings of the 14th Annual Conference on Uncertainty in Artificial Intelligence (UAI-98), 1998 139–14.
35. [35] Dahl, G., Ranzato, M.A, Mohamed, A.R, Hinton, G.E., Phone recognition with the mean-covariance restricted boltzmann machine. Advances in Neural Information Processing Systems 23, 2010, 469–477.
36. [36] Tsungnan, L., Horne, B.G., Tino, P., Giles, C.L., Learning long-term dependencies in narx recurrent neural networks. Neural Networks IEEE Trans. 7:6 (1996), 1329–1338.
37. [37] Gers, F., Schmidhuber, J., Lstm recurrent networks learn simple context-free and context-sensitive languages. Neural Networks IEEE Trans. 12:6 (2001), 1333–1340.
38. [38] Chow Jia Yi, K.Y.H., Chaturvedi, I., An analysis of herding behaviours in basketball as a function of skill level. ECSS, 2013.
39. [39] Chen, H.-T., Chou, C.-L., Fu, T.-S., Lee, S.-Y., Lin, B.-S. P., Recognizing tactic patterns in broadcast basketball video using player trajectory. J. Visual Commun. Image Represent. 23:6 (2012), 932–947.
40. [40] Chaturvedi, I., Ong, Y.-S., Arumugam, R.V., Deep transfer learning for classification of time-delayed gaussian networks. Signal Process. 110:0 (2015), 250–262.