Graph-induced restricted Boltzmann machines for document modeling

Information Sciences

Volumn 328, Issue , 2016, Pages 60-75

  Nguyen, Tu Dinh ^a   Tran, Truyen ^a,b   Phung, Dinh ^a   Venkatesh, Svetha ^a  

^a DEAKIN UNIVERSITY   (Australia)

^b CURTIN UNIVERSITY   (Australia)

Author keywords

Document modeling; Feature group discovery; Restricted Boltzmann machine; Topic coherence; Word graphs

Indexed keywords

ARTIFICIAL INTELLIGENCE; CLASSIFICATION (OF INFORMATION); GRAPHIC METHODS; LEARNING SYSTEMS; LINGUISTICS; SPEECH RECOGNITION;

CLASSIFICATION ACCURACY; DOCUMENT MODEL; FEATURE GROUPS; GRAPH-BASED REGULARIZATIONS; INTRINSIC DIMENSIONALITIES; PROBABILISTIC FORMULATION; RESTRICTED BOLTZMANN MACHINE; WORD GRAPHS;

DATA MINING;

EID: 84945529896     PISSN: 00200255     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ins.2015.08.023     Document Type: Article

References (83)

1. D. Andrzejewski, X. Zhu, M. Craven, and B. Recht A framework for incorporating general domain knowledge into latent dirichlet allocation using first-order logic Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence 2 2011 AAAI Press 1171 1177
2. M. Belkin, and P. Niyogi Laplacian eigenmaps for dimensionality reduction and data representation Neural Comput. 15 6 2003 1373 1396
3. Y. Bengio Learning deep architectures for AI Found. Trends Mach. Learning 2 1 2009 1 127 (Also published as a book. Now Publishers, 2009).
4. Y. Bengio, A. Courville, and P. Vincent Representation learning: a review and new perspectives IEEE Trans. Pattern Anal. Mach. Intell. 35 8 2013 1798 1828
5. Y. Bengio, and L. Yao Bounding the test log-likelihood of generative models arXiv:1311.6184 2013
6. M.S. Bernstein, B. Suh, L. Hong, J. Chen, S. Kairam, and E.H. Chi Eddi: interactive topic-based browsing of social status streams Proceedings of the 23nd Annual ACM Symposium on User interface Software and Technology 2010 ACM 303 312
7. D.M. Blei Probabilistic topic models Commun. ACM 55 4 2012 77 84
8. D.M. Blei, T.L. Griffiths, and M.I. Jordan The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies J. ACM 57 2 2010 7
9. D.M. Blei, A.Y. Ng, and M.I. Jordan Latent Dirichlet allocation J. Mach. Learning Res. 3 2003 993 1022 (Printed)
10. H.D. Bondell, and B.J. Reich Simultaneous regression shrinkage, variable selection, and supervised clustering of predictors with oscar Biometrics 64 1 2008 115 123
11. Y.-L. Boureau, F. Bach, Y. LeCun, and J. Ponce Learning mid-level features for recognition IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2010 IEEE 2559 2566
12. D. Cai, X. He, and J. Han Document clustering using locality preserving indexing IEEE Trans. Knowledge Data Eng. 17 12 2005 1624 1637
13. S.S. Chen, D.L. Donoho, and M.A. Saunders Atomic decomposition by basis pursuit SIAM J. Sci. Comput. 20 1 1998 33 61
14. Z. Chen, A. Mukherjee, B. Liu, M. Hsu, M. Castellanos, and R. Ghosh Leveraging multi-domain prior knowledge in topic models Proceedings of the International Joint Conference in Artificial Intelligence (IJCAI) 2013
15. F.R. Chung Spectral Graph Theory 92 1997 American Mathematical Soc.
16. A. Coates, and A. Ng The importance of encoding versus training with sparse coding and vector quantization Proceedings of the 28th International Conference on Machine Learning (ICML) 2011 921 928
17. A. Coates, and A.Y. Ng Learning feature representations with k-means Neural Networks: Tricks of the Trade 2012 Springer 561 580
18. R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa Natural language processing (almost) from scratch J. Mach. Learning Res. 12 2011 2493 2537
19. G.E. Dahl, R.P. Adams, and H. Larochelle Training restricted Boltzmann machines on word observations CoRR abs/1202.5695 2012
20. D.L. Donoho, and M. Elad Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization Proc. Nat. Acad. Sci. 100 5 2003 2197 2202
21. O. Dor, and Y. Reich Strengthening learning algorithms by feature discovery Informat. Sci. 189 2012 176 190
22. H. Fei, B. Quanz, and J. Huan Regularization and feature selection for networked features Proceedings of the 19th ACM International Conference on Information and Knowledge Management 2010 ACM 1893 1896
23. M.J. Gardner, J. Lutes, J. Lund, J. Hansen, D. Walker, E. Ringger, and K. Seppi The topic browser: an interactive tool for browsing topic models NIPS Workshop on Challenges of Data Visualization 2 2010
24. T. Griffiths, Z. Ghahramani, Infinite latent feature models and the Indian buffet process (2005).
25. F. Guo, and E.P. Xing Bayesian exponential family harmoniums Technical Report CMU-ML-06-103 2006 CMU
26. R. Hadsell, S. Chopra, and Y. LeCun Dimensionality reduction by learning an invariant mapping IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) 2 2006 IEEE 1735 1742
27. X. He, and P. Niyogi Locality preserving projections S. Thrun, L. Saul, B. Schölkopf, Advances in Neural Information Processing Systems 16 2004 MIT Press 153 160
28. G. Hinton, L. Deng, D. Yu, G.E. Dahl, A.-R. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T.N. Sainath, and et al. Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups IEEE Signal Process. Mag. 29 6 2012 82 97
29. G. Hinton, and R. Salakhutdinov Reducing the dimensionality of data with neural networks Science 313 5786 2006 504 507
30. G.E. Hinton Training products of experts by minimizing contrastive divergence Neural Comput. 14 8 2002 1771 1800
31. G.E. Hinton A practical guide to training restricted Boltzmann machines Montavon Neural Networks: Tricks of the Trade Lecture Notes in Computer Science 7700 2012 Springer Berlin Heidelberg 599 619
32. T. Hofmann Unsupervised learning by probabilistic latent semantic analysis Mach. Learning 42 1-2 2001 177 196
33. G. Hripcsak, and D.J. Albers Next-generation phenotyping of electronic health records J. Amer. Med. Informat. Assoc. 2012 amiajnl 2012
34. J. Huang, S. Ma, H. Xie, and C.-H. Zhang A group bridge approach for variable selection Biometrika 96 2 2009 339 355
35. A. Hyvärinen, and E. Oja Independent component analysis: algorithms and applications Neural Netw. 13 4 2000 411 430
36. R. Jenatton, G. Obozinski, and F. Bach Structured sparse principal component analysis International Conference on Artificial Intelligence and Statistics 2010 366 373
37. D. Kauchak Improving text simplification language modeling using unsimplified text data The 51st Annual Meeting of the Association for Computational Linguistics (ACL) 2013
38. A. Krizhevsky, I. Sutskever, and G. Hinton Imagenet classification with deep convolutional neural networks Advances in Neural Information Processing Systems 25 (NIPS) 2012 1106 1114 (Printed)
39. J.T. Kwok, and R.P. Adams Priors for diversity in generative latent variable models Advances in Neural Information Processing Systems (NIPS) 2012 3005 3013
40. D.D. Lee, and H.S. Seung Learning the parts of objects by non-negative matrix factorization Nature 401 6755 1999 788 791
41. H. Lee, R. Grosse, R. Ranganath, and A.Y. Ng Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations Proceedings of the 26th Annual International Conference on Machine Learning 2009 ACM 609 616
42. C. Li, and H. Li Network-constrained regularization and variable selection for analysis of genomic data Bioinformatics 24 9 2008 1175 1182
43. D. Mimno, H.M. Wallach, E. Talley, M. Leenders, and A. McCallum Optimizing semantic coherence in topic models Proceedings of the Conference on Empirical Methods in Natural Language Processing 2011 Association for Computational Linguistics 262 272
44. R.M. Neal Markov chain sampling methods for Dirichlet process mixture models J. Comput. Graphical Stat. 9 2 2000 249 265
45. D. Newman, E. Bonilla, and W. Buntine Improving topic coherence with regularized topic models Adv. Neural Informat. Process. Syst. 2011
46. T.D. Nguyen, T. Tran, D. Phung, and S. Venkatesh Latent patient profile modelling and applications with mixed-variate restricted Boltzmann machine J. Pei, V. Tseng, L. Cao, H. Motoda, G. Xu, Advances in Knowledge Discovery and Data Mining Lecture Notes in Computer Science 7818 2013 Springer-Verlag Berlin Heidelberg 123 135
47. T.D. Nguyen, T. Tran, D. Phung, and S. Venkatesh Learning parts-based representations with nonnegative restricted Boltzmann machine Proceedings of Asian Conference on Machine Learning (ACML), Canberra, Australia 2013 133 148
48. T.D. Nguyen, T. Tran, D. Phung, and S. Venkatesh Learning sparse latent representation and distance metric for image retrieval Proceedings of the IEEE International Conference on Multimedia and Expo (ICME) 2013 1 6
49. B.A. Olshausen, and et al. Emergence of simple-cell receptive field properties by learning a sparse code for natural images Nature 381 6583 1996 607 609
50. T. Pedersen, S. Patwardhan, and J. Michelizzi Wordnet::similarity: measuring the relatedness of concepts Demonstration Papers at HLT-NAACL 2004 HLT-NAACL-Demonstrations '04 2004 Association for Computational Linguistics Stroudsburg, PA, USA 38 41
51. M. Ranzato, and G.E. Hinton Modeling pixel means and covariances using factorized third-order Boltzmann machines IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2010 2010 IEEE 2551 2558
52. Y. Rao, Q. Li, X. Mao, and L. Wenyin Sentiment topic models for social emotion mining Informat. Sci. 266 2014 90 100
53. A. Rodriguez, D.B. Dunson, and A.E. Gelfand The nested Dirichlet process J. Amer. Stat. Assoc. 103 483 2008
54. S.T. Roweis, and L.K. Saul Nonlinear dimensionality reduction by locally linear embedding Science 290 5500 2000 2323 2326
55. R. Salakhutdinov, and G. Hinton Semantic hashing SIGIR Workshop on Information Retrieval and Applications of Graphical Models 500 2007 ACM Special Interest Group on Information Retrieva
56. R. Salakhutdinov, and G. Hinton Replicated softmax: an undirected topic model Adv. Neural Informat. Process. Syst. 22 2009 1607 1614
57. R. Salakhutdinov, A. Mnih, and G. Hinton Restricted Boltzmann machines for collaborative filtering Proceedings of the 24th International Conference on Machine Learning 2007 ACM 791 798
58. R. Salakhutdinov, and G. Hinton Deep Boltzmann machines Proceedings of the International Conference on Artificial Intelligence and Statistics 5 2009 448 455
59. T. Sandler, J. Blitzer, P.P. Talukdar, and L.H. Ungar Regularized learning with networks of features Proceedings of Advances in Neural Information Processing Systems 2009 1401 1408
60. D.B. Sharma, H.D. Bondell, and H.H. Zhang Consistent group identification and variable selection in regression with correlated predictors J. Comput. Graphical Stat. 22 2 2013 319 340
61. P. Smolensky Information processing in dynamical systems: Foundations of harmony theory Parallel Distributed Processing: Explorations in the Microstructure of Cognition 1 1986 194 281
62. R. Socher, B. Huval, C.D. Manning, and A.Y. Ng Semantic compositionality through recursive matrix-vector spaces Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning 2012 Association for Computational Linguistics 1201 1211
63. N. Srivastava, and R. Salakhutdinov Multimodal learning with deep Boltzmann machines Proceedings of Advances in Neural Information Processing Systems 25 (NIPS) 2012 2231 2239
64. N. Srivastava, R. Salakhutdinov, and G. Hinton Modeling documents with a deep Boltzmann machine UAI 2013
65. A.D. Szlam, K. Gregor, and Y.L. Cun Structured sparse coding via lateral inhibition Proceedings of Advances in Neural Information Processing Systems 2011 1116 1124
66. Y.W. Teh, M.I. Jordan, M.J. Beal, and D.M. Blei Hierarchical Dirichlet processes J. Amer. Stat. Assoc. 101 476 2006 1566 1581
67. R. Tibshirani Regression shrinkage and selection via the lasso J. Royal Stat. Soc. Series B (Methodological) 1996 267 288
68. T. Tieleman, and G. Hinton Using fast weights to improve persistent contrastive divergence Proceedings of the 26th Annual International Conference on Machine Learning 2009 ACM 1033 1040 http://onlinelibrary.wiley.com/doi/10.1111/j.1467-9868.2011.00771.x/full
69. N. Tomasev, M. Radovanović, D. Mladenić, and M. Ivanović The role of hubness in clustering high-dimensional data Advances in Knowledge Discovery and Data Mining 2011 Springer 183 195
70. T. Tran, D. Phung, and S. Venkatesh Ordinal Boltzmann machines for collaborative filtering Proceedings of 25th Conference on Uncertainty in Artificial Intelligence (UAI), Montreal, Canada 2009
71. T. Tran, D. Phung, and S. Venkatesh Learning Boltzmann distance metric for face recognition Proceedings of IEEE International Conference on Multimedia & Expo (ICME), Melbourne, Australia 2012
72. T. Tran, D. Phung, and S. Venkatesh Mixed-Variate Restricted Boltzmann Machines Proceedings of 3rd Asian Conference on Machine Learning (ACML) 2011
73. T. Tran, D. Phung, and S. Venkatesh Thurstonian Boltzmann machines: learning from multiple inequalities Proceedings of the 30th International Conference on Machine Learning (ICML) 2013
74. J. Turian, L. Ratinov, and Y. Bengio Word representations: a simple and general method for semi-supervised learning Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL) 2010 Association for Computational Linguistics 384 394
75. P.D. Turney, P. Pantel, and et al. From frequency to meaning: Vector space models of semantics J. Artificial Intell. Res. 37 1 2010 141 188
76. L. Van der Maaten, and G. Hinton Visualizing data using t-SNE J. Mach. Learning Res. 9 Nov 2008 2579 2605
77. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol Stacked denoising autoencoders: learning useful representations in a deep network with a local denoising criterion J. Mach. Learning Res. 11 2010 3371 3408
78. M. Welling, and S. Parise Bayesian random fields: the bethe-laplace approximation arXiv:1206.6868 2012
79. D.M. Witten, A. Shojaie, and F. Zhang The cluster elastic net for high-dimensional regression with unknown variable grouping Technometrics 56 1 2014 112 122
80. L. Younes Parametric inference for imperfectly observed Gibbsian fields Probability Theory Related Fields 82 4 1989 625 645
81. M. Yuan, and Y. Lin Model selection and estimation in regression with grouped variables J. Royal Stat. Soc.: Series B (Stat. Methodol.) 68 1 2006 49 67
82. L. Zeng, and J. Xie Group variable selection for data with dependent structures J. Stat. Comput. Simulation 82 1 2012 95 106
83. Z. Zhai, B. Liu, H. Xu, and P. Jia Constrained LDA for grouping product features in opinion mining Advances in knowledge discovery and data mining 2011 Springer 448 459