A network approach to topic models

Science Advances

Volumn 4, Issue 7, 2018, Pages

  Gerlach, Martin ^a,b   Peixoto, Tiago P ^c,d   Altmann, Eduardo G ^b,e  

^a Northwestern University   (United States)

^b MAX PLANCK INSTITUT FÜR PHYSIK KOMPLEXER SYSTEME   (Germany)

^c University of Bath   (United Kingdom)

^d ISI FOUNDATION   (Italy)

^e UNIVERSITY OF SYDNEY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

COMPLEX NETWORKS; LEARNING SYSTEMS; POPULATION DYNAMICS; STATISTICS; STOCHASTIC SYSTEMS;

COLLECTION OF DOCUMENTS; COMMUNITY DETECTION; CROSS FERTILIZATION; LATENT DIRICHLET ALLOCATIONS; MACHINE LEARNING APPROACHES; STATISTICAL MODELING; STATISTICAL PROPERTIES; STOCHASTIC BLOCK MODELS;

STOCHASTIC MODELS;

ARTICLE; CONTROLLED STUDY; CROSS FERTILIZATION; STOCHASTIC MODEL; TOPICAL DRUG ADMINISTRATION;

EID: 85050675107     PISSN: None     EISSN: 23752548     Source Type: Journal    
DOI: 10.1126/sciadv.aaq1360     Document Type: Article

References (61)

1. D. M. Blei, Probabilistic topic models. Commun. ACM 55, 77–84 (2012).
2. S. Deerwester, S. T. Dumais, G. W. Furnas, T. K. Landauer, R. Harshman, Indexing by latent semantic analysis. J. Am. Soc. Inf. Sci. 41, 391–407 (1990).
3. Z. Ghahramani, Probabilistic machine learning and artificial intelligence. Nature 521, 452–459 (2015).
4. T. Hofmann, Probabilistic latent semantic indexing, in Proceedings of the 22nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR’99), Berkeley, CA, 15 to 19 August 1999, pp. 50–57.
5. D. M. Blei, A. Y. Ng, M. I. Jordan, Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003).
6. T. L. Griffiths, M. Steyvers, Finding scientific topics. Proc. Natl. Acad. Sci. U.S.A. 101, 5228–5235 (2004).
7. C. D. Manning, P. Raghavan, H. Schütze, Introduction to Information Retrieval (Cambridge Univ. Press, 2008).
8. K. W. Boyack, D. Newman, R. J. Duhon, R. Klavans, M. Patek, J. R. Biberstine, B. Schijvenaars, A. Skupin, N. Ma, K. Börner, Clustering more than two million biomedical publications: Comparing the accuracies of nine text-based similarity approaches. PLOS ONE 6, e18029 (2011).
9. D. S. McNamara, Computational methods to extract meaning from text and advance theories of human cognition. Top. Cogn. Sci. 3, 3–17 (2011).
10. J. Grimmer, B. M. Stewart, Text as data: The promise and pitfalls of automatic content analysis methods for political texts. Polit. Anal. 21, 267–297 (2013).
11. B. Liu, L. Liu, A. Tsykin, G. J. Goodall, J. E. Green, M. Zhu, C. H. Kim, J. Li, Identifying functional miRNA–mRNA regulatory modules with correspondence latent Dirichlet allocation. Bioinformatics 26, 3105–3111 (2010).
12. J. K. Pritchard, M. J. Stephens, P. J. Donnelly, Inference of population structure using multilocus genotype data. Genetics 155, 945–959 (2000).
13. L. Fei-Fei, P. Perona, A Bayesian hierarchical model for learning natural scene categories, in IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2005 (CVPR’05), San Diego, CA, 20 to 25 June 2005, vol. 2, pp. 524–531.
14. E. G. Altmann, M. Gerlach, Statistical laws in linguistics, in Creativity and Universality in Language, M. Degli Esposti, E. G. Altmann, F. Pachet, Eds. (Springer, 2016), pp. 7–26.
15. G. K. Zipf, The Psycho-Biology of Language (Routledge, 1936).
16. A. Lancichinetti, M. I. Sirer, J. X. Wang, D. Acuna, K. Körding, L. A. N. Amaral, A high-reproducibility and high-accuracy method for automated topic classification. Phys. Rev. X 5, 011007 (2015).
17. T. L. Griffiths, M. Steyvers, D. M. Blei, J. B. Tenenbaum, Integrating topics and syntax, in Advances in Neural Information Processing Systems 17, L. K. Saul, Y. Weiss, L. Bottou, Eds. (MIT Press, 2005), pp. 537–544.
18. W. Li, A. McCallum, Pachinko allocation: DAG-structured mixture models of topic correlations, in Proceedings of the 23rd International Conference on Machine Learning (ICML’06), Pittsburgh, PA, 25 to 29 June 2006, pp. 577–584.
19. M. Rosen-Zvi, T. L. Griffiths, M. Steyvers, P. Smyth, The author-topic model for authors and documents, in Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence (UAI’04), Banff, Canada, 7 to 11 July 2004, pp. 487–494.
20. G. Doyle, C. Elkan, Accounting for burstiness in topic models, in Proceedings of the 26th Annual International Conference on Machine Learning (ICML’09), Montreal, Canada, 14 to 18 June 2009, pp. 281–288.
21. W. Zhao, J. J. Chen, R. Perkins, Z. Liu, W. Ge, Y. Ding, W. Zou, A heuristic approach to determine an appropriate number of topics in topic modeling. BMC Bioinformatics 16, S8 (2015).
22. H. M. Wallach, I. Murray, R. Salakhutdinov, D. Mimno, Evaluation methods for topic models, in Proceedings of the 26th Annual International Conference on Machine Learning (ICML’09), Montreal, Canada, 14 to 18 June 2009, pp. 1105–1112.
23. Y. W. Teh, M. I. Jordan, M. J. Beal, D. M. Blei, Hierarchical Dirichlet processes. J. Am. Stat. Assoc. 101, 1566–1581 (2006).
24. D. M. Blei, T. L. Griffiths, M. I. Jordan, The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies. J. ACM 57, 7 (2010).
25. J. Paisley, C. Wang, D. M. Blei, M. I. Jordan, Nested hierarchical Dirichlet processes. IEEE Trans. Pattern Anal. Mach. Intell. 37, 256–270 (2015).
26. E. B. Sudderth, M. I. Jordan, Shared segmentation of natural scenes using dependent Pitman-Yor processes, in Advances in Neural Information Processing Systems 21 (NIPS 2008), D. Koller, D. Schuurmans, Y. Bengio, L. Bottou, Eds. (Curran Associates Inc., 2009), pp. 1585–1592.
27. I. Sato, H. Nakagawa, Topic models with power-law using Pitman-Yor process, in Proceedings of the 16th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’10), Washington, DC, 25 to 28 July 2010, pp. 673–682.
28. W. L. Buntine, S. Mishra, Experiments with non-parametric topic models, in Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’14), New York, NY, 24 to 27 August 2014, pp. 881–890.
29. T. Broderick, L. Mackey, J. Paisley, M. I. Jordan, Combinatorial clustering and the beta negative binomial process. IEEE Trans. Pattern Anal. Mach. Intell. 37, 290–306 (2015).
30. M. Zhou, L. Carin, Negative binomial process count and mixture modeling. IEEE Trans. Pattern Anal. Mach. Intell. 37, 307–320 (2015).
31. S. Fortunato, Community detection in graphs. Phys. Rep. 486, 75–174 (2010).
32. E. M. Airoldi, D. M. Blei, S. E. Fienberg, E. P. Xing, Mixed membership stochastic blockmodels. J. Mach. Learn. Res. 9, 1981–2014 (2008).
33. B. Ball, B. Karrer, M. E. J. Newman, Efficient and principled method for detecting communities in networks. Phys. Rev. E 84, 036103 (2011).
34. M. E. J. Newman, M. Girvan, Finding and evaluating community structure in networks. Phys. Rev. E 69, 026113 (2004).
35. R. Guimerà, M. Sales-Pardo, L. A. N. Amaral, Modularity from fluctuations in random graphs and complex networks. Phys. Rev. E 70, 025101 (2004).
36. A. Lancichinetti, S. Fortunato, Limits of modularity maximization in community detection. Phys. Rev. E 84, 066122 (2011).
37. P. W. Holland, K. B. Laskey, S. Leinhardt, Stochastic blockmodels: First steps. Soc. Networks 5, 109–137 (1983).
38. B. Karrer, M. E. J. Newman, Stochastic blockmodels and community structure in networks. Phys. Rev. E 83, 016107 (2011).
39. E. M. Airoldi, D. M. Blei, E. A. Erosheva, S. E. Fienberg, Eds., Handbook of Mixed Membership Models and Their Applications (CRC Press, 2014).
40. T. P. Peixoto, Hierarchical block structures and high-resolution model selection in large networks. Phys. Rev. X 4, 011047 (2014).
41. T. P. Peixoto, Model selection and hypothesis testing for large-scale network models with overlapping groups. Phys. Rev. X 5, 011033 (2015).
42. T. P. Peixoto, Nonparametric Bayesian inference of the microcanonical stochastic block model. Phys. Rev. E 95, 012317 (2017).
43. T. P. Peixoto, Parsimonious module inference in large networks. Phys. Rev. Lett. 110, 148701 (2013).
44. M. Gerlach, E. G. Altmann, Stochastic model for the vocabulary growth in natural languages. Phys. Rev. X 3, 021006 (2013).
45. J. Rissanen, Modeling by shortest data description. Automatica 14, 465–471 (1978).
46. R. Arun, V. Suresh, C. E. V. Madhavan, M. N. N. Murthy, On finding the natural number of topics with latent Dirichlet allocation: Some observations, in Advances in Knowledge Discovery and Data Mining, M. J. Zaki, J. X. Yu, B. Ravindran, V. Pudi, Eds. (Springer, 2010), pp. 391–402.
47. J. Cao, T. Xia, J. Li, Y. Zhang, S. Tang, A density-based method for adaptive LDA model selection. Neurocomputing 72, 1775–1781 (2009).
48. A. Schoffield, M. Måns, D. Mimno, Pulling out the stops: Rethinking stopword removal for topic models, in Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Valencia, Spain, 3 to 7 April 2017, vol. 2, pp. 432–436.
49. A. Decelle, F. Krzakala, C. Moore, L. Zdeborová, Inference and phase transitions in the detection of modules in sparse networks. Phys. Rev. Lett. 107, 065701 (2011).
50. D. Hu, P. Ronhovde, Z. Nussinov, Phase transitions in random Potts systems and the community detection problem: Spin-glass type and dynamic perspectives. Philos. Mag. 92, 406–445 (2012).
51. T. P. Peixoto, Inferring the mesoscale structure of layered, edge-valued, and time-varying networks. Phys. Rev. E 92, 042807 (2015).
52. M. E. J. Newman, A. Clauset, Structure and inference in annotated networks. Nat. Commun. 7, 11863 (2016).
53. D. Hric, T. P. Peixoto, S. Fortunato, Network structure, metadata, and the prediction of missing nodes and annotations. Phys. Rev. X 6, 031038 (2016).
54. O. T. Courtney, G. Bianconi, Dense power-law networks and simplicial complexes. Phys. Rev. E 97, 052303 (2018).
55. R. N. Mantegna, S. V. Buldyrev, A. L. Goldberger, S. Havlin, C.-K. Peng, M. Simons, H. E. Stanley, Linguistic features of noncoding DNA sequences. Phys. Rev. Lett. 73, 3169–3172 (1994).
56. R. E. Kass, A. E. Raftery, Bayes factors. J. Am. Stat. Assoc. 90, 773–795 (1995).
57. T. Vallès-Català, T. P. Peixoto, R. Guimerà, M. Sales-Pardo, Consistencies and inconsistencies between model selection and link prediction in networks. Phys. Rev. E 97, 026316 (2018).
58. H. M. Wallach, D. M. Mimno, A. McCallum, Rethinking LDA: Why priors matter, in Advances in Neural Information Processing Systems 22 (NIPS 2009), Y. Bengio, D. Schuurmans, J. D. Lafferty, C. K. I. Williams, A. Culotta, Eds. (Curran Associates Inc., 2009), pp. 1973–1981.
59. A. Asuncion, M. Welling, P. Smyth, Y. W. Teh, On smoothing and inference for topic models, in Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence (UAI’09), Montreal, Canada, 18 to 21 June 2009, pp. 27–34.
60. E. G. Altmann, J. B. Pierrehumbert, A. E. Motter, Niche as a determinant of word fate in online groups. PLOS ONE 6, e19009 (2011).
61. M. Gerlach, thesis, Technical University Dresden, Dresden, Germany (2016).