Navigating the local modes of big data: The case of topic models

Computational Social Science: Discovery and Prediction

Volumn , Issue , 2016, Pages 51-97

  Roberts, Margaret E ^a   Stewart, Brandon M ^b   Tingley, Dustin ^c  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b PRINCETON UNIVERSITY   (United States)

^c HARVARD UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 85047352520     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/9781316257340.004     Document Type: Chapter

References (117)

1. Airoldi, Edoardo M., David M. Blei, Stephen E. Fienberg, and Eric P. Xing. 2009. "Mixed membership stochastic blockmodels." In Advances in Neural Information Processing Systems. pp. 33-40.
2. Anandkumar, Animashree, Rong Ge, Daniel Hsu, and Sham M. Kakade. 2014. "A tensor approach to learning mixed membership community models." Journal of Machine Learning Research 15:2239-2312. http://jmlr.org/papers/v15/anandkumar14a.html.
3. Anandkumar, Animashree, Rong Ge, Daniel Hsu, Sham M. Kakade, and Matus Telgarsky. 2014. "Tensor decompositions for learning latent variable models." Journal of Machine Learning Research 15:2773-2832. http://jmlr.org/papers/v15/ anandkumar14b.html.
4. Anandkumar, Animashree, Yikai Liu, Daniel J. Hsu, Dean P. Foster, and Sham M. Kakade. 2012. "A spectral algorithm for latent Dirichlet allocation." In Advances in neural information processing systems. pp. 917-925.
5. Anandkumar, Animashree, Daniel Hsu, Adel Javanmard, and Sham Kakade. 2013. "Learning linear bayesian networks with latent variables." In Proceedings of the 30th International Conference on Machine Learning. pp. 249-257.
6. Anandkumar, Animashree, Daniel Hsu, and Sham M. Kakade. 2012. "A method of moments for mixture models and hidden Markov models." arXiv preprint arXiv:1203.0683.
7. Arora, Sanjeev, Rong Ge, Yonatan Halpern, David Mimno, Ankur Moitra, David Sontag, Yichen Wu, and Michael Zhu. 2013. "A practical algorithm for topic modeling with provable guarantees." In Proceedings of the 30th International Conference on Machine Learning. pp. 280-288.
8. Arora, Sanjeev, Rong Ge, Ravindran Kannan, and Ankur Moitra. 2012. "Computing a nonnegative matrix factorization-provably." In Proceedings of the Forty-Fourth Annual ACM Symposium on Theory of Computing. ACM. pp. 145-162.
9. Arora, Sanjeev, Aditya Bhaskara, Rong Ge, and Tengyu Ma. 2014. "Provable bounds for learning some deep representations." In Proceedings of the 31st International Conference on Machine Learning. 32. pp. 584-592.
10. Arora, Sanjeev, Rong Ge, and Ankur Moitra. 2012. "Learning topic models - going beyond SVD." In Foundations of Computer Science (FOCS), 2012 IEEE. 53rd Annual Symposium on. IEEE. pp. 1-10.
11. Arora, Sanjeev, Rong Ge, and Ankur Moitra. 2014. "New algorithms for learning incoherent and overcomplete dictionaries." In Proceedings of The 27th Conference on Learning Theory. 35. pp. 779-806.
12. Arora, Sanjeev, Rong Ge, Ankur Moitra, and Sushant Sachdeva. 2012. "Provable ICA with unknown Gaussian noise, with implications for Gaussian mixtures and autoen-coders." In Advances in Neural Information Processing Systems. pp. 2375-2383.
13. Arthur, David and Sergei Vassilvitskii. 2007. "k-means++: The advantages of careful seeding." In Proceedings of the Eighteenth Annual ACM-SIAM Symposium on Discrete Algorithms. Society for Industrial and Applied Mathematics. pp. 1027-1035.
14. Asuncion Jr., Arthur Uy. 2011. "Distributed and accelerated inference algorithms for probabilistic graphical models." Technical report. California State University at Long Beach.
15. Bahmani, Bahman, Benjamin Moseley, Andrea Vattani, Ravi Kumar, and Sergei Vas- silvitskii. 2012. "Scalable k-means++." Proceedings of the VLDB Endowment 5(7):622-633.
16. Beck, Nathaniel, Gary King, and Langche Zeng. 2000. "Improving quantitative studies of international conflict: A conjecture." American Political Science Review 94:21-36.
17. Belloni, Alexandre, Victor Chernozhukov, and Christian Hansen. 2014. "High-dimensional methods and inference on structural and treatment effects." Journal of Economic Perspectives 28(2):29-50.
18. Bishop, Christopher M., et al. 2006. Pattern recognition and machine learning. Vol. 1. Springer: New York.
19. Blei, David M. 2012. "Probabilistic topic models." Communications of the ACM 55(4):77-84.
20. Blei, David M. 2014. "Build, compute, critique, repeat: Data analysis with latent variable models." Annual Review of Statistics and Its Application 1:203-232.
21. Blei, David M, Andrew Y. Ng, and Michael I. Jordan. 2003. "Latent Dirichlet allocation." Journal of Machine Learning Research 3:993-1022.
22. Boyd, Stephen and Lieven Vandenberghe. 2009. Convex optimization. Cambridge University Press.
23. Carpenter, Bob. 2010. "Integrating out multinomial parameters in latent Dirichlet allocation and naive Bayes for collapsed Gibbs sampling." Technical report. LingPipe.
24. Chaney, Allison June-Barlow and David M. Blei. 2012. "Visualizing topic models." In International Conference on Web and Social Media.
25. Chang, Jonathan. 2012. lda: Collapsed Gibbs sampling methods for topic models. R package version 1.3.2. http://CRAN.R-project.org/packageJda.
26. Chang, Jonathan, Sean Gerrish, Chong Wang, Jordan L. Boyd-Graber, and David M. Blei. 2009. "Reading tea leaves: How humans interpret topic models." In Advances in Neural Information Processing Systems. pp. 288-296.
27. Chuang, Jason, Christopher D. Manning, and Jeffrey Heer. 2012. "Termite: Visualization techniques for assessing textual topic models." In Proceedings of the International Working Conference on Advanced Visual Interfaces. ACM. pp. 74-77.
28. Chuang, Jason, Sonal Gupta, Christopher Manning, and Jeffrey Heer. 2013. "Topic model diagnostics: Assessing domain relevance via topical alignment." In Proceedings of the 30th International Conference on Machine Learning (ICML-13). pp. 612-620.
29. Cochocki, A. and Rolf Unbehauen. 1993. Neural networks for optimization and signal processing. John Wiley & Sons.
30. Dean, Jeffrey and Sanjay Ghemawat. 2008. "MapReduce: Simplified data processing on large clusters." Communications of the ACM 51(1):107-113.
31. Deb, Partha and Pravin K. Trivedi. 2002. "The structure of demand for health care: Latent class versus two-part models." Journal of Health Economics 21(4):601-625.
32. de Klerk, Etienne and Dmitrii V. Pasechnik. 2002. "Approximation of the stability number of a graph via copositive programming." SIAM Journal on Optimization 12(4):875-892.
33. De Marchi, Scott, Christopher Gelpi, and Jeffrey D. Grynaviski. 2004. "Untangling neural nets." American Political Science Review 98(2):371-378.
34. Dempster, Arthur P., Nan M. Laird, and Donald B. Rubin. 1977. "Maximum likelihood from incomplete data via the EM algorithm." Journal of the Royal Statistical Society. Series B (Methodological). pp. 1-38.
35. Ding, Weicong, Prakash Ishwar, Mohammad H. Rohban, and Venkatesh Saligrama. 2013. "Necessary and sufficient conditions for novel word detection in separable topic models." arXiv preprint arXiv:1310.7994.
36. Ding, Weicong, Mohammad H. Rohban, Prakash Ishwar, and Venkatesh Saligrama. 2013. "Topic discovery through data dependent and random projections." ar Xiv preprint ar Xiv:1303.3664.
37. Donoho, David and Victoria Stodden. 2003. "When does non-negative matrix factorization give a correct decomposition into parts?" In Advances in Neural Information Processing Systems. pp. 1141-1148.
38. Doshi-Velez, Finale, Yaorong Ge, and Isaac Kohane. 2014. "Comorbidity clusters in autism spectrum disorders: An electronic health record time-series analysis." Pediatrics 133(1):e54-e63.
39. Drton, Mathias and Thomas S. Richardson. 2004. "Multimodality of the likelihood in the bivariate seemingly unrelated regressions model." Biometrika 91(2): 383-392.
40. DuMouchel, William. 1999. "Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system." American Statistician 53(3):177-190.
41. Efron, Bradley et al. 1978. "The geometry of exponential families." Annals of Statistics 6(2):362-376.
42. Eisenstein, Jacob and Eric Xing. 2010. "The CMU 2008 Political Blog Corpus.".
43. Fan, Jianqing, Fang Han, and Han Liu. 2014. "Challenges of Big Data analysis." National Science Review 1:293-324.
44. Foulds, J. R. and P. Smyth. 2014. "Annealing paths for the evaluation of topic models." In Proceedings of the Thirtieth Conference Conference on Uncertainty in Artificial Intelligence.
45. Gardner, Matthew J., Joshua Lutes, Jeff Lund, Josh Hansen, Dan Walker, Eric Ringger, and Kevin Seppi. 2010. "The topic browser: An interactive tool for browsing topic models." In NIPS Workshop on Challenges of Data Visualization.
46. Gelman, Andrew, John B. Carlin, Hal S. Stern, David B. Dunson, Aki Vehtari, and Donald B. Rubin. 2013. Bayesian data analysis. CRC press.
47. Gillis, Nicolas. 2014. "The why and how of nonnegative matrix factorization." In Regularization, Optimization, Kernels, and Support Vector Machines. J.A.K. Suykens, M. Signoretto and A. Argyriou (eds), Chapman & Hall/CRC, Machine Learning and Pattern Recognition Series. pp. 257-291.
48. Gillis, Nicolas, and Robert, Luce. 2014. "Robust near-separable nonnegative matrix factorization using linear optimization." Journal of Machine Learning Research 15 (Apr). pp. 1249-1280.
49. Goldstone, Andrew and Ted Underwood, et al. 2014. "The Quiet Transformations of Literary Studies: What Thirteen Thousand Scholars Could Tell Us." New Literary History 45, no. 3:359-384.
50. Griffiths, D.M.B.T.L. and M.I.J.J.B. Tenenbaum. 2004. "Hierarchical topic models and the nested Chinese restaurant process." Advances in Neural Information Processing Systems 16:17.
51. Griffiths, Thomas L. and Mark Steyvers. 2004. "Finding scientific topics." Proceedings of the National Academy of Sciences 101(Suppl 1):5228-5235.
52. Grimmer, Justin. 2010a. "A Bayesian hierarchical topic model for political texts: Measuring expressed agendas in Senate press releases." Political Analysis 18(1):1-35.
53. Grimmer, Justin. 2010b. "An introduction to Bayesian inference via variational approx-imations." Political Analysis 19(1):32-47.
54. Grimmer, Justin. 2013. Representational style in Congress: What legislators say and why it matters. Cambridge University Press.
55. Grimmer, Justin and Gary King. 2011. "General purpose computer-assisted clustering and conceptualization." Proceedings of the National Academy of Sciences 108(7):2643-2650.
56. Grimmer, Justin and Brandon M. Stewart. 2013. "Text as data: The promise and pitfalls of automatic content analysis methods for political texts." Political Analysis 21(3): 267-297.
57. Hambleton, Ronald K. 1991. Fundamentals of item response theory. Vol. 2. Sage publications.
58. Hamilton, James D. 1989. "A new approach to the economic analysis of nonstationary time series and the business cycle." Econometrica: Journal of the Econometric Society 57(2): 357-384.
59. Hoff, Peter D., Adrian E. Raftery, and Mark S. Handcock. 2002. "Latent space approaches to social network analysis." Journal of the American Statistical Association 97(460):1090-1098.
60. Hopkins, Daniel J. and Gary King. 2010. "A method of automated nonparametric content analysis for social science." American Journal of Political Science 54(1):229-247.
61. Hornik, Kurt. 2005. "A clue for cluster ensembles." Journal of Statistical Software 14(12).
62. Hsu, Daniel and Sham M. Kakade. 2013. "Learning mixtures of spherical Gaussians: Moment methods and spectral decompositions. In Proceedings of the 4th Conference on Innovations in Theoretical Computer Science. ACM. pp. 11-20.
63. Hu, Yuening, Jordan Boyd-Graber, Brianna Satinoff, and Alison Smith. 2014. "Interactive topic modeling." Machine Learning 95(3):423-469.
64. Huang, Furong, U.N. Niranjan, M. Hakeem, and Animashree Anandkumar. Forthcoming. "Online tensor methods for learning latent variable models." Journal of Machine Learning Research. arXiv:1309.0787.
65. Jordan, Michael I., Zoubin Ghahramani, Tommi S. Jaakkola, and Lawrence K. Saul. 1998. An introduction to variational methods for graphical models. Springer.
66. Kalai, Adam Tauman, Ankur Moitra, and Gregory Valiant. 2012. "Disentangling Gaus-sians." Communications of the ACM 55(2):113-120.
67. Kass, Robert E. and Adrian E. Raftery. 1995. "Bayes factors." Journal of the American Statistical Association 90(430):773-795.
68. King, Gary. 1989. Unifying political methodology. Cambridge University Press.
69. King, Gary, Jennifer Pan, and Margaret E. Roberts. 2013. "How censorship in China allows government criticism but silences collective expression." American Political Science Review 107(02):326-343.
70. Kivinen, Jyrki and Manfred K. Warmuth. 1997. "Exponentiated gradient versus gradient descent for linear predictors." Information and Computation 132(1):1-63.
71. Koltcov, Sergei, Olessia Koltsova, and Sergey Nikolenko. 2014. "Latent Dirichlet allocation: Stability and applications to studies of user-generated content." In Proceedings of the 2014 ACM Conference on Web Science. ACM. pp. 161-165.
72. Krippendorff, Klaus. 2012. Content analysis: An introduction to its methodology. Sage.
73. Kuhn, Harold W. 1955. "The Hungarian method for the assignment problem." Naval Research Logistics Quarterly 2(1-2):83-97.
74. Kumar, Abhishek, Vikas Sindhwani, and Prabhanjan Kambadur. 2012. "Fast conical hull algorithms for near-separable non-negative matrix factorization." ar Xiv preprint ar Xiv:1210.1190.
75. Lancichinetti, Andrea, M. Irmak Sirer, Jane X. Wang, Daniel Acuna, Konrad Körding, and Luis A. Nunes Amaral. 2014. "A high-reproducibility and high-accuracy method for automated topic classification." ar Xiv preprint ar Xiv:1402.0422.
76. Lau, Jey Han, David Newman, and Timothy Baldwin. 2014. "Machine reading tea leaves: Automatically evaluating topic coherence and topic model quality." In Proceedings of the European Chapter of the Association for Computational Linguistics.
77. Lauderdale, Benjamin E. and Tom S. Clark. 2014. "Scaling politically meaningful dimensions using texts and votes." American Journal of Political Science 58(3):754-771.
78. Lazer, David, Alex Sandy Pentland, Lada Adamic, Sinan Aral, Albert Laszlo Barabasi, Devon Brewer, Nicholas Christakis, Noshir Contractor, James Fowler, Myron Gutmann, et al. 2009. "Life in the network: The coming age of computational social science." Science 323(5915):721.
79. Lim, Yew Jin and Yee Whye Teh. 2007. "Variational Bayesian approach to movie rating prediction." In Proceedings of KDD Cup and Workshop. Vol. 7. Citeseer. pp. 15-21.
80. Lloyd, Stuart. 1982. "Least squares quantization in PCM." IEEE Transactions on Information Theory 28(2):129-137.
81. Mahajan, Meena, Prajakta Nimbhorkar, and Kasturi Varadarajan. 2009. "The planar k-means problem is NP-hard." In WALCOM: Algorithms and Computation. Springer. pp. 274-285.
82. McLachlan, Geoffrey and David Peel. 2004. Finite mixture models. John Wiley & Sons.
83. Mikolov, Tomas, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. "Distributed representations of words and phrases and their compositionality." In Advances in Neural Information Processing Systems. pp. 3111-3119.
84. Mimno, David and David Blei. 2011. "Bayesian checking for topic models." In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. pp. 227-237.
85. Mimno, David, Hanna M. Wallach, Edmund Talley, Miriam Leenders, and Andrew McCallum. 2011. "Optimizing semantic coherence in topic models." In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. pp. 262-272.
86. Mullainathan, Sendhil. 2014. "What big data means for social science." Behavioral and Experimental Seminar.
87. Murphy, Kevin P. 2012. Machine learning: A probabilistic perspective. MIT press.
88. National Research. Council. 2013. Frontiers in massive data analysis. National Academies Press.
89. Nguyen, Thang, Yuening Hu, and Jordan, Boyd-Graber. 2014. "Anchors regularized: Adding robustness and extensibility to scalable topic-modeling algorithms." In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Baltimore, Maryland: Association for Computational Linguistics. pp. 359-369. http://www.aclweb.org/anthology/P14-1034.
90. Nielsen, Frank and Richard Nock. 2014. "Further heuristics for k-means: The merge-and-split heuristic and the (k, l)-means." arXiv preprint arXiv:1406.6314.
91. O'Connor, Brendan. 2014. "MiTextExplorer: Linked brushing and mutual information for exploratory text data analysis." In Proceedings of the Workshop on Interactive Language Learning, Visualization, and Interfaces. Association for Computational Linguistics. pp. 1-13.
92. O'Connor, Brendan, Brandon M. Stewart, and Noah A. Smith. 2013. "Learning to extract international relations from political context." In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Sofia, Bulgaria: Association for Computational Linguistics. pp. 1094-1104. http://www.aclweb.org/anthology/P13-1108.
93. Papadimitriou, Christos H. and Kenneth Steiglitz. 1998. Combinatorial optimization: Algorithms and complexity. Courier Dover Publications.
94. Park, Jong Hee. 2012. "A unified method for dynamic and cross-sectional heterogeneity: Introducing hidden Markov panel models." American Journal of Political Science 56(4):1040-1054.
95. Pearson, Karl. 1894. "Contributions to the mathematical theory of evolution." Philosophical Transactions of the Royal Society of London. A. pp. 71-110.
96. Poole, Keith T. and Howard Rosenthal. 1997. Congress: A political-economic history of roll call voting. Oxford University Press.
97. Quinn, Kevin M., Burt L. Monroe, Michael Colaresi, Michael H. Crespin, and Dragomir R. Radev. 2010. "How to analyze political attention with minimal assumptions and costs." American Journal of Political Science 54(1):209-228.
98. Rabiner, Lawrence and Biing-Hwang Juang. 1986. "An introduction to hidden Markov models." ASSP Magazine, IEEE 3(1):4-16.
99. Recht, Ben, Christopher Re, Joel Tropp, and Victor Bittorf. 2012. "Factoring nonnegative matrices with linear programs." In Advances in Neural Information Processing Systems. pp. 1214-1222.
100. Reich, Justin, Dustin Tingley, Jetson Leder-Luis, Margaret E. Roberts, and Brandon M. Stewart. 2015. Computer Assisted Reading and Discovery for Student Generated Text in Massive Open Online Courses. Journal of Learning Analytics. 2(1):156-184.
101. Rivers, Douglas. 2003. "Identification of multidimensional spatial voting models." Typescript. Stanford University.
102. Robert, Christian P. and George Casella. 2004. Monte Carlo statistical methods. Vol. 319. Springer: New York.
103. Roberts, Margaret E., Brandon M. Stewart, Dustin Tingley, Christopher Lucas, Jetson Leder-Luis, Shana Kushner Gadarian, Bethany Albertson, and David G. Rand. 2014. "Structural topic models for open-ended survey responses." American Journal of Political Science. 58(4):1064-1082.
104. Ruiz, Francisco J. R., Isabel Valera, Carlos Blanco, and Fernando Perez-Cruz. 2014. "Bayesian nonparametric comorbidity analysis of psychiatric disorders." Journal of Machine Learning Research 15:1215-1247. http://jmlr.org/papers/v15/ruiz14a.html.
105. Shneiderman, Ben. 1996. "The eyes have it: A task by data type taxonomy for information visualizations." In Proceedings of the IEEE Symposium on Visual Languages. pp. 336-343.
106. Song, Le, Eric P. Xing, and Ankur P. Parikh. 2011. "A spectral algorithm for latent tree graphical models." In Proceedings of the 28th International Conference on Machine Learning (ICML-11). pp. 1065-1072.
107. Sontag, David and Dan Roy. 2011. "Complexity of inference in latent Dirichlet allocation." In Advances in Neural Information Processing Systems. pp. 1008-1016.
108. Srivastava, Virendera K. and David E. A. Giles. 1987. Seemingly unrelated regression equations models: Estimation and inference. Vol. 80. CRC Press.
109. Teh, Yee Whye, Michael I. Jordan, Matthew J. Beal, and David M. Blei. 2006. "Hierarchical Dirichlet processes." Journal of the American Statistical Association 101(476).
110. Tenenbaum, Joshua B., Charles Kemp, Thomas L. Griffiths, and Noah D. Goodman. 2011. "How to grow a mind: Statistics, structure, and abstraction." Science 331(6022):1279-1285.
111. Tibshirani, Robert. 1996. "Regression shrinkage and selection via the lasso." Journal of the Royal Statistical Society. Series B (Methodological), pp. 267-288.
112. Vavasis, Stephen A. 2009. "On the complexity of nonnegative matrix factorization." SIAM Journal on Optimization 20(3):1364-1377.
113. Wallach, Hanna M., Iain Murray, Ruslan Salakhutdinov, and David Mimno. 2009. "Evaluation methods for topic models." In Proceedings of the 26th Annual International Conference on Machine Learning. ACM. pp. 1105-1112.
114. Ward, Michael D., Brian D., Greenhill, and Kristin M. Bakke. 2010. "The perils of policy by p-value: Predicting civil conflicts." Journal of Peace Research 47(4):363-375.
115. Wolpert, David H. and William G. Macready. 1997. "No free lunch theorems for optimization." IEEE Transactions on Evolutionary Computation 1(1):67-82.
116. Yao, Limin, David Mimno, and Andrew McCallum. 2009. "Efficient methods for topic model inference on streaming document collections." In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM. pp. 937-946.
117. Zhou, Tianyi, Jeff Bilmes, and Carlos Guestrin. 2014. "Divide-and-conquer learning by anchoring a conical hull." arXiv preprint arXiv:1406.5752.