Large scale distributed semi-supervised learning using streaming approximation

Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, AISTATS 2016

Volumn , Issue , 2016, Pages 519-528

  Ravi, Sujith ^a   Diao, Qiming ^b,c  

^a GOOGLE INC   (United States)

^b CARNEGIE MELLON UNIVERSITY   (United States)

^c Singapore University   (Singapore)

Author keywords

[No Author keywords available]

Indexed keywords

DEEP LEARNING; GRAPH THEORY; GRAPHIC METHODS; NATURAL LANGUAGE PROCESSING SYSTEMS; SUPERVISED LEARNING;

LABEL DISTRIBUTION; LEARNING ARCHITECTURES; NATURAL LANGUAGE APPLICATIONS; REAL-WORLD DATASETS; SEMI- SUPERVISED LEARNING; SEMI-SUPERVISED LEARNING (SSL); SPACE COMPLEXITY; STATE-OF-THE-ART ALGORITHMS;

MACHINE LEARNING;

EID: 85057307717     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (38)

1. R. Agrawal, A. Gupta, Y. Prabhu, and M. Varma. Multi-label learning with millions of labels: Recommending advertiser bid phrases for web pages. In Proceedings of the International World Wide Web Conference, 2013.
2. Apache giraph. http://giraph.apache.org/,2013.
3. S. Baluja, R. Seth, D. Sivakumar, Y. Jing, J. Yagnik, S. Kumar, D. Ravichandran, and M. Aly. Video suggestion and discovery for Youtube: Taking random walks through the view graph. In Proceedings of the 17th International Conference on World Wide Web, WWW’08, pages 895–904, 2008.
4. M. Belkin, P. Niyogi, and V. Sindhwani. On manifold regularization. In Proceeding of the Conference on Artificial Intelligence and Statistics (AISTATS), 2005.
5. Y. Bengio, O. Delalleau, and N. Le Roux. Label propagation and quadratic criterion. In O. Chapelle, B. Schölkopf, and A. Zien, editors, Semi-Supervised Learning, pages 193–216. MIT Press, 2006.
6. J. Blitzer and X. J. Zhu. Semi-supervised learning for natural language processing. In ACL-HLT Tutorial, June 2008.
7. A. Carlson, J. Betteridge, B. Kisiel, B. Settles, E. R. H. Jr., and T. M. Mitchell. Toward an architecture for never-ending language learning. In AAAI, 2010.
8. O. Chapelle, B. Schölkopf, and A. Zien, editors. Semi-Supervised Learning. MIT Press, Cambridge, MA, 2006.
9. M. Charikar. Similarity estimation techniques from rounding algorithms. In Proceedings of the thiry-fourth annual ACM symposium on Theory of computing, pages 380–388, 2002.
10. G. Cormode and S. Muthukrishnan. An improved data stream summary: The count-min sketch and its applications. Journal of Algorithms, 55(1):58–75, 2005.
11. D. Das and N. A. Smith. Graph-based lexicon expansion with sparsity-inducing penalties. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 677–687. Association for Computational Linguistics, 2012.
12. J. Dean, G. Corrado, R. Monga, K. Chen, M. Devin, Q. V. Le, M. Z. Mao, M. Ranzato, A. W. Senior, P. A. Tucker, K. Yang, and A. Y. Ng. Large scale distributed deep networks. In Proceedings of NIPS, pages 1232–1240, 2012.
13. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPR09, 2009.
14. B. V. Durme and A. Lall. Streaming pointwise mutual information. In Y. Bengio, D. Schuurmans, J. Lafferty, C. Williams, and A. Culotta, editors, Advances in Neural Information Processing Systems 22, pages 1892–1900. 2009.
15. A. Goyal, H. Daumé, III, and S. Venkatasubramanian. Streaming for large scale NLP: Language modeling. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, NAACL’09, pages 512–520, 2009.
16. T. Joachims. Transductive inference for text classification using support vector machines. In Proceedings of the Sixteenth International Conference on Machine Learning, pages 200–209, 1999.
17. T. Joachims. Transductive learning via spectral graph partitioning. In Proceedings of ICML, pages 290–297, 2003.
18. M. Kowalski and B. Torrésani. Sparsity and persistence: mixed norms provide simple signal models with dependent coefficients. Signal, Image and Video Processing, 3(3):251–264, Sept. 2009.
19. Z. Kozareva, K. Voevodski, and S.-H. Teng. Class label enhancement via related instances. In Proceedings of EMNLP, pages 118–128, 2011.
20. G. Malewicz, M. H. Austern, A. J. Bik, J. C. Dehnert, I. Horn, N. Leiser, and G. Czajkowski. Pregel: a system for large-scale graph processing. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of data, pages 135–146, 2010.
21. G. S. Manku and R. Motwani. Approximate frequency counts over data streams. In Proceedings of the 28th International Conference on Very Large Data Bases, VLDB’02, pages 346–357, 2002.
22. T. Mikolov, K. Chen, G. Corrado, and J. Dean. Efficient estimation of word representations in vector space. In Proceedings of Workshop at ICLR, 2013.
23. T. Mikolov, I. Sutskever, K. Chen, G. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In Proceedings of NIPS, 2013.
24. M. Osborne, A. Lall, and B. V. Durme. Exponential reservoir sampling for streaming language models. In Proceedings of The 52nd Annual Meeting of the Association for Computational Linguistics, ACL’2014, pages 687–692, 2014.
25. M. Seeger. Learning with labeled and unlabeled data. Technical report, 2001.
26. A. Subramanya and J. A. Bilmes. Entropic graph regularization in non-parametric semi-supervised classification. In Proceedings of NIPS, pages 1803–1811, 2009.
27. A. Subramanya, S. Petrov, and F. Pereira. Efficient graph-based semi-supervised learning of structured tagging models. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, EMNLP’10, pages 167–176, 2010.
28. P. Talukdar and W. Cohen. Scaling graph-based semi supervised learning to large number of labels using count-min sketch. In Proceedings of AISTATS, pages 940–947, 2014.
29. P. P. Talukdar and K. Crammer. New regularized algorithms for transductive learning. In Proceedings of the European Conference on Machine Learning and Knowledge Discovery in Databases: Part II, ECML PKDD’09, pages 442–457, 2009.
30. P. P. Talukdar and F. Pereira. Experiments in graph-based semi-supervised learning methods for class-instance acquisition. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, ACL’10, pages 1473–1481, 2010.
31. P. P. Talukdar, J. Reisinger, M. Pasca, D. Ravichandran, R. Bhagat, and F. Pereira. Weakly-supervised acquisition of labeled class instances using graph random walks. In EMNLP, pages 582–590, 2008.
32. R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society (Series B), 58:267–288, 1996.
33. A. C. Tommi and T. Jaakkola. On information regularization. In Proceedings of the 19th UAI, 2003.
34. J. Ugander and L. Backstrom. Balanced label propagation for partitioning massive graphs. In Proceedings of the Sixth ACM International Conference on Web Search and Data Mining, pages 507–516, 2013.
35. B. Van Durme and A. Lall. Efficient online locality sensitive hashing via reservoir counting. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Papers - Volume 2, HLT’11, pages 18–23, 2011.
36. Y. Wang, R. Ji, and S.-F. Chang. Label propagation from imagenet to 3d point clouds. In Proceedings of CVPR, pages 3135–3142. IEEE, 2013.
37. X. Zhu. Semi-supervised learning literature survey. Technical Report 1530, Computer Sciences, University of Wisconsin-Madison, 2005.
38. X. Zhu, Z. Ghahramani, and J. Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of ICML, pages 912–919, 2003.