Meta-learning probabilistic inference for prediction

7th International Conference on Learning Representations, ICLR 2019

Volumn , Issue , 2019, Pages

  Gordon, Jonathan ^a   Bronskill, John ^a   Nowozin, Sebastian ^b,c   Bauer, Matthias ^a   Turner, Richard E ^a  

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^b GOOGLE   (United Kingdom)

^c MICROSOFT RESEARCH   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DEPRECIATION;

ARBITRARY NUMBER; BENCHMARK DATASETS; CLASS OF METHODS; INFERENCE NETWORK; PROBABILISTIC INFERENCE; PROBABILISTIC INTERPRETATION; SECOND DERIVATIVES; STATE OF THE ART;

CLASSIFICATION (OF INFORMATION);

EID: 85083953617     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (59)

1. B. Bakker and T. Heskes. Task clustering and gating for Bayesian multitask learning. Journal of Machine Learning Research, 4(May):83-99, 2003.
2. ´ B. Schölkopf, and R. E. Turner. Discriminative k-shot learning using probabilistic models. arXiv preprint arXiv:1706.00326, 2017.
3. J. O. Berger. Statistical decision theory and Bayesian analysis. Springer Science & Business Media, 2013.
4. C. Blundell, J. Cornebise, K. Kavukcuoglu, and D. Wierstra. Weight uncertainty in neural network. In International Conference on Machine Learning, pages 1613-1622, 2015.
5. G. Casella and R. L. Berger. Statistical inference, volume 2. Duxbury Pacific Grove, CA, 2002.
6. A. X. Chang, T. Funkhouser, L. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva, S. Song, H. Su, J. Xiao, L. Yi, and F. Yu. ShapeNet: An Information-Rich 3D Model Repository. Technical Report arXiv:1512.03012 [cs.GR], Stanford University - Princeton University - Toyota Technological Institute at Chicago, 2015.
7. C. Cremer, X. Li, and D. Duvenaud. Inference suboptimality in variational autoencoders. arXiv preprint arXiv:1801.03558, 2018.
8. A. P. Dawid. The geometry of proper scoring rules. Annals of the Institute of Statistical Mathematics, 59(1):77-93, 2007.
9. H. Edwards and A. Storkey. Towards a neural statistician. In Proceedings of the International Conference on Learning Representations (ICLR), 2017.
10. L. Fei-Fei, R. Fergus, and P. Perona. One-shot learning of object categories. IEEE transactions on pattern analysis and machine intelligence, 28(4):594-611, 2006.
11. C. Finn, P. Abbeel, and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, pages 1126-1135, 2017.
12. C. Finn, K. Xu, and S. Levine. Probabilistic model-agnostic meta-learning. arXiv preprint arXiv:1806.02817, 2018.
13. V. Garcia and J. Bruna. Few-shot learning with graph neural networks. arXiv preprint arXiv:1711.04043, 2017.
14. M. Garnelo, D. Rosenbaum, C. J. Maddison, T. Ramalho, D. Saxton, M. Shanahan, Y. W. Teh, D. J. Rezende, and S. Eslami. Conditional neural processes. arXiv preprint arXiv:1807.01613, 2018a.
15. M. Garnelo, J. Schwarz, D. Rosenbaum, F. Viola, D. J. Rezende, S. Eslami, and Y. W. Teh. Neural processes. arXiv preprint arXiv:1807.01622, 2018b.
16. S. Gidaris and N. Komodakis. Dynamic few-shot visual learning without forgetting. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4367-4375, 2018.
17. E. Grant, C. Finn, S. Levine, T. Darrell, and T. Griffiths. Recasting gradient-based meta-learning as hierarchical Bayes. In Proceedings of the International Conference on Learning Representations (ICLR), 2018.
18. D. Ha, A. Dai, and Q. V. Le. Hypernetworks. arXiv preprint arXiv:1609.09106, 2016.
19. T. Heskes. Empirical bayes for learning to learn. 2000.
20. G. E. Hinton, P. Dayan, B. J. Frey, and R. M. Neal. The” wake-sleep” algorithm for unsupervised neural networks. Science, 268(5214):1158-1161, 1995.
21. F. Huszar. Scoring rules, divergences and information in Bayesian machine learning. PhD thesis, University of Cambridge, 2013.
22. E. T. Jaynes. Probability theory: the logic of science. Cambridge university press, 2003.
23. Ł. Kaiser, O. Nachum, R. Aurko, and S. Bengio. Learning to remember rare events. In International Conference on Learning Representations (ICLR), 2017.
24. T. Kim, J. Yoon, O. Dia, S. Kim, Y. Bengio, and S. Ahn. Bayesian model-agnostic meta-learning. arXiv preprint arXiv:1806.03836, 2018a.
25. Y. Kim, S. Wiseman, A. C. Miller, D. Sontag, and A. M. Rush. Semi-amortized variational autoencoders. In Proceedings of the 35th International Conference on Machine Learning, 2018b.
26. D. Kingma and J. Ba. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations (ICLR), 2015.
27. D. P. Kingma and M. Welling. Auto-encoding variational Bayes. In Proceedings of the International Conference on Learning Representations (ICLR), 2014.
28. D. P. Kingma, S. Mohamed, D. J. Rezende, and M. Welling. Semi-supervised learning with deep generative models. In Advances in Neural Information Processing Systems, pages 3581-3589, 2014.
29. D. P. Kingma, T. Salimans, and M. Welling. Variational dropout and the local reparameterization trick. In Advances in Neural Information Processing Systems, pages 2575-2583, 2015.
30. G. Koch, R. Zemel, and R. Salakhutdinov. Siamese neural networks for one-shot image recognition. In ICML Deep Learning Workshop, volume 2, 2015.
31. A. Lacoste, B. Oreshkin, W. Chung, T. Boquet, N. Rostamzadeh, and D. Krueger. Uncertainty in multitask transfer learning. arXiv preprint arXiv:1806.07528, 2018.
32. S. Lacoste-Julien, F. Huszár, and Z. Ghahramani. Approximate inference for the loss-calibrated Bayesian. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 416-424, 2011.
33. B. Lake, R. Salakhutdinov, J. Gross, and J. Tenenbaum. One shot learning of simple visual concepts. In Proceedings of the Annual Meeting of the Cognitive Science Society, volume 33, 2011.
34. Z. Li, F. Zhou, F. Chen, and H. Li. Meta-sgd: Learning to learn quickly for few shot learning. arXiv preprint arXiv:1707.09835, 2017.
35. L. V. D. Maaten and G. Hinton. Visualizing data using t-SNE. Journal of machine learning research, 9(Nov):2579-2605, 2008.
36. N. Mishra, M. Rohaninejad, X. Chen, and P. Abbeel. A simple neural attentive meta-learner. 2018.
37. S. Mohamed. Density ratio trick. http://blog.shakirm.com/2018/01/machine-learning-trick-of-the-day-7-density-ratio-trick/, 2018.
38. D. K. Naik and R. Mammone. Meta-neural networks that learn by learning. In Neural Networks, 1992. IJCNN., International Joint Conference on, volume 1, pages 437-442. IEEE, 1992.
39. S. Narayanaswamy, T. B. Paige, J.-W. van de Meent, A. Desmaison, N. Goodman, P. Kohli, F. Wood, and P. Torr. Learning disentangled representations with semi-supervised deep generative models. In Advances in Neural Information Processing Systems, pages 5927-5937, 2017.
40. A. Y. Ng and M. I. Jordan. On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes. In Advances in Neural Information Processing Systems, pages 841-848, 2002.
41. A. Nichol and J. Schulman. Reptile: a scalable metalearning algorithm. arXiv preprint arXiv:1803.02999, 2018.
42. B. N. Oreshkin, A. Lacoste, and P. Rodriguez. Tadam: Task dependent adaptive metric for improved few-shot learning. arXiv preprint arXiv:1805.10123, 2018.
43. C. R. Qi, H. Su, K. Mo, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. Proc. Computer Vision and Pattern Recognition (CVPR), IEEE, 1(2):4, 2017.
44. S. Qiao, C. Liu, W. Shen, and A. Yuille. Few-shot image recognition by predicting parameters from activations. arXiv preprint arXiv:1706.03466, 2017.
45. S. Ravi and H. Larochelle. Optimization as a model for few-shot learning. In Proceedings of the International Conference on Learning Representations (ICLR), 2017.
46. D. J. Rezende, S. Mohamed, and D. Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In International Conference on Machine Learning, pages 1278-1286, 2014.
47. A. A. Rusu, D. Rao, J. Sygnowski, O. Vinyals, R. Pascanu, S. Osindero, and R. Hadsell. Meta-learning with latent embedding optimization. arXiv preprint arXiv:1807.05960, 2018.
48. J. Schmidhuber. Evolutionary principles in self-referential learning. PhD thesis, Technische Univer-sität München, 1987.
49. J. Snell, K. Swersky, and R. Zemel. Prototypical networks for few-shot learning. In Advances in Neural Information Processing Systems, pages 4080-4090, 2017.
50. M. Sugiyama, T. Suzuki, and T. Kanamori. Density ratio estimation in machine learning. Cambridge University Press, 2012.
51. S. Thrun and L. Pratt. Learning to learn. Springer Science & Business Media, 2012.
52. E. Triantafillou, R. Zemel, and R. Urtasun. Few-shot learning through an information retrieval lens. In Advances in Neural Information Processing Systems, pages 2255-2265, 2017.
53. B. Trippe and R. Turner. Overpruning in variational bayesian neural networks. arXiv preprint arXiv:1801.06230, 2018.
54. R. E. Turner and M. Sahani. Two problems with variational expectation maximisation for time-series models. Bayesian Time series models, 1(3.1):3-1, 2011.
55. O. Vinyals, C. Blundell, T. Lillicrap, D. Wierstra, et al. Matching networks for one shot learning. In Advances in Neural Information Processing Systems, pages 3630-3638, 2016.
56. M. J. Wainwright and M. I. Jordan. Graphical models, exponential families, and variational inference. Foundations and TrendsR in Machine Learning, 1(1-2):1-305, 2008.
57. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing, 13(4):600-612, 2004.
58. F. S. Y. Yang, L. Zhang, T. Xiang, P. H. Torr, and T. M. Hospedales. Learning to compare: Relation network for few-shot learning. 2018.
59. M. Zaheer, S. Kottur, S. Ravanbakhsh, B. Poczos, R. R. Salakhutdinov, and A. J. Smola. Deep sets. In Advances in Neural Information Processing Systems, pages 3394-3404, 2017.