A Bayesian/Information Theoretic Model of Learning to Learn via Multiple Task Sampling

Machine Learning

Volumn 28, Issue 1, 1997, Pages 7-39

  Baxter, Jonathan ^a,b,c  

^a LONDON SCHOOL OF ECONOMICS   (United Kingdom)

^b UNIVERSITY OF LONDON   (United Kingdom)

^c AUSTRALIAN NATIONAL UNIVERSITY   (Australia)

Author keywords

Bias learning; Feature Learning; Hierarchichal Bayesian Inference; Information Theory; Neural Networks

Indexed keywords

HIERARCHICAL SYSTEMS; INFORMATION THEORY; MATHEMATICAL MODELS; NEURAL NETWORKS; SAMPLING;

FEATURE LEARNING;

LEARNING SYSTEMS;

EID: 0031187873     PISSN: 08856125     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1007327622663     Document Type: Article

References (24)

1. Abu-Mostafa, Y.S. (1989). Learning from Hints in Neural Networks. Journal of Complexity, 6:192-198.
2. Anthony, Martin & Bartlett, Peter. (1995). Function learning from interpolation. In Proceedings of the Second European Conference on Computational Learning Theory, Barcelona. Springer-Verlag.
3. Barron, Andrew & Clarke, Bertrand. (1994). Jeffreys' Prior is Asymptotically Least Favourable under Entropy Risk. Journal of Statistical Planning and Inference, 41:37-60.
4. Bartlett, Peter, Long, Philip & Williamson, Bob. (1994). Fat-Shattering and the Learnability of Real-Valued Functions. In Proceedings of the Seventh ACM Conference on Computational Learning Theory, New York. ACM Press.
5. Baxter, Jonathan. (1995a). A Model of Bias Learning. Technical Report LSE-MPS-97, London School of Economics, Centre for Discrete and Applicable Mathematics. Submitted for publication.
6. Baxter, Jonathan. (1995b). Learning Internal Representations. In Proceedings of the Eighth International Conference on Computational Learning Theory, pages 311-320, Santa Cruz, California. ACM Press.
7. Baxter, Jonathan. (1996a). A Bayesian/Information Theoretic Model of Bias Learning. In Proceedings of the Ninth ACM Conference on Computational Learning Theory, New York. ACM Press.
8. Baxter, Jonathan. (1996b). Learning Model Bias. In Advances in Neural Information Processing Systems 8, pages 169-175.
9. Berger, James O. (1985). Statistical Decision Theory and Bayesian Analysis. Springer-Verlag, New York.
10. Berger, James O. (1986) Multivariate Estimation: Bayes, Empirical Bayes, and Stein Approaches. SIAM.
11. Bridle, J.S. (1989). Probabilistic interpretation of feedforward classification network outputs, with relationships to statistical pattern recognition. In F Fogelman-Soulie and J Herault, editors, Neurocomputing: Algorithms, Architectures. Springer Verlag, New York.
12. Caruana, Richard. (1993). Learning Many Related Tasks at the Same Time with Backpropagation. In Advances in Neural Information Processing 5.
13. Clarke, Bertrand & Barron, Andrew. (1990). Information-Theoretic Asymptotics of Bayes Methods. IEEE Transactions on Information Theory, 36:453-471.
14. Cover, T.M. & Thomas, J.A. (1991). Elements of Information Theory. John Wiley & Sons, Inc., New York.
15. Fefferman, Charles. (1994). Reconstructing a neural network from its output. Rev. Mat. Iberoamericana, 10:507-555.
16. Good, I.J. (1980). Some History of the Hierarchical Bayesian Methodology. In J M Bernado, M H De Groot, D V Lindley, and A F M Smith, editors, Bayesian Statistics II. University Press, Valencia.
17. Haussler, David & Opper, Manfred. (1995a). General Bounds on the Mutual Information Between a Parameter and n Conditionally Independent Observations. In Proceedings of the Eighth ACM Conference on Computational Learning Theory, New York. ACM Press.
18. Haussler, David & Opper, Manfred. (1995b). Mutual Information, Metric Entropy and Risk in Estimation of Probability Distributions. Submitted to Annals of Statistics.
19. Hornik, K. (1991). Approximation capabilities of multilayer feedforward networks. Neural Networks, 4:251-257.
20. Mackay, David. (1991). Bayesian Interpolation. Neural Computation, 4:415-447.
21. Mackay, David. (1991). The Evidence Framework Applied to Classification Networks. Neural Computation, 4:698-714.
22. Mitchell, Tom M. (1990). The need for biases in learning generalisations. In Tom G Dietterich and Jude Shavlik, editors, Readings in Machine Learning. Morgan Kaufmann.
23. Mitchell, Tom M. & Thrun, Sebastian. (1994). Learning One More Thing. Technical Report CMU-CS-94-184, CMU.
24. Pratt, Lori Y (1992). Discriminability-based transfer between neural networks. In Stephen J Hanson, Jack D Cowan, and C Lee Giles, editors, Advances in Neural Information Processing Systems 5, pages 204-211, San Mateo. Morgan Kaufmann.