Multitask Learning

Machine Learning

Volumn 28, Issue 1, 1997, Pages 41-75

  Caruana, Rich ^a  

^a Carnegie Mellon University ^*  (United States)

Author keywords

Backpropagation; Generalization; Inductive transfer; K nearest neighbor; Kernel regression; Multitask learning; Parallel transfer; Supervised learning

Indexed keywords

BACKPROPAGATION; DECISION THEORY; INFORMATION THEORY; LEARNING ALGORITHMS; REGRESSION ANALYSIS; TREES (MATHEMATICS);

MULTITASK LEARNING;

LEARNING SYSTEMS;

EID: 0031189914     PISSN: 08856125     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1007379606734     Document Type: Article

References (62)

1. Abu-Mostafa, Y. S. (1990). "Learning from Hints in Neural Networks," Journal of Complexity, 6(2), pp. 192-198.
2. Abu-Mostafa, Y. S. (1993). "Hints and the VC Dimension," Neural Computation, 5(2).
3. Abu-Mostafa, Y. S. (1995). "Hints," Neural Computation, 7, pp. 639-671.
4. Baluja, S. & Pomerleau, D. A. (1995). "Using the Representation in a Neural Network's Hidden Layer for Task-Specific Focus of Attention," Proceedings of the International Joint Conference on Artificial Intelligence 1995, IJCAI-95, Montreal, Canada, pp. 133-139.
5. Baxter, J. (1994). "Learning Internal Representations," Ph.D. Thesis, The Flinders Univeristy of South Australia.
6. Baxter, J. (1995). "Learning Internal Representations," Proceedings of the 8th ACM Conference on Computational Learning Theory, (COLT-95), Santa Cruz, CA.
7. Baxter, J. (1996). "A Bayesian/Information Theoretic Model of Bias Learning," Proceedings of the 9th International Conference on Computational Learning Theory, (COLT-96), Desenzano del Gardo, Italy.
8. Breiman, L. & Friedman, J. H. (1995). "Predicting Multivariate Responses in Multiple Linear Regression," ftp://ftp.stat.berkeley.edu/pub/users/breiman/curds-whey-all.ps.Z.
9. Caruana, R. (1993). "Multitask Learning: A Knowledge-Based Source of Inductive Bias," Proceedings of the 10th International Conference on Machine Learning, ML-93, University of Massachusetts, Amherst, pp. 41-48.
10. Caruana, R. (1994). "Multitask Connectionist Learning," Proceedings of the 1993 Connectionist Models Summer School, pp. 372-379.
11. Caruana, R. (1995). "Learning Many Related Tasks at the Same Time with Backpropagation," Advances in Neural Information Processing Systems 7 (Proceedings of NIPS-94), pp. 656-664.
12. Caruana, R., Baluja, S., & Mitchell, T. (1996). "Using the Future to "Sort Out" the Present: Rankprop and Multitask Learning for Medical Risk Prediction," Advances in Neural Information Processing Systems 8 (Proceedings of NIPS-95), pp. 959-965.
13. Caruana, R. & de Sa, V. R. (1997). "Promoting Poor Features to Supervisors: Some Inputs Work Better as Outputs," to appear in Advances in Neural Information Processing Systems 9 (Proceedings of NIPS-96).
14. Caruana, R. (1997). "Multitask Learning," Ph.D. Thesis, School of Computer Science, Carnegie Mellon University.
15. Cooper, G. F. & Herskovits, E. (1992). "A Bayesian Method for the Induction of Probabilistic Networks from Data," Machine Learning, 9, pp. 309-347.
16. Cooper, G. F., Aliferis, C. F., Ambrosino, R., Aronis, J., Buchanan, B. G., Caruana, R., Fine, M. J., Glymour, C., Gordon, G., Hanusa, B. H., Janosky, J. E., Meek, C., Mitchell, T., Richardson, T., and Spirtes, P. (1997). "An Evaluation of Machine Learning Methods for Predicting Pneumonia Mortality," Artificial Intelligence in Medicine 9, pp. 107-138.
17. Craven, M. & Shavlik, J. (1994). "Using Sampling and Queries to Extract Rules from Trained Neural Networks," Proceedings of the 11th International Conference on Machine Learning, ML-94, Rutgers University, New Jersey, pp. 37-45.
18. Davis, I. & Stentz, A. (1995). "Sensor Fusion for Autonomous Outdoor Navigation Using Neural Networks," Proceedings of IEEE's Intelligent Robots and Systems Conference.
19. Dent, L., Boticario, J., McDermott, J., Mitchell, T., & Zabowski, D. (1992). "A Personal Learning Apprentice," Proceedings of 1992 National Conference on Artificial Intelligence.
20. de Sa, V. R. (1994). "Learning Classification with Unlabelled Data," Advances in Neural Information Processing Systems 6, (Proceedings of NIPS-93), pp. 112-119.
21. Dietterich, T. G., Hild, H., & Bakiri, G. (1990). "A Comparative Study of ID3 and Backpropagation for English Text-to-speech Mapping," Proceedings of the Seventh International Conference on Artificial Intelligence, pp. 24-31.
22. Dietterich, T. G., Hild, H., & Bakiri, G. (1995). "A Comparison of ID3 and Backpropagation for English Text-to-speech Mapping," Machine Learning, 18(1), pp. 51-80.
23. Dietterich, T. G. & Bakiri, G. (1995). "Solving Multiclass Learning Problems via Error-Correcting Output Codes," Journal of Artificial Intelligence Research, 2, pp. 263-286.
24. Fine, M. J., Singer, D., Hanusa, B. H., Lave, J., & Kapoor, W. (1993). "Validation of a Pneumonia Prognostic Index Using the MedisGroups Comparative Hospital Database," American Journal of Medicine.
25. Fisher, D. H. (1987). "Conceptual Clustering, Learning from Examples, and Inference," Proceedings of the 4th International Workshop on Machine Learning.
26. Ghahramani, Z. & Jordan, M. I. (1994). "Supervised Learning from Incomplete Data Using an EM Approach," Advances in Neural Information Processing Systems 6, (Proceedings of NIPS-93,) pp. 120-127.
27. Ghahramani, Z. & Jordan, M. I. (1997). "Mixture Models for Learning from Incomplete Data," Computational Learning Theory and Natural Learning Systems, Vol. IV, R. Greiner, T. Petsche and S.J. Hanson (eds.), Cambridge, MA, MIT Press, pp. 67-85.
28. Ghosn, J. & Bengio, Y. (1997). "Multi-Task Learning for Stock Selection," to appear in Advances in Neural Information Processing Systems 9, (Proceedings of NIPS-96).
29. Hinton, G. E. (1986). "Learning Distributed Representations of Concepts," Proceedings of the 8th International Conference of the Cognitive Science Society, pp. 1-12.
30. Holmstrom, L. & Koistinen, P. (1992). "Using Additive Noise in Back-propagation Training," IEEE Transactions on Neural Networks, 3(1), pp. 24-38.
31. Jordan, M. & Jacobs, R. (1994). "Hierarchical Mixtures of Experts and the EM Algorithm," Neural Computation, 6, pp. 181-214.
32. Koller, D. & Sahami, M. (1996). "Toward Optimal Feature Selection," Proceedings of the 13th International Conference on Machine Learning, ICML-96, Bari, Italy, pp. 284-292.
33. Le Cun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackal, L. D. (1989). "Backpropagation Applied to Handwritten Zip-Code Recognition," Neural Computation, 1, pp. 541-551.
34. Little, R. J. A. & Rubin, D. B. (1987). Statistical Analysis with Missing Data, Wiley, New York.
35. Liu, H. & Setiono, R. (1996). "A Probibilistic Approach to Feature Selection - A Filter Solution," Proceedings of the 13th International Conference on Machine Learning, ICML-96, Bari, Italy, pp. 319-327.
36. Martin, J. D. (1994). "Goal-directed Clustering," Proceedings of the 1994 AAAI Spring Symposium on Goal-directed Learning.
37. Martin, J. D. & Billman, D. O. (1994). "Acquiring and Combining Overlapping Concepts," Machine Learning, 16, pp. 1-37.
38. Mitchell, T. (1980). "The Need for Biases in Learning Generalizations," Rutgers University: CBM-TR-117.
39. Mitchell, T., Caruana, R., Freitag, D., McDermott, J., & Zabowski, D. (1994). "Experience with a Learning Personal Assistant," Communications of the ACM: Special Issue on Agents, 37(7), pp. 80-91.
40. Munro, P. W. & Parmanto, B. (1997). "Competition Among Networks Improves Committee Performance," to appear in Advances in Neural Information Processing Systems 9 (Proceedings of NIPS-96).
41. Omohundro, S. M. (1996). "Family Discovery," Advances in Neural Information Processing Systems 8, (Proceedings of NIPS-95), pp. 402-408.
42. O'Sullivan, J. & Thrun, S. (1996). "Discovering Structure in Multiple Learning Tasks: The TC Algorithm," Proceedings of the 13th International Conference on Machine Learning, ICML-96, Bari, Italy, pp. 489-497.
43. Pomerleau, D. A. (1992). "Neural Network Perception for Mobile Robot Guidance," Carnegie Mellon University: CMU-CS-92-115.
44. Pratt, L. Y., Mostow, J., & Kamm, C. A. (1991). "Direct Transfer of Learned Information Among Neural Networks," Proceedings of AAAI-91.
45. Pratt, L. Y. (1992). "Non-literal Transfer Among Neural Network Learners," Colorado School of Mines: MCS-92-04.
46. Quinlan, J. R. (1986). "Induction of Decision Trees," Machine Learning, 1, pp. 81-106.
47. Quinlan, J. R. (1992). C4.5: Programs for Machine Learning, Morgan Kaufman Publishers.
48. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). "Learning Representations by Back-propagating Errors," Nature, 323, pp. 533-536.
49. Sejnowski, T. J. & Rosenberg, C. R. (1986). "NETtalk: A Parallel Network that Learns to Read Aloud," John Hopkins: JHU/EECS-86/01.
50. Sharkey, N. E. & Sharkey, A. J. C. (1992). "Adaptive Generalisation and the Transfer of Knowledge," University of Exeter: R257.
51. Sill, J. & Abu-Mostafa, Y. (1997). "Monotonicity Hints," to appear in Neural Information Processing Systems 9 (Proceedings of NIPS-96).
52. Simard, P., Victorri, B., LeCun, Y., & Denker, J. (1992). "Tangent Prop - A Formalism for Specifying Selected Invariances in an Adaptive Neural Network," Advances in Neural Information Processing Systems 4 (Proceedings of NIPS-91), pp. 895-903.
53. Spirtes, P., Glymour, C., & Scheines, R. (1993). Causation, Prediction, and Search, Springer-Verlag, New York.
54. Suddarth, S. C. & Kergosien, Y. L. (1990). "Rule-injection Hints as a Means of Improving Network Performance and Learning Time," Proceedings of the 1990 EURASIP Workshop on Neural Networks, pp. 120-129.
55. Suddarth, S. C. & Holden, A. D. C. (1991). "Symbolic-neural Systems and the Use of Hints for Developing Complex Systems," International Journal of Man-Machine Studies, 35(3), pp. 291-311.
56. Thrun, S. & Mitchell, T. (1994). "Learning One More Thing," Carnegie Mellon University: CS-94-184.
57. Thrun, S. (1995). "Lifelong Learning: A Case Study," Carnegie Mellon University: CS-95-208.
58. Thrun, S. (1996a). "Is Learning the N-th Thing Any Easier Than Learning the First?," Advances in Neural Information Processing Systems 8 (Proceedings of NIPS-95), pp. 640-646.
59. Thrun, S. (1996b). Explanation-Based Neural Network Learning: A Lifelong Learning Approach, Kluwer Academic Publisher.
60. Tresp, V., Ahmad, S., & Neuneier, R. (1994). "Training Neural Networks with Deficient Data," Advances in Neural Information Processing Systems 6 (Proceedings of NIPS-93), pp. 128-135.
61. Valdes-Perez, R., & Simon, H. (1994). "A Powerful Heuristic for the Discovery of Complex Patterned Behavior," Proceedings of the 11th International Conference on Machine Learning, ML-94, Rutgers University, New Jersey, pp. 326-334.
62. Waibel, A., Sawai, H., & Shikano, K. (1989). "Modularity and Scaling in Large Phonemic Neural Networks," IEEE Transactions on Acoustics, Speech and Signal Processing, 37(12), pp. 1888-1898.