Client-Server multitask learning from distributed datasets

IEEE Transactions on Neural Networks

Volumn 22, Issue 2, 2011, Pages 290-303

  Dinuzzo, Francesco ^a,b   Pillonetto, Gianluigi ^c   De Nicolao, Giuseppe ^a  

^a UNIVERSITY OF PAVIA   (Italy)

^b SCUOLA UNIVERSITARIA SUPERIORE IUSS PAVIA   (Italy)

^c UNIVERSITY OF PADOVA   (Italy)

Author keywords

Collaborative filtering; conjoint analysis; inductive transfer; kernel methods; learning to learn; multitask learning; population methods; recommender systems; regularization theory

Indexed keywords

COLLABORATIVE FILTERING; CONJOINT ANALYSIS; INDUCTIVE TRANSFER; KERNEL METHODS; LEARNING TO LEARN; MULTITASK LEARNING; POPULATION METHODS; REGULARIZATION THEORY;

SIGNAL FILTERING AND PREDICTION;

RECOMMENDER SYSTEMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER; COMPUTER PROGRAM; COMPUTER SIMULATION; INFORMATION PROCESSING; LEARNING; METHODOLOGY; STANDARD;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; AUTOMATIC DATA PROCESSING; COMPUTER SIMULATION; COMPUTERS; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; SOFTWARE DESIGN; TRANSFER (PSYCHOLOGY);

EID: 79951674213     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2010.2095882     Document Type: Article

References (79)

1. S. Thrun, "Is learning the n-th thing any easier than learning the first," in Advances in Neural Information Processing Systems, vol. 8. Cambridge, MA: MIT Press, 1996.
2. R. Caruana, "Multitask learning," Mach. Learn., vol. 28, no. 1, pp. 41- 75, Jul. 1997. (Pubitemid 127507169)
3. S. Thrun and L. Pratt, Learning to Learn. Norwell, MA: Kluwer, 1997.
4. J. Baxter, "A Bayesian/information theoretic model of learning to learn via multiple task sampling," Mach. Learn., vol. 28, no. 1, pp. 7-39, Jul. 1997. (Pubitemid 127507168)
5. S. Ben-David, J. Gehrke, and R. Schuller, "A theoretical framework for learning from a pool of disparate data sources," in Proc. 8th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, Edmonton, AB, Canada, 2002, pp. 443-449.
6. B. Bakker and T. Heskes, "Task clustering and gating for Bayesian multitask learning," J. Mach. Learn. Res., vol. 4, pp. 83-99, Dec. 2003.
7. N. D. Lawrence and J. C. Platt, "Learning to learn with the informative vector machine," in Proc. Int. Conf. Mach. Learn., vol. 69. Banff, AB, Canada, 2004, p. 65.
8. A. Schwaighofer, V. Tresp, and K. Yu, "Learning Gaussian process kernels via hierarchical Bayes," in Advances in Neural Information Processing Systems, vol. 17. Cambridge, MA: MIT Press, 2005, pp. 1209-1216.
9. K. Yu, V. Tresp, and A. Schwaighofer, "Learning Gaussian processes from multiple tasks," in Proc. Int. Conf. Mach. Learn., Bonn, Germany, 2005, pp. 1012-1019.
10. S. Yu, V. Tresp, and K. Yu, "Robust multi-task learning with tprocesses," in Proc. 24th Int. Conf. Mach. Learn., Corvalis, OR, 2007, pp. 1103-1110.
11. Y. Xue, X. Liao, L. Carin, and B. Krishnapuram, "Multi-task learning for classification with Dirichlet process priors," J. Mach. Learn. Res., vol. 8, pp. 35-63, May 2007. (Pubitemid 46155123)
12. E. V. Bonilla, F. V. Agakov, and C. K. I. Williams, "Kernel multi-task learning using task-specific features," in Proc. 11th Int. Conf. Artif. Intell. Stat., 2007, pp. 1-8.
13. A. Argyriou, C. A. Micchelli, M. Pontil, and Y. Ying, "A spectral regularization framework for multi-task structure learning," in Advances in Neural Information Processing Systems 20, J. C. Platt, D. Koller, Y. Singer, and S. Roweis, Eds. Cambridge, MA: MIT Press, 2007, pp. 25-32.
14. S. Bickel, J. Bogojeska, T. Lengauer, and T. Scheffer, "Multi-task learning for HIV therapy screening," in Proc. 25th Annu. Int. Conf. Mach. Learn., Helsinki, Finland, 2008, pp. 56-63.
15. J. Zhang, Z. Ghahramani, and Y. Yang, "Flexible latent variable models for multi-task learning," Mach. Learn., vol. 73, no. 3, pp. 221-242, Dec. 2008.
16. Y. Qi, D. Liu, D. Dunson, and L. Carin, "Multi-task compressive sensing with Dirichlet process priors," in Proc. 25th Annu. Int. Conf. Mach. Learn., Helsinki, Finland, 2008, pp. 768-775.
17. Q. An, C. Wang, I. Shterev, E. Wang, L. Carin, and D. Dunson, "Hierarchical kernel stick-breaking process for multi-task image analysis," in Proc. 25th Annu. Int. Conf. Mach. Learn., Helsinki, Finland, 2008, pp. 17-24.
18. J. Baxter, "A model of inductive bias learning," J. Artif. Intell. Res., vol. 12, no. 1, pp. 149-198, Mar. 2000.
19. S. Ben-David and R. Schuller, "Exploiting task relatedness for multiple task learning," in Proc. Comput. Learn. Theory, 2003, pp. 1-8.
20. A. Maurer, "Bounds for linear multi-task learning," J. Mach. Learn. Res., vol. 7, pp. 117-139, Jan. 2006. (Pubitemid 43100916)
21. R. K. Ando and T. Zhang, "A framework for learning predictive structures from multiple tasks and unlabeled data," J. Mach. Learn. Res., vol. 6, pp. 1817-1853, Dec. 2005.
22. V. Srivastava and T. Dwivedi, "Estimation of seemingly unrelated regression equations: A brief survey," J. Econometrics, vol. 10, no. 1, pp. 15-32, Apr. 1971.
23. N. Arora, G. M. Allenby, and J. Ginter, "A hierarchical Bayes model of primary and secondary demand," Marketing Sci., vol. 17, no. 1, pp. 29-44, Jan. 1998.
24. G. M. Allenby and P. E. Rossi, "Marketing models of consumer heterogeneity," J. Econometrics, vol. 89, nos. 1-2, pp. 57-78, Nov. 1999. (Pubitemid 128416645)
25. W. Greene, Econometric Analysis, 5th ed. Englewood Cliffs, NJ: Prentice-Hall, 2002.
26. P. Resnick and H. R. Varian, "Recommender systems," Commun. ACM, vol. 40, no. 3, pp. 56-58, Mar. 1997. (Pubitemid 127441130)
27. D. Goldberg, D. Nichols, B. M. Oki, and D. Terry, "Using collaborative filtering to weave an information tapestry," Commun. ACM, vol. 35, no. 12, pp. 61-70, Dec. 1992.
28. P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom, and J. Riedl, "Grouplens: An open architecture for collaborative filtering of netnews," in Proc. ACM Conf. Comput. Supported Collaborative Work, Chapel Hill, NC, 1994, pp. 175-186.
29. J. A. Konstan, B. N. Miller, D. Maltz, J. L. Herlocker, L. R. Gordon, and J. Riedl, "Grouplens: Applying collaborative filtering to usenet news," Commun. ACM, vol. 40, no. 3, pp. 77-87, 1997. (Pubitemid 127441134)
30. H. Kautz, B. Selman, and M. Shah, "Referral web: Combining social networks and collaborative filtering," Commun. ACM, vol. 40, no. 3, pp. 63-65, Mar. 1997. (Pubitemid 127441132)
31. L. Terveen, W. Hill, B. Amento, D. McDonald, and J. Creter, "PHOAKS: A system for sharing recommendations," Commun. ACM, vol. 40, no. 3, pp. 59-62, Mar. 1997.
32. J. S. Breese, D. Heckerman, and C. Kadie, "Empirical analysis of predictive algorithms for collaborative filtering," in Proc. 14th Conf. Uncertainty Artif. Intell., 1998, pp. 43-52.
33. Y.-H. Chen and E. I. George, "A Bayesian model for collaborative filtering," in Proc. 7th Int. Workshop Artif. Intell. Stat., 1999, pp. 1-6.
34. T. Hofmann and J. Puzicha, "Latent class models for collaborative filtering," in Proc. 16th Int. Joint Conf. Artif. Intell., San Francisco, CA, 1999, pp. 688-693.
35. D. Heckerman, D. M. Chickering, C. Meek, R. Rounthwaite, and C. Kadie, "Dependency networks for inference, collaborative filtering, and data visualization," J. Mach. Learn. Res., vol. 1, pp. 49-75, Sep. 2000.
36. N. Srebro, J. D. M. Rennie, and T. S. Jaakkola, "Maximum-margin matrix factorization," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2005.
37. E. R. Carson, C. Cobelli, and L. Finkelstein, The Mathematical Modeling of Metabolic and Endocrine Systems. New York: Wiley, 1983.
38. J. A. Jacquez, Compartmental Analysis in Biology and Medicine. Ann Arbor, MI: Univ. Michigan Press, 1985.
39. L. B. Sheiner, B. Rosenberg, and V. V. Marathe, "Estimation of population characteristics of pharmacokinetic parameters from routine clinical data," J. Pharmacokinet. Biopharm., vol. 5, no. 5, pp. 445-479, Oct. 1977. (Pubitemid 8224518)
40. S. L. Beal and L. B. Sheiner, "Estimating population kinetics," Crit. Rev. Biomed. Eng., vol. 8, no. 3, pp. 195-222, 1982.
41. L. Yuh, S. Beal, M. Davidian, F. Harrison, A. Hester, K. Kowalski, E. Vonesh, and R. Wolfinger, "Population pharmacokinetic/ pharmacodynamic methodology and applications: A bibliography," Biometrics, vol. 50, no. 2, pp. 566-575, Jun. 1994.
42. L. B. Sheiner and J. L. Steimer, "Pharmacokinetic/pharmacodynamic modeling in drug development," Annu. Rev. Pharmacol. Toxicol., vol. 40, pp. 67-95, Apr. 2000. (Pubitemid 30340674)
43. S. Vozeh, J. L. Steimer, M. Rowland, P. Morselli, F. Mentre, L. P. Balant, and L. Aarons, "The use of population pharmacokinetics in drug development," Clin. Pharmacokinet., vol. 30, no. 2, pp. 81-93, Feb. 1996. (Pubitemid 26069034)
44. K. Park, D. Verotta, T. F. Blaschke, and L. B. Sheiner, "A semiparametric method for describing noisy population pharmacokinetic data," J. Pharmacokinet. Biopharm., vol. 25, no. 5, pp. 615-642, Oct. 1997. (Pubitemid 28355713)
45. S. Beal and L. Sheiner, NONMEM User's Guide. San Francisco, CA: NONMEM Project Group, 1992.
46. L. B. Sheiner, "The population approach to pharmacokinetic data analysis: Rationale and standard data analysis methods," Drug Metabolism Rev., vol. 15, nos. 1-2, pp. 153-171, 1984. (Pubitemid 14123936)
47. M. Davidian and D. M. Giltinan, Nonlinear Models for Repeated Measurement Data. London, U.K.: Chapman & Hall, 1995.
48. L. Aarons, "Software for population pharmacokinetics and pharmacodynamics," Clin. Pharmacokinet., vol. 36, no. 4, pp. 255-264, Apr. 1999. (Pubitemid 29201734)
49. J. C. Wakefield, A. F. M. Smith, A. Racine-Poon, and A. E. Gelfand, "Bayesian analysis of linear and non-linear population models by using the Gibbs sampler," Appl. Stat., vol. 41, no. 1, pp. 201-221, 1994.
50. D. J. Lunn, N. Best, A. Thomas, J. C. Wakefield, and D. Spiegelhalter, "Bayesian analysis of population PK/PD models: General concepts and software," J. Pharmacokinet. Pharmacodyn., vol. 29, no. 3, pp. 271-307, Jun. 2002. (Pubitemid 35333143)
51. W. R. Gilks, S. Richardson, and D. J. Spiegelhalter, Markov Chain Monte Carlo in Practice. London, U.K.: Chapman & Hall, 1996.
52. K. E. Fattinger and D. Verotta, "A nonparametric subject-specific population method for deconvolution: I. Description, internal validation and real data examples," J. Pharmacokinet. Biopharm., vol. 23, no. 6, pp. 581-610, Dec. 1995. (Pubitemid 26155335)
53. P. Magni, R. Bellazzi, G. De Nicolao, I. Poggesi, and M. Rocchetti, "Nonparametric AUC estimation in population studies with incomplete sampling: A Bayesian approach," J. Pharmacokinet. Pharmacodyn., vol. 29, nos. 5-6, pp. 445-471, Dec. 2002.
54. M. Neve, G. De Nicolao, and L. Marchesi, "Nonparametric identification of pharmacokinetic population models via Gaussian processes," in Proc. 16th IFAC World Congr., Praha, Czech Republic, 2005.
55. M. Neve, G. De Nicolao, and L. Marchesi, "Nonparametric identification of population models via Gaussian processes," Automatica, vol. 43, no. 7, pp. 1134-1144, Jul. 2007. (Pubitemid 46802443)
56. G. Pillonetto, G. De Nicolao, M. Chierici, and C. Cobelli, "Fast algorithms for nonparametric population modeling of large data sets," Automatica, vol. 45, no. 1, pp. 173-179, Jan. 2009.
57. B. Schölkopf and A. J. Smola, Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond (Adaptive Computation and Machine Learning). Cambridge, MA: MIT Press, 2001.
58. C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning. Cambridge, MA: MIT Press, 2006.
59. N. Aronszajn, "Theory of reproducing kernels," Trans. Amer. Math. Soc., vol. 68, no. 3, pp. 337-404, May 1950.
60. A. N. Tikhonov and V. Y. Arsenin, Solutions of Ill Posed Problems, W. H. Winston, Ed. New York: Wiley, 1977.
61. T. Poggio and F. Girosi, "Networks for approximation and learning," Proc. IEEE, vol. 7, no. 9, pp. 1481-1497, Sep. 1990.
62. G. Wahba, Spline Models for Observational Data. Philadelphia, PA: SIAM, 1990.
63. V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
64. F. Cucker and S. Smale, "On the mathematical foundations of learning," Bulletin Amer. Math. Soc., vol. 39, pp. 1-49, Oct. 2001.
65. T. Evgeniou, C. A. Micchelli, and M. Pontil, "Learning multiple tasks with kernel methods," J. Mach. Learn. Res., vol. 6, pp. 615-637, Dec. 2005.
66. C. A. Micchelli and M. Pontil, "On learning vector-valued functions," Neural Comput., vol. 17, no. 1, pp. 177-204, Jan. 2005. (Pubitemid 40305887)
67. G. Pillonetto, F. Dinuzzo, and G. De Nicolao, "Bayesian online multitask learning of Gaussian processes," IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 2, pp. 193-205, Feb. 2010.
68. M. Opper, Online Learning in Neural Networks. Cambridge, U.K.: Cambridge Univ. Press, 1998.
69. L. Csató and M. Opper, "Sparse on-line Gaussian processes," Neural Comput., vol. 14, no. 3, pp. 641-668, Mar. 2002.
70. N. Ramakrishnan, B. J. Keller, B. J. Mirza, A. Y. Grama, and G. Karypis, "Privacy risks in recommender systems," IEEE Internet Comput., vol. 5, no. 6, pp. 54-63, Nov.-Dec. 2001.
71. J. Canny, "Collaborative filtering with privacy via factor analysis," in Proc. 25th Annu. Int. ACM SIGIR Conf. Res. Develo. Inform. Retrieval, Tampere, Finland, 2002, pp. 238-245.
72. Z. Lu, T. Leen, Y. Huang, and D. Erdogmus, "A reproducing kernel Hilbert space framework for pairwise time series distances," in Proc. 25th Annu. Int. Conf. Mach. Learn., Helsinki, Finland, 2008, pp. 624- 631.
73. G. Kimeldorf and G. Wahba, "Some results on Tchebycheffian spline functions," J. Math. Anal. Appl., vol. 33, no. 1, pp. 82-95, Jan. 1971.
74. B. Schölkopf, R. Herbrich, and A. J. Smola, "A generalized representer theorem," in Proc. Annu. Conf. Comput. Learn. Theory, Portland, OR, 2001, pp. 416-426. (Pubitemid 33312837)
75. G. H. Golub and C. F. Van Loan, Matrix Computations. Baltimore, MD: The John Hopkins Univ. Press, 1996.
76. C. N. Nett and W. M. Haddad, "A system-theoretic appropriate realization of the empty matrix concept," IEEE Trans. Autom. Control, vol. 38, no. 5, pp. 771-775, May 1993. (Pubitemid 23706272)
77. E. Merlo-Pich and R. Gomeni, "Model-based approach and signal detection theory to evaluate the performance of recruitment centers in clinical trials with antidepressant drugs," Clin. Pharmacol. Ther., vol. 84, no. 3, pp. 378-384, Sep. 2008.
78. R. Gomeni, A. Lavergne, and E. Merlo-Pich, "Modelling placebo response in depression trials using a longitudinal model with informative dropout," Eur. J. Pharmaceutical Sci., vol. 36, no. 1, pp. 4-10, Jan. 2009.
79. C. Hu and M. E. Sale, "A joint model for nonlinear longitudinal data with informative dropout," J. Pharmacokinet. Pharmacodyn., vol. 30, no. 1, pp. 83-103, Feb. 2003. (Pubitemid 36571070)