Efficient regularized least-squares algorithms for conditional ranking on relational data

Machine Learning

Volumn 93, Issue 2-3, 2013, Pages 321-356

  Pahikkala, Tapio ^a   Airola, Antti ^a   Stock, Michiel ^b   De Baets, Bernard ^b   Waegeman, Willem ^b  

^a UNIVERSITY OF TURKU   (Finland)

^b GHENT UNIVERSITY   (Belgium)

Author keywords

Kernel methods; Learning to rank; Reciprocal relations; Regularized least squares; Symmetric relations

Indexed keywords

KERNEL METHODS; LEARNING TO RANK; LEAST SQUARE; RECIPROCAL RELATIONS; SYMMETRIC RELATIONS;

ALGORITHMS; BIOINFORMATICS; SOCIAL NETWORKING (ONLINE);

LEAST SQUARES APPROXIMATIONS;

EID: 84884286350     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-013-5354-7     Document Type: Article

References (67)

1. Abadir, M., & Magnus, J. (2005). Matrix algebra. Cambridge: Cambridge University Press.
2. Agarwal, S. (2006). Ranking on graph data. In W. W. Cohen & A. Moore (Eds.), ACM international conference proceeding series: Vol. 148. Proceedings of the 23rd international conference on machine learning (pp. 25-32). New York: ACM.
3. Airola, A., Pahikkala, T., & Salakoski, T. (2011a). On learning and cross-validation with decomposed Nyström approximation of kernel matrix. Neural Processing Letters, 33(1), 17-30.
4. Airola, A., Pahikkala, T., & Salakoski, T. (2011b). Training linear ranking SVMs in linearithmic time using red-black trees. Pattern Recognition Letters, 32(9), 1328-1336.
5. Basilico, J., & Hofmann, T. (2004). Unifying collaborative and content-based filtering. In C. E. Brodley (Ed.), ACM international conference proceeding series: Vol. 69. Proceedings of the twenty-first international conference on machine learning (ICML'04), New York: ACM.
6. Ben-Hur, A., & Noble, W. (2005). Kernel methods for predicting protein-protein interactions. Bioinformatics, 21(1), 38-46.
7. Brunner, C., Fischer, A., Luig, K., & Thies, T. (2012). Pairwise support vector machines and their application to large scale problems. Journal of Machine Learning Research, 13, 2279-2292.
8. Caetano, T., McAuley, J., Cheng, L., Le, Q., & Smola, A. (2009). Learning graph matching. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(6), 1048-1058.
9. Cao, Y., Xu, J., Liu, T. Y., Li, H., Huang, Y., & Hon, H. W. (2006). Adapting ranking SVM to document retrieval. In E. N. Efthimiadis, S. T. Dumais, D. Hawking, & K. Järvelin (Eds.), Proceedings of the 29th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2006) (pp. 186-193). New York: ACM.
10. Chapelle, O., & Keerthi, S. S. (2010). Efficient algorithms for ranking with SVMs. Information Retrieval, 13(3), 201-215.
11. De Baets, B., De Meyer, H., De Schuymer, B., & Jenei, S. (2006). Cyclic evaluation of transitivity of reciprocal relations. Social Choice and Welfare, 26, 217-238.
12. De Vito, E., Rosasco, L., Caponnetto, A., De Giovannini, U., & Odone, F. (2005). Learning from examples as an inverse problem. Journal of Machine Learning Research, 6, 883-904.
13. Engl, H., Hanke, M., & Neubauer, A. (1996). Mathematics and its applications: Vol. 375. Regularization of inverse problems. Dordrecht: Kluwer Academic.
14. Evgeniou, T., Pontil, M., & Poggio, T. (2000). Regularization networks and support vector machines. Advances in Computational Mathematics, 13(1), 1-50.
15. Fisher, L. (2008). Rock, paper, scissors: game theory in everyday life. New York: Basic Books.
16. Freund, Y., Yier, R., Schapire, R., & Singer, Y. (2003). An efficient boosting algorithm for combining preferences. Journal of Machine Learning Research, 4, 933-969.
17. Fürnkranz, J., Hüllermeier, E., & Vanderlooy, S. (2009). Binary decomposition methods for multipartite ranking. In W. L. Buntine, M. Grobelnik, D. Mladenic, & J. Shawe-Taylor (Eds.), Lecture notes in computer science: Vol. 5781. Proceedings of the European conference on machine learning and knowledge discovery in databases (ECML/PKDD'09) (pp. 359-374). Berlin: Springer.
18. Geerts, F., Mannila, H., & Terzi, E. (2004). Relational link-based ranking. In M. A. Nascimento, M. T. Özsu, D. Kossmann, R. J. Miller, J. A. Blakeley, & K. B. Schiefer (Eds.), Proceedings of the thirtieth international conference on very large data bases (pp. 552-563). San Mateo: Morgan Kaufmann.
19. Geurts, P., Touleimat, N., Dutreix, M., & d'Alché-Buc, F. (2007). Inferring biological networks with output kernel trees. BMC Bioinformatics, 8(2), S4.
20. Grangier, D., & Bengio, S. (2008). A discriminative kernel-based approach to rank images from text queries. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(8), 1371-1384.
21. Henderson, H. V., & Searle, S. R. (1981). On deriving the inverse of a sum of matrices. SIAM Review, 23(1), 53-60.
22. Horn, R. A., & Johnson, C. R. (1985). Matrix analysis. Cambridge: Cambridge University Press.
23. Horn, R. A., & Johnson, C. R. (1991). Topics in matrix analysis. New York: Cambridge University Press.
24. Hüllermeier, E., Fürnkranz, J., Cheng, W., & Brinker, K. (2008). Label ranking by learning pairwise preferences. Artificial Intelligence, 172(16-17), 1897-1916.
25. Joachims, T. (2002). Optimizing search engines using clickthrough data. In D. Hand, D. Keim, & R. Ng (Eds.), Proceedings of the 8th ACM SIGKDD conference on knowledge discovery and data mining (KDD'02) (pp. 133-142). New York: ACM.
26. Joachims, T. (2006). Training linear SVMs in linear time. In T. Eliassi-Rad, L. H. Ungar, M. Craven, & D. Gunopulos (Eds.), Proceedings of the 12th ACM SIGKDD international conference on knowledge discovery and data mining (KDD'06) (pp. 217-226). New York: ACM.
27. Kashima, H., Kato, T., Yamanishi, Y., Sugiyama, M., & Tsuda, K. (2009a). Link propagation: a fast semi-supervised learning algorithm for link prediction. In Proceedings of the SIAM international conference on data mining (SDM 2009) (pp. 1099-1110). Philadelphia: SIAM.
28. Kashima, H., Oyama, S., Yamanishi, Y., & Tsuda, K. (2009b). On pairwise kernels: an efficient alternative and generalization analysis. In T. Theeramunkong, B. Kijsirikul, N. Cercone, & T. B. Ho (Eds.), Lecture notes in computer science: Vol. 5476. Proceedings of the 13th Pacific-Asia conference on advances in knowledge discovery and data mining (pp. 1030-1037). Berlin: Springer.
29. Kersting, K., & Xu, Z. (2009). Learning preferences with hidden common cause relations. In W. L. Buntine, M. Grobelnik, D. Mladenic, & J. Shawe-Taylor (Eds.), Lecture notes in computer science: Vol. 5781. Proceedings of the European conference on machine learning and knowledge discovery in databases (ECML/PKDD'09) (pp. 676-691). Berlin: Springer.
30. Kimeldorf, G., & Wahba, G. (1971). Some results on Tchebycheffian spline functions. Journal of Mathematical Analysis and Applications, 33(1), 82-95.
31. Kotlowski, W., Dembczynski, K., & Hüllermeier, E. (2011). Bipartite ranking through minimization of univariate loss. In L. Getoor & T. Scheffer (Eds.), Proceedings of the 28th international conference on machine learning (ICML 2011) (pp. 1113-1120). New York: Omnipress.
32. Liu, T. Y. (2009). Learning to rank for information retrieval. Foundations and Trends in Information Retrieval, 3(3), 225-331.
33. Luce, R., & Suppes, P. (1965). Preference, utility and subjective probability. In Handbook of mathematical psychology (pp. 249-410). New York: Wiley.
34. MacLeod, N., Benfield, M., & Culverhouse, P. (2010). Time to automate identification. Nature, 467, 154-155.
35. Martin, C. D., & Van Loan, C. F. (2006). Shifted Kronecker product systems. SIAM Journal on Matrix Analysis and Applications, 29(1), 184-198.
36. Menon, A., & Elkan, C. (2010). Predicting labels for dyadic data. Data Mining and Knowledge Discovery, 21(2), 327-343.
37. Ng, M. K. P., Li, X., & Ye, Y. (2011). Multirank: co-ranking for objects and relations in multi-relational data. In C. Apté, J. Ghosh, & P. Smyth (Eds.), Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining (KDD'11) (pp. 1217-1225). New York: ACM.
38. Oyama, S., & Manning, C. (2004). Using feature conjunctions across examples for learning pairwise classifiers. In J. F. Boulicaut, F. Esposito, F. Giannotti, & D. Pedreschi (Eds.), Lecture notes in computer science: Vol. 3201. Proceedings of the 15th European conference on machine learning (pp. 322-333). Berlin: Springer.
39. Pahikkala, T., Tsivtsivadze, E., Airola, A., Järvinen, J., & Boberg, J. (2009). An efficient algorithm for learning to rank from preference graphs. Machine Learning, 75(1), 129-165.
40. Pahikkala, T., Waegeman, W., Airola, A., Salakoski, T., & De Baets, B. (2010a). Conditional ranking on relational data. In J. L. Balcázar, F. Bonchi, A. Gionis, & M. Sebag (Eds.), Lecture notes in computer science: Vol. 6322. Proceedings of the European conference on machine learning and knowledge discovery in databases (ECML/PKDD'10), part II (pp. 499-514). Berlin: Springer.
41. Pahikkala, T., Waegeman, W., Tsivtsivadze, E., Salakoski, T., & De Baets, B. (2010b). Learning intransitive reciprocal relations with kernel methods. European Journal of Operational Research, 206(3), 676-685.
42. Park, Y., & Marcotte, E. M. (2012). Flaws in evaluation schemes for pair-input computational predictions. Nature Methods, 9(12), 1134-1136.
43. Qin, T., Liu, T. Y., Zhang, X. D., Wang, D. S., Xiong, W. Y., & Li, H. (2008). Learning to rank relational objects and its application to web search. In J. Huai, R. Chen, H. W. Hon, Y. Liu, W. Y. Ma, A. Tomkins, & X. Zhang (Eds.), Proceedings of the 17th international conference on world wide web (pp. 407-416). New York: ACM.
44. Raymond, R., & Kashima, H. (2010). Fast and scalable algorithms for semi-supervised link prediction on static and dynamic graphs. In J. L. Balcázar, F. Bonchi, A. Gionis, & M. Sebag (Eds.), Lecture notes in computer science: Vol. 6323. Proceedings of the 2010 European conference on machine learning and knowledge discovery in databases: part III (pp. 131-147). Berlin: Springer.
45. Rudin, W. (1991). International series in pure and applied mathematics: Functional analysis (2nd ed.). New York: McGraw-Hill
46. Saunders, C., Gammerman, A., & Vovk, V. (1998). Ridge regression learning algorithm in dual variables. In J. W. Shavlik (Ed.), Proceedings of the fifteenth international conference on machine learning (pp. 515-521). San Mateo: Morgan Kaufmann
47. Shawe-Taylor, J., & Cristianini, N. (2004). Kernel methods for pattern analysis. Cambridge: Cambridge University Press.
48. Slabbinck, B., Waegeman, W., Dawyndt, P., De Vos, P., & De Baets, B. (2010). From learning taxonomies to phylogenetic learning: integration of 16S rRNA gene data into FAME-based bacterial classification. BMC Bioinformatics, 11(1), 69.
49. Srebro, N., Rennie, J. D. M., & Jaakkola, T. S. (2005). Maximum-margin matrix factorization. In L. K. Saul, Y. Weiss, & L. Bottou (Eds.), Advances in neural information processing systems (Vol. 17, pp. 1433-1440). Cambridge: MIT Press.
50. Steinwart, I. (2002). On the influence of the kernel on the consistency of support vector machines. Journal of Machine Learning Research, 2, 67-93.
51. Steinwart, I., & Christmann, A. (2008). Information science and statistics: Support vector machines. New York: Springer.
52. Stock, M., Pahikkala, T., Airola, A., Salakoski, T., De Baets, B., & Waegeman, W. (2012). Learning monadic and dyadic relations: three case studies in systems biology. In Proceedings of the ECML/PKDD 2012 workshop on learning and discovery in symbolic systems biology (pp. 74-84).
53. Suykens, J., Van Gestel, T., De Brabanter, J., De Moor, B., & Vandewalle, J. (2002). Least squares support vector machines. Singapore: World Scientific
54. Tsochantaridis, Y., Joachims, T., Hofmann, T., & Altun, Y. (2005). Large margin methods for structured and independent output variables. Journal of Machine Learning Research, 6, 1453-1484.
55. van der Vorst, H. A. (1992). BI-CGSTAB: a fast and smoothly converging variant of BI-CG for the solution of nonsymmetric linear systems. SIAM Journal on Scientific and Statistical Computing, 13(2), 631-644.
56. van Laarhoven, T., Nabuurs, S. B., & Marchiori, E. (2011). Gaussian interaction profile kernels for predicting drug-target interaction. Bioinformatics, 27(21), 3036-3043.
57. Van Loan, C. F. (2000). The ubiquitous Kronecker product. Journal of Computational and Applied Mathematics, 123(1-2), 85-100.
58. Varma, S., & Simon, R. (2006). Bias in error estimation when using cross-validation for model selection. Bioinformatics, 7(1), 91.
59. Vert, J., Qiu, J., & Noble, W. S. (2007). A new pairwise kernel for biological network inference with support vector machines. BMC Bioinformatics, 8, S8.
60. Waegeman, W., Pahikkala, T., Airola, A., Salakoski, T., Stock, M., & De Baets, B. (2012). A kernel-based framework for learning graded relations from data. IEEE Transactions on Fuzzy Systems, 20(6), 1090-1101.
61. Weston, J., Eliseeff, A., Zhou, D., Leslie, C., & Noble, W. S. (2004). Protein ranking: from local to global structure in the protein similarity network. Proceedings of the National Academy of Sciences of the United States of America, 101(17), 6559-6563.
62. Weston, J., Schölkopf, B., Bousquet, O., Mann, T., & Noble, W. (2007). Joint kernel maps. In G. Bakir, T. Hofmann, B. Schölkopf, A. Smola, B. Taskar, & S. Vishwanathan (Eds.), Neural information processing series: Predicting structured data (pp. 67-83). Cambridge: MIT Press.
63. Xia, F., Liu, T. Y., Wang, J., Zhang, W., & Li, H. (2008). Listwise approach to learning to rank: theory and algorithm. In W. W. Cohen, A. McCallum, & S. T. Roweis (Eds.), ACM international conference proceeding series: Vol. 307. Proceedings of the 25th international conference on machine learning (pp. 1192-1199). New York: ACM.
64. Xu, Z., Kersting, K., & Joachims, T. (2010). Fast active exploration for link-based preference learning using Gaussian processes. In J. L. Balcázar, F. Bonchi, A. Gionis, & M. Sebag (Eds.), Lecture notes in computer science: Vol. 6323. Proceedings of the European conference on machine learning and knowledge discovery in databases (ECML/PKDD'10), part III (pp. 499-514). Berlin: Springer.
65. Yamanishi, Y., Vert, J. P., & Kanehisa, M. (2004). Protein network inference from multiple genomic data: a supervised approach. Bioinformatics, 20(1), i363-i370.
66. Yang, Y., Bansal, N., Dakka, W., Ipeirotis, P., Koudas, N., & Papadias, D. (2009). Query by document. In R. A. Baeza-Yates, P. Boldi, B. A. Ribeiro-Neto, & B. B. Cambazoglu (Eds.), Proceedings of the 2nd international conference on web search and data mining (WSDM 2009) (pp. 34-43). New York: ACM.
67. Yue, Y., Finley, T., Radlinski, F., & Joachims, T. (2007). A support vector method for optimizing average precision. In W. Kraaij, A. P. de Vries, C. L. A. Clarke, N. Fuhr, & N. Kando (Eds.), Proceedings of the 30th annual international ACM SIGIR conference on research and development in information retrieval (SIGIR 2007) (pp. 271-278). New York: ACM.