On protocols and measures for the validation of supervised methods for the inference of biological networks

Frontiers in Genetics

Volumn 4, Issue DEC, 2013, Pages

  Schrynemackers, Marie ^a   Küffner, Robert ^b   Geurts, Pierre ^a  

^a UNIVERSITY OF LIÈGE   (Belgium)

^b UNIVERSITY OF MUNICH   (Germany)

Author keywords

Biological network inference; Cross validation; Evaluation protocols; Precision recall curves; ROC curves; Supervised learning

Indexed keywords

EID: 84892380473     PISSN: None     EISSN: 16648021     Source Type: Journal    
DOI: 10.3389/fgene.2013.00262     Document Type: Review

References (62)

1. Bauer, T., Eils, R., and Konig, R. (2011). Rip: the regulatory interaction predictor-a machine learning-based approach for predicting target genes of transcription factors. Bioinformatics 27, 2239-2247. doi: 10.1093/bioinformatics/btr366.
2. Ben-Hur, A., and Noble, W. (2006). Choosing negative examples for the prediction of protein-protein interactions. BMC Bioinformatics 7(Suppl. 1):S2. doi: 10.1186/1471-2105-7-S1-S2.
3. Bleakley, K., Biau, G., and Vert, J.-P. (2007). Supervised reconstruction of biological networks with local models. Bioninformatics 23, i57-i65. doi: 10.1093/bioinformatics/btm204.
4. Bleakley, K., and Yamanishi, Y. (2009). Supervised prediction of drug-target interactions using bipartite local models. Bioinformatics 25, 2397-2403. doi: 10.1093/bioinformatics/btp433.
5. Breiman, L. (2001). Random forests. Mach. Learn. 45, 5-32. doi: 10.1023/A:1017934522171.
6. Brodersen, K. H., Ong, C. S., Stephan, K. E., and Buhmann, J. M. (2010). "The binormal assumption on precision-recall curves," in International Conference on Pattern Recognition (Istanbul), 4263-4266. doi: 10.1109/ICPR.2010.1036.
7. Brouard, C., d'Alché-Buc, F., and Szafranski, M. (2011). "Semi-supervised penalized output kernel regression for link prediction," in Proceedings of ICML (Bellevue, Washington), 593-600.
8. Brunner, C., Fischer, A., Luig, K., and Thies, T. (2012). Pairwise support vector machines and their application to large scale problems. J. Mach. Learn. Res. 13, 2279-2292.
9. Ceccarelli, M., and Cerulo, L. (2009). "Selection of negative examples in learning gene regulatory networks," in IEEE International Conference on Bioinformatics and Biomedicine Workshop, BIBMW 2009 (Washington, DC), 56-61. doi: 10.1109/BIBMW.2009.5332137.
10. Cerulo, L., Elkan, C., and Ceccarelli, M. (2010). Learning gene regulatory networks from only positive and unlabeled data. BMC Bioinformatics 11:228. doi: 10.1186/1471-2105-11-228.
11. Chang, D. T.-H., Syu, Y.-T., and Lin, P.-C. (2010). Predicting the protein-protein interactions using primary structures with predicted protein surface. BMC Bioinformatics 11(Suppl. 1):S3. doi: 10.1186/1471-2105-11-S1-S3.
12. Chen, X.-W., and Liu, M. (2005). Prediction of protein-protein interactions using random decision forest framework. Bioinformatics 21, 4394-4400. doi: 10.1093/bioinformatics/bti721.
13. Cheng, F., Chuang, L., Jiang, J., Lu, W., Li, W., Liu, G., et al. (2012). Prediction of drug-target interactions and drug repositioning via network-based inference. PLos Compuat. Biol. 8:e1002503. doi: 10.1371/journal.pcbi.1002503.
14. Davis, J., and Goadrich, M. (2006). "The relationship between precision-recall and ROC curves," in Proceedings of the 23rd International Conference on Machine Learning (Pittsburgh, PA), 233-240. doi: 10.1145/1143844.1143874.
15. Denis, F., Gilleron, R., and Letouzey, F. (2005). Learning from positive and unlabeled examples. Theor. Comput. Sci. 348, 70-83. doi: 10.1016/j.tcs.2005.09.007.
16. Elkan, C., and Noto, K. (2008). "Learning classifiers from only positive and unlabeled data," in KDD '08: Proceeding of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (New Yor, NY), 213-220. doi: 10.1145/1401890.1401920.
17. Emmert-Streib, F., Glazko, G. V., Altay, G., and de Matos Simoes, R. (2012). Statistical inference and reverse engineering of gene regulatory networks from observational expression data. Front. Genet. 3, 1-15. doi: 10.3389/fgene.2012.00008.
18. Fawcett, T. (2006). An introduction to {ROC} analysis. Pattern Recogn. Lett. 27, 861-874. doi: 10.1016/j.patrec.2005.10.010.
19. Fluss, R., Faraggi, D., and Reiser, B. (2005). Estimation of the Youden Index and its associated cutoff point. Biom. J. (Biometrische Zeitschrift) 47, 458-472. doi: 10.1002/bimj.200410135.
20. Geurts, P. (2011). "Learning from positive and unlabeled examples by enforcing statistical significance," in JMLR: Workshop and Conference Proceedings. Vol. 15 (Lauderdale, FL), 305-314.
21. Geurts, P., Touleimat, N., Dutreix, M., and d'Alché Buc, F. (2007). Inferring biological networks with output kernel trees. BMC Bioinformatics 8(Suppl. 2):S4. doi: 10.1186/1471-2105-8-S2-S4.
22. He, Z., Zhang, J., Shi, X.-H., Hu, L.-L., Kong, X., Cai, Y.-D., et al. (2010). Predicting drug-target interaction networks based on functional groups and biological features. PLoS ONE 5:e9603. doi: 10.1371/journal.pone.0009603.
23. Hempel, S., Koseska, A., Nikoloski, Z., and Kurths, J. (2011). Unraveling gene regulatory networks from time-resolved gene expression data-a measures comparison study. BMC Bioinformatics 12:292. doi: 10.1186/1471-2105-12-292.
24. Hue, M., Riffle, M., Vert, J.-P., and Noble, W. S. (2010). Large-scale prediction of protein-protein interactions from structures. BMC Bioinformatics 11:144. doi: 10.1186/1471-2105-11-144.
25. Hue, M., and Vert, J.-P. (2010). "On learning with kernels for unordered pairs," in Proceedings of the 27th International Conference on Machine Learning (Haifa, Israel). 463-470.
26. Junaid, M., Lapins, M., Eklund, M., Spjuth, O., and Wikberg, J. E. S. (2010). Proteochemometric modeling of the susceptibility of mutated variants of the HIV-1 virus to reverse transcriptase inhibitors. PLoS ONE 5:e14353. doi: 10.1371/journal.pone.0014353.
27. Karni, S., Soreq, H., and Sharan, R. (2009). A network-based method for predicting disease-causing genes. J. Comput. Biol. 16, 181-189. doi: 10.1089/cmb.2008.05TT.
28. Kato, T., Tsuda, K., and Kiyoshi, A. (2005). Selective integration of multiple biological data for supervised network inference. Bioinformatics 21, 2488-2495. doi: 10.1093/bioinformatics/bti339.
29. Lapins, M., and Wikberg, J. E. (2010). Kinome-wide interaction modelling using alignment-based and alignment-independent approaches for kinase description and linear and non-linear data analysis techniques research article. BMC Bioinformatics 11:339. doi: 10.1186/1471-2105-11-339.
30. Lee, W., and Liu, B. (2003). Learning with positive and unlabeled examples using weighted logistic regression. Proc. Int. Conf. Mach. Learn. 20, 448.
31. Li, S., Xi, L., Wang, C., Li, J., Lei, B., Liu, H., et al. (2009). A novel method for protein-ligand binding affinity prediction and the related descriptors exploration. J. Comput. Chem. 30, 900-909. doi: 10.1002/jcc.21078.
32. Maetschke, S. R., Madhamshettiwar, P. B., Davis, M. J., and Ragan, M. A. (2013). Supervised, semi-supervised and unsupervised inference of gene regulatory networks. Brief. Bioinformatics doi: 10.1093/bib/bbt034. [Epub ahead of print].
33. Manning, C., Raghavan, P., and Schütze, H. (2009). An Introduction to Information Retrieval. New York, NY: Cambridge University Press.
34. Marbach, D., Costello, J., Küffner, R., Vega, N., Prill, R., Camacho, D., et al. (2012). wisdom of crowds for robust network inference. Nat. Meth. 9, 794-804. doi: 10.1038/nmeth.2016.
35. Martin, A., Doddington, G., Kamm, T., Ordowski, M., and Przybocki, M. (1997). "The DET curve in assessment of detection task performance," in Fifth European Conference on Speech Communication and Technology, EUROSPEECH 1997 (Rhodes, Greece), 1899-1903.
36. Mei, J.-P., Kwoh, C.-K., Yang, P., Li, X.-L., and Zheng, J. (2013). Drug-target interaction prediction by learning from local information and neighbors. Bioinformatics 29, 238-245. doi: 10.1093/bioinformatics/bts670.
37. Mordelet, F., and Vert, J.-P. (2008). Sirene: supervised inference of regulatory networks. Bioninformatics 24, i76-i82. doi: 10.1093/bioinformatics/btn273.
38. Mordelet, F., and Vert, J.-P. (2013). A bagging SVM to learn from positive and unlabeled examples. Pattern Recog. Lett. doi: 10.1016/j.patrec.2013.06.010. (in press).
39. Niijima, S., Yabuuchi, H., and Okuno, Y. (2011). Cross-target view to feature selection: identification of molecular interaction features in ligand-target space. J. Chem. Inf. Model. 51, 15-24. doi: 10.1021/ci1001394.
40. Paladugu, S. R., Zhao, S., Ray, A., and Raval, A. (2008). Mining protein networks for synthetic genetic interactions. BMC Bioinformatics 9:426. doi: 10.1186/1471-2105-9-426.
41. Pandey, G., Zhang, B., Chang, A. N., Myers, C. L., Zhu, J., Kumar, V., et al. (2010). An integrative multi-network and multi-classifier approach to predict genetic interactions. PLoS Comput. Biol. 6:e1000928. doi: 10.1371/journal.pcbi.1000928.
42. Park, Y., and Marcotte, E. M. (2011). Revisiting the negative example sampling problem for predicting protein-protein interactions. Bioinformatics 27, 3024-3028. doi: 10.1093/bioinformatics/btr514.
43. Qi, Y., Bar-Joseph, Z., and Klein-Seetharaman, J. (2006). Evaluation of different biological data and computational classification methods for use in protein interaction prediction. Proteins 63, 490-500. doi: 10.1002/prot.20865.
44. Ryan, C., Greene, D., Cagney, G., and Cunningham, P. (2010). Missing value imputation for epistatic maps. BMC Bioinformatics 11:197. doi: 10.1186/1471-2105-11-197.
45. Schölkopf, B., Platt, J., Shawe-Taylor, J., Smola, A., and Williamson, R. (2001). Estimating the support of a high-dimensional distribution. Neural Comput. 13, 1443-1471. doi: 10.1162/089976601750264965.
46. Stumpf, M. P. H., and Porter, M. A. (2012). Critical truths about power laws. Science 335, 665-666. doi: 10.1126/science.1216142.
47. Takarabe, M., Kotera, M., Nishimura, Y., Goto, S., and Yamanishi, Y. (2012). Drug target prediction using adverse event report systems: a pharmacogenomic approach. Bioinformatics 28, i611-i618. doi: 10.1093/bioinformatics/bts413.
48. Tastan, O., Qi, Y., Carbonell, J. G., and Klein-Seetharaman, J. (2009). Prediction of interactions between hiv-1 and human proteins by information integration. Pac. Symp. Biocomput. 14, 516-527.
49. Ulitsky, I., Krogan, N., and Shamir, R. (2009). Towards accurate imputation of quantitative genetic interactions. Genome Biol. 10, R140. doi: 10.1186/gb-2009-10-12-r140.
50. van Laarhoven, T., Nabuurs, S. B., and Marchiori, E. (2011). Gaussian interaction profile kernels for predicting drug-target interaction. Bioinformatics 27, 3036-3043. doi: 10.1093/bioinformatics/btr500.
51. Vert, J.-P. (2010). "Reconstruction of biological networks by supervised machine learning approaches," in Elements of Computational Systems Biology, eds H. Lodhi and S. Muggleton (Oxford: John Wiley & Sons, Inc.), 165-188 (Chapter 7). doi: 10.1002/9780470556757.ch7.
52. Vert, J.-P., Qiu, J., and Noble, W. S. (2007). A new pairwise kernel for biological network inference with support vector machines. BMC Bioninformatics 8(Suppl. 10):S8. doi: 10.1186/1471-2105-8-S10-S8.
53. Vert, J.-P., and Yamanishi, Y. (2005). "Supervised graph inference," in Advances in Neural Information and Processing System, (Vancouver, BC), 1433-1440.
54. Witten, I. H., and Frank, E. (2005). Data Mining: Practical Machine Learning Tools and Techniques. 2nd Edn. Boston, MA: Morgan Kaufmann.
55. Wong, S. L., Zhang, L. V., Tong, A. H. Y., Li, Z., Goldberg, D. S., King, O. D., et al. (2004). Combining biological networks to predict genetic interactions. PNAS 101, 15682-15687. doi: 10.1073/pnas.0406614101.
56. Yabuuchi, H., Niijima, S., Takematsu, H., Ida, T., Hirokawa, T., Hara, T., et al. (2011). Analysis of multiple compound-protein interactions reveals novel bioactive molecules. Mol. Syst. Biol. 7, 472. doi: 10.1038/msb.2011.5.
57. Yamanishi, Y. (2009). Supervised bipartite graph inference. Adv. Neural Inform. Process. Syst 21, 1841-1848.
58. Yamanishi, Y., Araki, M., Gutteridge, A., Honda, W., and Kanehisa, M. (2008). Prediction of drug-target interaction networks from the integration of chemical and genomic spaces. Bioinformatics 24, i232-i240. doi: 10.1093/bioinformatics/btn162.
59. Yamanishi, Y., and Vert, J.-P. (2005). Supervised enzyme network inference from the integration of genomic data and chemical information. Bioinformatics 21, i468-i477. doi: 10.1093/bioinformatics/bti1012.
60. Yip, K. Y., and Gerstein, M. (2008). Training set expansion: an approach to improving the reconstruction of biological networks from limited and uneven reliable interactions. Bioinformatics 25, 243-250. doi: 10.1093/bioinformatics/btn602.
61. Yousef, M., Jung, S., Showe, L. C., and Showe, M. K. (2008). Learning from positive examples when the negative class is undetermined-microRNA gene identification. Algorithms Mol. Biol. 3, 2. doi: 10.1186/1748-7188-3-2.
62. Yu, H., Chen, J., Xu, X., Li, Y., Zhao, H., Fang, Y., et al. (2012). A systematic prediction of multiple drug-target interactions from chemical, genomic, and pharmacological data. PLoS ONE 7:e37608. doi: 10.1371/journal.pone.0037608.