Inferring causal molecular networks: Empirical assessment through a community-based effort

Nature Methods

Volumn 13, Issue 4, 2016, Pages 310-322

  Hill, Steven M ^a   Heiser, Laura M ^b,c   Cokelaer, Thomas ^d,bg   Linger, Michael ^e   Nesser, Nicole K ^c   Carlin, Daniel E ^f   Zhang, Yang ^g,bh   Sokolov, Artem ^f   Paull, Evan O ^f   Wong, Chris K ^f   Graim, Kiley ^f   Bivol, Adrian ^f   Wang, Haizhou ^g,bi   Zhu, Fan ^a   Afsari, Bahman ⁱ   Danilova, Ludmila V ^i,j   Favorov, Alexander V ^c,i,k   Lee, Wai Shing ⁱ   Taylor, Dane ^l,m   Hu, Chenyue W ⁿ   Long, Byron L ⁿ   Noren, David P ⁿ   Bisberg, Alexander J ⁿ   Mills, Gordon B ^o   Gray, Joe W ^b,c   Kellen, Michael ^p   Norman, Thea ^p   Friend, Stephen ^p   Qutub, Amina A ⁿ   Fertig, Elana J ⁱ   Guan, Yuanfang ^h,q   Song, Mingzhou ^g   Stuart, Joshua M ^f   Spellman, Paul T ^c   Koeppl, Heinz ^e,bj   Stolovitzky, Gustavo ^r   Saez Rodriguez, Julio ^d,s   Mukherjee, Sach ^a,t,u   Al Ouran, Rami ^v   Anton, Bernat ^w   Arodz, Tomasz ^x   Sichani, Omid Askari ^y   Bagheri, Neda ^z   Berlow, Noah ^aa   Bohler, Anwesha ^ab   Bonet, Jaume ^w   Bonneau, Richard ^ac,ad,ae   Budak, Gungor ^ab   Bunescu, Razvan ^v   Caglar, Mehmet ^af   Cai, Binghuang ^ag   Cai, Chunhui ^ag   Carlon, Azzurra ^ah   Chen, Lujia ^ag   Ciaccio, Mark F ^z   Cooper, Gregory ^ag   Coort, Susan ^ab   Creighton, Chad J ^ai   Daneshmand, Seyed Mohammad Hadi ^y   De La Fuente, Alberto ^aj   Di Camillo, Barbara ^ah   Dutta Moscato, Joyeeta ^ag   Emmett, Kevin ^ak   Evelo, Chris ^ab   Fassia, Mohammad Kasim H ^z   Finotello, Francesca ^ah   Finkle, Justin D ^al   Gao, Xi ^x   Gao, Jean ^am   Ghosh, Samik ^an   Giaretta, Alberto ^ah   Großeholz, Ruth ^ao   Guinney, Justin ^p   Hafemeister, Christoph ^ac   Hahn, Oliver ^ao   Haider, Saad ^aa   Hase, Takeshi ^an   Hodgson, Jay ^p   Hoff, Bruce ^p   Hsu, Chih Hao ^ap   Hu, Ying ^ap   Huang, Xun ^aq   Jalili, Mahdi ^y   Jiang, Xia ^ag   Kacprowski, Tim ^ar   Kaderali, Lars ^as,at   Kang, Mingon ^am   Kannan, Venkateshan ^au   Kikuchi, Kaito ^an   Kim, Dong Chul ^av   Kitano, Hiroaki ^an   Knapp, Bettina ^as,aw   Komatsoulis, George ^ap   Krämer, Andreas ^ax   Kursa, Miron Bartosz ^ay   Kutmon, Martina ^ab   Li, Yichao ^v   Liang, Xiaoyu ^v   Liu, Zhaoqi ^az   Liu, Yu ^ba   Lu, Songjian ^ag   Lu, Xinghua ^ag   Manfrini, Marco ^ah   Matos, Marta R A ^as   Meerzaman, Daoud ^ap   Min, Wenwen ^az   Müller, Christian Lorenz ^ac,ad   Neapolitan, Richard E ^bb   Oliva, Baldo ^w   Opiyo, Stephen Obol ^bc   Pal, Ranadip ^aa   Palinkas, Aljoscha ^bd   Planas Iglesias, Joan ^w   Poglayen, Daniel ^w   Sambo, Francesco ^ah   Sanavia, Tiziana ^ah   Sharifi Zarchi, Ali ^be   Slawek, Janusz ^x   Streck, Adam ^bd   Strunz, Sonja ^aj   Tegnér, Jesper ^au   Thobe, Kirste ^bd   Toffolo, Gianna Maria ^ah   Trifoglio, Emanuele ^ah   Unger, Michael ^e   Wan, Qian ^aa   Welch, Lonnie ^v   Wu, Jia J ^al   Xue, Albert Y ^z   Yamanaka, Ryota ^an   Yan, Chunhua ^ap   Zairis, Sakellarios ^bf   Zengerling, Michael ^ao   Zenil, Hector ^au   Zi, Zhike ^aq   more..

^a UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^b Oregon Health and Science University Department of Biomedical Engineering   (United States)

^c OREGON HEALTH AND SCIENCE UNIVERSITY   (United States)

^d EUROPEAN BIOINFORMATICS INSTITUTE   (United Kingdom)

^e ETH ZURICH   (Switzerland)

^f University of California Santa Cruz   (United States)

^g New Mexico State University   (United States)

^h UNIVERSITY OF MICHIGAN MEDICAL SCHOOL   (United States)

ⁱ JOHNS HOPKINS UNIVERSITY   (United States)

^j VAVILOV INSTITUTE OF GENERAL GENETICS   (Russian Federation)

^k STATE RESEARCH INSTITUTE OF GENETICS AND SELECTION OF INDUSTRIAL MICROORGANISMS   (Russian Federation)

^l Statistical and Applied Mathematical Sciences Institute USA   (United States)

^m The University of North Carolina at Chapel Hill   (United States)

ⁿ Rice University   (United States)

^o UNIVERSITY OF TEXAS MD ANDERSON CANCER CENTER   (United States)

^p SAGE BIONETWORKS   (United States)

^q UNIVERSITY OF MICHIGAN   (United States)

^r IBM   (United States)

^s UNIVERSITY HOSPITAL RWTH AACHEN   (Germany)

^t UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^u GERMAN CENTER FOR NEURODEGENERATIVE DISEASES DZNE   (Germany)

^v US Institute of Navigation   (United States)

^w UNIVERSITAT POMPEU FABRA   (Spain)

^x Virginia Commonwealth University   (United States)

^y SHARIF UNIVERSITY OF TECHNOLOGY   (Iran)

^z Northwestern University   (United States)

^aa Texas Tech University   (United States)

^ab MAASTRICHT UNIVERSITY   (Netherlands)

^ac NEW YORK UNIVERSITY   (United States)

^ad NEW YORK UNIVERSITY   (United States)

^ae Simons Center for Data Analysis   (United States)

^af TEXAS TECH UNIVERSITY   (United States)

^ag UNIVERSITY OF PITTSBURGH SCHOOL OF MEDICINE   (United States)

^ah UNIVERSITY OF PADOVA   (Italy)

^ai BAYLOR COLLEGE OF MEDICINE   (United States)

^aj LEIBNIZ INSTITUTE FOR FARM ANIMAL BIOLOGY FBN   (Germany)

^ak National Cancer Institute ^*  (United States)

^al NORTHWESTERN UNIVERSITY   (United States)

^am University of Texas at Arlington   (United States)

^an Systems Biology Institute   (Japan)

^ao UNIVERSITY OF HEIDELBERG   (Germany)

^ap NATIONAL CANCER INSTITUTE   (United States)

^aq UNIVERSITY OF FREIBURG   (Germany)

^ar UNIVERSITY OF GREIFSWALD   (Germany)

^as DRESDEN UNIVERSITY OF TECHNOLOGY   (Germany)

^at UNIVERSITY MEDICINE GREIFSWALD   (Germany)

^au SCIENCE FOR LIFE LABORATORY   (Sweden)

^av UNIVERSITY OF TEXAS PAN AMERICAN   (United States)

^aw INSTITUTE OF EPIDEMIOLOGY   (Germany)

^ax Qiagen Bioinformatics   (United States)

^ay UNIVERSITY OF WARSAW   (Poland)

^az CHINESE ACADEMY OF SCIENCES   (China)

^ba ST JUDE CHILDREN S RESEARCH HOSPITAL   (United States)

^bb NORTHWESTERN UNIVERSITY   (United States)

^bc OHIO STATE UNIVERSITY   (United States)

^bd FREIE UNIVERSITÄT BERLIN   (Germany)

^be ROYAN INSTITUTE FOR REPRODUCTIVE BIOMEDICINE   (Iran)

^bf Och Spine at New York Presbyterian Hospitals   (United States)

^bg INSTITUT PASTEUR   (France)

^bh Amyris Inc   (United States)

^bi SimQuest   (United States)

^bj DARMSTADT UNIVERSITY OF TECHNOLOGY   (Germany)

more..

Author keywords

[No Author keywords available]

Indexed keywords

PHOSPHOPROTEIN; PROTEIN TYROSINE KINASE;

ARTICLE; CANCER CELL LINE; CAUSAL MODELING; CAUSAL MOLECULAR NETWORK; CONTROLLED STUDY; HUMAN; HUMAN CELL; LEARNING; MOLECULAR BIOLOGY; PHENOTYPE; PRIORITY JOURNAL; SIGNAL TRANSDUCTION; VALIDITY; ALGORITHM; BIOLOGICAL MODEL; BIOLOGY; CAUSALITY; COMPUTER SIMULATION; GENE EXPRESSION PROFILING; GENE REGULATORY NETWORK; GENETICS; NEOPLASM; PROCEDURES; PROTEIN ANALYSIS; SOFTWARE; SYSTEMS BIOLOGY; TUMOR CELL CULTURE;

ALGORITHMS; CAUSALITY; COMPUTATIONAL BIOLOGY; COMPUTER SIMULATION; GENE EXPRESSION PROFILING; GENE REGULATORY NETWORKS; HUMANS; MODELS, BIOLOGICAL; NEOPLASMS; PROTEIN INTERACTION MAPPING; SIGNAL TRANSDUCTION; SOFTWARE; SYSTEMS BIOLOGY; TUMOR CELLS, CULTURED;

EID: 84978621488     PISSN: 15487091     EISSN: 15487105     Source Type: Journal    
DOI: 10.1038/nmeth.3773     Document Type: Article

References (48)

1. Bansal, M., Belcastro, V., Ambesi-Impiombato, A. & di Bernardo, D. How to infer gene networks from expression profiles. Mol. Syst. Biol. 3, 78 (2007).
2. Markowetz, F. & Spang, R. Inferring cellular networks—a review. BMC Bioinformatics 8, S5 (2007).
3. Hecker, M., Lambeck, S., Toepfer, S., van Someren, E. & Guthke, R. Gene regulatory network inference: data integration in dynamic models—a review. Biosystems 96, 86-103 (2009).
4. De Smet, R. & Marchal, K. Advantages and limitations of current network inference methods. Nat. Rev. Microbiol. 8, 717-729 (2010).
5. Marbach, D. et al. Revealing strengths and weaknesses of methods for gene network inference. Proc. Natl. Acad. Sci. USA 107, 6286-6291 (2010).
6. Maetschke, S.R., Madhamshettiwar, P.B., Davis, M.J. & Ragan, M.A. Supervised, semi-supervised and unsupervised inference of gene regulatory networks. Brief. Bioinform. 15, 195-211 (2014).
7. Ideker, T. & Krogan, N.J. Differential network biology. Mol. Syst. Biol. 8, 565 (2012).
8. de la Fuente, A. From ‘differential expression’ to ‘differential networking’—identification of dysfunctional regulatory networks in diseases. Trends Genet. 26, 326-333 (2010).
9. Hill, S.M. et al. Bayesian inference of signaling network topology in a cancer cell line. Bioinformatics 28, 2804-2810 (2012).
10. Saez-Rodriguez, J. et al. Comparing signaling networks between normal and transformed hepatocytes using discrete logical models. Cancer Res. 71, 5400-5411 (2011).
11. Molinelli, E.J. et al. Perturbation biology: inferring signaling networks in cellular systems. PLoS Comput. Biol. 9, e1003290 (2013).
12. Chen, W.W. et al. Input-output behavior of ErbB signaling pathways as revealed by a mass action model trained against dynamic data. Mol. Syst. Biol. 5, 239 (2009).
13. Akbani, R. et al. A pan-cancer proteomic perspective on The Cancer Genome Atlas. Nat. Commun. 5, 3887 (2014).
14. Eduati, F., De Las Rivas, J., Di Camillo, B., Toffolo, G. & Saez-Rodriguez, J. Integrating literature-constrained and data-driven inference of signalling networks. Bioinformatics 28, 2311-2317 (2012).
15. Pearl, J. Causality: Models, Reasoning, and Inference 2nd edn. (Cambridge Univ. Press, 2009).
16. Freedman, D. & Humphreys, P. Are there algorithms that discover causal structure? Synthese 121, 29-54 (1999).
17. Husmeier, D. Sensitivity and specificity of inferring genetic regulatory interactions from microarray experiments with dynamic Bayesian networks. Bioinformatics 19, 2271-2282 (2003).
18. Friedman, N., Linial, M., Nachman, I. & Pe’er, D. Using Bayesian networks to analyze expression data. J. Comput. Biol. 7, 601-620 (2000).
19. Sachs, K., Perez, O. & Pe’er, D. Causal protein-signaling networks derived from multiparameter single-cell data. Science 308, 523-529 (2005).
20. Spirtes, P., Glymour, C.N. & Scheines, R. Causation, Prediction, and Search 2nd edn. (MIT Press, 2000).
21. Cantone, I. et al. A yeast synthetic network for in vivo assessment of reverse-engineering and modeling approaches. Cell 137, 172-181 (2009).
22. Marbach, D. et al. Wisdom of crowds for robust gene network inference. Nat. Methods 9, 796-804 (2012).
23. Stolovitzky, G., Monroe, D. & Califano, A. Dialogue on reverse-engineering assessment and methods: the DREAM of high-throughput pathway inference. Ann. NY Acad. Sci. 1115, 1-22 (2007).
24. Stolovitzky, G., Prill, R.J. & Califano, A. Lessons from the DREAM2 challenges. Ann. NY Acad. Sci. 1158, 159-195 (2009).
25. Prill, R.J. et al. Towards a rigorous assessment of systems biology models: the DREAM3 challenges. PLoS ONE 5, e9202 (2010).
26. Prill, R.J., Saez-Rodriguez, J., Alexopoulos, L.G., Sorger, P.K. & Stolovitzky, G. Crowdsourcing network inference: the DREAM predictive signaling network challenge. Sci. Signal. 4, mr7 (2011).
27. Meyer, P. et al. Network topology and parameter estimation: from experimental design methods to gene regulatory network kinetics using a community based approach. BMC Syst. Biol. 8, 13 (2014).
28. Tibes, R. et al. Reverse phase protein array: validation of a novel proteomic technology and utility for analysis of primary leukemia specimens and hematopoietic stem cells. Mol. Cancer Ther. 5, 2512-2521 (2006).
29. Mertins, P. et al. Ischemia in tumors induces early and sustained phosphorylation changes in stress kinase pathways but does not affect global protein levels. Mol. Cell. Proteomics 13, 1690-1704 (2014).
30. Derry, J.M.J. et al. Developing predictive molecular maps of human disease through community-based modeling. Nat. Genet. 44, 127-130 (2012).
31. Hill, S.M. et al. Context-specificity in causal signaling networks revealed by phosphoprotein profiling. bioRxiv doi:10.1101/039636 (2016).
32. Davis, J. & Goadrich, M. The relationship between Precision-Recall and ROC curves. in Proc. 23rd International Conference on Machine Learning 233-240 (ACM, 2006).
33. Costello, J.C. et al. A community effort to assess and improve drug sensitivity prediction algorithms. Nat. Biotechnol. 32, 1202-1212 (2014).
34. Margolin, A.A. et al. Systematic analysis of challenge-driven improvements in molecular prognostic models for breast cancer. Sci. Transl. Med. 5, 181re1 (2013).
35. Cerami, E.G. et al. Pathway Commons, a web resource for biological pathway data. Nucleic Acids Res. 39, D685-D690 (2011).
36. Wang, H. & Song, M. Ckmeans.1d.dp: optimal k-means clustering in one dimension by dynamic programming. R J. 3, 29-33 (2011).
37. Chresta, C.M. et al. AZD8055 is a potent, selective, and orally bioavailable ATP-competitive mammalian target of rapamycin kinase inhibitor with in vitro and in vivo antitumor activity. Cancer Res. 70, 288-298 (2010).
38. Maathuis, M.H., Colombo, D., Kalisch, M. & Bühlmann, P. Predicting causal effects in large-scale systems from observational data. Nat. Methods 7, 247-248 (2010).
39. Olsen, C. et al. Inference and validation of predictive gene networks from biomedical literature and gene expression data. Genomics 103, 329-336 (2014).
40. Shannon, P. et al. Cytoscape: a software environment for integrated models of biomolecular interaction networks. Genome Res. 13, 2498-2504 (2003).
41. Neve, R.M. et al. A collection of breast cancer cell lines for the study of functionally distinct cancer subtypes. Cancer Cell 10, 515-527 (2006).
42. Garnett, M.J. et al. Systematic identification of genomic markers of drug sensitivity in cancer cells. Nature 483, 570-575 (2012).
43. Barretina, J. et al. The Cancer Cell Line Encyclopedia enables predictive modelling of anticancer drug sensitivity. Nature 483, 603-607 (2012).
44. Hennessy, B.T. et al. A technical assessment of the utility of reverse phase protein arrays for the study of the functional proteome in non-microdissected human breast cancers. Clin. Proteomics 6, 129-151 (2010).
45. Eduati, F. et al. Prediction of human population responses to toxic compounds by a collaborative competition. Nat. Biotechnol. 33, 933-940 (2015).
46. Guitart-Pla, O., Kustagi, M., Rügheimer, F., Califano, A. & Schwikowski, B. The Cyni framework for network inference in Cytoscape. Bioinformatics 31, 1499-1501 (2015).
47. Benjamini, Y., Krieger, A.M. & Yekutieli, D. Adaptive linear step-up procedures that control the false discovery rate. Biometrika 93, 491-507 (2006).
48. Cokelaer, T. et al. DREAMTools: a Python package for scoring collaborative challenges. F1000Research 4, 1030 (2015).