Inferring Pairwise Interactions from Biological Data Using Maximum-Entropy Probability Models

PLoS Computational Biology

Volumn 11, Issue 7, 2015, Pages

  Stein, Richard R ^a   Marks, Debora S ^b   Sander, Chris ^a  

^a MEMORIAL SLOAN KETTERING CANCER CENTER   (United States)

^b HARVARD MEDICAL SCHOOL   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

GENE EXPRESSION; MAXIMUM ENTROPY METHODS; PROTEINS;

BIOLOGICAL DATA; DATATYPES; DIRECT INTERACTIONS; ENTROPY-BASED; GENE EXPRESSION DATA; INFERENCE METHODS; MAXIMUM-ENTROPY; PAIRWISE INTERACTION; PROBABILITY MODELLING; SIMILAR SOLUTION;

FORECASTING;

DATA PROCESSING; MATHEMATICAL MODEL; MAXIMUM LIKELIHOOD METHOD; MOLECULAR BIOLOGY; MOLECULAR INTERACTION; PAIRWISE MAXIMUM ENTROPY PROBABILITY MODEL; PREDICTIVE VALUE; PROBABILITY; PROTEIN STRUCTURE; REVIEW; SEQUENCE ALIGNMENT; SEQUENCE DATABASE; STATISTICAL DISTRIBUTION; STOCHASTIC MODEL; THERMODYNAMICS; ALGORITHM; AMINO ACID SEQUENCE; BINDING SITE; CHEMICAL MODEL; CHEMISTRY; COMPUTER SIMULATION; ENTROPY; MOLECULAR GENETICS; PROCEDURES; PROTEIN ANALYSIS; SEQUENCE ANALYSIS; STATISTICAL MODEL;

PROTEIN; PROTEIN BINDING;

ALGORITHMS; AMINO ACID SEQUENCE; BINDING SITES; COMPUTER SIMULATION; ENTROPY; MODELS, CHEMICAL; MODELS, STATISTICAL; MOLECULAR SEQUENCE DATA; PROTEIN BINDING; PROTEIN INTERACTION MAPPING; PROTEINS; SEQUENCE ANALYSIS, PROTEIN;

EID: 84938651255     PISSN: 1553734X     EISSN: 15537358     Source Type: Journal    
DOI: 10.1371/journal.pcbi.1004182     Document Type: Review

References (86)

1. Lezon TR, Banavar JR, Cieplak M, Maritan A, Fedoroff NV, Using the principle of entropy maximization to infer genetic interaction networks from gene expression patterns. Proceedings of the National Academy of Sciences of the United States of America. 2006;103(50):19033–19038. 17138668
2. Locasale JW, Wolf-Yadlin A, Maximum entropy reconstructions of dynamic signaling networks from quantitative proteomics data. PloS one. 2009;4(8):e6522. doi: 10.1371/journal.pone.0006522 19707567
3. Schneidman E, Berry II MJ, Segev R, Bialek W, Weak pairwise correlations imply strongly correlated network states in a neural population. Nature. 2006;440:1007–1012. 16625187
4. Tang A, Jackson D, Hobbs J, Chen W, Smith JL, Patel H, et al. A maximum entropy model applied to spatial and temporal correlations from cortical networks in vitro. The Journal of Neuroscience. 2008;28(2):505–518. doi: 10.1523/JNEUROSCI.3359-07.2008 18184793
5. Weigt M, White RA, Szurmant H, Hoch JA, Hwa T, Identification of direct residue contacts in protein—protein interaction by message passing. Proceedings of the National Academy of Sciences of the United States of America. 2009;106(1):67–72. doi: 10.1073/pnas.0805923106 19116270
6. Marks DS, Colwell LJ, Sheridan R, Hopf TA, Pagnani A, Zecchina R, et al. Protein 3D Structure Computed from Evolutionary Sequence Variation. PLoS One. 2011;6(12):e28766. doi: 10.1371/journal.pone.0028766 22163331
7. Morcos F, Pagnani A, Lunt B, Bertolino A, Marks D, Sander C, et al. Direct-coupling analysis of residue co-evolution captures native contacts across many protein families. Proceedings of the National Academy of Sciences of the United States of America. 2011;108:E1293–E1301. doi: 10.1073/pnas.1111471108 22106262
8. Jones DT, Buchan DWA, Cozzetto D, Pontil M, PSICOV: precise structural contact prediction using sparse inverse covariance estimation on large multiple sequence alignments. Bioinformatics. 2012;28(2):184–190. doi: 10.1093/bioinformatics/btr638 22101153
9. Stephens GJ, Bialek W, Statistical mechanics of letters in words. Physical Review E. 2010;81(6):066119.
10. Bialek W, Cavagna A, Giardina I, Mora T, Silvestri E, Viale M, et al. Statistical mechanics for natural flocks of birds. Proceedings of the National Academy of Sciences. 2012;109(13):4786–4791.
11. Wood K, Nishida S, Sontag ED, Cluzel P, Mechanism-independent method for predicting response to multidrug combinations in bacteria. Proceedings of the National Academy of Sciences of the United States of America. 2012;109(30):12254–12259. doi: 10.1073/pnas.1201281109 22773816
12. Whittaker J, Graphical models in applied multivariate statistics. Wiley Publishing; 2009.
13. Lauritzen SL, Graphical models. Oxford: Oxford University Press; 1996.
14. Butte AJ, Kohane IS. Unsupervised knowledge discovery in medical databases using relevance networks. In: Proceedings of the AMIA Symposium. American Medical Informatics Association; 1999. p. 711–715.
15. Toh H, Horimoto K, Inference of a genetic network by a combined approach of cluster analysis and graphical Gaussian modeling. Bioinformatics. 2002;18(2):287–297. 11847076
16. Dobra A, Hans C, Jones B, Nevins JR, Yao G, West M, Sparse graphical models for exploring gene expression data. Journal of Multivariate Analysis. 2004;90(1):196–212.
17. Schäfer J, Strimmer K, A shrinkage approach to large-scale covariance matrix estimation and implications for functional genomics. Statistical applications in genetics and molecular biology. 2005;4(1):1–32.
18. Roudi Y, Nirenberg S, Latham PE, Pairwise maximum entropy models for studying large biological systems: when they can work and when they can’t. PLoS Computational Biology. 2009;5(5):e1000380. doi: 10.1371/journal.pcbi.1000380 19424487
19. Cramér H, Mathematical methods of statistics. vol. 9. Princeton university press; 1999.
20. Guttman L, A note on the derivation of formulae for multiple and partial correlation. The Annals of Mathematical Statistics. 1938;9(4):305–308.
21. Krumsiek J, Suhre K, Illig T, Adamski J, Theis FJ, Gaussian graphical modeling reconstructs pathway reactions from high-throughput metabolomics data. BMC Systems Biology. 2011;5(1):21.
22. Giraud BG, Heumann John M, Lapedes Alan S, Superadditive correlation. Physical Review E. 1999;59(5):4983–4991.
23. Neher E, How frequent are correlated changes in families of protein sequences? Proceedings of the National Academy of Sciences of the United States of America. 1994;91(1):98–102. 8278414
24. Göbel U, Sander C, Schneider R, Valencia A, Correlated mutations and residue contacts in proteins. Proteins. 1994;18(4):309–317. 8208723
25. Taylor WR, Hatrick K, Compensating changes in protein multiple sequence alignments. Protein Engineering. 1994;7(3):341–348. 8177883
26. Shindyalov IN, Kolchanov NA, Sander C, Can three-dimensional contacts in protein structures be predicted by analysis of correlated mutations? Protein Engineering. 1994;7(3):349–358. 8177884
27. Dunn SD, Wahl LM, Gloor GB, Mutual information without the influence of phylogeny or entropy dramatically improves residue contact prediction. Bioinformatics. 2008;24(3):333–340. 18057019
28. Burger L, Van Nimwegen E, Disentangling direct from indirect co-evolution of residues in protein alignments. PLoS computational biology. 2010;6(1):e1000633. doi: 10.1371/journal.pcbi.1000633 20052271
29. Atchley WR, Wollenberg KR, Fitch WM, Terhalle W, Dress AW, Correlations among amino acid sites in bHLH protein domains: an information theoretic analysis. Molecular biology and evolution. 2000;17(1):164–178. 10666716
30. Lapedes A, Giraud B, Jarzynski C. Using Sequence Alignments to Predict Protein Structure and Stability With High Accuracy. eprint arXiv:12072484. 2002;.
31. Ciriello G, Miller ML, Aksoy BA, Senbabaoglu Y, Schultz N, Sander C, Emerging landscape of oncogenic signatures across human cancers. Nature genetics. 2013;45(10):1127–1133. doi: 10.1038/ng.2762 24071851
32. Koller D, Friedman N, Probabilistic graphical models: principles and techniques. MIT press; 2009.
33. Mead LR, Papanicolaou N, Maximum entropy in the problem of moments. Journal of Mathematical Physics. 1984;25:2404–2417.
34. MacKay DJ, Information theory, inference and learning algorithms. Cambridge university press; 2003.
35. Cover TM, Thomas AJ, Elements of information theory. John Wiley & Sons; 2012.
36. Agmon N, Alhassid Y, Levine RD, An algorithm for finding the distribution of maximal entropy. Journal of Computational Physics. 1979;30(2):250–258.
37. Shannon CE, A Mathematical Theory of Communication. Bell system technical journal. 1948;27(3):379–423.
38. Jaynes ET, Information Theory and Statistical Mechanics. Physical Review. 1957;106(4):620–630.
39. Jaynes ET, Information Theory and Statistical Mechanics II. Physical Review. 1957;108(2):171–190.
40. Jaynes ET, Probability theory: the logic of science. Cambridge: Cambridge university press; 2003.
41. Murphy KP, Machine learning: a probabilistic perspective. The MIT Press; 2012.
42. Balescu R, Matter out of Equilibrium. World Scientific; 1997.
43. Goldstein S, Lebowitz JL, On the (Boltzmann) entropy of non-equilibrium systems. Physica D: Nonlinear Phenomena. 2004;193(1):53–66.
44. Durbin R, Eddy SR, Krogh A, Mitchison G, Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids. Cambridge University Press; 1998.
45. Finn RD, Bateman A, Clements J, Coggill P, Eberhardt RY, Eddy SR, et al. Pfam: the protein families database. Nucleic acids research. 2014;42:D222–D230. doi: 10.1093/nar/gkt1223 24288371
46. Remmert M, Biegert A, Hauser A, Söding J, HHblits: lightning-fast iterative protein sequence searching by HMM-HMM alignment. Nature methods. 2012;9(2):173–175.
47. Finn RD, Clements J, Eddy SR, HMMER web server: interactive sequence similarity searching. Nucleic acids research. 2011;p. gkr367.
48. Baldassi C, Zamparo M, Feinauer C, Procaccini A, Zecchina R, Weigt M, et al. Fast and accurate multivariate Gaussian modeling of protein families: Predicting residue contacts and protein-interaction partners. PloS one. 2014;9(3):e92721. doi: 10.1371/journal.pone.0092721 24663061
49. Lapedes AS, Giraud BG, Liu LC, Stormo GD. A Maximum Entropy Formalism for Disentangling Chains of Correlated Sequence Positions. In: Proceedings of the IMS/AMS International Conference on Statistics in Molecular Biology and Genetics; 1998. p. 236–256.
50. Santolini M, Mora T, Hakim V, A general pairwise interaction model provides an accurate description of in vivo transcription factor binding sites. PloS one. 2014;9(6):e99015. doi: 10.1371/journal.pone.0099015 24926895
51. Ekeberg M, Lövkvist C, Lan Y, Weigt M, Aurell E, Improved contact prediction in proteins: Using pseudolikelihoods to infer Potts models. Physical Review E. 2013;87(1):012707.
52. Bishop CM, Pattern recognition and machine learning. New York: Springer-Verlag; 2006.
53. Meinshausen N, Bühlmann P, High-dimensional graphs and variable selection with the lasso. The Annals of Statistics. 2006;34(3):1436–1462.
54. Friedman J, Hastie T, Tibshirani R, Sparse inverse covariance estimation with the graphical lasso. Biostatistics. 2008;9(3):432–441. 18079126
55. Witten DM, Tibshirani R, Covariance-regularized regression and classification for high dimensional problems. Journal of the Royal Statistical Society: Series B (Statistical Methodology). 2009;71(3):615–636.
56. Ledoit O, Wolf M, A well-conditioned estimator for large-dimensional covariance matrices. Journal of multivariate analysis. 2004;88(2):365–411.
57. Kappen HJ, Rodriguez F, Efficient learning in Boltzmann machines using linear response theory. Neural Computation. 1998;10(5):1137–1156.
58. Tanaka T, Mean-field theory of Boltzmann machine learning. Physical Review E. 1998;58(2):2302–2310.
59. Roudi Y, Aurell E, Hertz JA, Statistical physics of pairwise probability models. Frontiers in computational neuroscience. 2009;3. doi: 10.3389/neuro.10.003.2009 19242556
60. Wainwright MJ, Jordan MI, Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning. 2008;1(1–2):1–305.
61. Broderick T, Dudik M, Tkacik G, Schapire RE, Bialek W. Faster solutions of the inverse pairwise Ising problem. arXiv preprint arXiv:07122437. 2007;.
62. Besag J, Statistical analysis of non-lattice data. The Statistician. 1975;24(3):179–195.
63. Liang P, Jordan MI. An asymptotic analysis of generative, discriminative, and pseudolikelihood estimators. In: Proceedings of the 25th international conference on Machine learning. ACM; 2008. p. 584–591.
64. Höfling H, Tibshirani R, Estimation of sparse binary pairwise markov networks using pseudo-likelihoods. The Journal of Machine Learning Research. 2009;10:883–906.
65. Balakrishnan S, Kamisetty H, Carbonell JG, Lee SI, Langmead CJ, Learning generative models for protein fold families. Proteins: Structure, Function, and Bioinformatics. 2011;79(4):1061–1078.
66. Kamisetty H, Ovchinnikov S, Baker D, Assessing the utility of coevolution-based residue—residue contact predictions in a sequence-and structure-rich era. Proceedings of the National Academy of Sciences. 2013;110(39):15674–15679.
67. Ovchinnikov S, Kamisetty H, Baker D, Robust and accurate prediction of residue—residue interactions across protein interfaces using evolutionary information. eLife. 2014;3: e02030. doi: 10.7554/eLife.02030 24842992
68. Wainwright MJ, Jordan MI, Log-determinant relaxation for approximate inference in discrete Markov random fields. IEEE Transactions on Signal Processing. 2006;54(6):2099–2109.
69. Banerjee O, El Ghaoui L, d’Aspremont A, Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data. The Journal of Machine Learning Research. 2008;9:485–516.
70. Ravikumar P, Wainwright MJ, Lafferty JD, High-dimensional Ising model selection using l1-regularized logistic regression. The Annals of Statistics. 2010;38(3):1287–1319.
71. Sessak V, Monasson R, Small-correlation expansions for the inverse Ising problem. Journal of Physics A: Mathematical and Theoretical. 2009;42(5):055001.
72. Lapedes AS, Giraud BG, Liu L, Stormo GD, Correlated mutations in models of protein sequences: phylogenetic and structural effects. In: Statistics in Molecular Biology/IMS Lecture Notes—Monograph Series. JSTOR; 1999. p. 236–256.
73. Rockman MV, Reverse engineering the genotype—phenotype map with natural genetic variation. Nature. 2008;456(7223):738–744. doi: 10.1038/nature07633 19079051
74. Ritchie MD, Holzinger ER, Li R, Pendergrass SA, Kim D, Methods of integrating data to uncover genotype-phenotype interactions. Nature Reviews Genetics. 2015;16(2):85–97. doi: 10.1038/nrg3868 25582081
75. Russ WP, Lowery DM, Mishra P, Yaffe MB, Ranganathan R, Natural-like function in artificial WW domains. Nature. 2005; 437(7058):579–583. 16177795
76. EVFold. http://evfold.org.
77. Direct Coupling Analysis. http://dca.rice.edu.
78. Ekeberg M. pseudolikelihood maximization Direct-Coupling Analysis. http://plmdca.csc.kth.se.
79. Pagnani A. Pseudo Likelihood Maximization for protein in Julia. https://github.com/pagnani/PlmDCA.
80. CCMpred. https://bitbucket.org/soedinglab/ccmpred.
81. Gremlin. http://gremlin.bakerlab.org.
82. Psicov. http://bioinfadmin.cs.ucl.ac.uk/downloads/PSICOV.
83. Friedman J, Hastie T, Tibshirani R. Graphical lasso in R and Matlab. http://statweb.stanford.edu/~tibs/glasso/.
84. Witten DM, Tibshirani R. scout: Implements the Scout method for Covariance-Regularized Regression. http://cran.r-project.org/web/packages/scout/index.html.
85. Schäfer J, Opgen-Rhein R, Strimmer K. Modeling and Inferring Gene Networks. http://strimmerlab.org/software/genenet/.
86. Schäfer J, Opgen-Rhein R, Zuber V, Ahdesmäki M, Silva APD, Strimmer K. Efficient Estimation of Covariance and (Partial) Correlation. http://strimmerlab.org/software/corpcor/.