Robust and efficient parameter estimation in dynamic models of biological systems

BMC Systems Biology

Volumn 9, Issue 1, 2015, Pages

  Gábor, Attila ^a   Banga, Julio R ^a  

^a INSTITUTO DE INVESTIGACIONES MARINAS   (Spain)

Author keywords

Dynamic models; Global optimization; Overfitting; Parameter estimation; Regularization

Indexed keywords

ALGORITHM; BIOLOGICAL MODEL; METABOLISM; NONLINEAR SYSTEM; PROCEDURES; SIGNAL TRANSDUCTION; SYSTEMS BIOLOGY;

ALGORITHMS; METABOLIC NETWORKS AND PATHWAYS; MODELS, BIOLOGICAL; NONLINEAR DYNAMICS; SIGNAL TRANSDUCTION; SYSTEMS BIOLOGY;

EID: 84945919257     PISSN: None     EISSN: 17520509     Source Type: Journal    
DOI: 10.1186/s12918-015-0219-2     Document Type: Article

References (130)

1. Mogilner A, Wollman R, Marshall WF. Quantitative modeling in cell biology: what is it good for? Dev Cell. 2006;11(3):279-87.
2. Epstein JM. Why model? J Artif Soc Social Simul. 2008;11(4):12.
3. Wolkenhauer O. Why model? Front Physiol. 2014;5:21.
4. Lander AD. The edges of understanding. BMC Biol. 2010;8(1):40.
5. Wolkenhauer O. Systems biology: the reincarnation of systems theory applied in biology? Brief Bioinform. 2001;2(3):258-70.
6. Stelling J. Mathematical models in microbial systems biology. Curr Opin Microbiol. 2004;7(5):513-8.
7. Wolkenhauer O, MesarovícM. Feedback dynamics and cell function: why systems biology is called systems biology. Mol BioSyst. 2005;1(1):14-6.
8. De Jong H. Modeling and simulation of genetic regulatory systems: a literature review. J Comput Biol. 2002;9(1):67-103.
9. Tyson JJ, Chen KC, Novak B. Sniffers, buzzers, toggles and blinkers: dynamics of regulatory and signaling pathways in the cell. Curr Oinion Cell Biol. 2003;15(2):221-31.
10. Kholodenko BN. Cell-signalling dynamics in time and space. Mol Cell Biol. 2006;7(3):165-76.
11. Aldridge BB, Burke JM, Lauffenburger DA, Sorger PK. Physicochemical modelling of cell signalling pathways. Nat Cell Biol. 2006;8(11):1195-203.
12. Doyle FJ, Stelling J. Systems interface biology. J R Soc Interface. 2006;3(10):603-16.
13. Schaber J, Klipp E. Model-based inference of biochemical parameters and dynamic properties of microbial signal transduction networks. Curr Opin Biotechnol. 2011;22:109-16.
14. Sunnå ker M, Zamora-Sillero E, Dechant R, Ludwig C, Busetto AG, Wagner A, et al. Automatic generation of predictive dynamic models reveals nuclear phosphorylation as the key Msn2 control mechanism. Sci Signal. 2013;6(277):41-1.
15. Link H, Christodoulou D, Sauer U. Advancing metabolic models with kinetic information. Curr Opin Biotechnol. 2014;29:8-14.
16. Le Novère N. Quantitative and logic modelling of molecular and gene networks. Nat Rev Genet. 2015;16:146-58.
17. Baltes M, Schneider R, Sturm C, Reuss M. Optimal experimental design for parameter estimation in unstructured growth models. Biotechnol Prog. 1994;10(5):480-8.
18. Apgar JF, Toettcher JE, Endy D, White FM, Tidor B. Stimulus design for model selection and validation in cell signaling. PLoS Comput Biol. 2008;4(2):30.
19. Banga JR, Balsa-Canto E. Parameter estimation and optimal experimental design. Essays Biochem. 2008;45:195-210.
20. Bandara S, Schlöder JP, Eils R, Bock HG, Meyer T. Optimal experimental design for parameter estimation of a cell signaling model. PLoS Comput Biol. 2009;5(11):1000558.
21. Hasenauer J, Waldherr S, Wagner K, Allgower F. Parameter identification, experimental design and model falsification for biological network models using semidefinite programming. IET Syst Biol. 2010;4(2):119-30.
22. Almquist J, Cvijovic M, Hatzimanikatis V, Nielsen J, Jirstrand M. Kinetic models in industrial biotechnology-improving cell factory performance. Metab Eng. 2014;24:38-60.
23. Marchisio MA, Stelling J. Computational design tools for synthetic biology. Curr Opin Biotechnol. 2009;20(4):479-85.
24. Arpino JAJ, Hancock EJ, Anderson J, Barahona M, Stan G-BV, Papachristodoulou A, et al. Tuning the dials of synthetic biology. Microbiology. 2013;159(Pt 7):1236-53.
25. Stanford NJ, Lubitz T, Smallbone K, Klipp E, Mendes P, Liebermeister W. Systematic construction of kinetic models from genome-scale metabolic networks. PLoS ONE. 2013;8(11):79195.
26. Karr JR, Sanghvi JC, Macklin DN, Gutschow MV, Jacobs JM, Bolival B, et al. A whole-cell computational model predicts phenotype from genotype. Cell. 2012;150(2):389-401.
27. Macklin DN, Ruggero NA, Covert MW. The future of whole-cell modeling. Curr Opin Biotechnol. 2014;28:111-5.
28. Cedersund G, Roll J. Systems biology: model based evaluation and comparison of potential explanations for given biological data. FEBS J. 2009;276(4):903-22.
29. Stumpf M, Balding DJ, Girolami M. Handbook of Statistical Systems Biology. Chichester, UK: John Wiley & Sons; 2011.
30. Cvijovic M, Almquist J, Hagmar J, Hohmann S, Kaltenbach HM, Klipp E, et al. Bridging the gaps in systems biology. Mol Gen Genomics. 2014;289(5):727-34.
31. Tarantola A. Inverse Problem Theory and Methods for Model Parameter Estimation. Philadelphia, PA: SIAM; 2005.
32. Clermont G, Zenker S. The inverse problem in mathematical biology. Math Biosci. 2015;260:11-5.
33. van Riel N. Dynamic modelling and analysis of biochemical networks: mechanism-based models and model-based experiments. Brief Bioinforma. 2006;7(4):364-74.
34. Jaqaman K, Danuser G. Linking data to models: data regression. Mol Cell Biol. 2006;7(11):813-9.
35. Chou IC, Voit EO. Recent developments in parameter estimation and structure identification of biochemical and genomic systems. Math Biosci. 2009;219(2):57-83.
36. Ashyraliyev M, Fomekong-Nanfack Y, Kaandorp JA, Blom JG. Systems biology: parameter estimation for biochemical models. FEBS J. 2009;276(4):886-902.
37. Vanlier J, Tiemann CA, Hilbers PAJ, van Riel NAW. Parameter uncertainty in biochemical models described by ordinary differential equations. Math Biosci. 2013;246(2):305-14.
38. Schittkowski K. Numerical Data Fitting in Dynamical Systems: a Practical Introduction with Applications and Software, vol. 77. Dordrecht, Germany: Springer; 2002.
39. Villaverde AF, Banga JR. Reverse engineering and identification in systems biology: strategies, perspectives and challenges. J R Soc Interface. 2014;11(91):20130505.
40. Kravaris C, Hahn J, Chu Y. Advances and selected recent developments in state and parameter estimation. Comput Chem Eng. 2013;51:111-23.
41. Ljung L, Chen T. Convexity issues in system identification. In: 10th IEEE International Conference on Control and Automation. Hangzhou, China: IEEE; 2013. p. 1-9.
42. Mendes P, Kell D. Non-linear optimization of biochemical pathways: applications to metabolic engineering and parameter estimation. Bioinformatics. 1998;14(10):869-83.
43. Esposito WR, Floudas CA. Global optimization for the parameter estimation of differential-algebraic systems. Ind Eng Chem Res. 2000;39: 1291-310.
44. Moles CG, Mendes P, Banga JR. Parameter estimation in biochemical pathways: a comparison of global optimization methods. Genome Res. 2003;13:2467-474.
45. Polisetty PK, Voit EO, Gatzke EP. Identification of metabolic system parameters using global optimization methods. Theor Biol Med Model. 2006;3(1):4.
46. Chachuat B, Singer AB, Barton PI. Global methods for dynamic optimization and mixed-integer dynamic optimization. Ind Eng Chem Res. 2006;45(25):8373-392.
47. Chen WW, Niepel M, Sorger PK. Classic and contemporary approaches to modeling biochemical reactions. Genes Dev. 2010;24(17):1861-75.
48. Geier F, Fengos G, Felizzi F, Iber D. Analyzing and constraining signaling networks: parameter estimation for the user. In: Liu X, Betterton MD, editors. Computational modeling of signaling networks. Methods in molecular biology, vol. 880. New York: Humana Press; 2012. p. 23-40.
49. Raue A, Schilling M, Bachmann J, Matteson A, Schelke M, Kaschek D, et al. Lessons learned from quantitative dynamical modeling in systems biology. PLoS ONE. 2013;8(9):74335.
50. Silver N. The Signal and the Noise: Why so Many Predictions Fail-but Some Don't. New York: Penguin; 2012.
51. Zi Z, Klipp E. Constraint-based modeling and kinetic analysis of the Smad dependent TGF-β signaling pathway. PLoS ONE. 2007;2(9):936.
52. Rocke DM, Durbin B. A model for measurement error for gene expression arrays. J Comput Biol. 2001;8(6):557-69.
53. Kreutz C, Bartolome-Rodriguez M, Maiwald T, Seidl M, Blum H, Mohr L, et al. An error model for protein quantification. Bioinformatics. 2007;23: 2747-753.
54. Schilling M, Maiwald T, Bohl S, Kollmann M, Kreutz C, Timmer J, et al. Quantitative data generation for systems biology: the impact of randomisation, calibrators and normalisers. In: IEE Proc-Syst Biol, vol.152. Stevenage, United Kingdom: IET; 2005. p. 193-200.
55. Huber W, Von Heydebreck A, Sültmann H, Poustka A, Vingron M. Variance stabilization applied to microarray data calibration and to the quantification of differential expression. Bioinformatics. 2002;18(suppl 1): 96-104.
56. Durbin BP, Hardin JS, Hawkins DM, Rocke DM. A variance-stabilizing transformation for gene-expression microarray data. Bioinformatics. 2002;18(suppl 1):105-10.
57. Walter E, Prorizato L. Identification of Parametric Models from Experimental Data. Berlin: Springer; 1997.
58. Seber GAF, Wild CJJ. Nonlinear Regression. New York: John Wiley & Sons, Inc; 2003.
59. Raue A, Kreutz C, Maiwald T, Bachmann J, Schilling M, Klingmüller U, et al. Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood. Bioinformatics. 2009;25(15):1923-9.
60. Balsa-Canto E, Alonso AA, Banga JR. An iterative identification procedure for dynamicmodeling of biochemical networks. BMC Syst Biol. 2010;4:11.
61. Joshi M, Seidel-Morgenstern A, Kremling A. Exploiting the bootstrap method for quantifying parameter confidence intervals in dynamical systems. Metab Eng. 2006;8(5):447-55.
62. Ljung L. System Identification: Theory for User. New Jersey: PTR Prentice Hall; 1987.
63. Hawkins DM. The problem of overfitting. J Chem Inf Comput Sci. 2004;44(1):1-12.
64. Guay M, McLean DD. Optimization and sensitivity analysis for multiresponse parameter estimation in systems of ordinary differential equations. Comput Chem Eng. 1995;19(12):1271-85.
65. Vassiliadis VS, Canto EB, Banga JR. Second-order sensitivities of general dynamic systems with application to optimal control problems. Chem Eng Sci. 1999;54(17):3851-860.
66. McKay MD, Beckman RJ, Conover WJ. A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics. 2000;42(1):55-61.
67. Rodriguez-Fernandez M, Egea JA, Banga JR. Novel metaheuristic for parameter estimation in nonlinear dynamic biological systems. BMC Bioinforma. 2006;7:483.
68. Kim KA, Spencer SL, Albeck JG, Burke JM, Sorger PK, Gaudet S, et al. Systematic calibration of a cell signaling network model. BMC Bioinforma. 2010. 11(202).
69. Papamichail I, Adjiman CS. Global optimization of dynamic systems. Comput Chem Eng. 2004;28:403-15.
70. Singer AB, Taylor JW, Barton PI, Green Jr WH. Global dynamic optimization for parameter estimation in chemical kinetics. J Phys Chem. 2006;110(3):971-6.
71. Miró A, Pozo C, Guillén-Gosálbez G, Egea JA, Jiménez L. Deterministic global optimization algorithm based on outer approximation for the parameter estimation of nonlinear dynamic biological systems. BMC Bioinforma. 2012;13(1):90.
72. Rodriguez-Fernandez M, Mendes P, Banga JR. A hybrid approach for efficient and robust parameter estimation in biochemical pathways. Bio Syst. 2006;83(2-3):248-65.
73. Sun J, Garibaldi JM, Hodgman C. Parameter estimation using metaheuristics in systems biology: a comprehensive review. Comput Biol Bioinforma, IEEE/ACM Trans on. 2012;9(1):185-202.
74. Balsa-Canto E, Peifer M, Banga JR, Timmer J, Fleck C. Hybrid optimization method with general switching strategy for parameter estimation. BMC Syst Biol. 2008;2(1):26.
75. Egea JA, Martí R, Banga JR. An evolutionary method for complex-process optimization. Comput Oper Res. 2010;37(2):315-24.
76. Jia G, Stephanopoulos G, Gunawan R. Incremental parameter estimation of kinetic metabolic network models. BMC Syst Biol. 2012;6:142.
77. Fan M, Kuwahara H, Wang X, Wang S, Gao X. Parameter estimation methods for gene circuit modeling from time-series mRNA data: a comparative study. Brief Bioinforma. 2015;015:.
78. Dennis JE, Gay DM, Welsch RE. An Adaptive Nonlinear Least-Squares Algorithm. ACM Trans Math Softw. 1981;7(3):348-68.
79. Serban R, Hindmarsh AC. CVODES: the sensitivity-enabled ode solver in sundials. In: ASME 2005 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. ASME, New York; 2005. p. 257-69. American Society of Mechanical Engineers.
80. Engl HW, Hanke M, Neubauer A. Regularization of Inverse Problems. Dordrecht: Kluwer Academic Publisher; 1996, pp. 1-329.
81. Jefferys WH, Berger JO. Ockham's razor and Bayesian analysis. Am Sci. 1992;80(1):64-72.
82. Chen Z, Haykin S. On different facets of regularization theory. Neural Comput. 2002;2846(12):2791-846.
83. Daunizeau J, David O, Stephan K. Dynamic causal modelling: a critical review of the biophysical and statistical foundations. NeuroImage. 2011;58(2):312-22.
84. Schölkopf B, Smola AJ. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. Cambridge, MA, USA: MIT press; 2002.
85. Bauer F, Lukas MA. Comparing parameter choice methods for regularization of ill-posed problems. Math Comput Simul. 2011;81(9): 1795-841.
86. Sra S, Nowozin S, Wright SJ. Optimization for Machine Learning. Cambridge, MA, USA: MIT Press; 2012.
87. Ljung L, Chen T. What can regularization offer for estimation of dynamical systems? In: Giri F, Van Assche V, editors. 11th IFAC International workshop on adaptation and learning in control and signal processing. Caen, France: IFAC; 2013. p. 1-8.
88. Engl HW, Flamm C, Kügler P, Lu J, Müller S, Schuster P, et al. Inverse problems in systems biology. Inverse Probl. 2009;25(12):123014.
89. Bansal L, Chu Y, Laird C, Hahn J. Regularization of inverse problems to determine transcription factor profiles from fluorescent reporter systems. AIChE J. 2012;58(12):3751-762.
90. Wang H, Wang XC. Parameter estimation for metabolic networks with two stage Bregman regularization homotopy inversion algorithm. J Theor Biol. 2014;343:199-207.
91. Tibshirani R. Regression shrinkage and selection via the Lasso. J R Stat Soc: Series B. 1996;58(1):267-88.
92. Zou H, Hastie T. Regularization and variable selection via the elastic net. J R Stat Soc: Series B. 2005;67(2):301-20.
93. Amato U, Hughes W. Maximum entropy regularization of Fredholm integral equations of the first kind. Inverse Probl. 1991;7:793-808.
94. Picard RR, Cook RD. Cross-validation of regression models. J Am Stat Assoc. 1984;79(387):575-83.
95. Snee RD. Validation of regression models: methods and examples. Technometrics. 1977;19(4):415-28.
96. Johansent TA. On Tikhonov Reguhrization, bias and variance in nonlinear system identification. Automatica. 1997;33(3):441-6.
97. Sjöberg J, Ljung L. Overtraining, regularization, and searching for minimum in neural networks In: Dugard L, Msaad M, Landau ID, editors. IFAC Symposia series, adaptive systems in control and signal processing. Grenoble: Pergamon Press, Oxford; 1992. p. 73-8.
98. Gupta NK, Mehra RK. Computational aspects of maximum likelihood estimation and reduction in sensitivity function calculations. IEEE Trans Autom Control. 1974;19(6):774-83.
99. Turányi T. Sensitivity analysis of complex kinetic systems. Tools and applications. J Math Chem. 1990;5(3):203-48.
100. Vajda S, Rabitz H. Identifiability and distinguishability of general reaction systems. J Phys Chem. 1994;98(20):5265-271.
101. Li R, Henson MA, Kurtz MJ. Selection of model parameters for off-line parameter estimation. IEEE Trans Control Syst Technol. 2004;12(3): 402-12.
102. Gutenkunst RN, Waterfall JJ, Casey FP, Brown KS, Myers CR, Sethna JP. Universally sloppy parameter sensitivities in systems biology models. PLoS Comput Biol. 2007;3(10):189.
103. Palm R. Numerical Comparison of Regularization Algorithms for Solving Ill-Posed Problems. 2010. PhD thesis, University of Tartu: Estonia.
104. Kaltenbacher B, Neubauer A, Scherzer O. Iterative Regularization Methods for Nonlinear Ill-Posed Problems. Radon Series on Computational and Applied Mathematics. Berlin, New York: Walter de Gruyter; 2008.
105. Hansen PC. Analysis of Discrete Ill-Posed Problems by Means of the L-Curve. SIAM Rev. 1992;34(4):561-80.
106. Hansen PC, O'Leary DP. The use of the L-Curve in the regularization of discrete ill-posed problems. SIAM J Sci Comput. 1993;14(6):1487-503.
107. Golub GH, Heath MT, Wahba G. Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics. 1979;21: 215-23.
108. Lukas MA. Robust GCV choice of the regularization parameter for correlated data. J Integr Equ Appl. 2010;22(3):519-47.
109. Lukas MA. Strong robust generalized cross-validation ot choosing the regularization parameter. Inverse Probl. 2008;24(3):34006-34021.
110. Gábor A, Banga JR. Improved parameter estimation in kinetic models: tuning of regularization methods In: Mendes P, Dada J, Smallbone K, editors. 13th Conference on computational methods in systems biology. Manchester: Springer; 2014.
111. Faraway JJ. Does data splitting improve prediction? Stat Comput. 2014;1-12.
112. Hollander M, Wolfe DA, Chicken E. Nonparametric statistical methods, 3rd edn. New Jersey: Wiley & Sons, Inc; 2014, pp. 1-844.
113. Morozov VA. Methods for solving incorrectly posed problems. New York: Springer; 1984.
114. Gfrerer H. An a posteriori parameter choice for ordinary and iterated Tikhonov regularization of ill-posed problems leading to optimal convergence rates. Math Comput. 1987;49(180):507.
115. Hämarik U, Raus T. On the choice of the regularization parameter in ill-posed problems with approximately given noise level of data. J Inverse Ill-posed Probl. 2006;14(3):251-66.
116. Hämarik U, Tautenhahn U. On the monotone error rule for parameter choice in iterative and continuous regularization methods. BIT. 2001;41(5):1029-38.
117. Lepskii O. On a problem of adaptive estimation in Gaussian White noise. Theory Probab Its Appl. 1991;35(3):454-66.
118. Bauer F. Some considerations concerning regularization and parameter choice algorithms. Inverse Probl. 2007;23(2):837-58.
119. Regínska T. A regularization parameter in discrete Ill-posed problems. SIAM J Sci Comput. 1996;17(3):740-9.
120. Brezinski C, Rodriguez G, Seatzu S. Error estimates for linear systems with applications to regularization. Numer Algo. 2008;49:85-104.
121. Bauer F, Mathe P. Parameter choice methods using minimization schemes. J Complex. 2011;27:68-85.
122. Wahba G. Practical approximate solutions to linear operator equations when the data are noisy. SIAM J Numer Anal. 1977;14(4):651-67.
123. Rodriguez-Fernandez M, Kucherenko S, Pantelides C, Shah N. Optimal experimental design based on global sensitivity analysis. In: 17th European symposium on computer aided process engineering; 2007. p. 1-6.
124. FitzHugh R. Impulses and physiological states in theoretical models of nerve membrane. Biophys J. 1961;1(6):445-66.
125. Nagumo J, Arimoto S, Yoshizawa S. An active pulse transmission line simulating nerve axon. Proc IRE. 1962;50(10):2061-070.
126. Kholodenko BN. Negative feedback and ultrasensitivity can bring about oscillations in the mitogen-activated protein kinase cascades. Eur J Biochem. 2000;267(6):1583-8.
127. Goodwin BC. Oscillatory behavior in enzymatic control processes. Adv Enzym Regul. 1965;3:425-38.
128. Bray D, Bourret RB, Simont MI. Computer simulation of the phosphorylation cascade controlling bacterial Chemotaxis. Mol Biol Cell. 1993;4(May):469-82.
129. Leander J, Lundh T, Jirstrand M. Stochastic differential equations as a tool to regularize the parameter estimation problem for continuous time dynamical systems given discrete time measurements. Math Biosci. 2014;251:54-62.
130. Gonze D, Abou-Jaoudé W. The Goodwin model: behind the Hill function. PLoS ONE. 2013;8(8):69573.