Maximum-likelihood maximum-entropy constrained probability density function estimation for prediction of rare events

AIChE Journal

Volumn 60, Issue 3, 2014, Pages 1013-1026

  Ahooyi, Taha Mohseni ^a   Soroush, Masoud ^a   Arbogast, Jeffrey E ^b   Seider, Warren D ^a   Oktem, Ulku G ^c  

^a Drexel University   (United States)

^b AIR LIQUIDE   (France)

^c UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

Bayesian network modeling; Fault detection; Probability density function estimation; Rare events; Risk assessment

Indexed keywords

EID: 84894089075     PISSN: 00011541     EISSN: 15475905     Source Type: Journal    
DOI: 10.1002/aic.14330     Document Type: Article

References (63)

1. Qin SJ. Statistical process monitoring: basics and beyond. J Chemom. 2003;17:480-502.
2. Mahadevan S, Shah SL. Fault detection and diagnosis in process data using one-class support vector machines. J Process Control. 2009;19:1627-1639.
3. Chiang LH, Braatz RD. Process monitoring using causal map and multivariate statistics: fault detection and identification. Chemom Intell Lab Syst. 2003;65:159-178.
4. Castillo I, Edgar TF, Dunia R. Nonlinear model-based fault detection with fuzzy set fault isolation. In; IECON 2010-36th Annual Conference on IEEE Industrial Electronics Society. Glendale, AZ, 2010:174-179.
5. Mhaskar P, McFall C, Gani A, Christofides PD, Davis JF. Isolation and handling of actuator faults in nonlinear systems. Automatica. 2008;44:53-62.
6. Pariyani A, Seider WD, Oktem UG, Soroush M. Improving process safety and product quality using large databases. In: Pierucci S, Buzzi Ferraris G, editors. Computer Aided Chemical Engineering. Naples, Italy: Elsevier, 2010;28:175-180.
7. Zhang Y, Qin SJ. Improved nonlinear fault detection technique and statistical analysis. AIChE J. 2008;54(12):3207-3220.
8. Mehranbod N, Soroush M, Panjapornpon C. A method of sensor fault detection and identification. J Process Control. 2005;15:321-339.
9. Mhaskar P, Gani A, McFall C, Christofides PD, Davis JF. Fault-tolerant control of nonlinear process systems subject to sensor faults. AIChE J. 2007;53:654-668.
10. Ohran B, Muñoz de la Peña D, Davis JF, Christofides PD. Enhancing data-based fault isolation through nonlinear control. AIChE J. 2008;54:223-241.
11. Taleb NN. The Black Swan: The Impact of the Highly Improbable, 2nd ed. New York: Random House Trade Paperbacks, 2010.
12. Härdle W, Müller M, Sperlich S, Werwatz A. Nonparametric and Semiparametric Models. New York: Springer, 2004.
13. Good IJ. The Estimation of Probabilities: An Essay on Modern Bayesian Methods (Research Monograph). Cambridge: The MIT Press, 2003.
14. McLachlan GJ, Peel D. Finite Mixture Models. New York: Wiley-Interscience, Inc., 2000.
15. Juneja S, Shahabuddin P. Rare-Event simulation techniques: an introduction and recent advances. In: Henderson SG, Nelson BL, editors. Handbooks in Operations Research and Management Science. The Netherlands: Elsevier, 2006;13:291-350.
16. Berryman JT, Schilling T. Sampling rare events in nonequilibrium and nonstationary systems. J Chem Phys. 2010;133:244101.
17. Hiemstra C, Kelejian HH. A rare events model: Monte Carlo results on sample design and large sample guidance. Econ Lett. 1991;37:255-263.
18. Doucet A, de Freitas N, Gordon N. Sequential Monte Carlo Methods in Practice. New York: Springer, 2001.
19. Asmussen S, Kroese DP, Rubinstein RY. Heavy tails, importance sampling and cross entropy. Stochastic Models. 2005;21:57-76.
20. Uneja S. Efficient rare-event simulation using importance sampling: an introduction. In Misra JC, editor. Computational Mathematics, Modeling and Algorithms. New Delhi: Narosa Publishing House, 2003:357-396.
21. Ahamed TPI, Borkar VS, Juneja S. Adaptive importance sampling technique for Markov chains using stochastic approximation. Oper Res. 2006;54:489-504.
22. Dean T, Dupuis P. Splitting for rare event simulation: a large deviation approach to design and analysis. Stochastic Processes Appl. 2009;119:562-587.
23. Shortle JF. Efficient simulation of blackout probabilities using splitting. Int J Electr Power Energy Syst. 2013;44:743-751.
24. Campillo F, Rakotozafy R, Rossi V. Parallel and interacting Markov chain Monte Carlo algorithm. Math Comput Simul. 2009;79:3424-3433.
25. Kaynar B, Ridder A. The cross-entropy method with patching for rare-event simulation of large Markov chains. Eur J Oper Res. 2010;207:1380-1397.
26. Cerou F, LeGland F, Del Moral P, Lezaud P. Limit theorems for the multilevel splitting algorithm in the simulation of rare events. In: Proceedings of the 2005 Winter Simulation Conference. Orlando, FL, 2005:682-691.
27. Qiu Y, Zhou H, Wu Y. An importance sampling method with applications to rare event probability. In: Proceedings of 2007 IEEE International Conference on Grey Systems and Intelligent Services. Nanjing, China, 2007:1381-1385.
28. Devroye L, Gábor L. Combinatorial Methods in Density Estimation. New York: Springer, 2001.
29. Tsybakov AB. Introduction to Nonparametric Estimation. New York: Springer, 2009.
30. Heylighen F, Joslyn C. Cybernetics and second order cybernetics. In: Meyers RA, editor. Encyclopedia of Physical Science and Technology, 3rd ed. New York: Academic Press, 2001:155-170.
31. Aldrich JRA. Fisher and the making of maximum likelihood 1912-1922. Stat Sci. 1997;12:162-176.
32. Lauritzen L, Spiegelhalter J. Local computations with probabilities on graphical structures and their application to expert systems (with discussion). J R Stat Soc. 1988;50:157-224.
33. Koski T, Noble JM. Bayesian Networks: An Introduction. Hoboken: Wiley, Inc., 2009.
34. Korb KB, Nicholson AE. Bayesian Artificial Intelligence, 2nd ed. Boca Raton: CRC Press, 2010.
35. Daly R, Qiang S, Stuart A. Learning Bayesian networks: approaches and issues. Knowl Eng Rev. 2011;26:99-157.
36. Casella G, Berger RL. Statistical Inference, 2nd ed. Australia: Cengage Learning, 2002.
37. Gillies D. Philosophical Theories of Probability. New York: Routledge, 2000.
38. Beesack PR. Inequalities for absolute moments of a distribution: from Laplace to von Mises. J Math Anal Appl. 1984;98:435-457.
39. Koralov L, Sinai YG. Theory of Probability and Random Processes, 2nd ed. Berlin: Springer, 2012.
40. Shannon CE. Prediction and entropy of printed English. Bell Syst Tech J. 1951;30:50-64.
41. Shannon CE. A mathematical theory of communication. Bell Syst Tech J. 1984;27:379-423.
42. Zellner A, Highfield AR. Calculation of maximum entropy distribution and approximation of marginal posterior distributions. J Econometrics. 1988;37:195-209.
43. Golan A, Judge G, Miller D. Maximum Entropy Econometrics Robust Estimation with Limited Data. New York: Wiley, Inc., 1996.
44. Lindsay BG. Moment matrices: applications in mixtures. Ann Stat. 1989;17:722-740.
45. Hall AR. Generalized Method of Moments. Oxford: Oxford University Press, 2005.
46. Carrasco M, Florens JP. Generalization of GMM to a Continuum of Moment Conditions. Econometric Theory. 2000;20:797-834.
47. Zacks S, Even M. Minimum variance unbiased and maximum likelihood estimators of reliability functions for systems in series and in parallel. J Am Stat Assoc. 1966;61:1052-1062.
48. Schmidt DF, Makalic E. Universal models for the exponential distribution. IEEE Trans Inf Theory. 2009;55:3087-3090.
49. Chris DO, Paul AR. On the uniqueness of the maximum likelihood estimator. Econ Lett. 2002;75:209-217.
50. Moons KGM, Rogier A, Donders T, Steyerberg EW, Harrell FE. Penalized maximum likelihood estimation to directly adjust diagnostic and prognostic prediction models for overoptimism: a clinical example. J Clin Epidemiol. 2004;57:1262-1270.
51. Benedikt M, Pötscher HL. On the distribution of penalized maximum likelihood estimators: the LASSO, SCAD, and thresholding. J Multivariate Anal. 2009;100:2065-2082.
52. Eliason SR. Maximum Likelihood Estimation: Logic and Practice. Newbury Park: SAGE Publications, Inc., 1993.
53. Piotr J, Durante F, Härdle WK, Rychlik T. Copula theory and its applications. In: Proceedings of the Workshop Held in Warsaw 2009. Heidelberg: Springer, 2010.
54. Li Q, Racine JS. Advances In Econometrics: Nonparametric Econometric Methods, Volume 25. Bingley, UK: Emerald, 2009.
55. Netica v. 4.16 [Software], Norsys Software Corporation. Vancouver, BC. 2010. Available at http://www.norsys.com.
56. Nelsen RB. An Introduction to Copulas. Vol. 139. Lecture Notes in Statistics. Berlin Heidelberg New York:Springer-Verlag, 1999.
57. Embrechts P, Lindskog F, McNeil A. Modelling dependence with copulas and applications to risk management. In: Rachev S, editor. Handbook of Heavy Tailed Distributions in Finance. New York: Elsevier, 2003:331-385.
58. McNeil AJ, Nešlehová J. From Archimedean to Liouville copulas. J Multivariate Anal. 2010;101:1772-1790.
59. Kolev N, Paiva D. Copula-based regression models: a survey. J Stat Plann Inference. 2009;139:3847-3856.
60. Silverman BW. Density Estimation for Statistics and Data Analysis. London: Chapman & Hall/CRC, 1998.
61. Epanechnikov VA. Non-parametric estimation of a multivariate probability density. Theory Probability Appl. 1969;14:153-158.
62. Duong T, Hazelton ML. Convergence rates for unconstrained bandwidth matrix selectors in multivariate kernel density estimation. J Multivariate Anal. 2005;93:417-433.
63. Kullback S. Letter to the Editor: The Kullback-Leibler distance. Am Stat. 1987;41:340-341.