Characterizing and visualizing predictive uncertainty in numerical ensembles through bayesian model averaging

IEEE Transactions on Visualization and Computer Graphics

Volumn 19, Issue 12, 2013, Pages 2703-2712

  Gosink, Luke ^a   Bensema, Kevin ^b   Pulsipher, Trenton ^a   Obermaier, Harald ^b   Henry, Michael ^a   Childs, Hank ^c   Joy, Kenneth ^b  

^a PACIFIC NORTHWEST NATIONAL LABORATORY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c UNIVERSITY OF OREGON   (United States)

Author keywords

Numerical ensembles; Statistical visualization; Uncertainty visualization

Indexed keywords

BAYESIAN FRAMEWORKS; BAYESIAN MODEL AVERAGING; ENSEMBLE FORECASTING; INDIVIDUAL MODELS; NUMERICAL ENSEMBLES; PREDICTIVE UNCERTAINTY; STATISTICAL VISUALIZATIONS; UNCERTAINTY VISUALIZATION;

BAYESIAN NETWORKS; COMPUTER SIMULATION; VISUALIZATION;

AGGREGATES;

ALGORITHM; AUTOMATED PATTERN RECOGNITION; BAYES THEOREM; COMPUTER GRAPHICS; COMPUTER INTERFACE; COMPUTER SIMULATION; PROCEDURES; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; STATISTICAL ANALYSIS; STATISTICAL MODEL; ARTICLE; METHODOLOGY;

ALGORITHMS; BAYES THEOREM; COMPUTER GRAPHICS; COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; MODELS, STATISTICAL; PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; USER-COMPUTER INTERFACE;

EID: 84886700438     PISSN: 10772626     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVCG.2013.138     Document Type: Article

References (49)

1. G. Apostolakis. The concept of probability in safety assessments of technological systems. Science, 250(4986):1359-1364, 1990. (Pubitemid 120031846)
2. J. M. Bates and C. W. J. Granger. The combination of forecasts. Opera- tional Research Quarterly, 20(4):451-468, 1969.
3. R. P. Botchen, D. Weiskopf, and T. Ertl. Texture-based visualization of uncertainty in flow fields. In IEEE Visualization, pages 647-654, 2005.
4. L. Breiman. Bagging predictors. Mach. Learning, 24(2):123-140, 1996. (Pubitemid 126724382)
5. M. A. Cardoso, L. J. Durlofsky, and P. Sarma. Development and application of reduced-order modeling procedures for subsurface flow simulation. In Int. J. Numer. Meth. Engng, volume 77, pages 1322-1350, 2009.
6. B. Clarke. Comparing bayes model averaging and stacking when model approximation error cannot be ignored. The Journal of Machine Learning Research, 4:683-712, 2003.
7. I. Davidson and W. Fan. When efficient model averaging out-performs boosting and bagging. In Principles and Practice of Knowledge Discov- ery in Databases (PKDD), volume 4213 of Lecture Notes in Computer Science, pages 478-486, 2006. (Pubitemid 44617748)
8. T. J. Davis and K. C. Peter. Modeling and visualizing multiple spatial uncertainties. Computers & Geosciences, 23(4):397-408, 1997.
9. J. Devooght. Model uncertainty and model inaccuracy. Reliability Engi- neering and System Safety, 59(2):171 - 185, 1998. (Pubitemid 128393934)
10. S. Djurcilov, K. Kim, P. Lermusiaux, and A. Pang. Visualizing scalar volumetric data with uncertainty. Comput. & Graphics, 26:239-248, 2002. (Pubitemid 34545587)
11. H.-H. Ehricke, U. Klose, and W. Grodd. Visualizing MR diffusion tensor fields by dynamic fiber tracking and uncertainty mapping. Comput. & Graphics, 30(2):255-264, April 2006. (Pubitemid 43344323)
12. M. P. Friedlander and M. A. Saunders. The Dantzig selector: statistical estimation when p is much larger than n. Annals of Statistics, 35(6):2385-2391, 2007.
13. A. Genell, S. Nemes, G. Steineck, and P. W. Dickman. Model selection in medical research: a simulation study comparing Bayesian model averaging and stepwise regression. BMC medical research methodology, 10:100-108, 2010.
14. S. G. Gilmour. The Interpretation of Mallows's Cp Statistic. J. of the Royal Statistical Society, 45(1):49-56, 1996.
15. G. Grigoryan and P. Rheingans. Probabilistic surfaces: Point based primitives to show surface uncertainty. In Proceedings of IEEE Visualization 2002, pages 147-153, 2002.
16. A. Hassan. A Validation for the Groundwater Flow and Transport Model of the Faultless Nuclear Test at Central Nevada Test Area. Desert Research Institute Publication. Desert Research Institute, 2003.
17. T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2001.
18. T. Hengl. Visualization of uncertainty using the HSI color model: computations with colors. Proc. GeoComputation, 23:8-17, 2003.
19. A. E. Hoerl and R. W. Kennard. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics, 12:55-67, 1970.
20. J. A. Hoeting, D. Madigan, A. E. Raftery, and C. T. Volinsky. Bayesian model averaging: A tutorial. Statistical Science, 14(4):382-417, 1999.
21. H. Jeffreys. Theory of Probability. Oxford: Oxford University Press, 1961.
22. S. Lodha, A. Pang, R. Sheehan, and C. Wittenbrink. Uflow: visualizing uncertainty in fluid flow. In Visualization '96. Proceedings., volume 27, pages 249 -254, November 1996.
23. A. M. MacEachren. Visualizing Uncertain Information. Cartographic Perspectives, 13:10-19, 1992.
24. A. M. MacEachren, A. Robinson, S. Hopper, S. Gardner, R. Murray, M. Gahegan, and E. Hetzler. Visualizing Geospatial Information Uncertainty: What We Know and What We Need to Know. Cartography and Geographic Information Science, 32(3):139-160, July 2005. (Pubitemid 41300254)
25. D. Madigan and A. E. Raftery. Model Selection and Accounting for Model Uncertainty in Graphical Models Using Occam's Window. J. of the American Statistical Assoc., 89(428):1535-1546, 1994.
26. E. Morales-Casique, S. P. Neuman, and V. V. Vesselinov. Maximum likelihood Bayesian averaging of airflow models in unsaturated fractured tuff using Occam and variance windows. Stochastic Environmental Research and Risk Assessment, 24:863-880, 2010.
27. S. P. Neuman and P. Wierenga. A comprehensive strategy of hydrogeologic modeling and uncertainty analysis for nuclear facilities and sites. Technical report, University of Arizona, 2003.
28. T. S. Newman andW. Lee. On visualizing uncertainty in volumetric data: techniques and their evaluation. J. of Visual Languages and Computing, 15(6):463-491, 2004. (Pubitemid 40256611)
29. D. Opitz and R. Maclin. Popular Ensemble Methods: An Empirical Study. J. of Artificial Intelligence Research, 11:169-198, 1999. (Pubitemid 129628763)
30. M. Otto, T. Germer, H.-C. Hege, and H. Theisel. Uncertain 2D vector field topology. Computer Graphics Forum, 29(2):347-356, 2010.
31. H. Owhadi, C. Scovel, and T. Sullivan. Bayesian Brittleness: Why no Bayesian model is "good enough". arXiv preprint arXiv:1304.6772, 2013, http://arxiv.org/pdf/1304.6772v1.pdf.
32. A. Pang. Visualizing Uncertainty in Geo-spatial Data. Technical report, University of California, Santa Cruz, 2001.
33. A. Pang, C. M. Wittenbrink, and S. K. Lodha. Approaches to uncertainty visualization. The Visual Computer, 13(8):370-390, 1997. (Pubitemid 127583168)
34. B. Plewe. The nature of uncertainty in historical geographic information. Transactions in GIS, 6(4):431-456, 2002. (Pubitemid 35369156)
35. K. Potter, A. Wilson, P.-T. Bremer, D. Williams, C. Doutriaux, V. Pascucci, and C. Johhson. Visualization of uncertainty and ensemble data: Exploration of climate modeling and weather forecast data with integrated ViSUS-CDAT systems. In Proceedings of SciDAC 2009, volume 180 of Journal of Physics: Conference Series, 2009.
36. J.-S. Prassni, T. Ropinski, and K. Hinrichs. Uncertainty-aware guided volume segmentation. Transactions on Visualization and Computer Graphics, 16(6):1358 -1365, Dec 2010.
37. A. Raftery. Bayesian Model Selection in Social Research. Sociological Methodology, 25:111-163, 1995.
38. A. Raftery, T. Gneiting, F. Balabdaoui, and M. Polakowski. Using bayesian model averaging to calibrate forecast ensembles. Monthly Weather Review, 133:1155-1174, 2005. (Pubitemid 40829946)
39. A. E. Raftery, J. A. Hoeting, and D. Madigan. Bayesian model averaging for linear regression models. J. of the American Statistical Assoc., 92:179-191, 1997.
40. R. Rojas, O. Batelaan, L. Feyen, and A. Dassargues. Assessment of conceptual model uncertainty for the regional aquifer Pampa del Tamarugal - North Chile. Hydrology and Earth System Sciences, 14(2):171-192, 2010.
41. L. Rokach. Ensemble-based classifiers. Artificial Intelligence Review, 33(1-2):1-39, 2010.
42. J. Sanyal, S. Zhang, J. Dyer, A. Mercer, P. Amburn, and R. J. Moorhead. Noodles: A tool for visualization of numerical weather model ensemble uncertainty. IEEE Trans. Visualization and Computer Graphics, 16(6):1421-1430, 2010.
43. G. Seber and A. Lee. Linear Regression Analysis, volume 1 of Wiley Series in Probability and Statistics. John Wiley & Sons, 2012.
44. J. Thomson, E. Hetzler, A. MacEachren, M. Gahegan, and M. Pavel. A typology for visualizing uncertainty. Society of Photo-Optical Instrumen- tation Engineers (SPIE) Conference Series, pages 146-157, March 2005. (Pubitemid 40949414)
45. C. T. Volinsky, D. Madigan, A. E. Raftery, and R. A. Kronmal. Bayesian model averaging in proportional hazard models: Assessing the risk of a stroke. Applied Statistics, 46:433-448, 1997.
46. D. Wang, W. Zhang, and A. Bakhai. Comparison of Bayesian model averaging and stepwise methods for model selection in logistic regression. Statistics in Medicine, 23(22):3451-3467, 2004. (Pubitemid 39467863)
47. C. Wittenbrink, A. Pang, and S. Lodha. Glyphs for visualizing uncertainty in vector fields. Transactions on Visualization and Computer Graphics, 2(3):266-279, Sep 1996. (Pubitemid 126781407)
48. M. Ye, S. P. Neuman, and P. D. Meyer. Maximum likelihood Bayesian averaging of spatial variability models in unsaturated fractured tuff. Water Resources Research, 40(5):n. pag., 2004.
49. Y. Zhang and G. Pau. Reduced-order model development for CO2 storage in brine reservoirs. Technical report, NRAP, 2012.