Criteria for evaluating model deficiency for groundwater models and the effects of eliminating deficient models on multi-model analysis using AICC, KIC, AIC, and BIC
Abstract
Multi-model analysis (MMA) considers multiple model interpretations of a system. MMA provides a more realistic assessment of uncertainty associated with model predictions because both uncertainty of individual models and uncertainty associated with different model structures are considered. Models are evaluated for the strength of evidence that they represent an unknown system using different Information Criteria (IC) equations. IC equations are designed to assess the likelihood that a model in a set of models represents the true but unknown system. IC equations do not include a component which identifies a deficient model. Therefore, inclusion of deficient models in the set of models leads to poor model-averaged results. Evaluation of models to assess whether available observation data sufficiently support the model structure is an important step in MMA. Measures for evaluating models include: 1) failure to reach proper convergence during non-linear regression; 2) unreasonable parameter estimates; 3) unreasonable confidence intervals on parameters or a coefficient of variation greater than ten for one or more parameters; 4) high correlations between parameters; 5) determinant of the correlation matrix less than 1x10-12; 6) condition number of the Jacobian matrix greater than 2000; and 7) unreasonable confidence intervals on predictions. Experiments presented herein are designed to evaluate how components of AIC, AICc, BIC, and KIC rank models and assign model probabilities, and to demonstrate how removing deficient models improves MMA results. Synthetic models are used to represent true but unknown systems in contrast to experimental models that are created to simulate a simplified version of the unknown system based on observation data taken from the synthetic models. AIC, AICc, BIC, and KIC generally assign high probability to deficient models. AICc generally assigns high probability to deficient models if 1) there are many observation data or 2) there are few observation data and the model fits the data well. KIC generally assigns high probability to deficient models because these models have low Fisher Information. AIC and BIC are influenced by the goodness-of-fit and are more likely to assign high probability to more complex models because these models are generally over-fitted. Removing deficient models results in improved MMA results using AIC, AICc, BIC and KIC.Rights
Copyright of the original work is retained by the author.Collections
Related items
Showing items related by title, author, creator and subject.
-
Improving U.S. national water modeling: an intercomparison of two high-resolution, continental scale models, ParFlow-CONUS and the National Water Model v1.2 configuration of WRF-HydroMaxwell, Reed M.; Tijerina, Danielle T.; Condon, Laura E.; Gochis, David; Zhou, Wendy (Colorado School of Mines. Arthur Lakes Library, 2018)Development of integrated hydrology modeling systems, where subsurface, land-surface, and energy budget processes are represented, is an increasing trend. In hydrologic science, there is a need for more intricate models for comprehensive hydrologic forecasting and water management over large spatial areas, specifically the Continental US (CONUS). We compare streamflow output from two models developed for the CONUS: ParFlow-CONUS, using the integrated model ParFlow and WRF-Hydro.NWM, a configuration of the National Water Model version 1.2 using the National Center for Atmospheric Research, Weather Research and Forecasting hydro extension package WRF-Hydro. Accurately representing large domains remains a challenge considering the difficult task of representing complex hydrologic processes, computational expense, and extensive data needs. Intercomparing models helps disentangle process, parameter, and formulation differences. Results show that WRF-Hydro.NWM and PF-CONUS generally capture flow magnitude, but WRF-Hydro.NWM better captures flow timing. Spatial differences exist as well—both models accurately simulate the humid east, but struggle with the Great Plains and intermountain west. Simulations such as these will help improve physical process representation in hydrologic models and give greater confidence in large-scale forecasts.
-
Modeling the High Plains aquifer, South Dakota: refinement of model hydrologic data and grid sizeTurner, A. Keith, 1941-; Tschupp, Edward Walter (Colorado School of Mines. Arthur Lakes Library, 1987)
-
Multi-scale modeling and analysis via surrogate modeling techniques for in vivo knee loading predictionsTurner, Cameron J.; Silverman, Anne K.; Adams, Matthew J.; Petrella, Anthony J.; Blacklock, Jenifer (Colorado School of Mines. Arthur Lakes Library, 2014)Total knee replacement is a viable treatment for end-stage knee arthritis. With a greater number of younger patients opting for total knee replacement surgery, their increasingly active lifestyles will result in higher wear rates while decreasing the life expectancy of the tibial insert component of the knee replacement implant. In response to the eventuality of patients with more active lifestyles requiring knee replacement surgery, this research proposed to accurately estimate in vivo knee loading over a gait cycle through a multi-scale modeling approach. Estimates for knee loading were compared to publicly-available in vivo knee loading measurements from a telemetric implant. A whole-body musculoskeletal modeling approach was used to simulate the gait cycle of a person who had undergone total knee replacement surgery. This approach was used to calculate net knee joint contact forces. Then, an explicit dynamic finite element analysis was used to estimate the load distribution in the medial and lateral compartments of the knee using a six degree of freedom knee joint. Surrogate modeling via spline interpolations was then utilized to reduce computational time and effort for calculation of the load distributions from finite element analysis to less than five seconds. Results suggest that generic whole-body modeling and hybrid forward dynamic simulation techniques for estimating knee joint loads may become clinically feasible in the near future. Finite element modeling and analysis produced two key results. First, the best results from varying the location of the femoral component reference point did not accurately reflect an ISO wear model location for the femoral reference point. Second, the finite element model accurately estimated the medial and lateral contact forces during the stance phase. However, surrogate modeling successfully interpolated the load distributions in the medial and lateral contact surfaces using results from the finite element analysis without requiring any knowledge of the geometry of the contact surfaces. This research concludes by proposing to couple whole-body modeling and simulation techniques with a surrogate optimization scheme to provide clinicians with a patient's knee loading behavior. A path toward improving predictive wear modeling and simulation is also provided.