QuoFem Parameter Estimation

[STOKLJOS]

Hello,

I have been using the parameter estimation for an opensees model and it seems to not yielding the best results. For instance look at the output of the dakotaTab.out; the parameters are varying by 1e-4 and the best parameter picked is 30 kips over the experimental value. I am currently using a convergence tolerance of 0.0001 and max iterations of 1000. The parameters also have a healthy lower and higher bounds on them (ie stc has lower of 0.01 and higher of 1).

 I am wondering why the simulation is yielding results after 7 evaluations and not varying the parameters more to hopefully converge to a closer result.

Thanks,

Josh




%eval_id  interface stc                srs            shr                  MaxStrength   
1        NO_ID     0.3                   0.33           1                     -159.9473689   
2        NO_ID     0.3001011366   0.33           1.000231482    30.27846112   
3        NO_ID     0.3001254848   0.33           1.000312594    -158.1980189   
4        NO_ID     0.3001035714   0.33           1.000239593    -153.3980689   
5        NO_ID     0.3001013801   0.33           1.000232293    -159.5161689   
6        NO_ID     0.300101161    0.33           1.000231563    -155.7201689   
7        NO_ID     0.3001011391   0.33           1.00023149     -157.0393689

[AakashBS]

Hello Josh,

By observing the results, we think there could be two potential issues:

1) The model might not be numerically stable (going by the large changes in the MaxStrength residuals for small changes in the parameter values, as seen in evaluation 2 vs all the others).

2) A gradient-based (i.e., local) optimization algorithm is used to perform parameter estimation. This algorithm works for continuously differentiable objective functions and will converge to a local minimum. It seems that there are multiple minima even in a very small range of the parameter values (caused by noisy MaxStrength results) - see attachment. That is why the algorithm is taking small steps and converging to a local optimum in the vicinity of the start point of the search.

We recommend that you first perform a parameter study (for e.g. forward propagation using LHS in quoFEM) to identify any such discontinuity/noisiness in the model outputs before performing a parameter estimation study. Sometimes the noisy behavior indicates that the model can be improved (for example by setting stricter convergence tolerance, if feasible). If this is not feasible, you can perform a Global Sensitivity Analysis using quoFEM and decide to not calibrate the values of parameters that are causing this noisy behavior, provided that parameter has a low sensitivity index. If the output is sensitive to the parameter which is causing the noisy behavior, then your option is to use a global parameter estimation method. Currently, you can use the TMCMC algorithm in quoFEM to perform probabilistic parameter estimation, which works for such noisy landscapes too. The trade-off is that it will require a large number of model evaluations, in comparison to a local search method. But you can use DesignSafe's computational resources to overcome this issue. In the future, we might make other global deterministic parameter estimation methods available in quoFEM.

P.S. Please double check if the MaxStrength values are being returned correctly. We ran simulations with the model script that you had previously shared with us over a large range of the parameter values and the MaxStrength residuals were around -4.

Thanks,
Aakash and Sang-ri

[STOKLJOS]

Hello,

Thank you for your reply. I also noticed the large difference in max strength. This is due to the error function looking at largest loads when pushed in the opposite direction as well as the normal direction. I fixed that so the output is always positive but still running into the same local minima solution so I will try the steps you laid out.

Thanks again,

Josh