Comparison between the Results of Pelicun and PACT

[rezvan]

Hello,

I just wanted to share this post to show the comparison between the damage and loss assessment results, which were performed with Pelicun and PACT software for a 6-story building.
The results seem to be close; however, the uncertainty in the results based on PACT analysis is more considerable than Pelicun.

Thank you,
Pooya

[adamzs]

Hi Pooya,

Thank you for sharing those results. Such comparisons are always very helpful in verifying newly developed code.

In this particular case, I can offer two ideas on where the difference might come from.

First, make sure the demand sample is identical. If PACT assumes an increase in the variance of the lognormally distributed EDPs, make sure that assumption is made in Pelicun as well. I expect that you've checked this.

The second option is a bit more complicated: There is an important steps in the loss estimation of FEMA P-58 that do not receive a lot of attention in the official volumes: the calculation of the quantity of damage for modeling the economies of scale.

There are multiple ways of performing this task and the different approaches can lead to substantially different results. I have been collaborating with researchers at McMaster University and IUSS Pavia to investigate these issues and their impact on FEMA P-58 assessments. We are going to present a conference paper about it in the upcoming 12NCEE.

I'll give you a brief overview of the issue and suggest a solution below.

When it comes to the quantity of damaged components you need to decide if you want to aggregate damages across all floors of the building and if you want to aggregate damages across all damage states. These choices lead to four options:
- all floors, all damage states
- all floors, but only the given damage state
- only the given floor, all damage states
- only the given floor and only the given damage state

The first option can lead to a much larger quantity of damage than the last option, especially for medium and high rise buildings. This often yields a substantial reduction in repair consequences and their variance - because the variance of consequence functions in FEMA P-58 depends on their median value.
PACT uses the first option: all floors all damage states approach.
Pelicun 3 allows you to choose the approach you'd like to use, but the default setting is the second option: all floors, individual damage state. When you initialize an Assessment, you can provide a list of settings in a dictionary. One of those settings is "EconomiesOfScale". I have already written up a description of it in an updated version of the example from the first Live Expert Tips that I'll release shortly. Here is what I write there:

EconomiesOfScale: Controls how the damages are aggregated when the economies of scale are calculated. Expects the following dictionary: {'AcrossFloors': bool, 'AcrossDamageStates': bool} where bool is either True or False. Default: {'AcrossFloors': True, 'AcrossDamageStates': False}
- 'AcrossFloors' if True, aggregates damages across floors to get the quantity of damage. If False, it uses damaged quantities and evaluates economies of scale independently for each floor.
- 'AcrossDamageStates' if True, aggregates damages across damage states to get the quantity of damage. If False, it uses damaged quantities and evaluates economies of scale independently for each damage state.

For example, initializing the assessment like this should reproduce PACT's behavior:

PAL = Assessment({
    "PrintLog": True,
    "Seed": 415,
    "EconomiesOfScale": {"AcrossFloors": True, "AcrossDamageStates": True}
})

I suggest running the calculation with the above settings and comparing the results to PACT again. If the difference persists, you might have found a bug.

Please let me know how it goes.

Thanks,
Adam

[adamzs]

Hi Pooya,

I just wanted to let you know that I've released a new version of Pelicun3, we are at 3.1.b6 now. It might be a good idea to run the comparison with the new version.

I also updated the FEMA P58 example notebook on DesignSafe with a lot of additional details and explanation: https://www.designsafe-ci.org/data/browser/public/designsafe.storage.published/PRJ-3411v5

Adam

[rezvan]

Hi Adam,

First and foremost, I am so grateful for such a valuable and detailed response. Also, I appreciate the new version with the added EDPs.
Moreover, I apologize for replying too late. I had a hectic month and wanted to try all the mentioned points before reaching out.

I thoroughly checked all the options and input data and examined different cases for a 6-story eccentrically braced frame. To make a fair comparison between PACT and PELICUN, I tried to use the same input data as follows:
a- The components' quantities are deterministic in both software (no dispersion was considered for the components).
b- In PELICUN, the number of blocks for each component is calculated by the total quantity divided by the unit quantity of the component defined in PACT. For example, if the total quantity of “B.20.22.001" is 1500 square feet (SF), the unit of this component in PACT is "30 SF". Therefore, the number of blocks in PELICUN was calculated to be 1500/30=50.
c- Similar fragilities for collapse and excessive RID were introduced (median and dispersion).
d- "Mont-Carlo" simulation with 1000 realization was used in both software.
e- Non-directional conversion factor equal to 1.2 was assigned in PACT (the default value in PELICUN).
f- IN PACT, the demands of the prototype subjected to a set of 22 ground motions were provided, while in PELICUN the median and logarithmic standard deviation of these demands were introduced. The log-stdev of each set of demands was calculated by first getting the natural logarithm of the set of 22 EDPs and then calculating the standard deviation of this set (please confirm if it is correct).
g- In PACT, the maximum residual drifts of the building (among all the stories) subjected to the set of 22 ground motions were provided. In PELICUN the ensemble median of the peak residual story drift of each story subjected to the set of 22 ground motions was introduced (similar to the other EDPs). Therefore, there is a difference in terms of providing the input data of the residual drift between PACT and PELICUN.

I examined different cases, and here are some of the findings:

Case-1: On the first try, I tried to make a simpler model for comparison, therefore I only included 2-3 components in the CMP_marginals, and excluded excessive RID and collapse cases by assigning a large number to the median of the corresponding fragilities. If the "Economies of scale" is set to "True", "True" (similar to PACT), the analysis fails (probably because no realization leads to neither collapse nor excessive drift), but if this option is set to "True", "False", the analysis converges. As can be seen in the figure, there is a small difference between CDFs of the repair cost in terms of uncertainty.

Case-2: Similar to case-1, but the collapse, and excessive RID are included. The "Economies of scale" was set to "True", "True" (similar to PACT). As shown in the figure, there is a break in the CDF of the repair cost. This break is due to the large difference between the total repair cost of the repairable cases and the replacement cost (irreparable and collapse cases). The difference between the height of the two horizontal lines infers the probability of either collapse or irreparable damage. It can be seen that PELICUN predicts a higher probability for those scenarios (which is mainly due to excessive RID and not collapse and maybe because of the different ways of providing the residual drift in the two software as mentioned in part "g" above).

Case-3: The remaining components were added, and probabilistic quantities were used for components. Excessive RID and collapse were included in the analysis and the "Economies of scale" was set to "True", "True".


Consequently, two differences can be observed between the results:
1- the probability of irreparable damage in PACT and PELICUN seems to be different (In the first post, such a difference was not observed because the prototype was a self-centering system with almost zero residual drift).
2- The uncertainty seems to be slightly different as shown in Case-1.

Please let me know if I am missing something.

Again, I appreciate your precious work, and I will be glad if I can do any help.

Pooya,