Pandemic modelling effectiveness examined by Wales’ TAC

The Technical Advisory Cell has released an exercise to compare historic COVID-19 modelling in Wales with actual COVID-19 data.

The report by the TAC is another reflective piece, similar to the in-depth mortality statistics report published in April, looking back at aspects of the pandemic.

The document explains, “It is important to note that the model outputs are not predictions or forecasts…. the models are used to capture plausible key assumptions about the complex way epidemics progress in a
formal mathematical framework, and then use this framework to explore future scenarios based on those assumptions”.

The TAC paper is Wales only, in June 2020, a policy modelling group was established to discuss different models and scenarios and to commission some bespoke modelling for Wales, which was produced by Swansea University and other academic groups.

Detail on the quick turn around Omicron modelling is explained:

A rapid modelling response was required in December 2021 due to the emergence
of the omicron variant. Transmissibility was extremely high, with the likelihood of
significant immune escape (loss of protection from previous infection). Escape from
vaccine effects were much more uncertain. Omicron became the dominant variant
within weeks and scenarios were quickly generated. All scenarios agreed that a
large wave of infections would be expected, but the uncertainty focused on how this
would be reflected in severe clinical effects, and hospital burden. Scenarios were
created featuring high and low severity assumptions where high severity had a case
to hospitalisation ratio of 2.5% (similar to the recent experience with Delta) and low
severity had 1%. High severity scenarios were particularly concerning, due to the
very high transmissibility and potential immune escape of omicron. However, by 23
December 2021, evidence was available to rule out the high severity scenario. It was
determined that the low severity scenario was a closer reflection of actual experience
for Omicron relative to the Delta variant, however considerable uncertainty remained,
especially given the very high expected transmissibility.
Additionally, the models were also then refined (‘Jan-22) to account for accelerated
booster rollouts that were not previously included (along with higher R0, from 5.1 to
6.5, to reflect early observed exponential increase in omicron against the
background of previous exposure to delta).

The TAC said there was “inevitable uncertainty” in the modelling of COVID-19 cases, deaths and admissions behaviour when considering the future trajectory the pandemic could take.

That uncertainty was around the timing and scale of the virus, transmissibility, and the choice and adherence of the restrictions.

Latterly in the pandemic the rollout and effectiveness of vaccinations also impacted the modelling,

Discussion the modelling, the TAC summarises, “The actual outcome has sometimes differed even from the most optimistic or pessimistic of the scenarios produced – generally the actual outcome has been ‘better’ than the optimistic scenarios rather than worse than the pessimistic scenarios. Where the actual outcomes are worse than the scenarios presented, this is generally mainly because of a new variant that has arisen that was not necessarily anticipated.”

It concludes, “Despite these uncertainties, it has been observed that modelling can provide a useful estimate of a picture of the future; modelling the spread of COVID-19 provides an invaluable means of projecting forwards to try and better understand what could happen, aid policy decision making, and help plan more effectively for the future in both the short and long term. ”

The full TAC document and range of graphs can be found here.

(Top image: Actual PCR -confirmed COVID-19 cases vs. Swansea model)